Technically, you don't occupy the uncolored square for any finite amount of time, since the instant you land there, you are immediately moved elsewhere. So you don't get to see the gold on those squares. And yeah, it would be a pain in the butt to implement otherwise.