Is there something wrong with me or my brain? I looked through my legs in my living room and the distances and colors seemed the same as right side up.
There is most likely something to this. A dramatic example I've seen is where you take a picture of a head, and invert the mouth and view the picture upside down (so the head is pointing down, but the mouth is still right side up). It looks slightly off, but no more than any other picture that has been altered.
Now flip the picture back around right side up (mouth upside down) and if you aren't expecting it then it will quite likely make you jump back in horror. It actually hits some deeply rooted part of our primitive brain that just messes you up. (I just looked this up again, it is the Thatcher effect, and to me it is more frightening when the face is smiling).
Reminds me of the copypasta that’s says the “uncanny valley” indicates that in some time during our evolution we had good reason to identify something that looked human but wasn’t.
See also the "vampires" in Peter Watt's excellent Blindsight. In fact I thought the novel did refer to uncanny valley but apparently it doesn't, at least not with those exact words.
When I was learning to draw, one technique to help spot errors is to flip the drawing upside down. This made misaligned features, bad proportions, weird perspective issues immediately jump out.
It sounds like the 'prism-goggles' from the article's experiment control for the effect of just looking at it upside down. Clearly what's needed is to hang some students by their feet and send some others to space to see if it's the inner-ear effects or the posture that changes your perception :P
Art teachers also have people copy existing art upside down, or turn their still life upside down (if possible). By trying to reproduce something upside down, your brain doesn't do as much pre-processing and chunking into preconceived shapes, and you're forced to actually look at the shape of the thing.