In practice if you're working in high dimensions and don't do PCA then you have very little chance of building a good model. It certainly isn't linearly valid if to you valid demands scale invariance, but it's an essential tool for unsupervised feature engineering.
Auto-encoders have some big advantages over PCA, but they suffer the same shortcoming (sensitivity to units of measurement) described in the original post.
Another common method is to scale to mean 0, variance 1. In my opinion this makes more sense since it handles outliers a bit better-e.g., consider a case where most of your values for a feature are scaled from 1 to 10 but there's one point with value 1,000,000.
I agree that you SHOULD normalize/scale data before running neural networks and autoencoders.... and this resolves the units issue in most cases (unless measurements in some units are non-linear functions of measurements in others).
But this scaling also resolves the issue for PCA. So, I don't see much difference between autoencoders and PCA with regards to original post's "dimensional invalidity" concern.
If anything, the scaling options you mention suggest "dimensional invalidity" isn't a big deal in practice for either method.
To expand on this a bit (please correct me if I'm wrong), there are three major uses of dimensionality reduction:
1. Reduce overfitting
2. Train models faster
3. Take advantage of unsupervised data
Regularization handles case (1) quite well. It can also be used in conjunction with most methods of dimensionality reduction such as PCA/auto-encoders.
PCA covers all three, but isn't as effective at dimensionality reduction compared to auto-encoders.
Auto-encoders tend to yield a better compression than PCA but take more time to train and produce output that's harder to understand. There is a bit of analogy here, auto-encoders are to PCA what neural nets are to linear regression.
L1 regularization can also be used for feature selection[0] and hence dimensionality reduction. I've in found practice that the speed of this can compare favorably even to fast implementations[1] of svd.
[0] Feature selection, L1 vs. L2 regularization, and rotational invariance, Andrew Y. Ng. In Proceedings of the Twenty-first International Conference on Machine Learning, 2004. http://ai.stanford.edu/~ang/papers/icml04-l1l2.pdf
[1]"Augmented Implicitly Restarted Lanczos Bidiagonalization Methods", J. Baglama and L. Reichel,
SIAM J. Sci. Comput. 2005.
