Obviously sophisticated screenreading software could extract secrets through VNC, or malware could be entered via keyboard.

But the "VNC gap" immediately obviates the risk of unsophisticated attacks. No more viruses spreading over SMB. Rather limited bandwidth (preventing raw copies of The Hobbit in 4k from being ripped via VNC).

It isn't bulletproof, but you can't pretend it doesn't help.