I'm considering using those yubikey modes for ssh or local login on certain servers and in certain situations, though it requires some changes on each server to enable it.

I'm quite fond of the Yubikey NEO's openpgp applet paired with gpg-agent's ability to act as a compatible ssh-agent, allows standard SSH key login to any server with no server changes at all. I love the idea of my GPG and SSH key being truly portable in a very reasonably sized formfactor as well.