SSH agent has other interesting possibilities too - it's an opaque channel from far server to your local machine, there must be various cool hacks you could do.

Years ago I wrote something to copy a file from a remote machine to my desktop over ssh-agent, worked well back then. https://matt.ucc.asn.au/ssh-xfer/