Does RunDeck keep a log of every response? With Commando.io you can flip back through previous executions and see a snapshot of which servers got which recipe, and what the results were.

It definitely does: when I launch a job in RunDeck, I can see the live output (essentially a form of tail -f in the RunDeck web interface) and I can see the output of every previous job as well.