If he wants to quit and compete with you he should be able to even if this destroys your business. The problem is he used his access while employed to redirect customers. It wasn't fine after he was fired not because it wasn't ok for him to compete with his prior employer but because he had already swiped your customer base. The prior behavior and challenges after he left are inextricably intertwined. Him quitting didn't separate them.
If he had done NOTHING while employed and quit to do his own thing do you think it would have been a problem? If so why?
1) It wasn't my business, and I'm not sure why you seem to think it was.
2) He only started the business because he was able to siphon off customers, and felt it was easy money.
3) If he had quit, and started his own business (even based on the prior one), and started from scratch, that wouldn't have been a problem. Him waiting a period of time (non-competes pretty much HAVE to be time limited to hold up anywhere) would not likely have changed it either.
But that would have required more work and more risk.