Mostly right. The better way to think of it is that you have a bucket of hours from which to draw phone screens, in-person interviews, and challenges. If you do no challenges today, you shouldn't expand the bucket of hours to accommodate them, but rather take offsets from other activities.

Phone screens are particularly useless and a good place to scavenge hours from.