I also agree. As someone who has recently co-founded a startup, this is something my co-founder and I have spent a lot of discussing and exploring.
My personal experience with this style of interview was that it really helped me to personally demonstrate my abilities. I was interviewing at a startup in San Francisco in 2012 who wanted me to do a quick version of "battleship" as part of their hiring exercise. I proposed, mid interview, "how about I help you solve a performance problem with your current system. What's your biggest issue right now?" My interviewer actually showed me, and we ended up solving the problem together and then deploying out the change.
The funny thing about that experience was that when we were done improving their platform, the VP of engineering was told what I did, and then had the interviewer go back and do the battleship exercise anyway. They needed to "benchmark" me against other candidates, and the only way to do that was with a consistent "test". I did far more poorly with the battleship exercise than I did helping them on their platform.
Can't really blame him, it might seem kind of arrogant to refuse to do the task they give and propose one yourself, although it is pretty impressive that you could do something useful on a codebase you haven't seen during the code interview.
It's also important to make sure people are well-rounded. If one can tweak an architecture to put a queue in the right spot, she/he may still be the person that cannot properly implement simple concepts like "separation of concerns" or forgets input validation or security checks.
At the end of the day, with start-ups I really value that mix of system and hands-on code. Without initial hires having both, it's harder to build a solid system while iterating so quickly. Amazing people skills are also key.
Over-leveling and bad hire situations are no good for anybody.
> They needed to "benchmark" me against other candidates, and the only way to do that was with a consistent "test".
If your employer were ever sued by a candidate for discrimination and it came to light that you had received different treatment than other candidates, some very uncomfortable questions regarding the reason would be asked in the courtroom. An unfortunate bit of fallout from the litigious environment we live in today, but it is what it is.
My personal experience with this style of interview was that it really helped me to personally demonstrate my abilities. I was interviewing at a startup in San Francisco in 2012 who wanted me to do a quick version of "battleship" as part of their hiring exercise. I proposed, mid interview, "how about I help you solve a performance problem with your current system. What's your biggest issue right now?" My interviewer actually showed me, and we ended up solving the problem together and then deploying out the change.
The funny thing about that experience was that when we were done improving their platform, the VP of engineering was told what I did, and then had the interviewer go back and do the battleship exercise anyway. They needed to "benchmark" me against other candidates, and the only way to do that was with a consistent "test". I did far more poorly with the battleship exercise than I did helping them on their platform.