This is seriously f'd up. When I was hired by IBM years ago, they also had a test called the IPAT (Initial Programming Aptitude Test) but, guess what?, there was no programming in it! Instead, the test was designed to see if one had the basic reasoning and logic skills which would be necessary to solve a wide variety of programming-related problems -- not one stupid, specific problem in a particular language. That's just asinine.
Making sure that developers have the "needle in a haystack" solution that you're looking for almost guarantees that you'll weed out the most talented folks -- the ones who are programming generalists that can reason about and solve any kind of programming problem.
I believe these run into legal grey areas because they have to be obviously relevant to the job in question, which likely leads to the tests getting progressively more domain-specific as time goes on.
Programming does seem to be a field in which general intelligence is a big predictor of success. Whereas a field like management probably personality and general intelligence are more equally weighted.
Well yes. And that is what the current crop of "programming puzzle" interviews essentially are; the next iteration of attempting to measure people on a cardinal scale instead of an ordinal scale.
The key to remember is that evaluating on a cardinal scale is much more efficient than evaluating on an ordinal scale, so companies will do everything they can to transform the "secretary problem" into one where they measure absolute metrics instead of relative metrics.
Now, whether those metrics align with business value is a separate issue. All evidence suggests that they are largely independent of business value.
HR told me I couldn't ask such questions without proving they were useful first.
Basically a scientific experiment, have someone else ask and grade the questions, but those results were not considered for hire/nohire. Then 6 months after a bunch of people were hired we pull the results of the test and compare to their performance reviews. If the questions were a good predictor or future success we could use them on future interviews.
The overhead of this was more than we were willing to pay so we went back to HR's allowed questions.
My previous company issued the Wondelic Test (https://en.wikipedia.org/wiki/Wonderlic_test) for everyone. I don't know the details but there was a minimum for different positions. There was also a personality portion and they were looking mostly for "A" type personalities, and a small subset of others. I don't have the "A" type personality, but I'd like to think the IQ portion made up for it LOL.
1) there are not even enough generalists, good generalists are expensive, and bad generalists... you don’t want them.
2) as a generalist, I am quite OK with hiring specialists, if they are really good at what they do. I am a “full-stack”, which means that while I can do both frontend and backend, I can’t shoot React and CSS like a machine gun, nor can I squeeze the last bits of performance from JVM. I hire specialists to do that, after I got the ball rolling.
I think this can really depend on the size of the business. At a small company, you probably can't afford a specialist for each area of responsibility. At a big company, you probably can't afford _not_ to have a specialist.
Am generalist, I hire generalists. My favourite feature is that they/we don't know what we're not supposed to do. So we just go, damn the torpedoes. Smart generalists GSD
Sometimes you really do need particular skills and experience that take decades to learn. Otherwise you don't even know what you don't know and you don't know what mistakes are usually made in a given area.
Not knowing what to do or what others did can be an advantage. Einstein developed his theory of relatively by simply discarding all the known theories about how physics worked. In my own experience at IBM, I solved a bug that had been repeatedly opened over a series of 9 years by simply asking the question, "Is this number a hex or a decimal?"
"Einstein developed his theory of relatively by simply discarding all the known theories about how physics worked."
That's ludicrous nonsense. The primary breakthrough of the theory of relatively was taking what the math had been telling physicists about the universe for years seriously. In hindsight, everything necessary was there before Einstein, it just was very deeply assumed not to be reasonable. Relativity builds on numerous pre-existing foundations and is very much in the "standing on the shoulder of giants" tradition.
If you took out the bits of relativity that were developed by others, you'd be left with not very much. Relativity without the Lorentz transform... I'm not even sure how one would resolve that. As I said, the vast majority of the insight was just taking what the math was already saying seriously. Einstein put it into a final form, and this is important and valid work, but it's not like he invented all the mathematical tools from scratch.
Either you've misinterpreted Adam Frank's work, or it isn't excellent.
I'll counter-cite Reflections on Relativity, which contains a lot of deep analysis of the mathematical history of relativity both before and after Einstein: https://mathpages.com/rr/rrtoc.htm which makes it quite clear both what he did, and did not do.
I'm no astrophysicist, though Frank is. The two concepts I'm referring to as being "discarded" were Newton's idea that motion (ie: time and space) exist in a fixed frame of reference. I think it's fair to say Einstein tossed that out wholeheartedly. The other idea Frank mentions that Einstein threw away was the idea of an æther that filled the voids in the universe. According to Frank, Einstein didn't like this idea so he just worked without it.
> by simply discarding all the known theories about how physics worked
But he wasn't ignorant of them. He knew the existing work. He was extremely well read in physics and maths. He wasn't a generalist!
If you get a group of random generalists to build your compiler, or your graphics engine, or your cryptography stack, they aren't going to know what they don't know, and it'd take them decades to come up to speed.
As somebody who wrote all of the above (and a lot more) I call B.S. on this one.
The amount of unique stuff in each subfield is not that great, and if you have a wide and solid base in different subfields, picking up the unique ideas and techniques isn't that hard or time consuming.
But after a generalist has it, he is a lot more productive and creative because he often applies stuff from other subfields to come up with better/novel solution to the problem at hand - something a specialist cannot do.
Most opportunities look for some high degree of specificity. For contract work or a pre-defined task, that makes sense but for companies that need a rich talent pool, I don't know why there appears to be little opportunity for generalists.
That seems much worse than programming tests IMO. If I have a bachelor's degree, and you're asking me to retake the SAT, you're just insulting my qualifications. At least programming tests have some legitimate rationale. Let's not pretend IBM has any fucking idea what they are doing.
It wasn't an IQ test, and IBM definitely knew what it was doing in ways today's "move fast and break things" culture cannot fathom. Their software was and still is used in many mission-critical applications because the company was rigorously focused on quality. When was the last time you heard of a bug in mainframe software affecting anyone?
While I agree with you conceptually, as a daily DataStage/InfoSphere user, there are plenty of buggy IBM products. I do think a general problem solving test is way more useful for vetting candidates than some stupid algo whiteboard assessment, and my company has been using something like that to great effect to recruit jr devs.
This was also back when companies were actually willing to train people. Using a test like that, they could try to hire people who had "potential" and then give them a career where they could grow, improve, and bring value to IBM
That doesn't mesh so well with the modern era of employers hating spending a dime on training, even if it would save them a dollar, and employees wanting to jump ship every other year for that sweet sweet 10% compensation bump
I don't know if people have generally been following Taleb's critique of IQ and other intelligence measures, but there are three tracks to it.
One is purely statistical -- that if you offer any metric of success, and try to correlate it with any measure of intelligence, even if the two are actually uncorrelated, then having dead people in the sample (that is, people who score 0 on both) will increase the quality of the correlation. Applicable because when you eliminate people who score low on IQ tests and low on performance tests from the sample, the correlations tend to disappear in these studies.
The second is more intuitive, that intelligence is multifaceted and attempts to project it into a low-dimensional space will mask the inherent complexity, but will be capable of identifying individuals that score low across many dimensions. That leads to the condition in the first issue.
The third is probably the most controversial; that IQ tests in particular do not measure generalized intelligence, but measure how well you can do "drone work", so for places where performance can be objectively measured, it will tend to be in tasks that do not require creativity or initiative in unpredictable directions because those are not easily measured, so thus reflective of the same sorts of tasks that IQ tests measure.
Making sure that developers have the "needle in a haystack" solution that you're looking for almost guarantees that you'll weed out the most talented folks -- the ones who are programming generalists that can reason about and solve any kind of programming problem.