> They all have to extrapolate from a subset of the entire population.
This only really works if you have a representative subset of he population you're extrapolating to. Is the distribution of experience roughly equivalent to the general population of "developers"? Age distribution? Company size? # of children? Home life? These kinds of demographics can have a big impact, and I see no discussion of how they were accounted for.
Heck, even the general work ethic culture difference between the UK vs the rest of the world could be a huge confounding factor that I know wasn't really accounted for.
Getting a perfectly representative subset of the population, means surveying the whole population of the world. Also, accounting for every imaginable variable is always impossible, because we cannot ethically grow people in an isolated environment.
Sure, a meta-analysis of several studies reveals more than one single study, but that doesn't make a single study bad or worthless - or as said in the parent - "a bullshit study".
There are plenty of ways to account for not having a representative sample distribution, I'm just saying that I didn't see any kind of discussion about demographics or confounding variables in the actual study.
This was a pilot study, a way to open an avenue of discussion, and to be sure I think the study warrants more investigation. They even say so: "In future we would like
to conduct larger studies over multiple countries."
Clearly, several people (including me) think that you did imply it - so, regardless of whether or not you intended to do so, that's what you communicated in your post.
In particular, the way in which you mentioned extrapolation in the context of the study (and the conversation in general) carried the implication that the extrapolation was valid.
This only really works if you have a representative subset of he population you're extrapolating to. Is the distribution of experience roughly equivalent to the general population of "developers"? Age distribution? Company size? # of children? Home life? These kinds of demographics can have a big impact, and I see no discussion of how they were accounted for.
Heck, even the general work ethic culture difference between the UK vs the rest of the world could be a huge confounding factor that I know wasn't really accounted for.