"Estimates" are for things you've done before - like you can estimate building a house, because people have built houses before. The more like an existing house, the better you can estimate it.
Software is invention and construction. The construction part is pretty easy to estimate. The invention part is ... very very hard. I'd like to say it's impossible. I'd like to see the software industry use a different word than estimate.
You've perfectly nailed the fundamental issue here.
For many software projects, you simply cannot make meaningful granular estimations for parts of them. It doesn't matter how many story-point poker sessions you hold. Some software work cannot be reduced to a positive integer. In our business scopes creep, bugs plague us, and myriad issues make precise estimations pointless. It's engineering Numberwang.
I get that folks want to tame a chaotic world. But sometimes you can't. And the software engineering field in particular feels wildly neurotic about pretending otherwise. Sometimes I imagine applying our project management tools to other technical fields and laugh at how insane we must look.
That estimation is hard does not make it impossible. That it may be inaccurate does not make it worthless. Special pleading that our industry is somehow the only one that faces uncertainty and novelty doesn't really hold water.
> "Special pleading that our industry is somehow the only one that faces uncertainty and novelty doesn't really hold water."
I'm sure you're right. I don't think we should abandon all hope. I just think we should be more honest. I also think that software has fewer constraints than other industries, which is a blessing and a curse for estimation
Software is unique because it can be copy pasted. A builder that makes a house in 4 months the first time might estimate 3.5 months for a near identical house. But to deploy the same desktop app on two computers you merely reuse the installer from the first computer, and to recreate the same page in a different app you can copy the code, so no one ever estimates doing the same work twice. Every software problem has at least a little novelty, because if it isn't at least a little different, the work is already done.
>I get that folks want to tame a chaotic world. But sometimes you can't.
No matter how little you know about something you want to estimate, there is an infinite difference between what you do know and knowing nothing. The right thing to do is to represent your knowledge without fabricating it.
Saying "I can't make a point estimate, therefore I can say nothing" is a false dichotomy.
Even if you say, x is between 0.001 and 10^10, you are narrowing things down.
There is a great paper mentioned periodically on HN, where they examined the famous Drake equation and show that when you get rid of point estimates and use reasonable ranges for each variable, the "paradox" that we don't see aliens vanishes. Despite many of the variables having uncertainty of many orders of magnitude.
I'd beg to differ. There is very little invention going on. Most software solutions tackle well-known problems, customized to a particular business need. Akin to building a house, but with specific owner requirements (three-car garage, etc).
It gets a little complicated partly because of the industry's penchant for reinventing its tools on a rolling basis. In the trades, technology remains largely unchanged over decades, and only truly useful new advances are integrated into workflows (for example, laser level vs plumb bob or spirit level).
In software, new technological approaches are adopted on a continuous basis, and so a lot of effort is spent on learning how to solve old problems with new tools.
> Most software solutions tackle well-known problems, customized to a particular business need. Akin to building a house, but with specific owner requirements (three-car garage, etc.).
Not really. The "building" part in software really is compiling and deploying your code. With CI/CD being set up, this takes a few minutes and requires almost no effort.
So what's left is not "building" the house but "designing" it. Unfortunately, the comparison you give fits more for buying a software and installing/configuring it. Then yes, this should be rather well estimatible, given that you have done it before.
But most software developers are not paid for that. They are paid to design a space station that has never existed before. Sure, they can use existing parts (libraries, frameworks) but it is still design in the sense that certain expectations are simply not possible. Or someone wants that the space station is connected to the other space station on another planet and we don't even know how the other space station looks like (it's built by another company). Who knows if it's even possible to connect them? And if not, the other space station might have to be adjusted to be connectable and this must be done by the other company. How long will that take? Who knows...
By the way, don't think about a space station like the ISS here. Please think about a space station like the star wars death star. Because that's the complexity of most software where estimates are desired and important.
And we are only talking about design, not building. The building part is easy - once we have designed it, we can actually copy it and have 100 of these exact space stations easily.
If you said there is very little fundamental research going on, I would agree with you; but even putting together building blocks in a new way is invention. If you look at patents (arguably a unit of invention), most of them don't have fundamentally new building blocks, just existing blocks put together in new ways.
Very true, I think those estimates are actually two ideas/types crammed into one value.
1. The construction part, as you said, can be estimated.
2. I'd just call the other thing "allocated time" instead of "estimated time".
Any time someone asks me how long it will take me to fix a bug that I haven't really looked at yet, or to plan some new feature or something like that, and they badly need a number, I ask them how much time I should allocate to that. I can't promise to have something like that done by that time, but it gives us both an idea about how to treat that problem.
For example, we could allocate two hours to fix a bug, with the understanding that if that turns out to not be enough then we'll need to talk about workarounds. Or we can allocate two days to plan a new feature, and the best solution we can think of in that time shall be the one we use.
This is brilliantly put. That's pretty much what engineering is – constraints and tradeoffs – giving software project planning/execution a healthy vocabulary to talk about makes it so much better. Much of the crappy feeling that developers go through by putting themselves into a corner can be avoided if everyone had better mental/language models to think/talk about it.
Yes. So you can more accurately estimate replicating components you’ve written before, eg another database client, but estimating new software is a successive sequence of uncertainty reduction. The first pass being highly intuitive, the second pass, with more of the details worked out, less so... until you’ve completed the first working version, and you can finally say in hindsight, it actually took this long.
Yes, and if you find yourself getting better at estimation, that's probably because you have failed to build proper abstractions. With proper abstractions, the cost of the same stuff should be minimal, so the part with no good priors predominates.
I mostly agree, but for research, estimating the estimation is often good enough in practice.
For me it often something like this:
I don't have a clue how the hell to do what you asking for. But maybe implementing .. might help. Can't guarantee but it might. I think I can confirm or disprove that spending .. on the prototype subject to following limitations.. If then we find out it actually works for you, we'll go from there but approximately gonna take .. extra to rework the prototype into production-quality stuff. If it won't, I'll think about something else to try.
Software is invention and construction. The construction part is pretty easy to estimate. The invention part is ... very very hard. I'd like to say it's impossible. I'd like to see the software industry use a different word than estimate.