Why Software Delivery time is hard to estimate?

Reading: Working with Coders: A Guide to Software Development for the Perplexed Non-Techie (English Edition)

Delay

プログラマーとお仕事をするということ

Under the name of engineer, software engineer is different from other engineering like construction where the cost will blow up beyond estimation. There are two general causes

Imagination Problem(outside eng part)

Building software is building human process. 1.the process itself is abstract so we have to imagine. Though we can use an MVP like user story to sketch out the though idea, most of the time we need to see the full cycle to judge yes or no the software work as we want. 2. Little spec change may lead to enormous change in codes. Some times the 5% of the not fulfill spec(automatic formatter or smart grouping) cannot be filled by the library taken somewhere, so eng have to rebuild an proprietary one. The result is long time with seemingly no progression. 3. One counteract of #2 is to design more detailed beforehand. But it will be paradoxical since more spec will lead to more uncertainty and therefore more change( 100spec * 5% < 200spec * 3%)

Estimation Problem(inside eng part)

Estimation from engineer is inaccurate. Because usually the estimation is something like: - Estimation time: 3Day But it turns out - 1st implementation: 3Days + 2nd impl after feedback 1Day + BugFix 1Day + Contingency(Server down or serious bug from else where) 1Day = 6Day See, 100% over. And this is a very conservative case when estimation goes young.

80% Syndrome

When eng think they are done 80%, actually they are at 50%.

One interpretation is that when planning, the main part( creating the api) looks 80%, but the remaining part(write test, update test, adjust config, integratino) seems minor as 20%, so when the 'main part' is done, engs feel 80% is done.

To deal with this estimation bias, we can apply - Buffer multiple, when told 3 days we count it with a multiple as 6, or more - Use scrum to calculate long term velocity of the team(not individual eng), and update the velocity in every sprint to reflect real velocity. This approach needs several sprint to estimate(require warm up time) and will changed greatly when new member joins.