Estimating time for a task in a project, and for a whole project, is not easy. Projects regularly tends to take more time than expected. In this article I will point out some things that can improve your estimations. And a way to deal with the remaining uncertainty.
In short, these things are:
Estimate with uncertainty
Compare with previous projects
Cultivate your culture
Be clear on what a task includes
Use more than one set of eyes
Use continuous planning and The PlanMinder
Evolution has made humans into optimists. Our brains are hard wired to believe that this time, this time it will go much better than the last. We tend to focus how we want things to go, rather than to worry about the things that may go wrong. Now this is a good thing, otherwise our species would probably have been extinct by depression.
The first step to handle this problem is to be aware of it. The second is to estimate with uncertainty. One number for how much time it would take if everything does go really well. And one for how much time you need to reserve if you are to be reasonably sure you can complete the task on time.
If you have records of similar tasks done before, compare with them. Even if you are sure that you will not do the same mistakes again, acknowledge that new unknown problems may arise. Do not be alone doing the estimate. Discussing with another person increases the probability to think of things that may cause problems.
The culture in the organization may be a contributor to bad estimates. I have been in a planning meeting where a developer thinks a task may take 100 hours, based on his experience. Our boss got upset wondering how it could take so much time, expressing that it should not take more than 40 hours. The developer backed off and said that it may be possible to solve it in 60. Or maybe even 40 if there were absolutely no problems. It ended up taking something like 120 hours, and our boss could again complain that everything takes pi times longer than you think.
In other instances it may be a co-worker that shames you into making overly optimistic estimates. If this is a prevailing phenomena in your project organization, take steps to change this culture. Changing culture takes time, but if it is not deliberately shaped, it will shape itself. And it may not take on a shape you like.
A quicker solution to implement that helps a lot is, again, to use two numbers. The developer can acknowledge that if everything goes perfectly, it may be possible to have it done in 40 hours. But to be reasonably sure, taking into account all the things that according to experience can cause problems, you need 100 hours.
Another common will to please problem arises when the project has started, when you ask if the project or task will be done on time. The answer will be yes, if there is any possibility it might be true. The person you ask knows that yes is the answer you want to hear, and answering no will be seen as a failure, and cause problems.
A better thing to ask, at task level, is how much work remains. That is a neutral question, and shifts focus to the future and to make an accurate estimate. This estimate too should be with uncertainty.
When discussing a task it is important to agree on what it encompasses. When a developer is asked how much time it takes to solve a problem, they naturally focuses on the difficult part of the task. It may be easy to forget the auxiliary work, like testing or documentation. There might also be work that falls between tasks, like air between balls in a box. Those working with one task assumes it it handled in the other, and vice versa. Or maybe neither remembers that this work needs to be done.
To handle this, be as explicit as necessary when describing the tasks. Do the estimation together with somebody else to reduce the risk of forgetting things. Look at previous projects to check if you have missed something.
This may be true for the project as a whole too, that some tasks are forgotten. Early in my career we almost always forgot about the packaging in our product development projects. This was until we wrote it in as a point not to forget in our guidelines.
As suggested before, using two numbers in your estimation has many benefits. The first number should be the “if everything goes well” number. It should not be an impossible fantasy number, but it should still be quite unlikely that it will take less time than this estimate. The second estimate should mean that you are reasonable sure that you will be able to finish the task in less time. I suggest defining reasonable sure as 70 %, seven times out of ten it will be done in less time. It is an arbitrary number, but it matches what most people would consider reasonable in this context. And it is a probability number we can grasp. If you ask for an estimate that is 99 % certain you will get a wide variety of guesses. Since people would have to imagine, consider, and count, many extreme cases.
Another benefit to defining estimates with these two well defined numbers is that you can get all developers to talk the same language. If you only get one fixed estimate, you do not know if that is the best case scenario, or the best case scenario multiplied by pi.
When you add up individual estimates to see what the project total is, you could do that correctly in a spreadsheet. Handling it correctly when scheduling tasks on a timeline is more difficult, and requires support of a dedicated tool, like The PlanMinder.
Getting a probability distribution for cost of a project lets you estimate how likely it is that you will meet your financial goals, and what risks you are taking. Uncertainty on the timeline lets you see how likely it is that you will meet deadlines. Or the other way around, lets you select what risk level you accept when you have to make a promise on when the project will be completed.
The timeline is not only affected by the individual tasks and their estimates. The planning and assignment on people to tasks also plays a major role. With a tool like The PlanMinder that visualizes this, you can tweak your planning and see what effect it has.
The plan, as you have estimated it, is just a starting point; what you believe before you have started. As work progresses you will learn more, and you should update your plan accordingly. Doing that, you always have a plan that is valid, and you can follow the trend of the probability that you will reach your deadline. If the probability is dropping, you can plan for countermeasures, and decide at what point you must activate plan B in order for it to be effective.
If you are just asking if the deadline will be met, you will get the answer yes – Until its too late to do anything about it. Adding a lot of people to a project to push it over the finish line is at best ineffective, and often contra-productive. Expanding the project team with a few people early on is much more cost effective, less stressful and more likely to actually succeed. Continuous planning is a method to make it more likely to detect the need to adjust plans earlier. And to determine when it makes most sense to do it.
I have seen a NASA study showing that when they spend 25 % of the total project time before the project start, it goes much less over budget than when they spend 5 % in preparation. And that is natural. The more you know the better the estimates will be. However, where I come from even 5 % would be considered a lot of time for preparations. Some of the time used for preparations will be extra time, increasing the cost for the project while making it less risky. Especially if the scope of the project may change after it has started.
When you can see how big the uncertainty is, and where it comes from, you can determine if there are areas you have to investigate further to reduce the uncertainty before committing fully to the project. You can also sequence the work to take care of the high risk task early on, and have a chance to abort if it turns out to be too costly.
With continuous planning you will continuously increase the knowledge about the project, and have estimates that reflects that. More accurate information leads to better decisions, and more successfull projects.