Effective Effort Estimation: The Importance of Stakeholder Collaboration in Project Planning

Place: Bengaluru

Meeting room of a Multinational Software Company

Stakeholders:

1)  Mr. Sahay- Product Owner

2)  Ms. Kaur- Architect

4)  Mr. Sinha- Quality Manager

5)  Mr. Steve- Scrum Master

Agenda: Selection of an effort estimate technique for the aerospace and defense project.

Mr. Steve: Let me drive the meeting! Friends, we all have gathered here to select the technique we will be using for our project that we would be using for our aerospace and defense industry project. I open the floor to each one of you to share their perspective on the agenda which will help decide on the technique

Mr. Adrian: Thanks, Steve! Let me take this opportunity in setting the context. As each of us would be aware that there are many alternatives available. There are many Mathematical models available. Let me give a brief on them. Mathematical models are a technique that can be used for effort estimation. These models are based on mathematical algorithms and statistical techniques, and they can be used to estimate the effort required for a project by analyzing data from similar projects.

Several different types of mathematical models can be used for effort estimation, including:

Three-point estimation: This model uses the most likely, optimistic, and pessimistic estimates of the effort required for a project to generate a range of possible estimates. This approach can be useful for capturing the uncertainty and risk associated with a project.

Function point analysis: This model is used to estimate the effort required for software development projects. It is based on the number and complexity of the functions provided by the software, and it can be used to estimate the effort required for both new software development and maintenance of existing software.

COCOMO: This model (Constructive Cost Model) is used to estimate the effort required for software development projects. It is based on the size of the software and can be used to estimate the effort required for both new software development and maintenance of existing software.

Parametric models: This model uses mathematical equations that are based on historical data from similar projects to estimate the effort required for a new project. The model’s parameters are calibrated using the historical data, and the model is then used to estimate the effort required for the new project.

Machine learning model: This model uses historical data and machine learning algorithms to predict the effort required for a project. These models can be very powerful and can consider many factors that might influence the effort required for a project. However, they can be complex and require a large amount of data to train.

Mr. Sinha: Thanks for the wonderful description, Steve. It’s worth noting that while mathematical models can be a useful technique for effort estimation, they are not always appropriate for all projects. They can be time-consuming to set up and may require specialized expertise. They can also be influenced by data quality, and if the data is insufficient or inaccurate, it can lead to inaccurate results.

Mr. Sahay: Rightly said! Mathematical models do not capture the uncertainty of the project and depend on the assumptions made based on historical data.

Mr. Adrian: That is correct. We as PMs always consider the specific characteristics of our project and the available data before deciding to use a mathematical model for effort estimation. This is as per the best practices to which we need to adhere. I would recommend the “Three-point estimation”.  It is a technique that can be used for effort estimation. It is a form of probabilistic estimation that uses the most likely, optimistic, and pessimistic estimates of the effort required for a project to generate a range of possible estimates. The technique is based on the PERT (Program Evaluation and Review Technique) method. I would further request Mr. Sahay to give a bit of explanation on the technique.

Mr. Sahay: Sure! Let me directly get into the process.

The process of three-point estimation involves the following steps:

1)  Identify the task or activity to be estimated.

2)  Determine the most likely estimate of the effort required for the task. This is the estimate that the team feels is most probable based on their experience and knowledge of the project.

3)  Determine an optimistic estimate of the effort required for the task. This is the best-case scenario and the team’s estimate of the least amount of effort required.

4)  Determine a pessimistic estimate of the effort required for the task. This is the worst-case scenario and the estimate that the team feels is the highest possible effort required.

We need to use a formula to calculate the expected effort, variance, and standard deviation:

Effort expected = (optimistic + 4 * most likely + pessimistic) / 6.

(Pessimistic – Optimistic) Variance = 2 / 36

The standard deviation is equal to the square root of the variance.

Mr. Sinha: Kindly hold Sahay. Let me check if my understanding is correct. The expected effort is the best estimate of the effort required for the task, and it is calculated by averaging the most likely, optimistic, and pessimistic estimates. The variance and standard deviation are calculated to provide a measure of the uncertainty around the estimate.

Mr. Sahay: Bang on Sinha! It’s worth noting that the three-point estimation is useful for capturing the uncertainty and risk associated with a project. However, we assume here, that the effort required for a task can be represented by a normal distribution, which may not always be the case. That might be a fallacy in our hypothesis. In addition, it assumes that the estimates are independent of each other, which is not always the case. Therefore, project managers should use this technique carefully and consider the specific characteristics of their project before deciding to use three-point estimation for effort estimation.

Mr. Sinha: A word of caution! Several real-life case studies demonstrate the importance of accurate effort estimates in the aerospace and defense industry. Let me cite a few examples of failures.

The Lockheed Martin F-35 Lightning II is a fifth-generation, single-seat, single-engine, stealth multirole fighter aircraft. Effort estimates for the F-35 program have been widely criticized for being unrealistic, leading to significant cost overruns and schedule delays.

The Boeing 747 is a large, long-range commercial jet airliner. Effort estimates for the 747 program were also criticized for being unrealistic, leading to significant cost overruns and schedule delays.

The Airbus A380 is a double-deck, wide-body, four-engine jet airliner. Effort estimates for the A380 program were also criticized for being unrealistic, leading to significant cost overruns and schedule delays.

The Space Shuttle program was a manned spaceflight program run by NASA, the space agency of the United States. Effort estimates for the Space Shuttle program were also criticized for being unrealistic, leading to significant cost overruns and schedule delays.

The Ariane 5 rocket is a European heavy-lift launch vehicle that is used to deliver payloads into geostationary transfer orbit (GTO). The effort estimates for the Ariane 5 program were also criticized for being unrealistic, leading to significant cost overruns and schedule delays.

Ms. Kaur: Wow! I appreciate the details. We need to be cautious! Do we have any more techniques?

Mr. Adrian: Yes, there are many options available. Let us now proceed with Function Point Analysis (FPA). It is a technique that can be used for effort estimation in software development projects. It is a functional measurement method that is used to quantify the functionality provided by a software system. I will not get into the history of it. Function Point Analysis (FPA) is based on the idea that functionality can be quantified and that software systems can be compared based on the functionality they provide.

It’s worth noting that function point analysis (FPA) is a relatively simple and straightforward method that can be applied to a wide variety of software development projects. However, it can be subject to interpretation and bias, and it may not always be appropriate for all projects. Additionally, it does not consider the complexity of the code, which can be a significant factor in determining the effort required for a project.

Mr. Sahay: A small question. Can we handle the complexities of the aerospace industry? I am a bit skeptical.

Mr. Adrian: Let me help you get a perspective. It is possible to use Function Point Analysis (FPA) in the aerospace and defense industry, particularly in the development of software systems for military applications. FPA can be used to estimate the size and complexity of software systems used in defense systems such as weapon systems, command and control systems, and intelligence systems.

Ms. Kaur: As far as I know, FPA has been actively used in the defense. It can be used in various areas. Let me tell you about a few real-time use cases.

The US Navy used FPA to estimate the size and complexity of a software system used in the control and guidance of submarines. The FPA results were used to estimate the cost and schedule for the development and maintenance of the system.

The US Air Force used FPA to estimate the size and complexity of a software system used in the control and guidance of unmanned aerial vehicles (UAVs). The FPA results were used to estimate the cost and schedule for the development and maintenance of the system.

The UK Ministry of Defense used FPA to estimate the size and complexity of a software system used in the control and guidance of military vehicles. The FPA results were used to estimate the cost and schedule for the development and maintenance of the system.

The Australian Defense Force used FPA to estimate the size and complexity of a software system used in command-and-control systems. The FPA results were used to estimate the cost and schedule for the development and maintenance of the system. There might be many other examples. I have done a bit of research on it.

Mr. Sinha: This information will help us a lot. However, let me update you a bit on Machine Learning. Of late, there are many initiatives taken on Machine Learning. We need to figure out if we can leverage our understanding to attain the goal. Let me take this opportunity to apprise all of you of the various techniques in machine learning which will help us.

Neural networks are a type of machine learning model that can be used to develop data-driven models for effort estimation in software development projects. Neural networks are inspired by the structure and function of the human brain and consist of interconnected nodes, or neurons, that process information. They can be used to model complex relationships between independent variables (also known as predictors or explanatory variables) and dependent variables (also known as the response or outcome variable). In the context of effort estimation, the independent variables might include factors such as the size of the project, the complexity of the project, the number of users, etc., and the dependent variable might be the effort required to complete the project.

Several different types of neural networks can be used, including:

Feedforward neural networks: This type of neural network is the simplest type of neural network. It consists of layers of interconnected nodes, where each layer receives input from the previous layer, and the last layer produces the output.

Recurrent neural networks: This type of neural network is designed to process sequential data, such as time series data. It consists of layers of interconnected nodes, where each node receives input from both the previous layer and the previous time step.

Convolutional neural networks: This type of neural network is designed to process image and video data. It consists of layers of interconnected nodes, where each layer is designed to extract features from the input data.

Mr. Adrian: Neural networks can be powerful and accurate for effort estimation in software development projects because they can model complex relationships and patterns in the data. They can also handle many input variables and adapt to new data and changing conditions. However, developing a neural network model for effort estimation can be a complex and resource-intensive task, requiring specialized expertise and large amounts of data to train the model. Additionally, neural networks can be difficult to interpret and understand, and it can be challenging to identify the factors that are driving the model’s predictions. Therefore, we should use this technique carefully and consider the specific characteristics of their project before deciding to use neural networks for effort estimation.

I would like to hear more from Ms. Kaur as she is an SME in Machine Learning. Over to you.

Ms. Kaur: Thanks, Adrian. A machine learning model is a set of algorithms that are designed to automatically learn from data, identify patterns, and make predictions or decisions without being explicitly programmed. They can be used to develop data-driven models for effort estimation in software development projects. Machine learning models can consider many factors that might influence the effort required for a project and can adapt to new data and changing conditions.

Several different types of machine learning models can be used, including:

Supervised learning: This type of machine learning is used when the data includes both input variables and the corresponding output variables, also called labels. The model is trained on this labeled data and can be used to make predictions on new, unseen data.

Unsupervised learning: This type of machine learning is used when the data only includes input variables and there are no corresponding output variables. The model is trained on this unlabeled data and can be used to identify patterns and structure in the data.

Reinforcement learning: This type of machine learning is used when the model learns through trial and error, receiving feedback in the form of rewards or penalties. It’s mainly used for decision-making and control problems.

Machine learning models can be powerful and accurate for effort estimation in software development projects because they can model complex relationships and patterns in the data. However, developing a machine learning model for effort estimation can be a complex and resource-intensive task, requiring specialized expertise and large amounts of data to train the model. Additionally, interpreting and understanding the decision-making process of a machine learning model can be challenging, and it can be difficult to identify the factors that are driving the model’s predictions. Therefore, we need to use this technique carefully and consider the specific characteristics of their project before deciding to use machine learning models for effort estimation.

Mr. Sinha: Many companies in the software industry use real-time effort estimation to plan and manage their projects. I have checked a bit about what is used by the leaders in the software industry.

Microsoft uses a combination of expert judgment and historical data to estimate the effort required for its software development projects.

IBM uses a range of techniques for effort estimation, including function point analysis, use case points, and the COCOMO (Constructive Cost Model) method.

Google uses a technique called “reference class forecasting” which involves using historical data from similar projects to estimate the effort required for a new project.

Atlassian uses a technique called “planning poker” where developers estimate the effort required for each task by holding a planning meeting and then using the consensus estimate as the final estimate.

Amazon: Amazon uses a technique called “two-pizza teams” where small teams (typically no more than eight people) are used to estimate the effort required for each task, and then use the consensus estimate as the final estimate.

Mr. Adrian: It is important to note that effort estimation is not an exact science and there will always be some level of uncertainty involved. However, by using a combination of different techniques and methodologies, and by continually monitoring and updating the estimate throughout the project, all the project stakeholders can take informed decisions and improve the chances of project success. But one thing is clear. We all need to keep in mind that effort estimation is not a one-time process.  We should at regular intervals review it. The estimates need to be updated as the project progresses, to consider any changes in project scope, requirements, or resources. Overall, effort estimation is a key aspect of project management that helps to ensure that a project is completed on time, within budget, and to the desired level of quality.

Mr. Steve: Let us close the session. Now we are clear on how to go about making the project a big success!

References:

1. “Software Engineering Economics” by Barry W. Boehm
2. “Agile Estimating and Planning” by Mike Cohn
3. “Project Management: A Systems Approach to Planning, Scheduling, and Controlling” by Harold Kerzner
4. “The New Dynamic Project Management: Winning Through the Competitive Advantage” by Anthony T. Newbold

Disclaimer: Any resemblance of characters or names used in this document to real individuals or entities is purely coincidental. These characters are fictional and are used solely for the purpose of illustrating concepts and scenarios. Any actions or statements attributed to these characters should not be interpreted as reflecting the views or actions of any real individuals or entities.

Related blogs