On March 22, 2012, I posted a blog article concerning KPIs for Business Analysis and Project Management. That was one of our best read blog articles, having over 1,000 reads. This article provides additional KPIs for business analysis.
Before we define KPIs for business analysis, let’s define what business analysis is. The International Institute of Business Analysts (IIBA) describes business analysis as “the set of tasks and techniques used to work as a liaison among stakeholders in order to understand the structure, policies, and operations of an organization, and recommend solutions that enable the organization to achieve its goals.” IIBA describes business analysis as a discipline, rather than as the responsibilities of a person with the job title of business analyst. According to IIBA, business analysis may be performed by people with various job titles, such as systems analyst, process analyst, project manager, product manager, developer, QA analyst, business architect, or consultant, among others. In this article, I may refer to a business analyst but I am actually referring to the discipline of business analysis.
Anyone who has ever worked on a complex and lengthy software development project knows that effective business analysis skills can mean the difference between success and failure. Generally, business analysts have many responsibilities on a project and their involvement starts at the beginning of a project. Most business analysts “own the requirements processes,” where they work with key line-of-business executives and users on just what it is they want from a new application”, says Carey Schwaber, a senior analyst of application development at Forrester Research. “If you believe that software projects succeed or fail based on the quality of the requirements,” Schwaber says, “then you believe that software projects succeed or fail on the basis of business analysts, too.”
However, business analysts have many other important duties beyond gathering requirements. However, the other business analysis duties are often not well known. In fact, according to Schwaber and fellow Forrester analyst Rob Karel, not many people, including many business analysts themselves, have determined a standard definition for the business analyst position. Below is a list of typical business analysis responsibilities for a project.
- Identify and understand the business problem and the impact of the proposed solution on operations of the organization.
- Document the solution scope including objectives, added value and benefit expectations.
- Identify and assess the impact of the solution on all project stakeholders.
- Review impacted business processes and work with business stakeholders to identify and make business process improvements that result in cost savings, higher quality, increased efficiencies, lower risk, and shorter cycle times.
- Elicit needs from stakeholders.
- Develop a clear set of requirements that are complete and understood by development resources and business stakeholders.
- Manage requirements and associated changes to the requirements through the project lifecycle.
- Work with stakeholders and associated business units to reduce resistance and effect organizational change.
Key performance indicators (KPIs) are high-level snapshots of an organization or a business process based on specific predefined measures. KPIs are typically captured and conveyed in a combination of reports, spreadsheets, or charts. In developing KPIs, a user or developer defines target performance levels and then decides the best way to represent variance from that target.
I define the business analyst’s primary role as working with stakeholders to gain an understanding of the problem or opportunity and then define and manage requirements for a solution that solves the problem and delivers value to the business. Based on this definition, business analysis KPIs can be organized into the following categories:
- Solution planning and analysis
- Stakeholder engagement
- Achieving business case benefits
- Business process optimization
- Requirements Quality
Solution Planning and Analysis KPIs
- Number of non-IT recommendations implemented – Often business analysts stay focused on just providing the IT solution and miss many low hanging fruit which are not IT related.
- Percentage of Business Objectives that were Achieved – SMART business objectives should be defined for every project. However, the real measure is how many of the business objectives were achieved.
- Scope Statement Quality Index – Carefully defining good solution scope statements is key to defining good requirements. Conduct an independent peer review of the scope statements and assess a numerical score using predefined critiera.
Stakeholder Engagement KPIs
- Stakeholder satisfaction index – Project stakeholders should be interviewed to determine their satisfaction with the solution. This might be done with a series of questions such as:
- Were your needs addressed?
- How effective were communications with the solution team?
- Did you have transparency into the project?
- Was your investment of time worthwhile?
- Number of stakeholder comments received? – How many comments or concerns were received about the requirements?
- Stakeholder lifecycle participation – What % of the stakeholders participated in the following lifecycle events?
- Solution scope definition
- Requirements development
- Documenting business rules
- Defining key terms
- Design review
- Providing clarifications during build
- Participating in demonstration sessions
- Acceptance Testing
- Instructional design review
- Number of stakeholder complaint (emails and letters) received – Sometimes stakeholders get frustrated and write emails or voice complaints. Many of these are because of poor communication problems.
- Number of key stakeholders that were either not identified or identified late in the project – How many stakeholders were missed during the stakeholder analysis activity?
- Number of participating stakeholders? – How many stakeholders were interviewed or participated in focus groups?
- Stakeholder Attendance % - The number of stakeholders that attended meetings versus the number of stakeholders that were invited
- Number of needs identified by stakeholders – Stakeholders provide needs that are translated into requirements. One of the best measures for stakeholder engagement is to see how many needs they provided.
Business Process Optimization KPIs
- Business Process Productivity Increase % – The actual increase in business productivity after the system has been implemented.
- Cycle Time Reduction % - The decrease in cycle time that occurred after the system was implemented expressed as a percentage.
- Unit Cost – The average unit cost decrease after the solution has been implemented and operational for 6 months.
- Increase in Revenue attributed to Solution – The actual increase of revenue that can be directly attributable to the solution.
- Decrease in Cost attributed to Solution – The actual decrease in cost that can be directly attributable to the solution.
Achieving Business Case Benefits
- Cost Variance – % of Actual Cost to Projected Cost identified in business case.
- Revenue Variance – % of Actual Revenue achieved versus revenue increase predicted in business case.
- User Satisfaction Index – Are the users satisfied with the delivered solution?
- Deviation of planned ROI – The deviation of the planned Return on Investment (ROI) is the difference between Return on Investment in the planned baseline and the actual Return on Investment. ROI is the return an investment will generate annually as a percentage of the total investment.
- Deviation of net present value (NPV) – The deviation of the planned net present value (NPV) is the difference in value between the planned baseline against the actual net present value. NPV is a method used in discounted cash flow analysis to find the sum of money representing the difference between the present value of all inflows and outflows of cash associated with the project by discounting each at a target yield.
- Deviation of planned break-even time – The deviation of the planned break-even time is the difference in time between the planned baseline against the actual break-even time. The break-even time is determined by the point where the business expenses equal the income (or cost reductions) generated, with neither profit nor loss.
Requirements Quality KPIs
Below are representative KPIs to measure the effectiveness of business analysis for projects.
- Percentage of rework attributable to requirements – Rework is a serious problem on most projects, representing about 40% of total project cost. According to industry studies, about 70% of this rework is related to ambiguous, inaccurate, or missing requirements.
- Percentage of projects with prioritized requirements – Prioritizing requirements is critical to ensure that project teams first focus on items that deliver high value to the business.
- Percentage of requirements fully implemented – This is part of requirements traceability; requirements must be traced through design, test, and deployment.
- Percentage of approved requirements not implemented- A test of the likelihood of user satisfaction with the final result.
- Developer Requirements Satisfaction Index- Developers should be surveyed to determine their satisfactions with requirements. This actually should be presented as a series of requirements questions concerning quality such as
- Project Stakeholder Satisfaction Index – Project stakeholders should be interviewed to determine if they felt their needs were met and their overall satisfaction with the requirements. This might be done with a series of questions such as:
- Do the requirements address the business needs?
- Were the requirements gold plated by BAs or developers?
- Were stakeholders adequately involved in the requirements process?
- QA Requirements Satisfaction index – QA should be surveyed to ensure that the requirements were testable.
- Percentage of Requirements Tested – One good measure to determine if the requirements were testable and implemented is to determine what % of the requirements were actually tested.
- Number of missing requirements – Missing or incomplete requirements is always a major problem for projects. This KPI tracks the number of requirements that were added after the baseline was approved.