Posted by Adam Jacobs
July 16, 2018
Seemingly every week, there’s a story of an epic failure of a well-intentioned IT project. The company, or government, or charity set out to make changes and incorporate modern technology. They ended up wasting gobs of money and having nothing to show for it.
Some stories can be truly mind-boggling, like the US Veterans Administration spending eight years and over $100 million on a scheduling system… which was never produced. In Ontario, the government’s integrated justice program met a similar fate: years of “development” and massive expenditures, but ultimately a failed project. This summary from the Auditor General should be sufficiently hair-raising:
Yes, you read that correctly: the costs were >$250,000,000, with a benefit equal to <4% of the costs. It’s even worse – the auditor notes that the public and private sector “have incurred additional transition costs in connection with the end of the Work Term,” and both parties sued each other, which wasted more time and money.
Similar to IT projects, Business Intelligence (BI) projects are not immune to failure. A BI project often has an apparently simple goal, like “implement a new reporting system” or “transition to a self-service analytics approach.” But within that lies a lot of complexity, a lot of dependent pieces, and some key decisions that can doom a project and frustrate everyone involved.
Why do BI projects fail so frequently? Unilytics has reviewed the evidence and come up with the top 7 reasons projects may stall or never achieve their goals:
In our experience and our review of the literature, this is perhaps the most common pitfall. Project management is done on best-case scenarios; rarely do we account sufficiently for the delays in acquiring data, or even purchasing the tools required to complete the project. Seemingly trivial details like getting database passwords or setting up VPN access for consultants can move a project off track and lead to corner-cutting to meet the aggressive goals. The Project Management Institute (PMI) posits that an employee who works 8 hours a day should only be assigned to a project for 6 hours a day maximum. This allows a realistic buffer of time for the administrative aspects of any job such as responding to email, logging timesheets, attending meetings, scheduling, and so on. Assuming that everyone works at 100% capacity almost guarantees a project will be behind and over budget.
Advice: realistic estimation of time requirements is one of the most difficult project planning steps to the point where there are even textbooks that address the topic. A rule of thumb that may be helpful is to double (yes, double) every initial estimate to account for unforeseen difficulties along the way.
As simple as it seems, the goals of a dashboard or report often aren’t clearly defined. What are we designing? How frequently will we update the data? Does the user need to filter the dashboard? Basic questions to be sure, but if they are not asked initially, the final product can disappoint everyone. The mere process of asking these questions can clarify and lead in new directions; often projects begin as “let’s replicate our old reports” but evolve into something more innovative and useful.
Advice: formalize requirements gathering processes and make it the initial design step. Unilytics employs the UD3 design methodology for all BI projects, with robust requirements gathering as the very first stage of project implementation.
According to IBM Systems magazine, “54 percent of IT project failures can be attributed to project management, whereas only 3 percent are attributed to technical challenges.” (emphasis added). Very rarely are the tools or even the people a limitation on the project; s the tracking and organization of the project t is where the weakness lies. There is no one simple way to manage projects; usually it’s a combination of technology and good communication.
Advice: when possible, employ a project manager for BI projects even when they appear small and manageable. Often the developers have a handle on the technical requirements, but issues of scope, cost, and time are better handled by a PM. When this isn’t possible, at the very least time needs to be allocated for the overhead of PM-type work – scheduling, project planning, and budgeting.
Speaking of the right technology…
Often decisions in organizations are made based on prior commitments; e.g., “we have been using Company X for some other aspect of the business and they now have a BI tool, so we’ll use that.” While there is no perfect tool, it’s important to select something that will give you the best chance of success. Some dashboard development tools are better for transforming and merging data; others excel in building interactive content, or working with mobile devices; still others emphasize accessibility standards or simplicity of upkeep.
Advice: use a vendor with a proven record of success and leadership. Choose a leader from Gartner’s report on Business Intelligence Platforms.
Can the size of a project significantly impact its chance of success? Surely other factors like Agile vs Waterfall, or the technology used play a larger role…? But research consistently shows that IT projects with large budgets are far likelier to fail than much smaller ones; therefore, it’s important to do “larger” projects as a series of smaller ones. For whatever reason, large projects can get away from everyone: PMs, executives, IT, and developers are all culpable. We know that people are bad at estimating future costs and works, and the larger the project, the more acute this problem becomes.
Advice: pursue a “gated” approach to BI projects; rather than approving a giant project initially, use smaller steps and proof-of-concepts to avoid over committing to an untenable project that can get bogged down.
As Forbes recently noted, “When projects are dubbed ‘IT ‘ and left to the IT department, there’s also a lack of accountability that can develop… If your tech team can’t adequately explain what’s happening on the project or why it’s needed, that’s a huge red flag.” Ultimately a successful BI project requires executive sponsorship; whether it’s for financial support, resourcing, or backing the project through the inevitable ups and downs.
Advice: engage upper management first, not after the project has been proposed and scoped. Even if executives are non-technical, they are often the end users and the evangelists for BI projects. Make sure they are getting something immediately useful – features, not specifications!
Perhaps the single most important feature in BI project success is engaging the user. The Standish Group “Chaos Reports” on IT failure regularly point this out: the number one factor identified in successful projects is “User Involvement.” Conversely, the number one factor identified as a “Project Challenged Factor” was lack of user input.
Advice: engage users in testing and feedback along the development process. A BI solution developed without user input, no matter how “correct” or well-designed, has a risk of failing because of lack of buy-in and understanding. Don’t make the user work hard to understand the solution, make it work for them.
BI projects don’t have to fail, of course. They require a good requirements gathering process, excellent project management, tools that match the task, and end-to-end communication by all involved. Please CONTACT US for further discussion about making your BI project a huge success.