Agile An Executive Guide for Real Results
February 28, 2024
Software development and the IT industry has historically been plagued by projects that
- budget overruns
- missed deadlines
- low quality outputs
- solutions that leave the users dissatisfied
To address these issues an alternative set of practices has evolved with the collective term of ‘Agile’. While there people and groups promote different flavors of ‘Agile’ they the following common principles:
- replace large up-front investment in software solutions with incremental investment based on business value returns
- focus project team members on delivering capabilities that generate the highest business value for the organization
- encourage ongoing communication between the business areas and project
- team members to increase the relevance, usability and quality of delivered software
- entrust and empower staff to deliver value
- position software solutions to be responsive to ongoing industry, organizational and technology changes.
A Short History of the Evolution of Agile
Early software development was modelled on the construction and engineering industry with ‘architecture’ and ‘software architects’. However these IT industry was plagued by the remarkably high failure rate of software development projects; projects that became notorious for their missed deadlines, substantially overrun budgets, faulty deliverables and dissatisfied customers.
Prior to the emergence of Agile methodologies,
- 31% of all projects were being discontinued before completion,
- 53% of projects ended up costing 189% of original estimates and
- only 16% of projects were completed on time and in budget.
- Completed projects ended up on average delivering only 42% of what was originally promised.
A handful of thought leaders in the industry believed that these IT project failures could be attributed to three key factors
- over-planning (rigid and unrealistic)
- insufficient communication
- ‘all-at-once’ delivery.
- Lack of user input (13% percent of all projects)
- Incomplete specifications/requirements (12%)
- Changing specifications/requirements (12%)
Table of Contents
Over-planning
IT software projects traditionally began with the development of extensive ‘up-front’ documentation, including
- project plans
- functional requirements
- system design specifications
- technical architectural designs.
These documents, which often took months to produce (and even longer to get approved), were intended to ensure that the developed software would align with user requirements.
These well intentioned plans born of best construction and engineering practices, missed that software often had
- far greater complexity with many more decisions
- higher risks to solve new problems
- factors of usability, personal preferences, performance
As a result these plans and documents only served to provide projects of a false sense security of expenditure of their IT budgets and time.
Keeping to the time and budget and rigid designs meant compromises and missed opertunities that delivered software would be substantially misaligned with the real needs of users and the business.
Realizing a need for Change
It took a lot for industry leaders to accept they were not good at planning and delivering large and complex projects
The biggest problems with organizations relying upon up-front documentation were:
- a lack of responsiveness to ongoing changes in user requirements
- internal resource availability, resulting in requirements left to the discretion of the technical team to interpret
- not understanding the capabilities or risks of the underlying technologies
- the tendency for stakeholders (e.g. business areas, customers) to not have their requirements captured clearly. Wordy documents rather than screen mockups
- not prioritizing requirements, request for everything under the sun (resulting in highly valuable business requirements being lost in a sea of extraneous requirements)
Insufficient communication
The second overwhelming driver in the ongoing failure of software development projects in the 1990s was the traditional, and often deliberate separation of the business areas that required the software and the technical staff responsible for delivering the solution (i.e. development in a vacuum).
Large projects moved through ‘departments’ from architecture to planning, to budget. These departments that seldom talked to the end users or each other.
Once approved these projects were handed over to the technical team for development. The technical team reported to the business on deadlines and features and not to the end users.
Only after development and at installation was the resulting software on seen and tested at users’ machines for acceptance testing (or, even worse, production use).
This isolation created inevitable issues with the resulting software, including:
- user requirements left to the interpretation of the technical team members, without the benefit of understanding the business context
- disconnect between the two-dimensional concept proposed in the documentation and the manifestation of that concept into tangible screens that the user could interact with
- not allowing for changes to business requirements that may have occurred between the time that the user was last consulted and the months (and sometimes years) that followed before the resulting software was installed on their
Big Bang or ‘All-at-once’ delivery
Software development projects in the 1990s depended heavily on ‘waterfall’ project management methodologies, where
- analysis
- design
- development
- testing
- delivery
stages were undertaken serially, requiring the full completion of one activity before the next one could begin.
The use of waterfall methodologies on these projects meant that software design could not begin until all of the requirements analysis was complete; software testing could not begin until software development was complete; and software was not delivered to the users until all of the preceding stages had been completed.
This process was intended to reduce risk by ensuring every step was ‘signed off’ however for these complex projects that required user engagement it failed to deliver rather it
- mandating big up-front documentation (with all of its related issues)
- discouraging responsiveness to changing requirements as the project evolved
- creating ‘silos’ of ownership that reduced communication across project team members.
Perhaps the most risky and costly impact was that problems in the software requirements were only seen after it was delivered causing expensive rewrites.
After so much expenditure business felt compelled to continue development despite numerous production issues.
The 10 core business benefits of Agile
The business value that Agile methods provide to organizations can be distilled into the following 10 core Agile business benefits:
- Ongoing risk management: by regularly confirming and adjusting requirements based on ongoing interaction with stakeholders; and by delivering fully functional and tested software features, so that technical risks are identified and mitigated throughout the process.
- Ongoing control of budget expenditure: by providing decision makers with the opportunity to review and assess the business value of delivered in each iteration, with the option to adjust, postpone or stop ongoing funding based on the return on investment (ROI) of delivered work.
- Rapid delivery of tangible outcomes: by focusing team efforts on regularly producing fully functional and tested software features.
- A focus on the highest-priority features: by continually working with key stake- holders to confirm and adjust the work undertaken by the project team to align with the most current priorities of the organization.
- Strong alignment with business requirements: by directly involving key stakeholders in the initial and ongoing review of developed software, and by incorporating their feedback throughout the process.
- Responsiveness to business change: by adjusting work throughout the process to incorporate organizational, industry and technology changes.
- Transparency in status tracking: by regularly providing tangible outputs for stakeholder review, combined with open communication channels and centrally available status-tracking tools.
- Substantially higher quality outputs: by incorporating rigid testing regimes throughout the software delivery process; and by working regularly with stakeholders to confirm the usability and business value of delivered features.
- Greater employee retention: It was found by creating work environments that are based on high communication, empowering the team and trusting their expertise, encouraging innovation, and regularly acknowledging the team’s contribution to the organization.¹⁴
- Minimized whole-of-life software costs: by incorporating usability, quality and extensibility into software solutions throughout the delivery process, which reduces support and maintenance overheads; and by allowing additional functionality to be integrated into the solutions more cost effectively.
Agile Myths
The adoption of any new process can allow individuals and departments to over-reach or interpret the process to suit themselves. As a result a number of common Agile myths have emerged especially during the adoption process
Myth 1: Agile projects do not require planning
While agile offers freedom from the large upfront planning. It does not mean no planning or ad hoc planning. Agile methods replace one-time up-front planning with the ongoing refinement, re-confirmation and adjustment of plans throughout the delivery process to reflect the most current information available to the organization.
Myth 2: Agile projects do not require documentation
Similar to the planning, avoiding large and often inaccurate documentation does not mean no documentation. Rather we chose small document sets especially in a format that can be understood and approved by the stakeholders. Hence we prefer
- Mockups and wireframes over wordy documents
- Mockup demos over wordy documents
- Collaborative planning and architecture drawings
- Small deliverable ‘stories’ over large complex documents
Myth 4 : Agile is too risky (or radical) for our organization
Not having the same documentation sets and approvals that an organization may be used to can feel risky. The problem is often compounded by much of the terminology around various Agile approaches. However by now you you should see that most of principles of agile are common sense goals of
- break a task down into small achievable goals
- allow the experts to solve those goals in the best means available
- verify the results of those solutions with the demos, and face-to -face communication with those affected.
Conclusion
Agile should be neither a religion nor a cult rather the adoption of some common sense practices born of often bitter experiences. They should be discussed and adjusted to fit your specific company, organization or project.
In the next post I will provide FIVE STEPS TO AGILE SUCCESS
Richard Donovan is a CEO and consultant who has led both technical and agile transformations in many industries from Banking, Fintech, Logistics, Media and Telecoms.
Having worked with leading enterprises in his field Richard found that technology is often the smallest challenge and the team(s) and executive alignment and by far the hardest and most important challenges.
After decades of research goThink has evolved a visual and interactive workshop to discover the collative purpose, teach and ensure the adoption of the practices.
