History's role is not to help us predict the future, but to prepare us for it.
Jim Highsmith's career over the last six decades is a good start.
In the Wild West Era (1966-1979), software engineering was in its infancy and the knowledge of how to "do" software development negligible. This era was wild.
The Structured Methods and Monumental Methodologies (1980-1989) era was an attempt to go from the undisciplined wild west to a more disciplined and formal approach, trying to be recognized as an engineering discipline.
IT projects were generally viewed as out of control, and business managers assumed building software was the same as building a warehouse. Focus on tasks rather than people and team dynamics.
A formation and expanded use of waterfall, control-oriented, and process-heavy methodologies emerged assuming that documentation was sufficient to communicate between groups with no need for further explanation. Variation from the plan was seen as poor execution.
Despite of this, the approaches managed to deliver value. The success and failures of one era form the base for the next.
"IT projects take too long, costs too much, and don't meet our needs". Welcome to The Roots of Agile (1990 - 2000). In response to executives cries for help, software developers began experimenting with iterative Rapid Application Development (RAD and RADical) methodologies, especially for Internet applications.
Focus on value, quality, speed, leadership and collaboration. Waterfall couldn't handle the level of uncertainty with new technology and changing requirements.- "It was just a small project",
- "It was a new project",
- "They didn't have to follow our standards".
Management wanted agility, but with traditional performance measures. To change behavior and culture, you must change measures of success.
In the current Digital Transformation period (2011 - present), the problem isn't scaling Agile, but enterprise-wide scaling of agility and innovation.
We need to move from predicting and planning for the future, to building a platform that enables us to adapt to whatever happens.
Preparing for the future challenges us to scale an agility mindset, not simply agile methods and methodologies.