lidenskap.blog

History's role is not to help us predict the future, but to prepare us for it. 

Today we talk about product management, agile, waterfall, projects, iterative and measures of success. I enjoy putting this in a historical context. Because those who fail to learn from history are condemned to repeat it. History can help us prepare for the future. Jim Highsmith's career over the last six decades is a good start.

Wild West (1966-1979)

Software engineering was in its infancy and the knowledge of how to software development negligible. Software project success was measured by completion and cost. Businesses operated on the premise that the world was predictable and if plans failed to materialize, the problem was execution, not planning. Change was the "unusual state".

Structured Methods and Monumental Methodologies (1980 - 1989)

This was an era where management theory was firmly stuck in the command-control model. IT projects were generally viewed as out of control. Managers assumed building software was similar to building a warehouse. They often knew little about the value of IT systems, even though productivity increased, and focused therefore on "out of control" cost and schedule overruns. 

Structured methods should bring discipline and control to software development.

But the attempt to go from the undisciplined Wild West to a more disciplined, formal approach overshot and focused on the wrong things - processes and documents. 

Formality was mistaken for discipline. Formal processes, phase reviews and documentation added a level of bureaucracy that overshadowed the benefits of structured techniques. It worked for tangible products and construction-type projects though.

The belief was that documentation was enough to convey all the information required by individuals in the next phase of work.

A paper by Winston Royce in 1970 is often credited with introducing the Waterfall model of software development (a term he never used in the paper). However, Royce did not advocate for the  model in its strict, linear form. He highlighted its limitations and risks, particularly the challenge of accommodating changes once the project is underway. To address these shortcomings he argued for feedback loops, overlapping development phases, and the need for more flexibility and adaptability.

But the waterfall lifecycle was greatly influenced by the surrounding ethos, manager's thinking and the serial "hardware" mindset of the times. Departments outside of IT also adapted their work to conform to the serial mindset. For example, accounting started to classify operating (OpEx) versus capital (CapEx) costs.

Too much structure reduced problem solving and innovation; too little created chaos and ineffectiveness.

The Roots of Agile (1990 - 2000)

Executives now cried for help. "It takes too long, costs too much, and doesn't meet our needs".

In response, software developers began experimenting with RAD (rapid application development) methodologies. They assembled a small team, delivered working software every week, and conducted "customer focus groups" every Friday to get feedback. This was about value, quality, speed, leadership, and collaboration.

The 1990s were the decade where software outsourcing to solve technical debt and the high-cost mess began to flourish. The assumption was that less expensive programming could be accomplished with minimal communication. But much of the outsourcing was based on the false premises of predictability and prescriptive practices.

Worse, it stripped internal IT teams of the expertise they would need in the coming Internet upheaval.

The methodology of software development continued to be deterministic and formal in the 1990s, but slivers of RAD and iterative development began to creep in, especially for rapidly expanding Internet applications.

The 1990s were the decade of process for IT. We sought not just perfection in the way we built software, but total predictability. It wasn't enough just to do things right, we also had to say in advance exactly what we intended to do, and then do exactly that.

The Agile Era (2001 - 2010)

Demand exploded. Acquiring and retaining technical talent proved difficult. Many companies didn't have the money or the talent pool to respond to the pressures of the 2000s. IT organizations faced pressure from the high costs. Companies had to innovate and expand their technical and product design capabilities.

Technical debt, which had been growing rapidly, was still almost invisible to business executives. IT struggled to convince business leaders to approve the investments required to reduce technical debt, thereby enabling continuous value delivery.

On February 11-13, 2001, 17 software developers met at the Snowbird ski resort in Utah to talk, ski, relax, and try to find common ground leading to the Agile manifesto.

Many software engineers were concerned about the nascent agile movement. They viewed agile development as a retreat to ad hoc practices of the past. But in reality, teams were highly disciplined, but worked somewhat informally. The emphasis shifted from documentation towards collaboration, replacing traditional formality.

From 2001 - 2004, individual teams received dispensation to try this "agile stuff". Teams focused on iterations, stories, daily stand-ups, backlogs and co-locating teams - but the core technical practices were often bypassed.

Agile projects' success was often downplayed by others: "it was just a small project", "it was a new project", "they didn't have to follow our standards". General comment from Agilists was "We don't need any project management". But the management of tasks rather than people was what they were complaining about.

From 2005 to 2010, success versus failure with organizational implementation of agile development rested with courageous executives who understood agility at their core. Courageous executives thrived on adventure, the corporate kind, and had the ability to sort through a myriad of opportunities, engage others with their enthusiasm, and demonstrate results through action.

Another success factor was that Agilists had enough background in change models to muddle through team-level implementations. At an enterprise level, it made a difference when they received help from organizational change experts.

Do we implement from top to bottom, or from bottom to top? Do we start with a few teams and user their experience to seed others, or do we "sheep dip" everyone? What was the strategy for moving agility up (the organizational hierarchy) and sideways? How do we instill both being agile and doing agile? Whose change model and approach do we use? Agilists were simply not experts in change management.

Other success factors were demonstrated business benefits, practices that appealed to engineers and a manifesto that stated a clear purpose and principles for the movement.

Digital Transformation (2011 - present)

The problem now wasn't scaling agile, it was scaling agility and innovation at enterprise levels.

Remember the lessons from agile projects: To change behavior and culture, you must change measures of success. Many managers wanted agility, but with traditional performance measures (planned scope, schedule and costs).

Over the last six decades, measures of software development success have evolved from completion to customer value. Nothing drives change like the way organizations measure success, but few things are harder to change than those measures.

The Digital Transformation period explored what becoming a digital enterprise meant. Enterprise executives developed digital strategies but found their transition from strategy to operating models lacking.

Enterprises today should be seeking to become digital transforming rather than to achieve digital transformation. Transformation indicates completion of a stage, whereas transforming suggests a constant state of becoming.

More than six decades of change have required organizations to constantly adapt by addressing the following areas:

• Measures of success - this can drive or restrict change.
• Being digital demands a better business and technology partnership.
• An operating model linking strategy to action.
• Creating a quickly malleable organizational model.
• Management suited for the digital era.

Preparing for the future

Over the 60 years, have anyone notice the rate of change in the world, in technology, and in business slow down? Change has spiraled constantly upward, not only linearly, but bordering on exponentially.

One conclusion from history is that agility, not agile, should be the goal. Agility is a mindset; agile is a type of methodology. Failure to make this differentiation has left organizations with prescriptive methodologies, while leaders who embrace agility have the best chance of thriving in the future.

Change takes more than a one-day workshop. No one changes deeply held mental models easily. Even when organizations and managers intend to start the journey with the right mindset, many don't realize the magnitude and complexity of this transformation.

The root cause of success is people and their interactions, and the root cause of failure is also people and their interactions.

What advice do you wish you had when you pursued your first leadership role?

Steffan, you will now go from the guy making recommendations to the guy making decisions. You go from being an individual contributor to delivering through others. You now lead a team of people doing what you used to be good at, that got you promoted in the first place.

That's a big change. How to make good decisions, facilitate a meeting, craft a strategy or delegate work is not something that everyone naturally just know how to do. If you think they are natural abilities, then you might feel there's something wrong with you if you don't have them. Please don't. It's a skill you can learn. You will do mistakes, but over time you will nail it. Leadership is an education.

People is your work now. Invest time in them, be present, be available. Listen. Ask how you can help all the time. Then comes communication, communication, communication, setting budgets, performance reviews, 1:1 meetings, meetings with stakeholders, recruitment, setting goals, prioritize, help finding solutions on complex problems, solving conflicts.

Don't forget to take care of yourself. What gives you energy? What drains you for energy? What do you need to do yourself, and what can you delegate to others? Do one thing at a time. Use your team. Create a support system around you, and use it. Block time in your calendar to think and reflect.

Never create a bunch of tasks that you assign to your team for execution. Provide context, intent. Let your brilliant team and problem solvers define and own their tasks and actions. A good plan developed and understood by the team is better than a "brilliant" plan by you.

Don't stress if you don't have a vision on the first day. You are not Steve Jobs. Visions can be found. Find it together with your team and let it be the compass in everything you do. What would you like to achieve together? What's the future we are trying to create? How do we plan to accomplish that? Let it become theirs, not only yours.

Watch out for compromises in this process. If you try to come up with something that pleases everyone, you end up pleasing no one.

Steffan, you are not a super human and don't pretend to be one. You have your strengths, but also your weaknesses and gaps. Your job is to build a team and surround yourself with people closing those gaps. Your job is not to know everything.

There will be periods where you doubt yourself and question everything, but suddenly you will see someone on your team achieve more than they thought they were capable of.

You will see your team come together to solve impossible problems. You will see a team that would do anything to help each other out. Impact. That's the joy of leadership. That's your reward. That's the reason why you became a leader.

Good luck my friend!

Best Regards,

Steffan Sørenes, a year older and (somewhat) wiser

A potential minefield. 

The high energy from ideation and brainstorming quickly turns into frustration when it's time to make a decision.

We don't fully control the outcomes from a decision, but what we have some control over and what we can improve is the quality of our decision process.

A challenge is that we often want to accomplish two things at the same time. 

We don't want to waste too much time, and we don't want to sacrifice too much accuracy. The key to balancing this trade-off is figuring out the penalty for not getting the decision exactly right. What's the worst case that can happen?

So you have a list of ideas coming from your team, but which of them shall we pursue? There will always be one that is accountable for the final call, but there are many ways to accomplish it.

Shall we do an individual ranking where each idea must pass relevant thresholds, or shall we rank them relative to each other? What are the selection criteria to assess? What's your current constraints and non-negotiables?

Shall you do it democratically through voting? Voting is time effective and consistent. Will everyone feel a part of the decision through voting? Do you get rid of groupthink? In a voting there will be limited information and knowledge flow between the team members. Do we want that?

Or shall we just appoint a leader that makes the decision? That's probably effective, but the quality of the decision depends heavily on the leader's style and knowledge.

Will the leader involve and listen to each team member's opinions, and change their own opinions if needed? Will the leader share their thought process afterwards?

Or shall we aim for consensus? Done right, consensus can result in a high-quality decision. You share information within the team, everyone hears what everyone else has to say and can ask questions to each other. Team members also share responsibility (if not, is it true consensus).

However, consensus takes time. It's slow. At least for a new team. It can easily break down without the right team dynamics and guidelines. Does it feel a bit "leaderless"? 

And is it not a short distance from consensus to compromise where three good ideas are turned into one bad one to please everybody.

What about the consensus trap where everyone seems to be in agreement but in reality the majority disagrees without speaking up. What do you do if not everyone agrees with you at the end? Some companies swear to "disagree and commit".

Gerald Weinberg said that the trick is not to know the best method, but the best method under the present circumstances. The most effective leaders are the ones who help the team to recognize when circumstances change and to find a new decision making method that fits.

How do you make decisions in diverse groups and teams? Which forms do you use under which circumstances?

Many have witnessed failed transformations, but few have witnessed true successes. 

Moving to the product operating model is a transformation. At the end, it's about consistently creating technology-powered solutions that your customers love, yet work for your business. It's about delivering real results. But what do you change exactly?

You change how you build. Products are managed as an ongoing effort - improving every week until it's decided to sunset it. You do frequent, small releases so you know that it's working and how it's being used.

You change how you solve problems. Empowered product teams are given problems to solve and outcomes to achieve instead of a list of prioritized features, projects and perceived solutions decided by various stakeholders.

You change how you decide which problems to solve. A strong product company has a compelling product vision and insight-based product strategy identifying the most critical problems that need to be solved to deliver on the business objectives. Your strategy cannot be to serve as many business stakeholders as possible.

Pushing the decisions and responsibilities for finding the best solution to the problem down to the relevant product team, and then holding that team accountable for the results also drives the need for new product core competencies (that normally takes years to learn). 

Unless you are willing to establish these new competencies, your transformation hopes will likely end here.

For the product team together with the product manager to discover and deliver effective solutions, it is absolutely critical that they have direct access to:

1. Users and customers,
2. Product data, 
3. Business stakeholders and
4. Engineers (the tech lead as a minimum).

Fight any attempt to place a well-meaning person or cumbersome process between the product manager and these constituencies.

An honest and accurate assessment of the organization’s current situation is essential to any plan to successfully transform. Be realistic, talk to all levels and look for evidence. 

Start with pilot teams volunteering to be on the leading edge of these changes. Develop the skills of the product teams (bottom up) and the skills of the product leaders (top down). Ensure your CEO supports and is a champion of the change.

Transformation is a long game, and for that reasons it helps to have some quick wins. It could simply be a team that has never visited users starts doing do and shares it experiences and insights. 

Constantly beat the drum, evangelize and show everyone the progress being made.

"Transformed - Moving to the product operating model" by Marty Cagan.