Scrum: It’s a full-time job.

“An adequate ScrumMaster can handle two or three teams at a time...A great ScrumMaster can handle one team at a time.”

-Michael James

One of the lessons we learned early in our Scrum roll-out was that the Product Owner and Scrum Master roles required much more involvement than originally thought.  Coming from a Waterfall development environment, where releases were long in duration and milestones spread out, people in the organization were used to having key roles on multiple projects, context switching as needed.   Juggling responsibilities in a Waterfall environment seemed to work well, or at least acceptable.  Our Product Managers, for example, could own several product lines, including associated customer meetings and field interactions, and easily satisfy occasional project touch-points such as requirements definition (at the beginning of the release) and Beta and FCS duties (at the end of the release).  These project touch-points required their full-time involvement periodically, a few times a year, making it relatively easy to context-switch periodically and satisfy the needs of each effort.



With Scrum, however, things turned out a bit different.  Though we “took the training” and “read the literature” which repeatedly said the roles of Scrum Master and Product Owner were full time jobs, we weren’t true believers.  Our “muscle memory” way of working had everyone juggling multiple roles and responsibilities, it was the way we were used to working. During the course of our Agile Pilot projects, however, we got convinced pretty quickly that Scrum roles could not be part-time.

Scrum is more intensive than waterfall development.   

With frequent “potentially shippable product increments” created every 2-4 weeks,  there’s essentially a full product release at least once a month if not twice.   During our Agile pilots, our Product Managers, filling the role of Product Owners, soon realized that creating and maintaining a Product Backlog required a significant amount of time.  Every few weeks a new set of fully-defined “product requirements” need to be ready for the team for the next Sprint. Product Owners found it challenging to satisfying their Scrum PO role and concurrently complete other project work.  Our Scrum Masters, too, found themselves booked full-time, facilitating team interactions, producing Sprint Burn Down charts, working impediments, and delivering frequent product releases at regular Sprint Review.  Team members, often with other-project responsibilities, found themselves spending more time delivering on Sprint Goals, often at the expense of their other responsibilities.  

As we scale Scrum within the organization, we’ve (obviously!) needed to identify new Scrum Masters and Product Owners.  We’ve held fast to the one project per Scrum Master recommendation. Identifying new dedicated Scrum Masters has been fairly easy - our Pilot teams was that the efforts produced several Scrum Master candidates, a nice side-benefit!   Finding full-time Product Owners for each team was more challenging.  Because we’re applying Scrum to large multi-team projects, our mapping of Product Manager positions to Product Owners simply didn’t scale.    Our solution was to look to our lead engineers. Most of our new teams have technical Product Owners, usually Software Architects, and they are dedicated full-time to the team.  Product Managers are involved at either a co-Product Owner (in an outward facing role) or at a higher level in the project effort, via a Product Council-type role that involves them as stakeholders.  

I expect our organization to fine tune (via inspect and adapt) these non-Scrum roles as we mature in our Agility.  But regardless of the organizational “noise” around the Scrum Teams, I believe that participation at the Scrum Team-level will continue to require full-time effort from all members in order to be successful.

Scrum Cadence

It’s unfortunate that the the Scrum product iteration increment is called a Sprint, implying that each increment is delivered by the team in a mad rush, full throttle.  The image I have in my head is a 100 yard dash, everyone running as fast as they can to reach the finish line.  Developing software is more like running a marathon than it is a back-to-back series of 100 yard dashes.  In order to successfully complete a marathon you have to find a one mile pace you are comfortable running at, and keep running at that speed for 26 miles.  Sprinting hard for the first several miles often means you are walking (or even crawling) the last mile, if you even make it that far.

Me, still running at mile 26, in the 1998 Boston Marathon

One of the challenges many new Scrum teams face is developing a sustainable pace, sometimes called cadence, over multiple sprints over the course of a project.    

New to Scrum, teams often struggle to find a sustainable cadence for delivering potentially shippable product increments.  What I’ve observed happens is this:  Early in a new Scrum project the teams is (I hope) excited to adopt a new way of working.  They understand the Scrum principles and the time-boxed Sprint iteration, and more importantly, they feel a strong commitment to succeed on their newly committed Sprint deliverables within the first few Sprints. After all, at the end of the Sprint they will be proudly demonstrating their work to their stakeholders, which usually includes there boss and department head.  However, since everyone is new to this different way of working, it is inevitable that certain work, usually that requiring specialized skills such as QA and documentation writing skills, ends up being delivered late in the game.  Undeterred, the team works extra hours, heroically delivering the Sprint goals.  This extra effort delivers a much needed early success, builds organization confidence, and can kick-start viral adoption.  However, this type heroic effort doesn’t scale - after 2 or 3 Sprints the team starts feeling burned out and may fail to deliver Sprint-committed work.

In an ideal situation, the Team inspects and adapts, and, over the course of several sprints, build a sustainable cadence.  However, many teams will continue to struggle, and can fall back to the “old way of doing things”.

While every team, project and environment is different, here are some points to consider to help get to a sustainable cadence:

• Work, as a full team, on only one story at a time.  Make sure that story is DONE before starting the next story.  In other words, implement stories sequentially.
• Avoid picking stories targeted for one individual due to their specialized skills.  Make sure the whole team is involved in delivering each story.
• Develop an “in it together” feeling among the team.  If you aren’t feeling this, your team isn’t operating as a true TEAM.  
• Commit to less stories to make sure your Sprint backlog really is DONE at the end of a Sprint.
• Become more of a generalist. Scrum certainly favors “jack of all trades” type of developers.  Coming from a Waterfall environment, where there are separate groups for QA and documentation, work can fall to those with that domain specialization.  As a team member, don’t look at work as “that person’s”, view it as a team deliverable. Learn a new skill to help a team member (and thus the Team) out, be it coding, QA or documentation.
• Be honest and innovative during your Sprint Retrospectives.  It’s your chance to change how you work together to deliver software.
• Get additional training or coaching.  Read Scrum blogs and ask questions on Agile forums.  Likely you are not the first team to run into the issues you are experiencing.
• If you are manager reading this, realize that team cadence, a predictable velocity, is key to delivering on corporate objectives.  Work hard to remove team obstacles and allow them to focus and succeed.  This is the true value of a manager in an Agile world.

Developing a sustainable cadence is one of the keys to the success of the Scrum Team and project effort.  Remember, running a marathon full throttle is unsustainable, unless, of course, you are an ultra-elite runner.  I know I’m not one.

Inspect and Adapt

“SCRUM assumes that the systems development process is an unpredictable, complicated process that can only be roughly described as an overall progression.”

-Ken Schwaber, Scrum Development Process, 1995

One of the things that I like about Scrum is that its creators defined it knowing that no development process is perfect.  The Scrum Guide does not tell us how to build software, rather, it openly acknowledges that building software is complicated and unpredictable.  This may seem odd, with Scrum being thought of as a development process (it’s not).  In a recent post to the Yahoo! Scrum Development group, Ken Schwaber stated that “we purposefully constructed Scrum with a lot of holes in which a person or team or organization has to continually figure out the best thing to do.”

So how does a Scrum team “continually figure out” how to do the best thing?   To facilitate iterative and ongoing improvement, Scrum specifies that each Scrum Team hold a Sprint Retrospectives. Further, these retrospectives must happen frequently, occurring immediately following each Sprint Review.  

It is often overlooked when starting out with Scrum, but I believe the Sprint Retrospective is the most important meetings a Scrum team participates in.  While the Sprint Review is a valuable “public” meeting where working product is demonstrated to stakeholders and customers, the Sprint Retrospective is a “private” team meeting, allowing the team to safely discuss what’s working and want could be improved with regards to how they are delivering their work.  In the Sprint Retrospective, anything that relates to how the team operates can be discussed and possibly tuned or improved.  Up for debate could be how the team communicates, technology issues, done-ness criteria, or even choice of tools, as was recently discussed for one of our new Scrum teams in my organization (They reverted back to whiteboard task charts and spreadsheet burndown charts!).

What this ultimately means is that each Scrum team is evaluating, refining, and improving how they work together and build software every 2 to 4 weeks.  Almost continuously improving throughout the life of the project!  Over a 12 month project there can be upwards of 24 opportunities to “do things better” and improve how the software is built and delivered.   Compare that to our our Waterfall-like development model where we generally conduct formal retrospectives twice, once after Beta and once after we release.   In our current model, we, at best, have one predefined opportunity to improve our practices during the release, at Beta when there is usually no more than 3-4 months of (QA and bug fixing time) time remaining before the software is released.  Process improvement is clearly not highly integrated into our Waterfall process.

With frequent inspection and adaptation we can become more efficient at building and delivering high quality software that meets or even exceeds customer expectation.  So my advice is: take full advantage of your Sprint Retrospectives.   As a Scrum team member, it’s your (frequent!) opportunity to make a difference...

kaizen  (kaɪˈzɛn) - n  a philosophy of continuous improvement 

of working practices that underlies total quality management 

and just-in-time business techniques [Dictionary.com]

Scrum @ Progress

Earlier in 2010, my company began the process of defining a Unified Development Process for the 500+ engineers in the Product Group.  We formed a committee containing key representatives from across the product group. After extensive investigation and deliberation, the committee chose to focus on an Agile process, and in particular, Scrum.

What is Scrum and how does it fit within our Company?   

Scrum is a software development framework, and not a prescribed process. It’s a framework that focuses on cross-functional, self-organizing teams producing potentially shippable business value in increments called Sprints.   A quick Google search will uncover reams of literature on what Scrum is, with the Scrum Alliance has a nice short overview here: http://www.scrumalliance.org/learn_about_scrum.  

What’s important about this process effort is how such an Agile framework fits within our existing 25+ year old corporate environment.  On the surface it sounds simple:  educate the engineering teams, flip a switch or shout”on your mark, get set, go!” and voila!, we’re now Agile!  In reality, evolving from a Waterfall software development lifecycle to Agile is a monumental undertaking, but one that is, in my opinion and experience, worth taking.  

As part of the our effort, we’ve began running a number of Scrum Pilot Projects, projects in each division who have “volunteered” to build software using the Scrum framework.  I’ve had the good fortune to “own” one of the pilots and to also observe others.  Most of the benefits and challenges that I have observed during this process matched my own experience as well as those I have heard about from friends or read about from others in the software industry.  And some of what I observed surprised me.  I hope to capture these observations as the subject this blog, aptly titled “Agile making Progress”.

Scrum Roles

There are three roles that the Scrum framework identifies:  ScrumMaster, Product Owner, Team Member.  How the roles do or could fit into my company's Product Group is discussed below, ihowever it should be noted that every group, every product, has potentially different needs.

The ScrumMaster.  The ScrumMaster is the person who ensures that the team is functional and productive.  The ScrumMaster role can be viewed as the facilitator or ringmaster.  At my company the role of a GEM (General Engineering Manager, a project coordinator) may align well with ScrumMaster.  For the Pilot Tooling team, the pilot project that I own, we’ve got a great ScrumMaster who is in fact a GEM.  However, this shouldn’t be a hard and fast role mapping.  Other teams have ScrumMasters who are engineers, or even engineering managers (just not the manager of anyone on the team).  The ScrumMaster cannot hold a position of reporting authority over any of the team members.  On any team, it is very likely that the “right” person for the role will rise to the occasion - someone who is a facilitator or coordinator of the group.  With the pilots, sometimes the choice was obvious, as it was with the Tooling team, sometimes after a sprint or two, the team realizes that someone else on the team might be better suited as the ScrumMaster and the team adjusts, as happened with another Pilot team.

The Product Owner.  The Product Owner is responsible for the business value of the project, and represents the “voice of the customer.”  The PO is also identified as the “single wring-able neck”, having ultimate responsibility for the project.  On of the most important tasks for the Product Owner is for identifying and prioritizing the “stories” (release requirements) that the team will be working on. Here in in my company, one initial thought expressed was that a Product Manager should be a part of every team in the role of Product Owner.  In theory this makes a lot of sense.  In practice, however, it quickly becomes problematic.  We’ve seen in the Pilots that the user stories and product backlog require near-constant refinement (also known as “grooming”).  Additionally, some user stories are very technical in nature, due to the type of software we are building.  A high degree of low-level technical understanding has often pulled in a Software Architect to help with backlog grooming and prioritization.   The Tooling team found that the backlog grooming and story refinement regularly required deeper technical expertise. Scrum favors empowered and self-organizing teams, and a this team quickly readjusted to solve this need by making the Product Owner a joint role, shared between a Product Manager (outward view) and a Software Architect (inward, technical view).  To date that partnership has worked well over 4 sprints.  It will be interesting to see how the Product Owner role will scale within the larger organization where there could be a dozen or so Scrum teams working on the same product line.  Perhaps we’ll see some form of Product Council or co-Product Owner or a hybrid approach.  I look forward to seeing how each team self-organizes!

The Team.  The team consists of the people doing the work. As a general rule, the team should be cross-functional, because at the end of each Sprint a potentially shippable piece of software is produced.  For software to be shippable it must be written, tested, documented, packaged, and have training materials created.  As you can imagine, this generally means that you need Developers, QA, Doc and Readiness on the team.  The Scrum team is given the authority and empowerment to self-organize and do what they need to do to get the job done.  In return for this self-governing power, the business owner asks for a team commitment to deliver some level of potentially shippable software in a predetermined amount of time.  In other words, the team commits to implementing a some number of stories (requirements prioritized by the Product Owner) within some predetermined amount of time (generally 2-4 weeks - for the pilots it was 2 weeks).  Because I had a high level of confidence in Scrum to begin with, I wasn’t personally surprised by the (tremendous) output of the pilot teams.  I was surprised, however, by a higher than expected commitment by each Team to make this effort work.  This was Scrum magic, in my eyes.

Organizational Transformation

Due to corporate evolution, the environment I work in makes it hard for many us to focus on one project.   Because we’ve all got a number (often different) “top priority” work items, we must often time slice our attention across these efforts to achieve some level of success.  This strategy is at odds with Scrum, as Scrum requires a Team to commit to deliver an unchanging list of functionality, over a fixed period of time.  In other words, Scrum teams need unwavering focus on commitments they have made.  For a Scrum team to succeed, Business Owners (management) need to leave the team alone during that time-boxed sprint so that the team can succeed. Here, to me, lies the biggest organizational challenge.

The pilot teams demonstrated to me that even in the face of major impediments and overriding “external” priorities that they were able to “figure it out” on their own and make it work.  This is a good thing and gives me hope for the success of our ongoing organizational transformation.

 

Agile Transformation and Me

I've worked for big companies like Digital Equipment Corporation, medium, like Sybase and Progress (my current job) all the way down to small start-ups, like EasyAsk and Novera Software.  Each job has offered me different opportunity to grow, from create value from nothing at start-ups, to managing product lines generating hundreds of millions of dollars.  Through it all, one thing has remained constant regardless as to the size of the company or my role in it:  continuously improve and become more efficient.  The only difference I've experienced is that with smaller companies it is generally easy, or quicker, to change what (and how) you are doing.  The larger the company, the more difficult it is to change.  Pure size, inertia and existing customers and revenue provide significant reasons for doing things the way they have always been done.  

It is in this type of "bigger company" environment that the greatest challenge and perhaps the greatest rewards lie.  Improvements in the efficiency of the software development process can reap significant dividends.  Over the past decade+, a major evolution in software development practice, in the name of Agile Software Development, has been taking hold in the industry.  I've been fortunate to have experience with delivering product using Agile frameworks and methodologies, both as an individual contributor, project leader, up to VP of Engineering.  In my present role of Director of Engineering for 3 product lines generating over $300M in revenue, I am one of the leaders working to convert my the engineering organization of over 500 software engineers from a Waterfall-centric development process to Agile (Scrum) process.  As this transformation got underway and I began coaching our pilot teams, I found there were many seemingly random thoughts and ideas bouncing around my head that I started writing them down, and ultimately started blogging within my organization.   I've decided to post these blogs publicly, charting my organization's journey, our Progress, from Waterfall to Agile.

Thanks for reading...

John Piekos