Reading Time: 10 minutes
I recently moved teams and role following a company restructure and merger, which led me to reflect on my last three years. One of the reasons I had taken the role was to gain more experience in agile product development. I’d worked in open source product development for over a decade, and on a number of agile projects, but in order to grow and develop as a software engineer and technology lead, I wanted more direct experience of product development and technology leadership with a small, agile scrum team.
So, I organised my reflections around the 12 agile principles, which seems like a sensible way to reinforce the learnings I’ve come away with. Obviously there’s lots that I cant talk about, but I’ve selected some things that aren’t commercially sensitive and aren’t airing anyones dirty washing! It was a useful exercise for me to reflect on, hopefully it can be useful to others too.
1 Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
One of the most important hires I made was a dedicated Product Owner (PO). The teams work had always been delivered into a two weekly release cycle and with the new PO we instigated new procedures for release comms to ensure customers knew what was just released and what was coming next. We focussed on more effective story sizing so that work of value could be delivered and released in one sprint rather than across many. We also worked on a quarterly product roadmap under a Now, Next, Future model which really guided the teams decision making and focussed minds on customer value.
Working on a 10yr old product we had a lot of tech debt to tackle, which isn’t often visible to customers, so we had to present that value in other ways, telling customers what we were doing ‘under the hood’ and how this would benefit them in the long term. Working with an experienced and talented PO was a great learning experience. I strongly believe that leaders must surround themselves with experts who are better than them and who they can learn from, and I certainly learned loads from my PO. I’m really proud of the roadmap model we introduced and the stakeholder engagement that was built around it, and will use these again in future.
2 Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Reducing story size made the team more able to handle change. When developers are taking three months to deliver a story, they were working to a specification that was three months old at the point of release, that’s too slow in such a fast moving world and you can’t react well enough to changing needs. We got large stories reorganized into epics, and the resulting smaller story sizes meant that work could be deployed and tested many times over in the time it would previously have taken to release something.
But our biggest challenge with regards to change, was outside of the team. Change was everywhere. The market we operated in was evolving and expanding so rapidly that it was hard to keep up. The product itself was a small part of our business, and the business strategy was undergoing major change while the parent group was undergoing its own transformation programme. Aligning product strategy to business strategy in this environment is challenging, to put it mildly. Our UX research and roadmap creation led us to align all product development under five core pillars which represented four broad functional areas and a fifth pillar which reflected our own business needs around scalability. Everything the team worked on had to slot into one of these pillars, which provided a useful framework for handling the wide variety of requirements the team would regularly receive. This proved to be a good way to fend off the more left-field requirements that previously the team may have agreed to take on but which did not represent good value to our customer base, while also giving the product team stability and direction when everything around them seemed to be constantly changing. I’ve already reused the five pillars model on another product I’m involved with.
3 Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
The team had always delivered every two weeks and maintained this release cadence no matter what. It was extremely rare to miss a release.
The key words in this agile principal for me are ‘working software’. It’s easy enough to do a release, but a good quality one is more difficult, especially given the complexity of our product. We had a manual testing process which worked well for us at a certain team size, but with growth ambitions we really needed an automated test suite so we could test fast and deliver rapidly. Despite a strong business case, life just kind of got in the way of delivering this and it remains an ambition of the team, along with a fully automated CI pipeline. These are not small tasks and require a level of expertise our team did not have, but were keen to learn.
I’ve not yet worked in a business that has mastered Test and CI Automation. As a software engineer I want to develop a well-rounded set of skills across the entire discipline, and this really feels like a gap in my knowledge especially given my background in software testing several decades ago. When you follow tech blogs, news and podcasts it can sometimes feel like everyone is doing automation except you, but then you interview for engineering roles on your own team and realise how badly things are done elsewhere! Recruitment can provide a useful perspective sometimes. I hope the team can get to this in future, and it’s certainly a gap I am looking to fill personally.
4 Business people and developers must work together daily throughout the project.
Our team was cross functional and based in a single office. The developers never worked in isolation, the daily stand-up included all of the scrum team and because of the small team size we even included the support and customer success functions in the stand-up. As the team grew we eventually limited stand-up just to those working on the release, and put in a new process to ensure the support and the dev team still collaborated well. In one of my new teams the developers have historically worked largely in isolation with little stakeholder involvement, which is something I am aiming to change.
5 Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
One thing that separates Agile from other approaches to software development is the focus on the people doing the work and how they work together. Having performed team leadership roles for nearly 20 years I think this part of agile came easier to me than other areas. I enjoy creating a culture where people can grow and develop, giving them ownership to work by themselves and a support framework they can call on as needed.
At various times our team lacked key roles – PO, Scrum Master, UX Designer – or had people performing dual roles for various reasons who weren’t able to give either role the focus it needed. Therefore much of my time was spent on trying to get the right people into the right role, and when it worked it was enlightening to see the results. The perfect Scrum team as defined in the Agile Manifesto has been rare in my experience, I’ve never worked at a business that had the ideal team setup, for a variety of reasons, so you always have to be a bit creative. But with a well motivated team who are willing to roll up their sleeves and get stuck in, it can always be made to work.
6 The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Sounds a bit rich now that we’re all working remotely, doesn’t it! But my time with this team was 100% office based, we were all together in an open plan office and made the most of it: daily stand-ups, collaborative backlog refinement, sprint planning, paired code reviews, team retrospectives. Of course it can all be done remotely, so with that in mind this principle becomes more about communicating in real time and providing a framework for that to happen. My new team is maintaining a product using a Kanban board, we don’t have the pleasure of time-boxed sprints and the associated team-based ceremonies, the developers are working on paid projects most of the time and then using time between projects to do a bit of product work, picking items off the board in isolation from each other. I need to work out how we can work collaboratively in order to foster ongoing discussion, learning and improvement as these are some of things I enjoyed most about working on the product scrum team.
7 Working software is the primary measure of progress.
A big lesson here from the PO I hired. The team had a history of working on major new features that would take many months to release. The new PO changed this, finding the smallest releasable piece of functionality, building and releasing that and then iterating it over time. In doing so we would constantly release value to customers, rather than forever telling them the feature was on its way. A big takeaway and one that I will need to bring to my new team, where there are already some major tasks brewing. We’ll aim to break those down into “minimum viable features” and deliver value to users quickly!
8 Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
We had a good two weekly release cadence in the team but were historically poor at estimation and tasks would end up creeping over multiple sprints. The PO got the story sizes down so they were mostly deliverable and releasable in a single sprint and this made a huge difference, particularly to team motivation as more work got shipped more quickly. Some of the team led an initiative to try out new estimation methods, eventually settling on planning poker and story pointing. It took time for the developers to fully understand it, but it eventually jelled and we started to arrive at a consensus on what was achievable in each sprint. This is a great tool when you’ve got a set number of developers for a set time period. One of my new teams has three developers but with the bulk of their time on paid project work, the amount of time remaining for core product development is unknown, so sustainable development is likely to be our single biggest challenge.
9 Continuous attention to technical excellence and good design enhances agility.
Good design and technical excellence can be really tough with a product built on a 10 year old codebase! Your life can sometimes feel more about tech debt than new features, especially with only a small team of 3 or 4 developers. Obviously you have to find a balance. Before I joined, the team had already taken on a huge refactoring of the product front-end by migrating to React but there was lots of work left to to tackle, particularly making the monolithic product more scalable for future growth. Again using the premise of surrounding yourself with people better than yourself, I hired a tech director with prior experience in re-architecting legacy platforms. She did a cracking job in leading the developers to define the approach that would put the team on a path to technical excellence into the future.
Developing for the future while fixing the past is a hard balance to strike, but fortunately we got investment signed off to hire a developer to lead the re-architecting work, fundamental to which was migrating to a microservice architecture. It was a tough journey but by the time I left the team, the fundamentals were in place and it was great to see the other developers starting to create new features as microservices. As this work continues many of the historical product complexities will gradually lessen which will have so many positive knock-on effects. With the addition of an automated CI pipeline it’s easy to see how product agility can be hugely enhanced by pursuing this sort of technical excellence. I’m sad to leave the team before this work is complete, but proud of what was achieved on my watch and I hope the team will complete this work over time, it really will revolutionise the product and the team!
10 Simplicity–the art of maximizing the amount of work not done–is essential.
In working out our roadmap we looked hard at our wider business strategy and the parent group strategy. They had to be linked, our product had to help meet the aims of the wider organisation. There were many long discussions about how best to map product strategy onto business strategy. My PO and I were both long time fans of Roman Pichler who provided some good canvasses to use as part of this process. The significant effort we put into product strategy and the five pillars meant that we could be ruthless with the requirements and ideas that came our way; if it didn’t map into the pillars it was thrown out. Those that remained were prioritised, sometimes a proof of concept was built. It was all about getting the right features into the product for our customers. Given the amount of tech debt we were tackling too, we didn’t have a lot of dev time on new features so we had to be sure that what we chose to develop delivered maximum value to our customers. I think we did that pretty well. I wish we could have delivered more, but that’s probably the same for product managers everywhere.
11 The best architectures, requirements, and designs emerge from self-organizing teams.
I was keen to encourage a self organising team, if a little selfishly due to my responsibilities in other parts of the business! I mentioned earlier how at one point the daily stand-up changed from the full team of developers and support staff, to just ‘those involved in the next release’, following which the CSM and PO worked out a new process to ensure each team was kept well informed of each others work. No-one told the team to do any of this, they decided it for themselves following a sprint retro. This was a turning point for me, I didn’t have to step in to say ‘do it this way’ – the team took responsibility and decided when to adjust their working practice. It started to happen more and more, usually in response to discussions in a retro. I’m not one to micro manage, the team are adults and professionals, they knew they had the authority to make their own operational changes, they knew I trusted them to make the right decision and as a safety net I also knew the PO understood the business interests and they all knew where I was if they had any questions. There were many other examples of self organising, but this one stands out for me as exactly how an agile team should work.
12 At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Every sprint concluded with a retrospective, without fail. Different people led these at different times, it turned out the Lead Developer was particularly good at facilitating them. I’ve already described above how retros resulted in action. Sure, not everything that got raised got fixed, but we did a better job at continuously learning and improving than anywhere I’ve ever worked. I realised that I should have shouted more about this when a manager from another part of the business was asking about how different teams learned lessons. Our team was an agile island among a sea of waterfall-based project teams in the larger business, and not many people understood the practices we were following. My bad, I prefer reflection to self-promotion. But it made me realise what we were doing needed to be told in order for the business as a whole to learn and move forward. Will do better next time!
Conclusion
The three years has been a blast and the product team were an absolutely awesome bunch of people. I’m proud of what we did together. And yes, I feel like I achieved what I set out to do, to immerse myself in agile product development.
I did have one other major lesson, which was seeking sound advice. I hired specialists who were better than me, which is important. But I also sought advice (and sollace!) outside work in a local CTOs meetup, Brighton CTOs. This work is tough on mental health and we all need someone to talk to. For me the Brighton CTO meetups in the pub, comparing war stories in confidence about tech debt and re-platforming and the like, were tremendously useful in providing ideas, sounding boards and just knowing that I wasn’t alone in all of this and that everyone finds it difficult! I hope that we can reconvene in person once we are through this pandemic and life gets back to normal!