Leading Quality is one of the best pragmatic books I have come across, similar to another book titled “Team Topologies”.
Authors Ronald Cummings and Owais Peer have compiled and categorized the book into 3 different sections catering to audiences
from different backgrounds.
Here are my insights from the book:
The original SDLC process mimicked the lean manufacturing and kaizen principles.
Lean manufacturing evolved into total quality management principles that focused on providing value to the customer.
One of my favorite quotes from the book: Continuous deployment without quality is just delivering continuous
bugs to your customer.
The 3 C’s are key to successful implementation of quality: customer issues lead to company issues which eventually
impact your career.
Connect the work of your quality teams to business outcomes.
Section I outlines how one can become a leader of quality by defining the quality narrative.
Audience: This section is crucial for someone starting out their career as an individual contributor either as a new college graduate
or with 1-2 years of experience. This section defines how one can take their quality initiatives to the next level and eventually
lead to successful implementation.
All companies in essence claim that they want to improve the quality of the product they are producing. But you can better
understand if they are actually working towards it by observing how they work. The book quotes a very good example from Hubspot
where the developers work closely with the support team to understand customer issues and thereby fix bugs and improve
customer experience.
In order to influence your quality narrative, one needs to master the following:
Know your audience
Create empathy to increase alignment
Support your narrative with evidence
Cultivate internal champions
There is no one size fits all solution to test automation and copying someone else’s tactics may result in more harm. The testing
environment typically will help dictate your strategic approach.
The value narrative is wonderfully explained using the American Airlines scheduling bug example. Again this narrative varies and
can be one of the following:
Revenue potential
Improving company’s core growth metric
Monetary savings from automation
Risk mitigation
depending upon the challenges faced at your company.
Section II defines how to master your strategic decisions.
Audience: This section is ideal for managers and directors of quality aiming to implement and change the course of quality
in their organizations. Doesn’t limit an IC from thinking along these lines though!
Multiple factors impact why for example test automation succeeds in 1 organization vs fails at another:
Team culture and their approach to problem solving
Test automation expectations
Knowledge of appropriate tools based on product maturity and lifecycle
They invested in test automation infrastructure to get the most of it
Dan Ashby’s framework on when to use different types of testing helps to uncover problems via investigation and how we
verify if that is the expected result or not. The key is you can’t automate creativity. This framework helps to determine when
to apply manual testing vs automation.
Your approach to quality also changes with product maturity. Each company must then adapt their product and view
on quality to suit the types of users they support.
Another favorite quote from the book: If your quality strategy fails, it indicates that your needs have changed and so you
need to change your strategy to suit.
One of the examples outlined in the book from Ashley’s team, explains that the earlier you identify and address a problem, the
more time and effort it will save you in the long run. For example, if your unit tests fail soon after you made a change in the
codebase, it is easier to rework your logic, rather than realize when your QE tests the code manually. There could be dependencies
that could cause a major refactor to fix the issue! That’s why it is important to have the mindset of continuous testing throughout the
development cycle.
Feedback loops are critical to continuous testing to prevent bugs. “In terms of feedback loops, faster doesn’t always mean better.”
There are 4 ways to improve your feedback loop:
Prioritize value over speed
Run tests simultaneously to increase scale
Learn through continuous improvement
Create infrastructure that leverages the team
Investing in a stable and robust testing infrastructure is key to successful automation implementation. Testing in production may
or may not be suitable based on your product.
Investing in testing infrastructure is key because of the 4 things that users really care about:
Basic availability and correctness
Latency
Completeness and durability
Features
Section III defines how great leaders deliver high quality software.
Audience: This section is ideal for head of quality/ VP of quality who want to focus on the overall big picture initiatives to improve
quality in an organization. Again, doesn’t limit an IC from thinking along these lines though!
Always leverage the core value to the customer as a metric to align quality initiatives.
How do you determine which number to focus on? The author makes an observation that the best companies have a
single company wide metric that drives every decision, every department and every individual’s efforts - it is called the growth metric.
There are different types of growth metrics:
Attentions based growth metrics
Transaction based growth metrics
Productivity based growth metrics
Other metrics include:
Mean time to recovery(MTTR)
Mean time to failure(MTTF)
Mean time to acknowledge(MTTA)
Mean time between failures(MTBF)
By focusing on outcome led metrics, there’s a greater chance that the quality teams could contribute to business growth in a
positive way.
Tristan’s notes from our book club discussion:
Key takeaways
- Niranjani Look at the core value that you provide to your customer - that's the key metric to drive
- your business.
- Seema: When things break doesn't mean that QA is failing, just means that the needs have changed. We think we need to
- fix people, processes, automation, but when do we pause to see what is missing here?
- Sara: Think about quality in terms of how it affects the rest of the company. Issue for me: companies don't always see how
- quickly sales see the impact and change.
- Bill: Continuous testing not just being about automation, but from the design phase. Important to
have the whole org align together. Investment in the organization and around the whole organization.
- How do you determine which type of testing is best suited for your product?
- What kind of metrics should your testing team focus on?
- Niranjani: focus on making tests more stable, but never thought of aligning our goals with the company goals.
- Seema: with mobile apps (we collect data about telecom networks). The problem is when we test in-house, it's our
ioS devices. Too many scenarios. I was the first QA member of a brand new team. Device farms have been for UI,
not backend.
- We had to strategize just enough testing at the right status. We found pragmatic ways to create checklists. Have a JIRA ticket
to show that you got the test, but you don't need long written checklists. Automation doesn't just
mean UI or E2E test. We focused time testing partner apps. GDPR compliance, California privacy laws.
Small things
that made a huge difference to going into half-a-day.
- Every quarter we have a tiered focus of countries that is targeted by the marketing team.
We have aligned our test goals based on marketing.
- Tristan: can you connect values of company to the product? i.e. inclusion is a company value, but can we tie this to
product mission for inclusive user experience aka accessibility.
- Sara: our automation has been around for a while via Java mid-90s with solid j-unit. We have already passed the ideal of
100% automation. We already have a base for this. We are now building a new web based app that takes the lessons
that we learned from our desk-top based app; the value of prioritization. What is the impact?
- What kinds of metrics would you use in a quality organization. What are the benefits of having a whole organization
focused on quality and how do we tie this into a product?
- Seema: Put price tag on # of bugs, time to release, developer time invested, and how declined. Time taken to fix these is a
measurement for quality. On the other side, we are building layers of automation.
- How can we turn this into an opportunity for where we focus on quality as a whole.
Make our impact more visible and how we helping? (Sara)
- How do we know if we are bringing value to development teams if we are not on same teams.
- Bill: rotated in and out of sprint teams. We would rotate to technical debt focus, different parts of the app development.
We got a holistic view (load testing, API); people could come in and out of product lifecycle.
Lead with questions and what could go wrong. We were on small teams so this worked for annual rotations.
- We would do in-process demos of what they were working on currently and what would be delivered.
- Co-Author Ron -Take one idea from this book and think about how this applies to your teams. As we move into leadership,
it becomes more about culture, strategy and the impact of the work that we are doing.
- Quality goals aligned with manager; make sure that improving growth metric, while scaling quality. What are our submetrics
and how can that apply to quality?
For more information about the book, check out here.
Thanks Bill, Sara and Seema for sharing your thoughts and making this a lively discussion! Special shout out to Ron for his
personal message to our book club members! None of this would have been possible without our amazing co-host Tristan!!
If you’re interested in becoming a member of our book club, please reach out to Tristan or myself. Check out our book club here.
