https://www.linkedin.com/pulse/do-we-need-test-team-nathan-shearer
This has been a difficult post to write. Not the writing; the controversy.
“Traditional” systems are being disrupted at an unprecedented pace. This is true of not only the Test Team, but many other knowledge task-based roles in organisations.
The “End Of Jobs” is a very real prospect in our current generation, moving much of what we class as work right now into robotics and eventually AI.
Before this turns into some sort of opinion piece on macroeconomics, let’s bring this back to Quality Assurance.
With literally hundreds of friends and colleagues in the Testing industry, it feels like putting targets on their backs.
That’s not the intention. As with every job, business, strategy, if you aren’t growing, you’re dying.
How To Adapt To Radical Change In Testing
After spending 10 years in senior roles in almost every sector and industry it’s still fascinating to find many medium and larger enterprises that still have no concept of having a Test Team.
We have a QA lady that comes in 2 days a week
Equally as fascinating is how many larger organisations have bloated, disparate “Testing” teams with no clear accountability and commonly conflicting motivations with Development Managers, Project Managers/Sponsors, and The Business/Users.
How often do you have robust conversations with your head of QA that seem to escalate to the Trump card “this could break everything, I’m just trying to raise the risk..”
The truth is, this is by design.
Quality Assurance should be a task and strategic goal for every team (and member’s of).
Quality Assurance should be responsible for the quality of code.
Quality Assurance should push projects forward.
Quality Assurance should escalate to the Users.
But, that leads us to our original problem – bloat.
When Gartner’s famous Magic Quadrant for Outsourced Testing Services back in 2011 said we were less than 2 years away, coming out of the Trough of Disillusionment, from this being the norm, well-organised, centralised, outsourced testing as a service should be in every organisation by 2014.
Without getting into any predictions, it’s probably safe to say that one was a miss guys.
The big players in the Testing As A Service are the most impacted by that failing of prediction.
They built large flexible teams and heavy hitting front end consultant teams for post-sales but couldn’t deliver on a promise that was based on a rapidly evolving development model.
Those 2 developers over there take it in turns to test on the last day of the sprint
Test Driven Development and Behaviour Driven Development have grown significantly in popularity and adoption over the last 2 years and in turn have pushed much of the work of the traditional testing resource back up the project chain to Developers and Business Analysts.
Why not, if the processes and resources required in a Test Team will extend a project and increase costs, this approach to spread these tasks and responsibilities makes sense.
In fact, PlanIT’s recent Testing Index showed that organisations across Australia put “Time To Market as the biggest trend to impact projects over the next 5 years”
Does that mean we drop testing and the test team altogether?
Is there a magic pill? How do organisations and current test resources evolve?
The Lean QA Startup
Everyone’s favourite bellwether methodology can be applied directly to Quality Assurance and the Testing Lifecycle and the efficiency gains will make tight pursed PM’s and CTO’s alike sit up in their chairs. (Or their stand up desks…)
Minimum Viable Testing
If starting from scratch, building a set of regression tests to provide coverage across the core functionality of the system can be a monumental task.
Similarly, testing every permutation of a new change can quickly escalate the number of tests to a level that will see the test effort double, triple, (or even more) the development effort.
To move fast (and break stuff) the approach to MVT is similar to that of Risk Based Testing.
Determine the level of risk based on changes and weigh the testing effort accordingly toward change and core functionality.
The Testers Role Here: Familiarise with modern risk based testing approaches. Work closely with Business representatives. Plan, define, coordinate, and report on the execution of these tests.
How much testing do we need to do – to be confident?
Continuous Smoke Testing
When Philip Lew said earlier this year that Testing Is Not Dead, It’s Just Changing and “traditional blackbox manual testing where you have people going through the software and finding unexpected or broken behavior certainly has been reduced but will never go away” I think he wasn’t looking at the bigger picture.
The idea that this is classed as “Exploratory Testing” or “Smoke Testing” is outdated.
Many development teams have matured Continuous Integration, with many of the development stacks now offering robust solutions for even the smallest of teams.
By opening up these systems and processes to the testers to utilise, the testers can bring a QA mindset and approach to the development of these tests and in turn execute these as part of their functional test sets.
This also has the flow on effect of enabling a Test Automation function to grow organically within the team and in cadence with the development team.
This does require testers to break open the black box somewhat and become more technical in their skillset.
The Testers Role Here: Upskill in the Continuous Integration methodology and tooling used by the development team and pull the discrete unit tests together into a set of functions that serve to validate
A/B TESTing
Split Testing is an experiment in which “different versions of a product are offered to customers at the same time” with the goal being to observe differences in behaviour and measure the impact of each version on an actionable metric.
Within this framework, we are aiming for Actionable Test Reporting therefor A/B Testing will have a goal of testing Risk assumptions, the Risk of the pending changes or system being implemented.
What could go wrong with these changes?
What’s a possible unintended functionality with this core function?
By applying critical thinking against risk and use case paths, testers can reduce the number of repeated tests and scenarios and produce a greater level of confidence sooner.
As with peer reviews in the development space, splitting a test team or resource pool into defining vs execution groups will provide an additional level of criticism of the system under test.
The Testers Role Here: Critical thinking to define and create these scenarios. Plan the scheduling of these tests.
Actionable Test Reporting
Instead of traditional Test Completion Reporting, which typically restates most of the test plan, project context, and other such superfluous material, Actionable Test Reporting is based on a Quality Assurance view of risk to the system and users.
A huge list of passed tests can have as much bearing on the confidence in the system as a small list of passes and fails.
Paramount to this accountability is the testers skills in adequately representing both the systems development/maintenance and the Users/Business interests and deeply understanding what each group will need to be confident enough in the solution/sprint to allow the project/change to progress.
The Testers Role Here: Report risks and mitigation with actionable steps to reach the required level of confidence in moving forward.
What’s the Risk?
QA Pivot
In the Lean Startup world, a pivot is a “structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth.”
Steve Blank defines a pivot as “changing (or even firing) the plan instead of the executive (the sales exec, marketing or even the CEO).”
From a QA perspective, the Pivot is the stage gate process of whether to complete the current phase or sprint based on the results of the testing.
The important focus being that you fire the plan (in this case the code branch or sprint) and not come down on the development team or continue to raise bugs against a dead code path.
The Testers Role Here: Accountability for the decision to proceed.
Innovation QA Accounting
A testers role is to maintain accountability and maximise outcomes for system changes.
This means ensuring each test, risk mitigation, defect, and report is created with the interests of Development, the Business, and the User in mind.
Reporting is not only done at the end.
Key to improving the pace of the testing team, along with the recovery and remediation time of defects by developers, testers should be working almost side by side with the development team.
This includes being part of daily standups and coordination of defect triage and resolution sessions.
The Testers Role Here: Planning, scheduling, coordination, and reporting of Quality Assurance items.
Test – Measure – Learn
The Lean Startup Build Measure Learn loop emphasises speed as a critical ingredient to product development.
Speed is not usually associated with Quality Assurance.
Yet given adequate technical skills and tools, a complementary set of processes to the development team, and a cross project/business accountability to confidence in the system, a tester can maintain and improve these lean based skills and processes to keep pace with change.
The Testers Role Here: Maintain documented test processes and show measurable ongoing improvements to Actionable Test Reporting and Innovation QA Accounting above.
Implementing Lean QA
Typical consulting engagements will roll out the red carpet and sales machines with a view to standing up a traditional testing model that ensures stickiness and longevity for the consultancy. (See my previous post on Better Outsourced Testing)
For new teams or those looking to add QA as a necessary evil, engaging a short term expert to develop a Lean QA Framework will ensure an adequate level of documentation and workflows with a pragmatic balance of actionable processes, procedures, and resource planning.
Wrapping this framework around existing development and test teams can enable an in-place transition without workflow or project disruptions.
Want a copy of this framework? PM me.
Further assistance in tailoring and or implementing a Lean QA within your organisation, large or small is also available. Reach out for more details and to schedule a informal review for your Development or QA team or organisation today.