4 Reasons Why End Users Need to Participate in Testing Before UAT

As end users, many of us who have tested a product or service just before it goes to market are familiar with that unpalatable anxiety in the back of our minds suggesting there may be some function or feature we did not test, simply because we didn’t know about it. That is why it is so important for end users need to be involved with testing before user acceptance testing (UAT) begins.

It’s a catch twenty-two. Since end users are typically not skilled testers, they have their own full-time jobs to focus on. However, any software product that is going to have a successful release in today’s agile environment will need a significant amount of their time, and not just at the tail end of the development phase.

Oftentimes functional or performance testing, preceding UAT, is assigned to a test team who might not have been involved in business requirements gathering or have minimal knowledge about the project’s objective, relying solely on their test scripts. Those testers may have even been outsourced due to staffing shortages. The saving grace is that experienced testers are used in these sink-or-swim situations, and have a dynamic skill set to traverse those waters. However, they may not always fully understand the waters that surround them, so there are limits to what they can do.

Such limitations can become very costly for the project, very fast. The most cost-effective way to avoid defects is by having end users involved in testing throughout the product development life cycle.

1. End users understand exactly what the system needs to do (for them).

An experienced tester can be advised on requirements, however they might not know if the product they are testing effectively meets those requirements or resolves all the necessary issues, especially those that were never mentioned during requirements gathering sessions.

“We need to do testing at every stage by launching incremental modules out to the tester. It can’t wait until the end; if the product fails, we want to fix it fast,” said Bashir Fancy, Chairman and CEO of Canada’s Association of IT Professionals (CIPS). He also believes that this approach, often referred to as the extreme programming agile methodology, provides the project with a higher participation rate during UAT.

2. End users’ acceptance is more likely if they’re involved during earlier stages of testing.

Robin Goldsmith, owner of Go Pro Management Consultancy, does not recommend trying to turn end users into “miniature testers,” however he understands the importance of bringing them into the picture early. “Users will need to start thinking seriously about what they want to see in the delivered product before they can feel confident enough to rely on it.”

Engaging users early on not only allows them to become more competent and confident as testers, it also allows them feel empowered as key stakeholders. “They want to know that project team is facilitating the users’ version of acceptance criteria, not their own,” said Goldsmith.

3. End users can identify bugs before they get too costly to fix.

The earlier we catch a defect, the less money it costs to fix it. An infamous study published by the National Institute of Standards and Technology (NIST) in 2002 indicated that the cost of fixing one bug found during the production phase is 15 hours, compared to five hours of effort if the same bug was found during development.

And we’re only seeing the tip of the iceberg.

It is much easier to backfill an operational role as opposed to a UAT tester’s role as the latter requires someone with very specific experience, and hindsight, to validate a product specifically developed for their use. A quick peek on any major Canadian jobsite indicates the average salary for a QA tester ranges between $55,000 and $80,000. These numbers can easily skyrocket for a company that is outsourcing their testers, whose rates can go as high as $100 per hour when working on a high-profile project. The average salary for an administrative role, such as a clinical administrative coordinator for example, ranges from $35,000 to $45,000.

One thing we can’t estimate is the reputational damage that results when end users, who were not engaged until the UAT phase, see the product they must use for the first time. At this point, they have been set up for a number of hurdles already, such as being forced to deal with change or possibly even transformation, and now have a short time span to become experts with a new product or service they were not asked about. Unless the project team can offer a very fine tuned product, there is a high risk that negative feedback provided by end users can stain the project’s reputation.

4. End users offer a broader perspective.

While the project team works around the clock on project deliverables, such as facilitating meetings, identifying requirements, and updating Gantt charts, end users can stay focused on ensuring the product everyone is working so hard to deliver actually does what it needs to.

Fortune 500 companies, such as Apple and Google, conduct beta testing where, with some form of compensation, the public can access an early release of the product, bugs and all, in return for their participation in helping to identify defects and providing useful feedback.

Director of Centre Code Mario Sancho, who manages beta testing programs for many of Canada’s Fortune 1000 companies, believes customer validation should happen on at least three general levels of testing: alpha tests, where users test the earliest release possible, beta tests, where users can test every major feature of the product, and field tests, also known as UAT.

Conclusion

The importance of end users’ acceptance can not be understated. Anything that is socialized about the UAT experience to management, colleagues and the public could mark the project’s legacy as a success or failure for a long time to come. Even while the project benefited from the experience of an excellent project team, met all its deadlines, and stayed within budget, a stakeholder’s view on how well the project achieved (or didn’t achieve) its scope is heavily weighed upon by the end users’ experience.