The Positive Outcomes Of Negative Testing
As software testers and automation engineers, we often think about the happy path: the path the user will most likely take when they are using our application. When we write our automated UI tests we want to make sure we are automating those happy paths, and when we write API automation we want to verify that every endpoint returns a “200 OK” or similar successful response.
But it’s important to think about negative testing in both our manual and automated tests. Here are a few reasons why.
Our Automated Tests Might Be Passing for the Wrong Reasons
When I first started writing automated UI tests in JavaScript, I didn’t understand the concept of the promise. I just assumed that when I made a request to locate an element, it wouldn’t return that element until it was actually located. I was so excited when my tests started coming back with the green “Passed” result until a co-worker suggested I try to make the test fail by asserting on a different value. It passed again because it was actually validating against the promise that existed, which was always returning “True”. That taught me a valuable lesson: never assume that your automated tests are working correctly just because they are passing. Be sure to run some scenarios where your tests should fail, and make sure they do so. This way, you can be sure you are really testing what you think you are testing.
Negative Testing Can Expose Improperly Handled Errors That Could Impact a User
In API testing, any client-related error should result in a 400-level response rather than a 500-level server error. If you are doing negative testing and you discover that a 403 response is now coming back as a 500, this could mean the code is no longer handling that use case properly. A 500 response from the server could keep the user from getting the appropriate information they need for fixing their error, or worse, it could crash the application.
Negative Testing Can Find Security Holes
Just as important as making sure a user can log in to an application is making sure a user can’t log in to an application when they aren’t supposed to. If you only run a login test with a valid username and password, you are missing this crucial area! I have seen a situation where a user could log in with anything as the password, a situation where a user could log in with a blank password, and a situation where a user could log in with an incorrect username and password.
It’s also crucial to verify that certain users don’t have access to parts of an application. Having a carefully tested and functional Admin page won’t mean much if it turns out that any random user can get to it.
Negative Testing Keeps Your Database Clean
As I mentioned in Chapter 12, having good, valid data in your database will help keep your application healthy. Data that doesn’t conform to expectations can cause web pages to crash or fail to load, or cause the information to be displayed incorrectly. The more negative testing you can do on your inputs, the more you can ensure that you will only have good data.
For every input field, I am responsible for testing, I like to know exactly which characters are allowed. Then I can run a whole host of negative tests to make sure entries with the forbidden characters are refused.
Sometimes Users Take the Negative Path
It is so easy, especially with a new feature that is being rushed to meet a deadline, to forget to test the user paths where they will click the Cancel or Delete button. But users do this all the time; just think about times when you have thought about making an online purchase and then changed your mind and removed an item from your cart. Imagine your frustration if you weren’t able to remove something from your cart, or if a Cancel button didn’t clear a form to allow you to start again. User experience in this area is just as crucial as the happy path.
Software testing is about looking for unexpected behaviors so that we find them before a user does. When negative testing is combined with happy path testing, we can ensure that our users will have no unpleasant surprises.
If you're looking to optimize your testing processes further, consider integrating a top-tier database management tool to handle your complex data needs.
The Complete Software Tester Book
When I discovered my love of software testing in 2009, I wanted to learn everything I could about software testing, but I found very few books that could teach me. I wound up learning through trial and error, reading blog posts, and talking with my co-workers. Today there are some excellent books about specific areas of software testing, such as exploratory testing, agile testing, and test automation, but I am not aware of any book that aims to be a complete reference on testing.
I wrote The Complete Software Tester because I wanted to provide both new and experienced testers with the ideas and tools they need to be as effective as possible. For quick QA tips and more book recommendations, subscribe to The QA Lead newsletter.
Related Read: WHAT IS ZEPHYR SCALE?
Related List of Tools:
Also Worth Checking Out: