Using try/catches to handle exceptions has become quite fashionable when writing tests with Java. However, this approach is also a frequent source of having false positives while running tests. Many times when writing the tests people forget to consider both sections of this code block: they forget to write the appropriate code in both sections - the code that deals with not encountering the exception (if the code in try executes) and the code that deals with encountering the exception (if the code in catch executes).
I won't go into what try/catch or Exceptions are. Those explanations can be found in the official Java documentation. This post is really about how try/catch is used in tests, what possible usages it has, and how, if not careful, it can lead to false positives in the tests.
One thing that should be mentioned is that catching exceptions must be made with care. You don't really want to catch the greatest exception of them all, "Exception", but instead specific ones (like NoSuchElementException for example). Using "Exception" will lead to executing the code from catch, no matter what exception was really thrown. So the some treatment will be applied no matter the exception, which is not ideal. You should treat each exception in a relevant way. Also, you need to consider the case when the code you wrote in the try block contains several lines where that same exception is thrown. In this case you need to be careful that you are aware which part of the code from try caused the exception. You would ideally have just one line of code in try that throws that specific exception.
Now, when it comes to writing the code, there might be 5 cases of using try/catch in your tests, as shown in the below image.
Let's take a look of all these situations and see how they should be properly written.
Test should only pass if the exception is not thrown
This scenario assumes that you will put some code in the try branch. If, and only if, that code does not trigger the specified exception, will the test pass. This means you need to write that code inside the try section, but you need not forget to put a failing counterpart in the catch section too. For example, a call to TestNG's fail() method or throwing a new exception is something done frequently.
From a code perspective this looks something like this:
try {
codeToBeRunHere;
}
catch (specificException e) {
fail("Code failed due to specifiedException");
} How this runs: the code in try is run. If the specified exception is encountered, the remainder of the code from try is not run (the lines of code below the line that generated the exception). Instead the code inside the catch section is run, namely the fail() method is called. This will fail the test, as per TestNG's behavior.
Note that if the code from the try section throws a different exception than the one declared in catch, the test will still fail, but by throwing that exception. If you want to also fail the test if another specified exception is thrown, you can add another catch clause.
HOWEVER - if you think about it, for this situation, you don't even need to use try/catch! Without it, you will simply write the code, and if any exception is thrown it will end and fail the test right there. It will display the appropriate, corresponding message to the console, with the stacktrace, so you can understand what happened and where it broke. Using try catch for this scenario would only be "useful" if you want to throw a specific exception or display a specific message in case of the failure. But this does not really justify using try/catches.
How you can get it wrong: What frequently happens is that a test will pass, even though an exception that should not happen is encountered, because in the catch section no code has been written for failing the test if the exception arises. This means that the test will pass even if the happy flow happened, but also if the error occurred. It will pass no matter what! Hence, a false positive. Don't do this.
2. Test should only pass if the exception is thrown
This is an interesting one: in try you have the code that needs to run. You expect it to throw an exception, in which case the test, up to this point at least, is considered successful (green, passing). However, if the exception is not thrown, you need to make sure the test is marked as failed (as success would mean that the exception has been thrown, which did not happen). Therefore, in try, right after the code is written, a call to TestNG's fail() method should be made.
From a code point of view this would look like this:
try {
codeToBeRunHere;
fail("Code should have thrown exception; it did not, therefore this test is marked as failed!");
}
catch (specificException e) {
} How this runs: the "codeToBeRunHere" is run. If it throws an exception, the remaining code from try is not run anymore, and instead the catch block of code is executed. However, if the execution of "codeToBeRunHere" is successful, it means no exception has been thrown, and the next line of code from try is executed, which in this situation is a fail() command, to mark that the test did not behave as it was supposed to (it did not throw the expected exception).
One thing worth mentioning: in the catch section there is no need to write any code. What happens is that, in the happy case, when the code has thrown the exception, the program will continue running with the next code statements that come after the catch section. Therefore, whatever code you think needs to be run in the catch section, will actually be run outside of the try/catch block, right after the catch section.
How you can get it wrong: Not putting the "fail" in the try section will lead to the test passing. That's because, in this case, no exception is thrown (as was required) and no check is made for it being thrown either.
3. Test should pass no matter if the exception is thrown or not
This scenario means that you want to run the code from the try section. Sometimes an exception might occur when the code is run, but sometimes it might not. You want the test to pass in both cases, and no further processing needs to be done when the exception is thrown. This means the catch block will remain empty. For this case, the code looks like this:
try {
codeToBeRunHere;
}
catch (specificException e) {
} 4. Test should pass no matter if the exception is thrown or not, with processing if it is
This scenario is similar with the one at point 3. The code from the try section sometimes throws an exception, sometimes it doesn't. The difference here is that you want to do some processing if the exception was thrown. A frequent such situation is when a popup might randomly appear when you are interacting with a page, and in it case it does, you want to close it. For this scenario, you simply need to add the processing bits into the catch section.
try {
codeToBeRunHere;
}
catch (specificException e) {
processing;
} 5. Test should fail in any case
Impossible. This scenario can never occur. If you find you wrote "fail" both in the try and catch sections, you are missing something. That should never happen.
As a summary, whenever you are writing try/catches, make sure to think of what happens when the exception is thrown, but also when it is not, and write the appropriate code for both these situations.