SoftAssert - don't make your test fail on the first assertion failure

When you have more than one assertion in your test, you might want one of two things:

1. Have your tests fail once the first assertion failure is encountered.

2. Have all your assertions run, no matter if they have passed or failed. Of course, after they are run, if there are failures, you want the test to fail, and also show you where the issues were.

For the first scenario, regular asserts from the TestNG library are used (assertTrue, assertEquals and so on). These are found in the Assert class.

For the second scenario, the same types of asserts can be done (you can check whether two items are equal, whether a condition is true, and so on). However in this case,  the methods come from another class, SoftAssert. One difference from the first kind of assertions is that after all the assertions have been written, an additional assert is needed: assertAll(). The way soft asserts work is that all the asserts will be executed. All of them will be run , even if they fail, but after all are run, the results are displayed. These results include all the failures that were encountered during the test execution. If there is at least one failure, the test will be marked as failed. If you forget to add the assertAll() check, the test will pass, and no failures will be printed, just as if everything works properly.

Importing the assertion methods/class

There is also another difference between the two kinds of assertions: for regular asserts it would be enough for you to make a static import on the class / assert methods and just use the assertions (without a prefix).  For softAsserts no static import is possible, hence you need to instantiate the SoftAssert class. A little example helps to understand what i mean:

1. How to import the methods/class for regular asserts:

import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertFalse;
import static org.testng.Assert.assertTrue;
In this case, you only have access to the three assert methods: assertEquals, assertFalse, assertTrue. To use other asserts from the Assert class, you will need to import those as well.
If you would like to use only one import statement and to have access to all the assert methods from the Assert class, you can use the following import instead:

import static org.testng.Assert.*;
 

1. How to import the class for softAsserts:

import org.testng.asserts.SoftAssert;
Before using any asserts from this class, you will need to instantiate it, right at the beginning of the class, like this:

private SoftAssert softAssert = new SoftAssert(); 

Usage and examples

A few examples are shown below, with the usage and results of running regular asserts versus soft asserts.

1. Regular asserts

@Test
    public void regularAssert() {
        assertEquals(1, 2);
        assertTrue(false);
        assertFalse(true);
    }
The result:

java.lang.AssertionError: 
Expected :2
Actual   :1
In this case, even though there are several asserts to be run, the test will stop running when it encounters the first assertions failure.

1. Soft asserts

@Test
    public void softAssert() {
        softAssert.assertEquals(1, 2);
        softAssert.assertTrue(false);
        softAssert.assertFalse(true);
        softAssert.assertAll();
    }
The result: 

java.lang.AssertionError: The following asserts failed:
    expected <2> but found <1>, 
    expected  but found , 
    
Expected :false
Actual   :true

Here all checks were made and all failures reported to the user.
Another example, where only two of the assertions are failing:

@Test
    public void softAssert() {
        softAssert.assertEquals(1, 2);
        softAssert.assertTrue(false);
        softAssert.assertFalse(false);
        softAssert.assertAll();
    }
In this case, the results are: 

java.lang.AssertionError: The following asserts failed:
    expected <2> but found <1>, 
    
Expected :true
Actual   :false 

Note: Ideally when working a project you should use the latest version of your dependencies. However some of the latest TestNG versions (including 6.9.10) have some issues, so if you encounter a message that says: "org.testng.TestNGException: org.xml.sax.SAXParseException; lineNumber: 3; columnNumber: 44; Attribute "parallel" with value "none" must have a value from the list "false methods tests classes instances " --> you should simply downgrade the library to a good version. You could try 6.9.4.