Better Test Code Principles: #2 Don't generate ALL your test data in @BeforeClass

A considerable amount of tests will need some test data to be generated previous to them running. Some people prefer to put all the data creation for all the tests in a class into the @BeforeClass method, some others prefer to keep the prerequisite data creation inside the tests themselves.

The @BeforeClass mechanism goes something like this:

1. you have a method allocated to this annotation, where you will write code that will be run before any code from the test methods are run

2. if all the code in the @BeforeClass method is run successfully, you will be able to run the tests in the class

3. if any step in the @BeforeClass method throws an exception, no other subsequent code will be run

Here are a few reasons why i would not suggest placing all the prerequisite data generation into the @BeforClass section:

1. If there is an exception encountered while running any step in the @BeforeClass method, the execution of the @BeforeClass will be stopped.  That means that some of the steps of the @BeforeClass might have been successful, but the first step that will fail will cause the whole method to finish abruptly. Since this step failed, no test from the test class will run. All tests from the class having this @BeforeClass method will be skipped. This means not even tests whose test data generation was successful will not start (those steps that where successful, before the failure was encountered). In this case, all the tests are dependent on the generating of test data for all other tests. Hence you can say that they are not independent, which is not good. You need to be able to run tests separately, but in this case you cannot. This is a situation you want to avoid - either all tests will be able to start, or none will start at all, given where their prerequisite data is generated.

2. If for some reason you only want to run a single test out of the whole test class, you will still need to wait for the data generation corresponding to all tests in the class. That is because when you run just that one test, the entire @BeforeClass is still run. Hence you might need to wait for a few minutes just to run a test that might take seconds to finish.

3. If you are working with testng.xml files, that means you want to select only some test methods to be run from your whole test project. Since these tests each require their @BeforeClass to generate their prerequisite data, yet again you will have to wait for all the lines of code from the @BeforeClass method to be executed before any test can be run. Now consider, as an example, that you want to extract 3 methods from 100 classes. That means about 300 methods to be run. Let's also assume that the @BeforeClass method of each class contains the data generation steps for all 10 method of each class. That means that in order to run 300 tests, you will actually need to wait for the data generation of 1000 tests. And that will take much more time than if you had separated the data generation steps into each test method, as is required. Having said all the above, it is better to have the test data generation code written inside the test methods that require it. Hence tests will be independent and your @BeforeClass will be smaller and cleaner.