Everything About Automated Testing

What is test automation?

Wikipedia says: 'In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes.'

Therefore, we automate the way how we - the user or just a consumer of an API - uses the application. There are two big groups:

GUI testing

When we do automated testing in this group, then we actually record an ordered set of user actions, then we run them against the softer under test, and then we check the result against the expected result.

API driven testing

When we do automated testing in this group, then we have no GUI. We have only an interface that can be accessed by our automated tests, then we just pass many kinds of parameters into it and verify the results against the expected.

Example

In our example, we have a SchoolManagerGUI, where the User can change the type of the IKitchenService with a few clicks on the UI. Note that, the user can see and access only the GUI, but there are many more services, managers in the background.

Why is it good for me to create automated tests?

Because it will:

1. - save up time and energy for you to safely add or extend features in your app,
2. - save up time and energy for your tester to focus on figuring out more advanced scenarios,
3. - save up time and energy for your team to have more time on the interesting new features instead of fixing bugs most of the time during an iteration.

How should I organize my automated tests?

Probably, you have already faced a situation, when you felt like you write more and more tests but you still have many issues after your tester verifies your changes. Maybe you have an awesome tester, or maybe you just do not organize your tests properly.

There is a suggestion on how you should organize your tests, but first, let's define a couple of test areas with the help of Wikipedia.

• - As a solid foundation, Unit testing provides robustness to the software products. Testing individual parts of the code makes it easy to write and run the tests.
• - The service layer refers to testing the services of an application separately from its user interface, these services are anything that the application does in response to some input or set of inputs.
• - At the top level, we have UI testing which has fewer tests due to the different attributes that make it more complex to run, for example, the fragility of the tests, where a small change in the user interface can break a lot of tests and adds maintenance effort.

It worths mentioning, Unit tests are usually very fast; while UI tests are slower. The reason is simple, the UI tests might need many more components, layers or modules to be run; while Unit tests focus on small units of the features, components, layers, or modules.

I did not mention classes on purpose, because you might face many more levels in the test pyramid in some workplaces. It can easily happen some teams define their own understanding of a layer, or they just simply introduce a new one. For example, I usually split the Service level into two: Integration and Functional tests, where Functional tests are like big unit tests, that run multiple classes, services; while Integration tests run modules or layers with the help of mocks.

Risk based testing

If it is hard to figure out which test case should be covered with what level of tests, then I suggest you use the RBT table:

In my example, there are only two important things in the project: Is this test case important for the users? and Is this test case important for the developers? And in my project, my goal with the RBT is to analyze which test case should be covered at what level. The more risky - the more important test case for the user needs to be tested by UI tests because that's the way how the user uses the application at first. The more risky - the more important test case for the developer needs to be tested with lower level tests because the user might not be able to repro those test cases easily, but we can do it from the code for sure.

Let's take the last line of the table, Test Case III. It is not important for the user, but it is important for us and we are so afraid of it, we will cover it at all levels. Or let's check Test Case II, it is kind of important for both the user and the developer, so we will ensure there is a UI test for the user and we also add service tests where we try to break down our feature with ways that the user even not capable of.

A use-case, when to use unit tests

In our profession, it is usually we have to deal with old, legacy code. It is not a problem to have them, there is always a reason why these codes exist, there is no reason to blame the creators of that code.

However, you still need to deal with it somehow, let's say you have to add, extend or refactor one legacy feature. Your first step should be adding as many automated tests as you can, so when you jump in changing the legacy code, you will have a stable checkpoint, constant feedback about your latest changes.

For example, first, you add UI tests to a login screen, then you start to modify the login process behind the UI. While you do your changes, you can frequently verify whether your latest modifications are correct and the user can still log in.

Some useful links

• Test automation in general
• An awesome tutorial about automated testing
• The Practical Test Pyramid
• Complete Guide to Test Automation

You will find many awesome and more sophisticated articles about this topic; here I have tried to introduce it from a different perspective, using simple kitchen language.