It’s almost New Year. Traditionally, this is a great time to take a step back and look from a distance to see upcoming trends and changes. In this article, I would like to describe some test automation tools that are worth experimenting with in the coming year. Some are new, others have already been here for a while, but if you haven’t heard about them, you should give them a try.
My review covers some low/no-code tools, programming libraries and packages that can be used by enterprises, as well as by smaller companies. Coding tools may seem to be overrepresented, but that is simply because it is easier to fit them out with new features.
1. TOSCA
One of the two off-the-shelf tools in this review, Tricentis Tosca is among fastest-growing enterprise-level tools on the market. It allows for functional testing on the UI and API levels. It also includes some advanced AI-based mechanisms, like visual testing and risk-based testing.
With TOSCA, a test automation engineer can create automated tests using best industry practices — including model-based testing. This approach separates the testing scripts from test data. A set of test data creates a set of different test cases that are tested with a single script.
TOSCA comes with an integrated test runner which allows parallel test execution. This is particularly important for larger applications, where end-to-end tests can take several hours. Parallelization makes it possible to shorten this time significantly.
Creating tests does not require any development skills. All options can be clicked through in the tool’s interface. Additionally, TOSCA makes it easy to find elements of tested applications with its built-in scanner.
On the other hand, in a dynamic real-world project, where applications are under constant development, test maintenance in TOSCA may be time consuming. This is mainly due to the way creating test scripts is implemented in TOSCA. Currently, you must rebuild the existing test almost entirely. Of course, testing data remains the same, but still the maintenance process takes quite a lot of time.
Another drawback of TOSCA is that it lacks clear support for testing on different environments of different versions of the same application, for example, in DEV there is version 1.2, but in the QA environment version 1.1 is still being tested. Also, TOSCA doesn’t handle dynamic test configurations well. Consider a situation where your application might be configured differently for different customers and you want to test these different configurations. In such a case you would be forced to create different tests for each configuration.
2. Postman
The remaining off-the-shelf tool in this review, Postman is one of the most popular tools for automation testing on the API level. While definitely not new on the market, it still is one of the top-of-mind tools when you think of functional API tests.
Postman has a pleasant and intuitive user interface, which makes it a very good choice for novice test automation engineers. The tester can easily create an API endpoint test with just a few clicks . Then such a test can be parametrized. The testing itself (asserting results) requires some coding, but at entry and mid-level at most. To make it even easier, Postman provides ready-to-use code snippets that can be easily adapted to your needs. For more advanced users, Postman provides robust support for JavaScript-based programming to make really powerful testing possible.
Equipped with a built-it test runner, Postman allows the tester to manually run the whole test suite and analyze results. But for more sophisticated uses, it has its own command-line test runner — Newman. Newman is also available as a node.js package, so you can develop your own collection runner/framework and reporter that integrates for example with test management tools.
Well-documented, Postman has great learning materials available online free of charge. What is also important for novice users, it has a great community, which can help with more difficult scenarios.
While Postman is really a great tool, there are a few drawbacks. Test traceability might be challenging, which is especially important in regulated environments, where among other formal requirements, test traceability is required to proof testing results. The other thing is use of the tool by power users and deep customization with custom reporters, scripting etc… While it is generally possible, if the amount of custom code is significant, it is probably a better idea to move entirely to code-based testing with, for example, RestAssured (Java) or axios (in the case of node.js).
3. PlayWright
One of the most exciting new automation libraries to observe in 2022, this library is similar to the Puppeteer library developed by Google. In fact, PlayWright is being developed by the same people and maintained by Microsoft. Recently, it has become a truly cross-browser automation library, and now supports Chrome, Edge (Chromium-based), Firefox and Safari.
What makes PlayWright special is that it uses a different browser communication protocol than Selenium does. Where the popular Selenium library communicates over WebDriver, PlayWright communicates over the DevTools protocol (a.k.a. Debugging Protocol). This is an entirely different architecture for automating the browser. The DevTools protocol has significant advantages over the WebDriver protocol. First, it is truly bi-directional. It also provides better control over the browser — everything that is available in DevTools is also available through the DevTools protocol. This allows for more advanced manipulation in modern web applications, especially when it comes to SPA — Single Page Applications developed in modern frameworks like Angular, React or Vue. PlayWright implements advanced synchronization techniques for better stability of tests; you can also easily run concurrent multi-user tests.
Apart from enabling cross-browser testing over the DevTools protocol (which is a great step forward from Puppeteer, which allows testing for Chromium-based browsers only — Chrome, Edge, Opera), it also emulates popular mobile devices for mobile testing. So, you can even test your JS-based mobile applications with just a few adjustments.
However, PlayWright is quite new to the environment, and working with it may be a bit challenging, as there is still limited support and finding code snippets on the web for certain situations might be difficult.
4. Selenium 4.0
Selenium is the golden standard for end-to-end test automation. For many, Selenium is synonymous with automated tests. Lately, we have finally got a new version of Selenium, currently 4.0, and this version brings significant changes compared to version 3.x.
The main improvement is that with Selenium you will be able to communicate with the browser over both WebDriver and DevTools protocols. Especially the latter makes Selenium more modern and more capable of handling testing of modern SPA applications. The DevTools protocol gives you more control over the web browser and enables the development of more conscious tests. You can now observe network activity from the test perspective, or do some performance testing. There are tons of interesting use cases for DevTools in testing, for example programmatic activation of options such as “onHover” for all elements on the page, to assert tooltips.
Another feature that was added to Selenium 4 is a function called Relative Locators. The new methods introduced in Selenium allow you to find a web element that is situated relative to another element, for example toLeftOf() or above() a known element. This makes the development of some cases (e.g. finding elements on the list) more streamlined and potentially less flaky compared to hardcoded locators.
With the new version of Selenium, Selenium Grid and Selenium IDE were also updated. The former enables parallel testing, and the latter gives you the ability to record and play tests. Such tests can be then exported to several programming languages, which (when used carefully) can help test development faster.
Among Selenium’s drawbacks we should mention the lack of auto-synchronization features to work with modern SPA applications. Using the DevTools protocol may help, but compared, for example, to PlayWright, there is still a lot to be developed manually.
5. Cypress
Cypress is getting some good press lately, and for a good reason: it approaches end-to-end testing in a unique way. When many tools today provide browser automation based on the WebDriver protocol (developed by the Selenium project), tests in Cypress are written in JavaScript and Cypress runner integrates with the browser. Tests are then run directly in the browser. This approach makes end-to-end testing more stable and robust.
Even though Cypress is a test library/package for node.js, it is very opinionated, which can streamline the whole test development process. There is no need to select the assertion library, test runner, and other libraries. Cypress comes with a whole package ready to use.
Tests are developed only in JavaScript. This may be perceived both as a positive and a negative. JavaScript is actually the only language that runs inside the web browser (I am abstracting away from Web Assembly for now), which makes it native for browser-based testing. Also, because front-end applications are developed with JavaScript/TypeScript, this makes test development more integrated with features development. Developers and test automation engineers can work in the same technology.
However, there are some actual drawbacks of Cypress you should know about. Probably the main one is that Cypress doesn’t provide a fully cross-browser testing environment. It supports several web browsers (mainly Chromium-based ones), but it doesn’t support Safari. Also, Cypress does not support multi-tab testing, which is particularly useful for testing multi-user scenarios, and it does not support mobile testing.
6. Pact
While Postman makes functional testing on the API level easy and streamlined, Pact provides a totally different approach to API testing. Instead of creating functional tests, Pact is focused on contract testing. Contracts between producer (most often — back-end application) and consumer (most often — front-end application) are shared structures and contents of messages exchanged between them. Pact makes it possible to run integration tests for different parts of an application in isolation. This way back-end can be tested without front-end and simultaneously front-end can be tested with back-end mocked by Pact. This approach to application testing can enable deep integration and involvement of dev teams and testing teams.
The drawback of this approach is that it might be difficult to introduce, because it requires a highly skilled team. Also, even though development might be simplified, the initial setup and configuration is a bit heavy.
Summary
The coming year is set to bring us not only development in well-known and established tools, but also some new tools entering the mainstream. As always, there is no such thing as the one best tool for all test automation jobs. You need to be aware of what project you are working on (what your business goals are) and what are other technical circumstances. Condsidering all this , make an informed decision to get tools best aligned with your needs.
