A common theme in testing browser-based applications is the browser matrix. Most projects are targeted at a small set of browsers and versions. This is intended to align to a majority of users that will use the application. The browser matrix is an import consideration in web applications. Unfortunately, testing a browser matrix indroduces several problems into the teams quality automation.
First, multiple browsers increase the complexity of test automation. The developer building automation must handle the matrix and browser expectations in code. This makes the code more costly to produce and maintain. This reduction in implementation speed can lean a project toward reduces the quantity of browser based tests.
Second, the time to run a full test suite increases dramatically. The tests must be run for every combination in the matrix. While this can be parallelized, the test must run within an active browser. This is increases the time to get feedback. The faster a set of tests run, the quicker the team can respond to breakages. One key goal of agile is reducing the feedback times to make changes less costly to implement.
So, how do you reduce the cost of building tests and the time for feedback? The test pyramid is a common metaphor for placing efforts on cheaper, faster tests before more expensive, integrated tests. While the test pyramid has the UI tests on the top, it is possible to break the UI tests into a simple pyramid of their own.
In order to have a high value suite of tests that run quickly, a new tool is required. This tool is PhantomJS. PhantomJS runs a headless browser and is completely automatable. It is based on WebKit and runs from the command line. It is also very fast.
The testing pyramid for UI automation is based on a broad suite of tests targeted at a single browser: PhantomJS. The PhantomJS tests are the unit tests of browser automation. The cover the logic and flows deeply. The also cover it quickly. This suite is run as often as possible. Ideally this should be on every project commit or code release.