We recently released Erik (name origin), an open-source Node package for running jasmine tests headlessly with PhantomJS and gulp.
Background
Prior to developing Erik, we ran our client tests semi-manually by running an npm script that opened our jasmine SpecRunner.html file (configured similarly to what's shown in the jasmine installation instructions) in Chrome. jasmine showed an easy-to-understand output of the test results and re-running the client spec suite was as straightforward as refreshing the page.
There were downsides to this process, though. We didn't like having to remember to run npm run client-tests (and often didn't remember) and we weren't able to run the tests in CI (and were therefore unable to block merges and deploys on failing client tests).
Earlier this year, we wrote some code that caused some of our client tests to fail. We missed running the client tests before merging the changes into master and later deployed the changes to production, where we realized that they had broken some functionality.
After fixing the issue, we decided that it was time to pull the trigger on automating our client tests so that we could avoid similar regressions in the future.
Requirements
When beginning to identify a solution for running our client tests headlessly, we laid out our requirements:
- Remote dependencies. In addition to our locally processed and bundled JS, our
SpecRunner.htmlfiles included several remote dependencies loaded from CDNs. We wanted to continue using those CDN-served libraries rather than bringing the libraries into our own projects. - Stick with
jasmine. All of our existing tests were written withjasmine. We didn't want to have to rewrite all of our tests in order to start running our client tests headlessly. - CI integraton. The main motivation for running our client tests headlessly was to catch failures in CI.
Additionally, we wanted to avoid writing custom scripts for running tests in PhantomJS. We preferred to use an existing driver/wrapper to interface with PhantomJS so that we didn't need to worry about ensuring the functionality of a new driver/wrapper or take time to learn a new library (PhantomJS).
Deciding on a solution
PhantomJS proposes a few packages for tesing headlessly but unfortunately, none of them were sufficient for us. Chutzpah is specific to Windows, guard-jasmine is for Ruby apps, we didn't want to introduce Grunt just for grunt-contrib-jasmine, and phantom-jasmine uses an outdated version of Jasmine and is unmaintained.
Eventually, we discovered Karma, a library that allows you to run tests written in a number of testing frameworks in different environments (including Chrome and PhantomJS).
Karma turned out to be perfect for us, but we didn't want to duplicate our configuration across all of our microservices. Additionally, we figured that we weren't the only ones with these needs, so we decided to develop an open-source package that would solve our issues as well as those of other developers: Erik.
Erik
Erik uses Karma and Karma's karma-phantomjs-launcher to register a gulp task for running tests headleslly. It assumes some default configuration to make getting started easy, while exposing the important stuff that package users might want to configure themselves.
Erik enables easy specification of remote dependencies, overwrites some Karma defaults for improved performance and test result communication, and works around a couple of bugs (1, 2) we discovered while developing Erik.
It also offers a "watch" mode, which re-runs your tests when your local dependencies change.
Shoot us a message if you're interested in joining an engineering culture that defaults to open-sourcing its tools.
