Unit Tests in WordPress: Making It Work on Travis

Now that we have configured succesfully our environment to run PHP and JavaScript unit tests, it’s time to automate the testing process. Travis CI is a service that allow you to automatically run your tests remotely. If you work alone, you can also avoid using it, thought I would use it anyway for the advantages that it brings. But if you work in a team, even in a team of two, you cannot afford not to use it. Let me explain in a few words how it works. First of all, you must have your plugin or theme on Github. Travis will monitor your Github repository and when it notices that the code has been updated, it downloads the plugin and runs all the tests against the new version of the plugin. In case all the tests succeed you won’t even know. In case any of your tests fails you will be notified by email. The first advantage that I see in this system is that you don’t have to run your tests manually every time that you modify your code. That alone is a huge time saver. To add to that, the whole testing process is simplified if you work in a team. If your partner modifies the code and pushes the modification to Github, the system will run the tests without you having to lift a finger. So yes, automation has a lot of advantages.

Configuring Travis

Configuring Travis to run tests is very simple…if you have only one language to test. It wouldn’t even make sense writing a whole article dedicated to Travis if we had only PHP or JavaScript to test. Configuring Travis to test only PHP or JavaScript is well documented, you just declare the language that you want to use, and Travis will take care of installing everything needed to run tests against that language. But my goal is to run tests in both PHP and JavaScript together, and you are allowed to declare only one language in the configuration file, so we must find a way to install the second language manually. Unfortunately, manual installation is not documented on Travis, and you have to figure it out by yourself. At the beginning, I wasn’t even sure that what I wanted could be done. I mean, why if it is possible it’s not even documented inside Travis? Then I found this issue opened on Github, and from here I started my research.

It makes sense to run tests against PHP and JS together if you think about it. Suppose that you modify a JavaScript function. You push your new code to Github and Travis…runs PHP unit tests. Ok, so you are supposed to switch the configuration file to run the JavaScript unit tests, and you do it. Then your colleague pushes a bug fix on a PHP function and Travis…runs JavaScript tests. And since a modification in the PHP side could also break some JavaScript function (they can interact and pass data to each other after all) you are supposed to run tests both in PHP and JavaScript every time that you modify something. The process of switching the configuration file every time that we push something to Github is boring and defeats the whole concept of automation that we are trying to pursue.

For this reason it totally makes sense to run PHP and JavaScript unit tests together in the same run. And I’ve been very surprised that I couldn’t find almost any information on how to achieve that result.

In Travis the process of running the tests is called ‘build’. I’m not going to explain in detail how to configure a build, you can go here and read a detailed documentation that will teach you everything you need to know. What we need to know here is that a build is configured in the file .travis.yml (the leading dot is not a typo). In that file we put all the instructions needed to run our tests in Travis. There’s actually no code stored in the Travis servers. When a test must run Travis builds a Linux virtual machine (VM from now on) on the fly, and installs and runs the software according to the instructions in the file .travis.yml. After the tests have finished the VM is destroyed and what you get is the log of what happened.

To build our .travis.yml file we will go step by step. I’ve decided to have PHP as pre-installed language, so I will use the PHP configuration file given by Travis as a starting point, and will build everything on that. The first thing to do is to find out how to install WordPress in our VM.

Developers are lazy. We don’t like to reinvent the wheel, and I’m no exception. I was sure that someone had already solved the problem of installing WordPress in a Travis VM, and in fact I was right. In this Github repository (and probably many others if you search well) there is exactly what I was looking for, so I’ve modified that file to add my target plugin. Here is the result:

# Travis CI Configuration File

# Tell Travis CI we're using PHP
language: php

# PHP version used in first build configuration.
php:
    - "5.5"

# WordPress version used in first build configuration.
env:
    - WP_VERSION=master

# Next we define our matrix of additional build configurations to test against.
# The versions listed above will automatically create our first configuration,
# so it doesn't need to be re-defined below.

# WP_VERSION specifies the tag to use. The way these tests are configured to run
# requires at least WordPress 3.8. Specify "master" to test against SVN trunk.

# Note that Travis CI supports listing these above to automatically build a
# matrix of configurations, but we're being nice here by manually building a
# total of four configurations even though we're testing 4 versions of PHP
# along with 2 versions of WordPress (which would build 8 configs otherwise).
# This takes half as long to run while still providing adequate coverage.

matrix:
  include:
    - php: "5.3"
      env: WP_VERSION=master
    - php: "5.4"
      env: WP_VERSION=3.8.1

# Clones WordPress and configures our testing environment.
before_script:
    - export PLUGIN_SLUG=$(basename $(pwd))
    - git clone --depth=50 --branch="$WP_VERSION" git://develop.git.wordpress.org/ /tmp/wordpress
    - git clone --branch="master" https://github.com/inboundnow/landing-pages.git /tmp/wordpress/wp-content/plugins/landing-pages
    - git clone --branch="master" https://github.com/inboundnow/leads.git /tmp/wordpress/wp-content/plugins/leads
    - cd ..
    - mv "$PLUGIN_SLUG" "/tmp/wordpress/src/wp-content/plugins/$PLUGIN_SLUG"
    - cd /tmp/wordpress
    - mysql -e "CREATE DATABASE wordpress_tests;" -uroot
    - cp wp-tests-config-sample.php wp-tests-config.php
    - sed -i "s/youremptytestdbnamehere/wordpress_tests/" wp-tests-config.php
    - sed -i "s/yourusernamehere/travis/" wp-tests-config.php
    - sed -i "s/yourpasswordhere//" wp-tests-config.php
    - cd "/tmp/wordpress/src/wp-content/plugins/$PLUGIN_SLUG"
    - ls -l

script: phpunit

If you look well into the code you’ll notice that I also clone two plugins from git and add them in WordPress. These two plugins are part of the Inboundnow suite of plugins which is composed by three plugins in total. I want all the three plugin to be active while I run tests on one so I can catch any conflicts or compatibility problems among them. Plugins can be activated in the bootstrap.php file that we’ve seen in a previous article in this series.

Now, with just a few lines we will be able to add Node.js, Karma and run JavaScript unit tests.

before_install:
    - export DISPLAY=:99.0
    - sh -e /etc/init.d/xvfb start

install:
    - nvm install 0.10
    - npm install
    - npm install karma-qunit karma-firefox-launcher

The ‘before_install’ section activate the display to allow the browser run. Then in the ‘install’ section we install Node.js and Karma. To run the tests we will add this line to the ‘script’ section at the end

- node_modules/karma/bin/karma start my.conf.js --single-run

Our final working file will be like the following:

# Travis CI Configuration File

# Tell Travis CI we're using PHP
language: php

# PHP version used in first build configuration.
php:
    - "5.5"

# WordPress version used in first build configuration.
env:
    - WP_VERSION=master

# Next we define our matrix of additional build configurations to test against.
# The versions listed above will automatically create our first configuration,
# so it doesn't need to be re-defined below.

# WP_VERSION specifies the tag to use. The way these tests are configured to run
# requires at least WordPress 3.8. Specify "master" to test against SVN trunk.

# Note that Travis CI supports listing these above to automatically build a
# matrix of configurations, but we're being nice here by manually building a
# total of four configurations even though we're testing 4 versions of PHP
# along with 2 versions of WordPress (which would build 8 configs otherwise).
# This takes half as long to run while still providing adequate coverage.

matrix:
  include:
    - php: "5.3"
      env: WP_VERSION=master
    - php: "5.4"
      env: WP_VERSION=3.8.1

before_install:
    - export DISPLAY=:99.0
    - sh -e /etc/init.d/xvfb start

install:
    - nvm install 0.10
    - npm install
    - npm install karma-qunit karma-firefox-launcher

# Clones WordPress and configures our testing environment.
before_script:
    - export PLUGIN_SLUG=$(basename $(pwd))
    - git clone --depth=50 --branch="$WP_VERSION" git://develop.git.wordpress.org/ /tmp/wordpress
    - git clone --branch="master" https://github.com/inboundnow/landing-pages.git /tmp/wordpress/wp-content/plugins/landing-pages
    - git clone --branch="master" https://github.com/inboundnow/leads.git /tmp/wordpress/wp-content/plugins/leads
    - cd ..
    - mv "$PLUGIN_SLUG" "/tmp/wordpress/src/wp-content/plugins/$PLUGIN_SLUG"
    - cd /tmp/wordpress
    - mysql -e "CREATE DATABASE wordpress_tests;" -uroot
    - cp wp-tests-config-sample.php wp-tests-config.php
    - sed -i "s/youremptytestdbnamehere/wordpress_tests/" wp-tests-config.php
    - sed -i "s/yourusernamehere/travis/" wp-tests-config.php
    - sed -i "s/yourpasswordhere//" wp-tests-config.php
    - cd "/tmp/wordpress/src/wp-content/plugins/$PLUGIN_SLUG"
    - ls -l

script:
    - phpunit
    - node_modules/karma/bin/karma start my.conf.js --single-run

With this file I am able to run unit tests simultaneously in PHP and JavaScript and check the compatibility with the other two plugins that we are working on. If you develop complex WordPress plugins and want to run unit test on them, this is a fantastic configuration. However, as you know nothing is set on stone, and I’ve discovered a tool to run unit tests that in my opinion is even better and more complete than PHPunit. I’ll write one or two articles about it, so this series about unit testing is not yet finished but with the first four articles you can configure a perfectly working environment to run tests locally and remotely.

Other Articles in This Series

Unit Tests in WordPress: an Introduction

Unit Tests in WordPress: Installing PHPunit

Unit Tests in WordPress: Testing JavaScript With Karma


So, what do you think ?