WordPress integration tests with Pest and Buddy

In one of his articles on WordPress testing, Maciek covered running unit tests for WordPress in Buddy. In this article, I’d like to focus on running integration tests for a WordPress application in your CI/CD pipeline.

Difference between unit and integration tests

There's a peculiar tendency in the WordPress community to run unit and integration test interchangeably, as if they were the same thing. But the truth is that unit and integration tests are widely different – not only in terms of the name.

Unit tests always test a single unit of code (function call, method in a class, single module) in isolation from the rest of the code. You can think of unit tests as a sort of validation tests: your function does a thing, and you need to make sure that thing is performed correctly when you throw all the edge cases at it (testing the happy and the unhappy path as well).

On the other hand, integration tests, as their name suggests, test how your code behaves when integrated into an application. For instance, testing that your plugin really registers a custom post type in WordPress. For integration tests you need to set up a specific environment - usually a test app and a test database, which more often than not makes them a bit slower than unit tests.

Why Pest?

Honestly? No real reason other than the fact I find the Pest testing syntax prettier than the native PHPUnit. In the end, PHPUnit is the thing that runs in the background, while Pest is just a wrapper around it.

I find Pest a bit easier to set up and easier to write, however. It has an excellent expressive syntax that allows you to write tests a bit more fluently, like writing regular sentences.

Setting the stage

Let’s say we want to test a plugin that has a custom post type (CPT) called Books, and a custom REST API endpoint for the CPT.

The plugin is super simple and looks like this (books.php ):

php
<?php
/**
 * Plugin Name: Books plugin
 * Description: Plugin that will add a custom post type called books
 * Version: 1.0.0
 * License: MIT
 */
function book_test_plugin_register_books_cpt()
{
    $args = [
        'label' => esc_html__('Books', 'test-plugin'),
        'public' => true,
        'publicly_queryable' => true,
        'show_ui' => true,
        'show_in_menu' => true,
        'query_var' => true,
        'rewrite' => ['slug' => 'book'],
        'capability_type' => 'post',
        'has_archive' => true,
        'hierarchical' => false,
        'menu_position' => null,
        'menu_icon' => 'dashicons-book',
        'show_in_rest' => true,
        'supports' => ['title', 'editor', 'author', 'thumbnail', 'excerpt'],
    ];
    register_post_type('book', $args);
}
add_action('init', 'book_test_plugin_register_books_cpt');

If you activate it locally, you’ll see the Books CPT in your WordPress admin. Granted, this is a really simple example, but it’s just to prove a point (we can expand on it a bit later).

Setting the tests

In order to set up the tests, we shall use the Pest package I’ve created using Composer.

bash
composer require dingo-d/wp-pest-integration-test-setup --dev$

The next step is setting up the plugin tests:

bash
vendor/bin/wp-pest setup plugin --plugin-slug=books$

Under the hood, the package sets up a WordPress instance with an in-memory database, and also provides several test examples.

Before writing the tests, let’s make sure the Composer’s autoloader will do its magic and instantiate all the classes we may need for testing (so we don’t have to do anything manually). In the composer.json file, add the autoload-dev key:

json
{
  "require-dev": {
    "dingo-d/wp-pest-integration-test-setup": "^1.4"
  },
  "config": {
    "allow-plugins": {
      "composer/installers": true,
      "pestphp/pest-plugin": true
    }
  },
  "autoload-dev": {
    "psr-4": {
      "Tests\\\\": "tests/"
    }
  }
}

Let’s add a new test in the tests/Integration/BookCptTest.php file:

php
<?php
namespace Tests\\Integration;
beforeEach(function () {
 parent::setUp();
});
afterEach(function () {
 parent::tearDown();
});
test('Books custom post type is registered', function () {
 // We can use assertions from PHP_Unit.
 $this->assertNotFalse(has_action('init', 'book_test_plugin_register_books_cpt'));
 $registeredPostTypes = \\get_post_types();
 // Or we can use expectations API from Pest.
 expect($registeredPostTypes)
  ->toBeArray()
  ->toHaveKey('book');
});

Running the vendor/bin/pest --group=integration should show you the following results:

shell
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
   PASS  Tests\\Integration\\BookCptTest
  ✓ Books custom post type is registered
   PASS  Tests\\Integration\\ExampleTest
  ✓ REST API endpoints work
  Tests:  2 passed
  Time:   0.19s

We already have a setup for REST API, so let’s rename it to RestApiTest.php and add another test for the REST API:

php
test('Books REST API endpoint is registered', function () {
 $routes = $this->server->get_routes();
 expect($routes)
  ->toBeArray()
  ->toHaveKey('/wp/v2/book');
});

Running the tests again will show us this:

shell
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
   PASS  Tests\\Integration\\BookCptTest
  ✓ Books custom post type is registered
   PASS  Tests\\Integration\\RestApiTest
  ✓ REST API endpoints work
  ✓ Books REST API endpoint is registered
  Tests:  3 passed
  Time:   0.21s

Now, let's spice things up a bit before setting our CI pipeline, which is, after all, the goal of this article.

We shall try to fake some book posts and see if we can get some information from the WordPress app about them. For that, we’ll need a factory. The WordPress test suite (to which we have access to) has a set of factories that can be used to quickly create certain WordPress objects, such as posts, pages, users, or terms and taxonomies, among others.

Because we created a custom post type that is not native to WordPress, we need to create our own BookFactory.

In the tests/Factories folder, create a BooksCptFactory.php:

php
<?php
namespace Tests\\Factories;
use WP_UnitTest_Factory_For_Thing;
use WP_UnitTest_Generator_Sequence;
class BooksCptFactory extends WP_UnitTest_Factory_For_Thing {
    function __construct( $factory = null ) {
        parent::__construct( $factory );
        $this->default_generation_definitions = array(
            'post_status' => 'publish',
            'post_title' => new WP_UnitTest_Generator_Sequence( 'Book title %s' ),
            'post_content' => new WP_UnitTest_Generator_Sequence( 'Book content %s' ),
            'post_excerpt' => new WP_UnitTest_Generator_Sequence( 'Book excerpt %s' ),
            'post_type' => 'book'
        );
    }
    function create_object( $args ) {
        return wp_insert_post( $args );
    }
    function update_object( $post_id, $fields ) {
        $fields['ID'] = $post_id;
        return wp_update_post( $fields );
    }
    function get_object_by_id( $post_id ) {
        return get_post( $post_id );
    }
}

Now we should use this factory in our test. Back in the BookCptTest.php file, add the following test:

php
<?php
namespace Tests\\Integration;
use Tests\\Factories\\BooksCptFactory;
// beforeEach, afterEach and one test goes here...
test('Creating book posts work', function(){
    $booksFactory = new BooksCptFactory();
    $books = $booksFactory->create_many(5);
    expect($books)
        ->toBeArray()
        ->toHaveCount(5);
    $title = get_the_title($books[0]);
    expect($title)
        ->toContain('Book title');
});

Running the integration test suite should give us this:

shell
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
   PASS  Tests\\Integration\\BookCptTest
  ✓ Books custom post type is registered
  ✓ Creating books work
   PASS  Tests\\Integration\\RestApiTest
  ✓ REST API endpoints work
  ✓ Books REST API endpoint is registered
  Tests:  4 passed
  Time:   0.27s

This is a super simple example of writing integration tests. A more interesting example would involve a custom REST API endpoint with complicated logic for fetching some third party API data that can be mocked and verified that the intended response is outputted in your endpoint. However, we’d need another article for that. 😀

Let’s see how we can hook our tests to our pipeline with Buddy

Pipeline configuration

The idea of running tests in a CI/CD pipeline is to have a quick way to check the codebase for accidental regression bugs. You develop a feature, cover it with tests, and push it to the pipeline to see if the tests indicate any failures.

Additionally, CI pipelines allow you to expand the environments for the tests with different PHP versions, different WP versions, and so on. Setting those up on your local computer can be a chore – and this is where Buddy enters the scene.

Adding repository and pipeline

First, we need to set up our repository with Buddy and add a new pipeline:

Image loading...

Action 1: Composer install

After creating the pipeline, select the PHP action. This will launch an isolated container with preinstalled PHP where we can run our tests. The first thing to install are the packages. This can be done with the default composer install command:

Image loading...

Action 2: Tests setup

With the packages installed, we can add another PHP action in which we'll set up the tests:

Image loading...

Add the following script in the bash window:

bash
vendor/bin/wp-pest setup plugin --plugin-slug=books --skip-delete$

Image loading...

In Buddy, containers are not stateless. This means that once you set up the pipeline, it stays that way. Because of that, we need to make sure that the script is allowed to fail, because on every subsequent run, the setup script will exit and say that the tests and WordPress are already installed/set up.

We can do that by forcing Buddy to suppress the error and continue to the next action, a setting you can find in the pipeline options:

Image loading...

Action 3: Running the tests

The third and last PHP action will run the tests:

bash
vendor/bin/pest --group=integration$

Image loading...

This is how the whole pipeline looks like:

Image loading...

Running the pipeline

The pipeline is all set and ready to run. Since the wp-pest package doesn’t depend on concrete but in-memory database, you don’t need to do anything else. Depending on the trigger settings, click the Run button or make a push to the assigned branch. The results should look like this:

Image loading...

Adding more tests

Let’s add another test to confirm our tests will run correctly in the pipeline. Since we’ve added the factory for the books, they will be set during the test run.

We shall see if our REST API endpoint retrieves correct information and that the keys we expect from our API exist.

In RestApiTest.php, add the following test:

php
test('Books REST API endpoint returns correct data', function () {
    $response = $this->server->dispatch(
        new \\WP_REST_Request('GET', '/wp/v2/book')
    );
    $data = $response->get_data();
    expect($data)
        ->toBeArray()
        ->toHaveCount(5)
        ->and($data[0])
        ->toHaveKey('title')
        ->and($data[0]['title']['rendered'])
        ->toContain('Book title');
});

Push the changes to the repository and see what happens on the pipeline run:

Image loading...

As predicted, despite the warning on the setup part, the tests are passing successfully.

Congratulations! 🎉 You have created a fully functional CI/CD pipeline that run integration tests on your WordPress instance.

Summary

As awesome as it sounds and as time-saving as it is, automation is the last step in application testing. As tedious as it sounds and as time-consuming as it, however, is you need to write the tests first. Still, the reward is well worth the price: high-performing software, free of bugs, and bound to work on every platform it's destined for.

So get out there and start writing, and when you're ready, invite Buddy to do the rest – you won't believe what a perfect couple you will form.