Factory Lollipop, speeding up Kernel Tests on Drupal

11 min readJul 24, 2023

Like many open-source projects, Drupal comes with automated tests that help prevent breaking changes while promoting code quality.

I have been doing a bit of Drupal development for many years now, and I’m loving it, but there is still some massive lack inside Drupal automated testing capabilities.

Keep on reading, and I will present you that problem and how Factory Lollipop may help you solve some of them.

A brief history of testing

Drupal 8 has changed its testing framework for unit and functional tests from its own Simpletest framework to PHPUnit, following Drupal 8’s strategy of using more third-party libraries in Core.

The PHPUnit framework is very well implemented and can handle different types of tests in Drupal core: unit tests, kernel tests, and functional tests. With these three tests, you can confirm the quality and reaction of code on edge cases in different layers.

A cat biting its tail

When writing tests for Drupal Core or any contribution themes/modules, you are writing specific features isolated to your module.
Meaning you will create the bare minimum setup for your tests (Entity, Entity Type, Fields, …) and the current Drupal automated testing strategy works like a charm with this paradigm.
This breaks down though when the tests need to rely on a site’s entire configuration (eg, display modes, access, etc), rather than a small subset of modules.

Indeed, you may face a very big challenge when writing tests for a custom module coded for an individual website - a module that needs very specific content-type, fields, taxonomies, … (aka: to run on a very specific Drupal installation) you will need a whole-site build (or at least a partial build).

To achieve that goal you will certainly follow the same path as me before you:

Set up a partial functioning version (some content-type not all of them, some fields sometimes not the exact type) of the website for this specific test;
Create data Fixtures for that test.
Run the test suits.

Believe me, that’s a lot of code for one test.
It’s hard to maintain, long to read, and the cognitive complexity may be insane (too much content types, 20+ fields, …).

In short, you end with many test cases using the same configuration which creates duplicate code.

The first thing that comes to mind is to eliminate the duplicate code. As we know, the Don’t repeat yourself (DRY) principle states that:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

So I get to work and remove the duplicate code by creating some base Class (or even Traits) which configures my tests and provides useful testing utility methods to its subclasses.

Unfortunately, it was very naive solution. Please read that blog post from Petri Kainulainen presenting three reasons why you should not use inheritance in your tests.

Back to our business. We are now convinced that we should not use inheritance in our tests. Still, we don’t want to repeat our Drupal configurations or all our tests. Rights ?
It would be helpful to use the test classes to run against your own installation of Drupal (meaning the exported configurations with all your Nodes Types, Taxonomy, Fields, …) — isn’t ?

Well, some very smart people try, but it never reaches the Core.
Some people, like chx, have a different approach to this problem which I found to be fast and effective.

(…) I use KernelTestBase and a test module. The test module has a config/install directory which contains symlinks to files in the config sync directory of the live site.
This allows recreating a fraction of the live site precisely and quickly. The tests are runs very quick since it doesn’t need to fully bootstrap Drupal even once.
It also allows me to mock anything that needs mocking.

Some other created that awesome Drupal Test Traits library for testing Drupal sites that have user content (versus unpopulated sites).

Drupal Testing Traits allows writing tests for an already installed site. A site with content types and content. You can write tests for the all the pages in your site in different viewports and test user interactions aka JavaScript tests.

The chx hack & the Drupal Test Traits are both two very creative and ingenious way to perform tests on a site’s entire/partial configuration.
Both of them are also compliant with the unit testing — isolation principle.

If your test is not independent and repeatable, then the chance that you will have negative fails …

The chx trick have a severe drawback about readability and maintainability.
You will indeed need to load “en masse” every potential configuration used in your tests or split your test modules into many sub-modules.
This can lead to slow tests suits, if many unnecessary configurations are being loaded. This may also have a significant impact on performance because each test will reimport all the declared configurations.

Then we have Drupal Test Traits, which does not solve the mass loading configurations, but preserve performance by providing a database-retention capability between each test suite, avoiding the whole database rebuild.

Both of those solutions are great, but you may still generate data incoherency, as every test will generate data without asserting those Nodes, Taxonomies are valid (mandatory fields filled, …).

Here came Factory Lollipop !

Factories, Factories everywhere

Test data factories solve that data inconsistency problem by making data and data loading more dynamic and programmable.

A factory is written in code, and while it can simply mimic a fixture by supplying default values for an object, factory libraries offer many other useful features.

Here’s a simple example factory:

$factoryLollipop->define('node', 'node_article', [
  'field_slug' => FixtureFactory::sequence("Slug %d"),
  'title' => 'Foo',
]);

An instance of this factory could be created on the fly like: node = \$this->factoryLollipop->create('node_article').

Following the builder pattern, a factory generates data for all the attributes in its definition, constructs any necessary associated objects, and allows a test case to override these values.

Here’s a slightly more complex example:

$factoryLollipop->define('node type', 'node_type_article', [
  'type' => 'article',
]);

$factoryLollipop->define('node', 'node_article', [
  'field_slug' => FixtureFactory::sequence("Slug %d"),
  'status' => 1,
  'type' => $factoryLollipop->association('node_type_article'),
  'field_bar' => function() { return now(); },
]);

$node = $factoryLollipop->create('node_article', [
  'title' => 'Tortor posuere ornare quisque mi vehicula nostra',
]);

# $node->getTitle() => 'Tortor posuere ornare quisque mi vehicula nostra'
# $node->get('status')->value => 1
# $node->getBundle() => article

Some typical features in factory libraries are:

Integration with Database; FixtureFactory::create(...) will typically build and save the object to a database,
Factory Association: Allowing factories to be associated to each other; e.g. FixtureFactory::association(...),
Lazy attributes; e.g. 'field_bar' => function() { return now(); },

Factory Lollipop a Factory for Drupal

For years, the most common way to provide test data for automated tests has been fixtures — hard-coded values, usually stored in text/yaml files.

But fixtures and the frameworks that rely on them have several drawbacks. They are not easily modifiable, which tends to lead to duplicate fixtures with unwieldy names like person-with-birthdate.yaml.

Defining Factories

Creating factory definitions is fairly straightforward, Factory Lollipop use the Chain of Responsibility design pattern to loop through declared Factory and get the first most appropriate Factory to define your Factory.

Let’s suppose we want to create a Factory for our Article Node with the following structure:

The following example will illustrate the use of default values, fieldable entity, and associations.

We’ll create a new Factory Definition Resolver by implementing the \Drupal\factory_lollipop\FactoryInterface.

Here is the basic structure:

<?php

namespace Drupal\my_custom_module\Factories;

use Drupal\factory_lollipop\FactoryInterface;
use Drupal\factory_lollipop\FixtureFactory;

/**
 * Creates Drupal Article Nodes Factory for use in tests.
 */
class NodeArticleFactory implements FactoryInterface {

  /**
   * {@inheritdoc}
   */
  public function getName():string {
    return 'my_project.definitions.node_article';
  }

  /**
   * {@inheritdoc}
   */
  public function resolve(FixtureFactory \$lollipop): void {
    // ...
  }
}

In our example code, we named the Definition Factory Class my_project_node_article. This name must be unique across all your Factories, it will be used as Identifier of this resolver for Lazy-Loading, see the next part "Loading Factories". Therefore, we highly suggest prefixing the Definition Factory Class name with the module name, the project name will avoid confusion with the Factory Name itself (see below).

Now we can implement the resolve() method to define our Article Node Factory:

/**
 * {@inheritdoc}
 */
public function resolve(FixtureFactory $lollipop): void {
  // Define the node type "Article".
  $lollipop->define('node type', 'node_type_article', [
    'type' => 'article',
  ]);

  // Add the "Scheduled at" field without default value.
  $lollipop->define('entity field', 'node_article_field_scheduled_at', [
    'entity_type' => 'node',
    'name' => 'field_scheduled_at',
    'bundle' => $lollipop->association('node_type_article'),
    'type' => 'datetime',
  ]);
  $lollipop->create('node_article_field_scheduled_at');

  // Add the "Is Paid" field with a default value of False.
  $lollipop->define('entity field', 'node_article_field_is_paid', [
    'entity_type' => 'node',
    'name' => 'field_is_paid',
    'bundle' => $lollipop->association('node_type_article'),
    'type' => 'boolean',
  ]);
  $lollipop->create('node_article_field_is_paid');

  // Define the Node Factory for Article".
  $lollipop->define('node', 'node_article', [
    'type' => $lollipop->association('node_type_article'),
    // Setup the default Status to Published.
    'status' => 1,
    // Setup the "Is Paid" field default value to FALSE.
    'field_is_paid' => FALSE,
  ]);
}

In our example code, we defined 4 Factories:

node_type_article: The Node Type, necessary for Drupal;
node_article_field_is_paid: The "Is paid" field attached to the Node Type Article;
node_article_field_scheduled_at: The "Scheduled at" field attached to the Node Type Article;
node_article: The Node itself, the one we will use for Data creation on the below "Using Factories".

Those 4 Factory Definition must be unique across all your loaded Definition Factories. As they will be used as Blueprint to generate Strcuture/Data.

With our Factory class complete, we’re ready to add this service to our custom module’s service file.

services:
  my_custom_module.factories.node.article:
    class: Drupal\my_custom_module\Factories\NodeArticleFactory
    tags:
      - { name: factory_lollipop.factory_resolver, priority: 1 }

Loading Factories

This can be accomplished by adding any necessary Definition Factory Class Name to the loader.

/**
 * Example of Factory Lollipop usage for a KernelTest.
 */
class MyKernelTest extends LollipopKernelTestBase {

  /**
   * {@inheritdoc}
   */
  public static $modules = [
    'node',
  ];

  public function testCreateNode(): void {
    $this->factoryLollipop->loadDefinitions(['my_project.definitions.node_article']);
  }
}

Using Factories

When a definition is loaded, then you will be able to use it to build any object from the definition pool.

When using a factory, it is possible to override any of the options provided by the definition. Consequently, you always have control over the data at the time you create it. Using factories is as simple as:

public function testCreateNode(): void {
  $this->factoryLollipop->loadDefinitions(['my_project.definitions.node_article']);
  $node = $this->factoryLollipop->create('node_article', ['title' => 'Magna cursus tempor']);
}

Drupal’s property-access component is used to populate the properties, therefore I can override the title or any available field on my Node Article entity.

Factory Lollipop a companion to Drupal Test Traits

Factory Lollipop gives you a way to define this configuration and default-data into a single place.

Factory Lollipop may solve your configuration declaration and data consistency — by the code — with a clear, extendable and strict architecture. Still, you may want to load an entire site’s configuration.

Even if Factory Lollipop is intended to be used as a standalone library to mimic a site’s entire/partial configuration, you can use it with the *chx tricks or Drupal Test Traits to only generate Data Factories instead of dealing with Data and Configuration.

// Define the Node Factory for Article".
$lollipop->define('node', 'node_article', [
  'type' => $lollipop->association('node_type_article'),

  // Setup the default Status to Published.
  'status' => 1,

   // Setup the "Is Paid" field default value to FALSE.
   'field_is_paid' => FALSE,
]);

$lollipop->create('node_article');

The last word

I’ve talked here about speeding up unit tests writing and performance by relying on Factory Lollipop, but there’s another surefire way to speed up your test suite.

Don’t use it.

Most Unit Tests Don’t Need Persisted Data

There are plenty of times when data needs to exist in the database to accurately test an application; most acceptance tests will require some amount of data persisted (either via Factory or by creating data driven through UI interactions). When unit-testing most methods, however, Factory (and even persisting data to the database) is unnecessary.

When testing an object and collaborators, consider doubles like fakes or stubs.

My general advice, though, is to avoid Kernel test as much as is reasonably possible. Not because it’s bad or unreliable software, but because its inherent persistence mechanism requires a whole working Drupal environment.

Sources

The official documentation.

This article has been first write for the Blog of Antistatique — Web Agency in Lausanne, Switzerland.
A place where I work as Full Stack Web Developer. Feel free to read it here or check it out there: https://antistatique.net/en/we/blog

Retrieve all the code on my Gist: https://gist.github.com/WengerK/b95aedd906ccf91c7c7f509dff90c399

For the most curious of you, here are some sources of additional information that inspired the creation of this article.

Noah Harrison (04 August 2014). Factories, not Fixtures.
Retrieved from artandlogic.com/…/factories-not-fixtures

Stackoverflow (03 March 2011). Factory Girl — what’s the purpose?
Retrieved from stackoverflow.com/q/…/factory-girl-(…)-purpose

Maria Cristina Simoes (21 February 2019). My Introduction To Factory Bot.
Retrieved from medium.com/…/my-introduction-to-factory-bot

Matt Sumner (26 April 2016). Factories Should be the Bare Minimum .
Retrieved from thoughtbot.com/…/factories-should-be-the-bare-minimum

Drupal Core Issue Queue (29 Apr 2020). Allow BTB to test an existing, already installed Drupal site instead of installing from scratch.
Retrieved from drupal.org/project/drupal/issues/2793445

Drupal API Doc (13 January 2021). PHPUnit in Drupal .
Retrieved from drupal.org/docs/automated-testing/phpunit-in-drupal

Sonnym (13 January 2021). Drupal module Factory Drone.
Retrieved from drupal.org/project/factorydrone

Damien Mckenna (14 March 2018). Writing Simple (PHPUnit) Tests for Your D8 module.
Retrieved from mediacurrent.com/…/writing-simple-phpunit-tests-your-d8-module

Hendra Uzia (20 April 2018). Working Effectively with Data Factories Using FactoryBot.
Retrieved from semaphoreci.com/…/working-effectively-with-data-factories-using-factorygirl

Philip Brown (27 May 2013). Laravel 4 Fixture Replacement with Factory Muffin.
Retrieved from culttt.com/…/laravel-4-fixture-replacement-with-factorymuff

Josh Clayton (14 August 2014). Speed Up Tests by Selectively Avoiding Factory Girl.
Retrieved from thoughtbot.com/…/speed-up-tests-by-selectively-avoiding-factory-girl

Sean Hamlin (24 March 2016). Writing PHPunit tests for your custom modules in Drupal 8.
Retrieved from pixelite.co.nz/…/writing-phpunit-tests-for-your-custom-modules-in-drupal-8

James Robertson (24 April 2019). A Full Guide to PHPUnit in Drupal 8.
Retrieved from atlanticbt.com/…/a-full-guide-to-phpunit-in-drupal-8/

Petri Kainulainen (20 April 2014). Three Reasons Why We Should Not Use Inheritance In Our Tests.
Retrieved from petrikainulainen.net/…/3-reasons-(..)-not-use-inheritance-(…)-tests

Moshe Weitzman (19 February 2021). Library Drupal Test Traits.
Retrieved from gitlab.com/weitzman/drupal-test-traits

Jibran Ijaz (04 December 2018). Introducing Drupal Testing Traits: Drupal extension for testing existing sites.
Retrieved from www.previousnext.com.au/.../introducing-drupal-testing-traits-(...)-existing-sites

O’Reilly online learning (19 February 2021). Test Isolation and Interaction.
Retrieved from oreilly.com/…/phpunit-essentials/…/ch06

Bo Jeanes (26 February 2012). Factories breed complexity.
Retrieved from bjeanes.com/…/factories-breed-complexity