Writing regression tests is (not) hard

Setting the stage

Imagine for a moment you work on an email analytics, automation, and enhancement platform, and a customer writes in about a bug they’re experiencing. We know the bug is in a long, complex function:

async function complexFunction(value) {
  if (!value) return;
  if (!value.name) throw new Error('Name is required for `complexFunction`.');
  const count = await db.values.count({ name: value.name });
  let newValue = value;
  if (count <= value.length) newValue = foo();
  return newValue;
} 

Fixing this bug involved a small change to fix an off-by-one error:

- if (count <= value.length) newValue = foo();
+ if (count < value.length) newValue = foo();

Great! We fixed the bug, and you can move on to the next task after doing your due diligence and updating the unit tests for this function to accommodate the change in logic. However, it turns out that the function doesn’t have any unit tests — and worse, it’s a long function that relies on external libraries and a database connection. You know you should write a test… but it looks like a lot of work, and what are the chances this will happen again? (Spoiler alert: it will happen again)

This situation above comes up a lot, and writing tests can seem time-consuming and intimidating, especially in large or complex codebases. At Mixmax, we care about testing! It contributes to reliable, maintainable code and makes our lives as developers easier in the long run. Writing those first tests in a particularly difficult piece of code makes you a hero next time something breaks and the test catches it in CI rather than hearing it from customer reports. So, in the interest of making writing tests easier, here are some strategies we use at Mixmax!

Create a test context

Existing code often relies on lots of external dependencies, data, functions, or even entire services. Whichever of those your code relies on, it’s helpful to create helper functions to setup and teardown external state so you and fellow developers aren’t stuck writing the same or similar boilerplate for each test. This external setup is often called the testing context since it’s the state all tests expect. This is an example of what a test context helper function that sets up a database connection looks like:

const mongoist = require('mongoist');
const { MongoMemoryServer } = require('mongodb-memory-server');
async function start() {
  const mongod = new MongoMemoryServer({
    binary: { version: '3.4.6' }
  });
  const uri = await mongod.getConnectionString();
  // The mongo client.
  const db = MongoClient(uri);
  return { db, mongod };
}
function stop(mongod) {
  return mongod.stop();
}
module.exports = { start, stop };

Here, start sets up a mongo server unique to the test context so other tests don’t modify the database during testing. Usually there’s one context per test file. Now, you can use the helper when setting up your new unit test:

const { start, stop } = require('../../helpers/context');
describe('testFunction', () => {
  let t;
  beforeAll(async () => {
    t = await start();
  });
  afterAll(() => stop(t.mongod));
  // it('TODO: write a test', () => {});
});

Now it’s time to write an actual test. But where to begin?

Test early returns first

It’s helpful to write functions that return early rather than having nested conditionals for clarity and readability. However, the first test that usually comes to mind is one that runs the entire function, ending with the last line, but these are also likely the most difficult tests to write since they exercise the most code. They also require understanding all the prerequisites and side effects of the function – no small task for large, existing code bases. Instead, it’s often easier to test places that return early or throw exceptions first, leaving a complete run of the function for the end. In our complexFunction it’s far easier to test the first two conditionals than the final one:

async function complexFunction(value) {
  if (!value) return;
  if (!value.name) throw new Error('Name is required for `complexFunction`.');
  const count = await db.values.count({ name: value.name });
  let newValue = value;
  if (count < value.length) newValue = foo();
  return newValue;
} 

Adding two tests for an undefined value and catching an exception requires no extra setup and already covers one third of the statements in complexFunction:

it('should return null when passed an empty value', async () => {
  expect(await complexFunction()).toBe(undefined);
});
it('should throw when name is undefined', async () => {
  await expect(complexFunction({})).rejects.toThrow('Name is required for `complexFunction');
});

Now that the easy tests are taken care of, it’s time to use a few other techniques for the rest of the function.

Taking advantage of mocking

In testing, mocking usually refers to fake implementation or data that’s set up specifically for one or more test cases. While there are many different libraries for, and types of mocks, we’ll focus primarily on mocking data and functions. To finish testing complexFunction we’ll want to mock the data in db.values as well as what foo() returns.

To mock data, add expected values before running your test and ensure that the values are unique to your test itself to prevent conflicts with existing or future tests.

it('should return the original value when count is >= length', async () => {
  const name = Math.random();
  await t.db.values.insert([{ name }, { name }]);
  const value = { name, length: 3 };
  const newValue = await complexFunction(value);
  expect(value.name).toEqual(newValue.name);
  expect(value.length).toEqual(newValue.length);
});

Taking this incremental approach, tests cover all but one statement in the function and the remaining bit of code exercises the updated logic. Testing the final line, ideally involves understanding foo and ensuring it runs correctly, returning expected values. However, to ease writing these tests, mocking the function (using jest’s builtin mocks or another mocking library) should suffice. We can add mocking to a hypothetical function foo by making a few modifications to the context start function:

// Import the required module
const module = require('./module');
// Replace `foo`'s implementation with a mock function
module.foo = jest.fn();
// Add foo to the returned values
return { db, mongod, foo: module.foo };

Interested in writing testable code? Join us at Mixmax, we’re hiring!