Skip to main content

Testing

To develop and test your integrations, the automator has several capabilities:

  • Profiles are organized in environments, usually "sandbox" and "production" (and "development" can also be made available if desired). You develop your integrations in the sandbox environment, where all accounts are connected to test systems. Once you're ready to deploy, use the transfer assistant to sync the data to the production environment.
  • Packages in the sandbox environment can be triggered by events in exactly the same way as they would be triggered in production, but because all accounts are connected to test systems, there is no risk of accidentally changing live data.
  • You can manually start a test run of a package via the Start test run button in the log panel of the package editor. Next to this button, there is a configuration button that allows you to configure the parameters of the test run and the contents of the (fake) trigger event.
  • You can create test packages for automated testing. Tests are defined with test, make assertions with the assert_* functions, and can replace external calls with mocks so that a test never touches an external system. This is explained in detail below.

Defining tests​

Tests are collected in packages with the package type "Test package".

A test is defined with the test function, taking a description and a function containing the test code.

test('adds up the line totals', () => {
  let total = sumLineTotals([{ amount: 10 }, { amount: 5 }]);
  assert_equal(total, 15, 'Line totals were not summed correctly');
});

To execute a test package, use the "Start test run" button in the log panel of the package editor. It is also possible to link a trigger event to a test package, so that test runs can be triggered automatically via an external event or on schedule.

When every assertion in a test passes, the test is marked passed and a SUCCESS message is written to the log. If any assertion fails, or the test code throws, the test is marked failed and a FAILED message is written. In both cases, the log message contains a link to the line of code of the corresponding test. In case of failure, the log message also contains a stack trace that can help you pinpoint the exact location of the failure.

Parameterized tests​

Pass an array as the optional second argument and the test runs once per item, receiving each item in turn. This is handy for table-driven tests:

let cases = [
  { input: 1, expected: 1 },
  { input: 4, expected: 16 },
  { input: 5, expected: 25 },
];

test('squares its input', cases, (c) => {
  assert_equal(square(c.input), c.expected, 'Wrong square');
});

Assertions​

Three assertion functions are available:

The failedText is written to the log when the assertion fails, so make it descriptive.

Grouping tests​

Tests can be nested to form groups. The group name is prefixed to the test name in the log, so keep group names short.

test('order processing', () => {
  test('valid order', () => {
    assert_ok(isValid(validOrder), 'Order should be valid');
  });

  test('missing customer', () => {
    assert_ok(!isValid(orderWithoutCustomer), 'Order should be invalid');
  });
});

When a test fails, its group fails immediately: execution stops at the first failed test, and the group result is logged.

Mocking with setMock​

setMock replaces a function with a fixed value or an alternative implementation. This is the key to testing packages that would otherwise create, update, or call external systems: mock those functions and the test runs without side effects.

// Return a fixed value for every call
setMock('update', { ok: true });

// Or a replacement function that receives the same arguments
setMock('update', (module, id, data) => {
  log('would update', id, 'with', data);
  return data;
});

// Remove the mock and restore the original
setMock('update');

Mocking works on your own functions and on most AutomatorScript functions — including the ones that change data, such as create, update, the rest_* and http_* functions, and the Xurrent functions — which is exactly what you want to neutralize in a test.

Ignoring with setIgnore​

setIgnore is a lighter alternative: an ignored function is not executed at all, and every call returns the string "ignored". Calling setIgnore again on the same function restores it. Use this when you simply want a call to do nothing and don't care about its return value.

Mocks and ignores reset automatically​

Any mock or ignore set inside a test function is automatically reset after the test finishes — and, for a parameterized test, between each item in the parameter array. You don't need to undo your mocks manually; each test starts from a clean slate.

Code coverage​

When you run a test package, the automator reports code coverage: the percentage of the package's (and its libraries') lines that were actually executed by the tests. The coverage percentage is written to the log, for example:

Code coverage: 87.5%

along with the lines that were never reached, grouped by package:

Code without coverage: do_on_request_update - lines: 12, 18-20

Use these reports to find logic your tests don't yet exercise.

Test organization​

A good starting point is to create one test package for each library, which contains one or more tests for each function in the library. For each library function, create a top level test() function, and then create nested test() functions for each test case. This provides a natural organization and help to make sure that your integration code is well covered.

Since test packages can cover libraries but not standard packages, it is recommended that your standard packages contain as little logic as possible, instead delegating the work to one or more libraries.

Test libraries​

As your test suite grows, you may find that you are often repeating the same complicated code to set up mocks. If that's the case, you can create a package of type "test library" to encapsulate that code. Using a separate package type helps to distinguish this kind of setup code from the actual library code under test.