Custom ExUnit Assertion

Writing an assert_invalid macro for ExUnit assertions. Implementation at the bottom.

I was working a lot with Ecto Changesets - using them for validation in my HTTP application. The application code (the code in my lib/ folder) was very nice, but I was not happy with the verbosity of the tests. Each test looked something like this:

An example without my custom assertion

This was okay for a single test about a single field, but quickly lead to a tonne of copy-and-paste across multiple tests that were asserting on multiple fields.

The solution was to create a custom assertion that could combine and abstract the multiple assertions away.

Here is an example test suite which uses assert_invalid (my creation) and shows off the error messages when the assertions fail — I think having good ExUnit error messages is the most important part!

Example Ecto.Changeset tests using assert_invalid that show off all of the various assertion error cases

Here is the resulting output:

Example output displaying the error messages you would see when your assertion fails

Most importantly, here is the code:

The implementation of assert_invalid

It is necessary for the assertion to be a macro for two reasons:

1. We need access to the flunk/1 and assert/1 functions that are provided by ExUnit when we run the tests
2. We would like to expose the error_message value so that users can write expressions against it

The most interesting part of the code is the call to Kernel.var!/2 on line 15. By default, variables declared inside of a macro are hygienic and will not impact variables defined where the macro is expanded. Using Kernel.var!/2 is one way to explicitly disable that hygiene and have the error_message value be available in our tests.

Additionally, using the IO.ANSI module to color my stdout really helped make the assertion errors more readable:

Example IO.ANSI coloring

I am not a metaprogramming expert, but hopefully this example code will be useful for other folks getting started with writing their own assertions.