ExCheck

Property-based testing for Elixir (QuickCheck style). It uses Erlang's triq library for underlying checking engine, and ExCheck's modules provide wrapper macros for ExUnit tests.

Installation

First add ExCheck and triq to your project's dependencies in mix.exs.

  defp deps do
    [
      {:excheck, "~> 0.6", only: :test}
    ]
  end

You need to have erlang-eunit installed in order to build triq.

and add the following to test_helper.exs:

ExCheck.start
# ... other helper functions
ExUnit.start

Configuration

You can also specify the amount of tests that you want to run for each property by adding the following to your config.exs:

use Mix.Config
# import "#{Mix.env}.exs"  # If you want to specify different amount for each environment

# And then in this file (or different amount in each config file):
config :excheck, :number_iterations, 200

Usage

The following is an testing example. ExCheck.SampleTest is the testing code for ExCheck.Sample.

Test

defmodule ExCheck.SampleTest do
  use ExUnit.Case, async: false
  use ExCheck
  alias ExCheck.Sample

  property :square do
    for_all x in int(), do: x * x >= 0
  end

  property :implies do
    for_all x in int() do
      #implies skip samples that does not satisfy the predicate. Also, it prints x when skip a sample
      implies x >= 0 do
        x >= 0
      end
    end
  end

  property :such_that do
    for_all {x, y} in such_that({xx, yy} in {int(), int()} when xx < yy) do
      x < y
    end
  end

  property :concat_list do
    for_all {xs, ys} in {list(int()), list(int())} do
      Enum.count(Sample.concat(xs, ys)) == Enum.count(xs) + Enum.count(ys)
    end
  end

  property :push_list do
    for_all {x, y} in {int(), list(int())} do
      result = Sample.push(x, y)
      Enum.at(result, 0) == x and Enum.count(result) == Enum.count(y) + 1
    end
  end

  # specify iteration count for running test
  @tag iterations: 30
  property :square_with_iteration_count do
    for_all x in int(), do: x * x >= 0
  end
end

Code

defmodule ExCheck.Sample do
  @moduledoc """
  Sample logic to be tested by ExCheck (refer to sample_test.exs for tests)
  """

  @doc "concatenate the list"
  def concat(x, y) do
    x ++ y
  end

  @doc "push element in the list"
  def push(x, y) do
    [x|y]
  end
end

Run

 mix test test/sample_test.exs
..................................................................................................................................................................................................................xxx....x.xx..xxx.x.xx.xx..x...x..x..xx.xx..xx.xx.x..x.x..x..x......x.xx............x.x..x..x...xxx..x..x..xx.x..xx.xx.x......x.xxx.xx..xx.x.x.x.xx.x.xx......xx..xxxx..x....xxx.xxxxx.xxxxx..xx...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 0.2 seconds (0.1s on load, 0.08s on tests)
948 tests, 0 failures

You can also add the optional flag --trace to get additional output

mix test test/sample_test.exs --trace

ExCheck.SampleTest
  * implies_property........xx.x.x..x..xx.xxx..x.x.x.x..x..x..xx....x.x.xx.xxxxxx..xx...x...xx.x...x.x..xxxxx..x...x......xxxxx.x.x..xxx.x..x.xx..xx..xxxx.x.x..xx.......x..xx.xx...x...x...xxx.xxx.x.xx.xx  * implies_property (14.7ms)
  * square_property........................................................................................................................................................................................  * square_property (4.1ms)
  * push_list_property.....................................................................................................................................................................................  * push_list_property (14.7ms)
  * square_with_iteration_count_property (0.7ms)......................
  * such_that_property.....................................................................................................................................................................................  * such_that_property (4.8ms)
  * concat_list_property...................................................................................................................................................................................  * concat_list_property (25.4ms)


Finished in 0.1 seconds (0.1s on load, 0.06s on tests)
942 tests, 0 failures

There are some more examples at test directory.

Generators

The following generators defined in :triq are imported through "use ExCheck" statement.

• list/1, tuple/1, int/0, int/1, int/2, byte/0, real/0, sized/1, elements/1, any/0, atom/0, atom/1, choose/2, oneof/1, frequency/1, bool/0, char/0, return/1, vector/2, binary/1, binary/0, non_empty/1, resize/2, non_neg_integer/0, pos_integer/0,
• unicode_char/0, unicode_string/0, unicode_string/1, unicode_binary/0, unicode_binary/1, unicode_binary/2, unicode_characters/0, unicode_characters/1,
• bind/2, bindshrink/2, suchthat/2, pick/2, shrink/2, sample/1, sampleshrink/1, seal/1, open/1, peek/1, domain/3, shrink_without_duplicates/1

Notes

• It's a trial implementation, and has limited functionalities yet.
• Files in test folder contains some more property examples.