Implement Property-Based Testing

Overview

Traditional example-based tests check one input at a time (assert reverse("abc") == "cba"). Property-based tests describe universal properties (reverse(reverse(s)) == s) and the framework generates hundreds of random inputs looking for violations. This approach surfaces edge cases — empty strings, Unicode combining characters, integer overflow, nulls — that hand-picked examples rarely cover.

When to Use

Property-based testing pays off when correctness is defined by invariants rather than by specific outputs. For the theory behind it and a framework comparison, see the complete property-based testing guide.

• Pure functions with clear mathematical properties (sorting, parsing, encoding, serialization)
• Input validation and sanitization routines that must handle arbitrary data
• State machine behavior where transitions should preserve invariants
• Algorithms that must be reversible (compress/decompress, encrypt/decrypt, encode/decode)
• You’ve had production bugs caused by edge-case inputs (empty collections, MAX_INT, special characters)

When NOT to Use

• The code is heavily I/O-dependent or side-effectful — properties are hard to state and verify
• Tests need to assert exact behavior for specific business scenarios — use example-based tests
• The property can’t be stated formally (“looks right to a human”)
• Execution time matters — property tests run hundreds of iterations and can be slow

How It Works

Every property-based run is a loop with three stages: generate, check, shrink.

The generator draws inputs from a strategy (st.text() in Hypothesis, fc.string() in fast-check, Arbitrary<String> in jqwik). When the property fails, the shrinker simplifies the failing input step by step until no smaller case still fails — a 100-element array collapses to the three elements that actually matter. The framework also prints the seed of the random run, so any failure can be replayed deterministically.

Step-by-Step Implementation

Python (Hypothesis)

import json
from hypothesis import given, strategies as st
from hypothesis.stateful import (
    RuleBasedStateMachine, rule, precondition, invariant,
)

# Basic property: reversing twice returns the original
@given(st.text())
def test_reverse_is_involution(s):
    assert reverse(reverse(s)) == s

# Constrained strategy
@given(st.integers(min_value=0, max_value=1000))
def test_square_is_non_negative(n):
    assert n * n >= 0

# Composite strategy for domain objects
@st.composite
def users(draw):
    return {
        "name": draw(st.text(min_size=1, max_size=100)),
        "age": draw(st.integers(min_value=0, max_value=150)),
        "email": draw(st.emails()),
    }

@given(users())
def test_user_serialization_roundtrip(user):
    assert json.loads(json.dumps(user)) == user

# Stateful testing: exercise a stack and check an invariant after each command
class StackMachine(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.items = []

    @rule(x=st.integers())
    def push(self, x):
        self.items.append(x)

    @precondition(lambda self: len(self.items) > 0)
    @rule()
    def pop(self):
        self.items.pop()

    @invariant()
    def size_is_never_negative(self):
        assert len(self.items) >= 0

TestStack = StackMachine.TestCase

JavaScript (fast-check)

import fc from 'fast-check';

// Property: reverse(reverse(s)) === s
fc.assert(
  fc.property(fc.string(), (s) => {
    return reverse(reverse(s)) === s;
  }),
  { numRuns: 1000 }
);

// Property: sorting keeps the length and produces a monotonic list
fc.assert(
  fc.property(fc.array(fc.integer()), (arr) => {
    const sorted = arr.slice().sort((a, b) => a - b);
    for (let i = 1; i < sorted.length; i++) {
      if (sorted[i - 1] > sorted[i]) return false;
    }
    return sorted.length === arr.length;
  })
);

// Model-based testing: commands implement check() and run()
class ListModel {
  constructor() { this.items = []; }
  push(x) { this.items.push(x); }
  pop() { return this.items.pop(); }
  get length() { return this.items.length; }
}

class PushCommand {
  constructor(value) { this.value = value; }
  check() { return true; }
  run(model, real) {
    model.push(this.value);
    real.push(this.value);
  }
}

class PopCommand {
  check(model) { return model.length > 0; }
  run(model, real) {
    if (real.pop() !== model.pop()) {
      throw new Error('pop mismatch between model and implementation');
    }
  }
}

fc.assert(
  fc.property(
    fc.commands([
      fc.integer().map((n) => new PushCommand(n)),
      fc.constant(new PopCommand()),
    ]),
    (cmds) => {
      fc.modelRun(() => ({ model: new ListModel(), real: new MyList() }), cmds);
    }
  )
);

// Shrinking: fast-check reduces a failing input to the smallest case
fc.assert(
  fc.property(fc.array(fc.integer()), (arr) => {
    return sum(arr) >= 0; // fails when generated negatives sum below zero
  })
);

Java (jqwik)

import net.jqwik.api.*;
import net.jqwik.api.stateful.*;
import java.util.List;
import java.util.Stack;

class StringProperties {

    @Property
    boolean reverseOfReverseIsOriginal(@ForAll String s) {
        return reverse(reverse(s)).equals(s);
    }

    @Property
    boolean concatenationLengthIsSum(
        @ForAll @StringLength(min = 0, max = 100) String a,
        @ForAll @StringLength(min = 0, max = 100) String b
    ) {
        return (a + b).length() == a.length() + b.length();
    }

    @Property
    boolean sortedListIsOrdered(@ForAll List<@IntRange(min = -1000, max = 1000) Integer> numbers) {
        List<Integer> sorted = numbers.stream().sorted().toList();
        for (int i = 1; i < sorted.size(); i++) {
            if (sorted.get(i - 1) > sorted.get(i)) return false;
        }
        return true;
    }

    // Custom arbitraries (generators)
    @Provide
    Arbitrary<Email> validEmails() {
        return Combinators.combine(
            Arbitraries.strings().alpha().ofLength(5),
            Arbitraries.of("gmail.com", "yahoo.com", "example.com")
        ).as((local, domain) -> new Email(local + "@" + domain));
    }

    @Property
    boolean emailParsingRoundTrip(@ForAll("validEmails") Email email) {
        return Email.parse(email.toString()).equals(email);
    }
}

// Stateful testing with ActionChain
class StackProperties {

    @Property
    void stackNeverCorrupts(@ForAll("stackActions") ActionChain<Stack<Integer>> chain) {
        chain.run();
    }

    @Provide
    ActionChainArbitrary<Stack<Integer>> stackActions() {
        return ActionChain.startWith(Stack::new)
            .withAction(pushActions())
            .withAction(new PopAction());
    }

    Arbitrary<Action<Stack<Integer>>> pushActions() {
        return Arbitraries.integers().between(-1000, 1000)
            .map(PushAction::new);
    }

    static class PushAction implements Action<Stack<Integer>> {
        private final int value;
        PushAction(int value) { this.value = value; }
        @Override
        public Stack<Integer> run(Stack<Integer> stack) {
            stack.push(value);
            return stack;
        }
    }

    static class PopAction implements Action<Stack<Integer>> {
        @Override
        public boolean precondition(Stack<Integer> stack) {
            return !stack.isEmpty();
        }
        @Override
        public Stack<Integer> run(Stack<Integer> stack) {
            stack.pop();
            return stack;
        }
    }
}

What works

• Start with properties, not generators. The hard part is finding the right property (encode(decode(x)) == x), not writing the generator.
• Use shrinking religiously. The value of the approach is the minimal failing case, so make sure your framework’s shrinking is enabled and check its output instead of debugging the raw random input.
• Combine with example-based tests. Properties check invariants; examples check specific business scenarios. You need both.
• Keep properties pure. A property that writes to a database or reads the current time isn’t reproducible and can’t be shrunk well.
• Use a deterministic seed in CI. Property tests are random by nature; a seed makes failures reproducible across runs.
• Constrain generators to realistic data. Well-formed inputs reach meaningful failures faster — see the test data generation recipe for strategies that encode domain rules.

Common Mistakes

• Testing the implementation, not the specification. Writing property: sort(arr) == mySortFunction(arr) is tautological and finds no bugs. A suite that can’t catch injected faults is decoration — mutation testing measures exactly that.
• Properties that are too weak. length(f(x)) >= 0 is always true and tells you nothing. Properties should be strong enough to catch real bugs.
• Ignoring the shrunk output. A 100-element array that fails is hard to debug; the shrunk three-element array is what you should analyze.
• Slow or non-terminating generators. Generating recursive structures without depth limits can hang the test run.
• Flaky properties due to global state. A property that mutates a module-level counter fails unpredictably depending on execution order.

Troubleshooting

• The same property passes and fails randomly: the function under test touches time, randomness or shared state. Isolate the state and pin the reported seed before debugging.
• Generation hangs or is very slow: recursive or unbounded strategies produce huge values. Cap depth (st.recursive(..., max_leaves=10)), bound collection sizes, and measure generation time separately from assertion time.
• A failure disappears on rerun: the seed changed between runs. Copy the seed Hypothesis prints with @reproduce_failure, or pass { seed } to fc.assert, then write an example-based test for the shrunk input.
• Shrinking takes longer than the test run: deeply nested strategies make shrinking expensive. Simplify the strategy or bound maxShrinks — a slightly larger minimal case is still actionable.
• The suite is too slow for CI: lower max_examples, numRuns or tries in the CI profile and keep the full-size runs for a nightly job.

Further Reading

• Hypothesis documentation: strategies, stateful testing and example database replay in Python.
• fast-check documentation: arbitraries, model-based testing and shrinking in TypeScript/JavaScript.
• jqwik user guide: properties, providers and ActionChain stateful testing on the JVM.
• Complete guide to property-based testing: the long-form version of this recipe with design trade-offs.
• Runnable companion project: the examples in this recipe as executable Python, JavaScript and Java projects.