Property-Based Testing: Hypothesis, fast-check, QuickCheck

Introducción

Property-based testing invierte el modelo de testing tradicional. En vez de escribir test cases individuales con inputs y outputs específicos, definís properties — invariantes que deberían valer para todos los inputs válidos — y el framework genera cientos o miles de test cases automáticamente. Cuando un test falla, el framework shrinkéa el input que falla al caso más chico posible que reproduce el bug. A continuación: Hypothesis para Python, fast-check para TypeScript/JavaScript y los principios subyacentes de QuickCheck que alimentan todos los frameworks de property-based testing.

Conceptos Core

Example-based vs. property-based testing

Example-based (tradicional):
  test_add(2, 3) == 5
  test_add(0, 0) == 0
  test_add(-1, 1) == 0
  → Pensás edge cases manualmente. Te faltan algunos.

Property-based:
  for all integers a, b:
    add(a, b) == add(b, a)          — commutativity
    add(a, 0) == a                   — identity
    add(add(a, b), c) == add(a, add(b, c))  — associativity
  → El framework genera cientos de inputs. Encuentra edge cases que te faltaron.

El ciclo de property-based testing

1. Generate: Creá random inputs dentro de constraints definidas
2. Execute: Corré la function bajo test con esos inputs
3. Verify: Checkeá si la property vale
4. Shrink: En failure, reducí el input al caso minimal que reproduce
5. Report: Presentá el counterexample shrunkéado

Hypothesis (Python)

Instalación

pip install hypothesis
# O con pytest integration (recomendado)
pip install hypothesis pytest

Properties básicas

# tests/test_string_properties.py — Basic property tests
from hypothesis import given, strategies as st

@given(st.text())
def test_string_uppercase_is_uppercase(s):
    """Uppercase de cualquier string no contiene lowercase characters."""
    assert s.upper().isupper() or not s

@given(st.text())
def test_string_reversal_is_inverse(s):
    """Revertir un string dos veces da back el original."""
    assert s[::-1][::-1] == s

@given(st.lists(st.integers()))
def test_list_sort_preserves_length(lst):
    """Sortear una list no cambia su length."""
    assert len(sorted(lst)) == len(lst)

@given(st.lists(st.integers()))
def test_list_sort_idempotent(lst):
    """Sortear una list ya sorteada da la misma list."""
    sorted_lst = sorted(lst)
    assert sorted(sorted_lst) == sorted_lst

@given(st.lists(st.integers()))
def test_list_max_is_in_list(lst):
    """El max de una list siempre es un elemento de la list."""
    if lst:
        assert max(lst) in lst

Composite strategies

# tests/test_user_properties.py — Composite strategies
from hypothesis import given, strategies as st, composite

@composite
def valid_email(draw):
    """Generá valid email addresses."""
    local = draw(st.text(min_size=1, max_size=20, alphabet=st.characters(whitelist_categories=("Ll", "Nd"))))
    domain = draw(st.text(min_size=1, max_size=10, alphabet=st.characters(whitelist_categories=("Ll", "Nd"))))
    tld = draw(st.sampled_from(["com", "org", "net", "io", "dev"]))
    return f"{local}@{domain}.{tld}"

@composite
def user_strategy(draw):
    """Generá user objects con valid fields."""
    return {
        "id": draw(st.integers(min_value=1, max_value=1000000)),
        "name": draw(st.text(min_size=1, max_size=50)),
        "email": draw(valid_email()),
        "age": draw(st.integers(min_value=0, max_value=150)),
        "is_active": draw(st.booleans()),
    }

@given(user_strategy())
def test_user_serialization_roundtrip(user):
    """Serializar y deserializar un user preserva todos los fields."""
    serialized = json.dumps(user)
    deserialized = json.loads(serialized)
    assert deserialized == user

@given(user_strategy())
def test_user_email_contains_at(user):
    """Cada user email generado contiene @."""
    assert "@" in user["email"]

Assumptions y conditional properties

# tests/test_conditional_properties.py — Filtrando inputs
from hypothesis import given, strategies as st, assume

@given(st.lists(st.integers()))
def test_binary_search_finds_element(lst):
    """Binary search encuentra cualquier elemento en una sorted list."""
    assume(len(lst) > 0)  # Skipeá empty lists
    sorted_lst = sorted(lst)
    target = sorted_lst[len(sorted_lst) // 2]  # Pick middle element

    def binary_search(arr, target):
        lo, hi = 0, len(arr) - 1
        while lo <= hi:
            mid = (lo + hi) // 2
            if arr[mid] == target:
                return mid
            elif arr[mid] < target:
                lo = mid + 1
            else:
                hi = mid - 1
        return -1

    result = binary_search(sorted_lst, target)
    assert result != -1
    assert sorted_lst[result] == target

@given(st.text())
def test_only_alphanumeric_sort(s):
    """Sortear funciona correctamente para alphanumeric strings."""
    assume(s.isalnum())  # Solo testeá alphanumeric strings
    assert "".join(sorted(s)) == "".join(sorted("".join(sorted(s))))

Stateful testing

# tests/test_stack_properties.py — Stateful property testing
from hypothesis.stateful import RuleBasedStateMachine, rule, invariant, initialize
from hypothesis import given, strategies as st

class StackMachine(RuleBasedStateMachine):
    """Testeá una stack implementation con stateful properties."""

    def __init__(self):
        super().__init__()
        self.stack = []

    @rule(value=st.integers())
    def push(self, value):
        self.stack.append(value)

    @rule()
    def pop(self):
        if self.stack:
            self.stack.pop()

    @invariant()
    def stack_size_non_negative(self):
        assert len(self.stack) >= 0

    @invariant()
    def pop_returns_last_pushed(self):
        if self.stack:
            top = self.stack[-1]
            assert self.stack.pop() == top
            self.stack.append(top)  # Restore para other rules

TestStack = StackMachine.TestCase

Settings y configuration

# tests/test_configured.py — Hypothesis settings
from hypothesis import given, strategies as st, settings, HealthCheck

@given(st.lists(st.integers(), min_size=1))
@settings(
    max_examples=500,          # Generá hasta 500 test cases
    deadline=2000,             # 2 second timeout per test
    suppress_health_check=[HealthCheck.too_slow],
    print_blob=True,           # Printá el failing input como blob
)
def test_large_list_processing(lst):
    """Procesá large lists dentro del time budget."""
    result = process_list(lst)
    assert len(result) == len(lst)

fast-check (TypeScript/JavaScript)

Instalación

npm install -D fast-check vitest

Properties básicas

// tests/properties/string.test.ts — Basic property tests con fast-check
import { describe, it, expect } from "vitest";
import fc from "fast-check";

describe("String properties", () => {
  it("uppercase contains no lowercase characters", () => {
    fc.assert(fc.property(fc.string(), (s) => {
      expect(s.toUpperCase().match(/[a-z]/)).toBeNull();
    }));
  });

  it("reversing twice returns the original", () => {
    fc.assert(fc.property(fc.string(), (s) => {
      expect(s.split("").reverse().reverse().join("")).toBe(s);
    }));
  });

  it("concatenation length is sum of lengths", () => {
    fc.assert(fc.property(fc.string(), fc.string(), (a, b) => {
      expect((a + b).length).toBe(a.length + b.length);
    }));
  });
});

describe("Number properties", () => {
  it("addition is commutative", () => {
    fc.assert(fc.property(fc.integer(), fc.integer(), (a, b) => {
      expect(a + b).toBe(b + a);
    }));
  });

  it("addition is associative", () => {
    fc.assert(fc.property(fc.integer(), fc.integer(), fc.integer(), (a, b, c) => {
      expect((a + b) + c).toBe(a + (b + c));
    }));
  });

  it("multiplication distributes over addition", () => {
    fc.assert(fc.property(fc.integer(), fc.integer(), fc.integer(), (a, b, c) => {
      expect(a * (b + c)).toBe(a * b + a * c);
    }));
  });
});

Custom arbitraries

// tests/properties/user.test.ts — Custom arbitraries
import { describe, it, expect } from "vitest";
import fc from "fast-check";

const validEmail = fc.tuple(
  fc.string({ minLength: 1 }).map((s) => s.replace(/[^a-zA-Z0-9]/g, "a")),
  fc.string({ minLength: 1 }).map((s) => s.replace(/[^a-zA-Z0-9]/g, "a")),
  fc.constantFrom("com", "org", "net", "io", "dev")
).map(([local, domain, tld]) => `${local}@${domain}.${tld}`);

const userArbitrary = fc.record({
  id: fc.integer({ min: 1, max: 1000000 }),
  name: fc.string({ minLength: 1, maxLength: 50 }),
  email: validEmail,
  age: fc.integer({ min: 0, max: 150 }),
  isActive: fc.boolean(),
});

describe("User properties", () => {
  it("serialization roundtrip preserves data", () => {
    fc.assert(fc.property(userArbitrary, (user) => {
      const serialized = JSON.stringify(user);
      const deserialized = JSON.parse(serialized);
      expect(deserialized).toEqual(user);
    }));
  });

  it("email contains @", () => {
    fc.assert(fc.property(userArbitrary, (user) => {
      expect(user.email).toContain("@");
    }));
  });
});

Stateful testing con fast-check

// tests/properties/stack.test.ts — Stateful property testing
import { describe, it, expect } from "vitest";
import fc from "fast-check";

type StackCommand =
  | { type: "push"; value: number }
  | { type: "pop" };

const stackCommands = fc.array(
  fc.oneof(
    fc.record({ type: fc.constant("push"), value: fc.integer() }),
    fc.record({ type: fc.constant("pop") })
  )
);

describe("Stack stateful properties", () => {
  it("stack size is always non-negative", () => {
    fc.assert(fc.property(stackCommands, (commands) => {
      const stack: number[] = [];
      for (const cmd of commands) {
        if (cmd.type === "push") {
          stack.push(cmd.value);
        } else if (stack.length > 0) {
          stack.pop();
        }
        expect(stack.length).toBeGreaterThanOrEqual(0);
      }
    }));
  });

  it("pop returns last pushed value (LIFO)", () => {
    fc.assert(fc.property(stackCommands, (commands) => {
      const stack: number[] = [];
      for (const cmd of commands) {
        if (cmd.type === "push") {
          stack.push(cmd.value);
        } else if (stack.length > 0) {
          const top = stack[stack.length - 1];
          expect(stack.pop()).toBe(top);
        }
      }
    }));
  });
});

Shrinking y debugging

// tests/properties/shrinking.test.ts — Entendiendo shrinking
import { describe, it, expect } from "vitest";
import fc from "fast-check";

describe("Shrinking behavior", () => {
  it("finds minimal failing case", () => {
    // Esto va a fallar y shrinkéar al caso minimal
    fc.assert(
      fc.property(fc.array(fc.integer()), (arr) => {
        // Property: sorted array has no element greater than the next
        const sorted = [...arr].sort((a, b) => a - b);
        for (let i = 0; i < sorted.length - 1; i++) {
          expect(sorted[i]).toBeLessThanOrEqual(sorted[i + 1]);
        }
      }),
      { verbose: 2 } // Mostrá shrinking steps
    );
  });

  it("with custom seed for reproducibility", () => {
    fc.assert(
      fc.property(fc.integer(), (n) => {
        expect(Math.abs(n)).toBeGreaterThanOrEqual(0);
      }),
      { seed: 12345 } // Test runs reproducibles
    );
  });
});

Patrones Prácticos de Properties

Roundtrip properties

# Python: serialize → deserialize → compare
@given(st.text())
def test_json_roundtrip(s):
    """JSON serialization es un roundtrip para strings."""
    assert json.loads(json.dumps(s)) == s

@given(st.lists(st.integers()))
def test_sort_roundtrip(lst):
    """Sortear es idempotent."""
    assert sorted(sorted(lst)) == sorted(lst)

// TypeScript: encode → decode → compare
it("base64 roundtrip", () => {
  fc.assert(fc.property(fc.string(), (s) => {
    const encoded = btoa(s);
    const decoded = atob(encoded);
    expect(decoded).toBe(s);
  }));
});

Invariant properties

@given(st.lists(st.integers(), min_size=1))
def test_max_is_geq_all(lst):
    """El max de una list es >= todos los elementos."""
    m = max(lst)
    for x in lst:
        assert m >= x

@given(st.lists(st.integers(), min_size=1))
def test_sum_equals_max_times_count(lst):
    """Sum de una list de valores idénticos equals value * count."""
    value = lst[0]
    assume(all(x == value for x in lst))
    assert sum(lst) == value * len(lst)

Oracle properties

@given(st.lists(st.integers()))
def test_custom_sort_matches_builtin(lst):
    """Custom sort produce mismo resultado que Python built-in sort."""
    custom_sorted = my_custom_sort(lst[:])
    builtin_sorted = sorted(lst)
    assert custom_sorted == builtin_sorted

Stateful properties

from hypothesis.stateful import RuleBasedStateMachine, rule, invariant

class DictStateMachine(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.dict = {}
        self.model = {}

    @rule(key=st.text(), value=st.integers())
    def set_value(self, key, value):
        self.dict[key] = value
        self.model[key] = value

    @rule(key=st.text())
    def get_value(self, key):
        if key in self.model:
            assert self.dict[key] == self.model[key]

    @invariant()
    def sizes_match(self):
        assert len(self.dict) == len(self.model)

TestDict = DictStateMachine.TestCase

Integración con CI

GitHub Actions con Hypothesis

# .github/workflows/property-tests.yml
name: Property Tests
on: [push, pull_request]

jobs:
  property-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install hypothesis pytest
      - run: pytest tests/properties/ -v --hypothesis-show-statistics

GitHub Actions con fast-check

# .github/workflows/property-tests.yml
name: Property Tests
on: [push, pull_request]

jobs:
  property-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npx vitest run tests/properties/

Best Practices

• For a deeper guide, see Implement Property-Based Testing.

• Empezá con properties simples — commutativity, associativity, identity

• Usá roundtrip properties para serialization — encode then decode debería devolver original

• Usá oracle properties cuando existe una reference implementation — compará contra built-in

• Seteá max_examples más alto para código crítico — 1000 en vez del default 100

• Usá assume sparingly — demasiadas assumptions reducen test coverage

• Pineá el random seed en CI — failures reproducibles

• Combiná con example-based tests — properties encuentran edge cases, examples documentan behavior

• Usá stateful testing para stateful systems — databases, caches, queues

• Mirá el shrinking output — el counterexample minimal revela el root cause

• No testees framework code — testeá tu business logic, no el sort del language

Common Mistakes

• Properties demasiado débiles: sort(lst) has same length as lst — true pero no testea sorting. Agregá sort(lst) is sorted.
• Properties demasiado fuertes: add(a, b) > 0 — falla para negative numbers. Usá add(a, 0) == a en vez.
• No usar assume: testear properties que solo valen para algunos inputs sin filtrar. Usá assume(condition).
• Ignorar shrinking: el caso shrunkéado es el output más valioso. Leelo para entender el bug.
• Testear demasiado por property: una property por test function. Si falla, sabés exactamente qué invariante breakó.