Integration Testing with Real Dependencies

Overview

Unit tests check one function at a time with everything around it faked. Production code, though, lives wired to databases, HTTP services, queues and file systems, and the wiring is where the bugs hide. Integration tests check that these components connect correctly: that a repository can actually read from a database, that an API client survives real network behavior, that events propagate through a broker.

Integration tests are slower and more complex than unit tests, but they catch a different class of bugs: connection string errors, schema mismatches, serialization problems, and network timeout handling. A healthy suite keeps both, and this recipe covers the integration layer in Python, JavaScript and Java.

When to Use

Use this recipe when:

• Checking repositories and migrations against a real database engine. Connection Pooling covers the connection layer.
• Testing HTTP API endpoints through real request/response cycles. Call REST API covers the client side.
• Confirming message queue producers and consumers integrate properly. See Kafka Event Streaming for the messaging setup.
• Validating third-party SDK or API client behavior. Handle Errors covers the failure paths.
• Checking that configuration and environment setup are correct. See Environment Variables for config patterns.
• Running pre-deployment smoke tests in a CI/CD pipeline.

Skip it when the risk lives inside a single class: pure logic, transformations, and validation rules are cheaper and more precise under a unit test with a mocked boundary. Integration tests pay off only when the bug would hide in the wiring between two real components. For how this layer fits the broader suite — pyramid proportions, coverage goals, CI gates — see the testing strategy guide.

Solution

Python (pytest + Testcontainers)

import pytest
from testcontainers.postgres import PostgresContainer
from sqlalchemy import create_engine, text

@pytest.fixture(scope="module")
def db_engine():
    with PostgresContainer("postgres:16") as postgres:
        engine = create_engine(postgres.get_connection_url())
        yield engine

def test_user_repository(db_engine):
    with db_engine.connect() as conn:
        conn.execute(text("CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT)"))
        conn.execute(
            text("INSERT INTO users (name) VALUES (:name)"),
            {"name": "Alice"}
        )
        conn.commit()

        result = conn.execute(text("SELECT * FROM users"))
        users = result.fetchall()
        assert len(users) == 1
        assert users[0][1] == "Alice"

The scope="module" fixture starts one PostgreSQL container per test module instead of per test, which is the single biggest speed win once you’ve got more than a handful of tests.

JavaScript (Jest + Supertest)

const request = require('supertest');
const app = require('./app'); // Express app

describe('POST /api/users', () => {
  afterAll(async () => {
    await app.db.close(); // close test database
  });

  test('creates a user and returns 201', async () => {
    const response = await request(app)
      .post('/api/users')
      .send({ name: 'Alice', email: 'alice@example.com' })
      .set('Accept', 'application/json');

    expect(response.status).toBe(201);
    expect(response.body).toHaveProperty('id');
    expect(response.body.email).toBe('alice@example.com');
  });

  test('returns 400 for invalid email', async () => {
    const response = await request(app)
      .post('/api/users')
      .send({ name: 'Alice', email: 'not-an-email' });

    expect(response.status).toBe(400);
  });
});

Java (JUnit + Testcontainers)

import org.junit.jupiter.api.*;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

@Testcontainers
public class UserRepositoryTest {

    @Container
    static PostgreSQLContainer<?> postgres =
        new PostgreSQLContainer<>("postgres:16");

    @Test
    void shouldInsertAndRetrieveUser() {
        UserRepository repo = new UserRepository(
            postgres.getJdbcUrl(),
            postgres.getUsername(),
            postgres.getPassword()
        );

        User user = new User("Alice", "alice@example.com");
        repo.save(user);

        User found = repo.findByEmail("alice@example.com");
        assertEquals("Alice", found.getName());
    }
}

getJdbcUrl() returns the mapped port, so the test works even when Docker assigns a random host port. Never hardcode localhost:5432.

Explanation

The diagram shows the boundary an integration test covers: the app talks to real infrastructure (database, broker) and to a stub for anything you don’t control.

• Testcontainers boots throwaway Docker instances of the real services, like PostgreSQL, Kafka or Redis, scoped to your test run. You get genuine integration coverage without polluting a shared development database, and a cleanup container (Ryuk) removes everything even when a test crashes. The first run pays for the image pull; after that, startup is a few seconds.
• Real engine vs in-memory: an H2 or SQLite stand-in diverges from PostgreSQL or MySQL in transaction isolation, type coercion, and JSON support. Tests that pass against the fake engine can fail in production, which defeats the purpose. Use the same engine you ship; Testcontainers exists precisely for that.
• Supertest sends real HTTP requests to an Express app without binding to a network port. You exercise the full request lifecycle: middleware, routing, validation, and serialization.
• Spring Boot Test with @SpringBootTest(webEnvironment = RANDOM_PORT) boots the whole application context on a random port, so you can hit @RestController endpoints with TestRestTemplate or WebTestClient.

Variants

Tool	Language	Dependency Type	Best For
Testcontainers	Java/Python/Go/Node	Real Docker containers	Databases, Kafka, Redis
Supertest	JavaScript	In-process HTTP	Express/Fastify APIs
pytest-django	Python	Real test database	Django ORM integration
Spring Boot Test	Java	Full application context	Spring microservices
WireMock / MockServer	Java/JS	Stubbed HTTP responses	Third-party APIs you don’t control

Mock only the systems you don’t own. A stubbed payment gateway keeps the test deterministic; a stubbed database removes the very thing the test is supposed to verify.

Structuring the Suite

Keep integration tests physically separate from unit tests so a fast loop stays fast:

• Python: tests/integration/ plus a marker, so you can select or deselect the slow tests explicitly.

# pytest.ini
[pytest]
markers =
    integration: needs Docker containers (deselect with -m "not integration")

• JavaScript: give integration tests their own Jest config (or a filename convention) and separate npm scripts.

{
  "scripts": {
    "test:unit": "jest --testPathIgnorePatterns integration",
    "test:integration": "jest --testMatch '**/integration/**/*.test.js'"
  }
}

• Java: name classes *IT.java and bind them to the Maven Failsafe plugin (mvn verify) instead of Surefire, so mvn test stays unit-only.

In CI, run unit tests on every commit and integration tests on pull requests or before merge. For test data, pick one isolation strategy per suite: truncate tables between tests, wrap each test in a rolled-back transaction, or recreate the schema per class. Mixing strategies inside one suite is how flaky ordering bugs start.

What works

• Use real dependencies, not mocks: the whole point of integration testing is verifying real interactions. Mock only external systems you don’t control (payment gateways, third-party APIs).
• Reset state between tests: truncate the tables, drain the queues or recreate the container. Otherwise the previous test’s leftovers decide whether the next one passes.
• Keep integration tests in a separate directory: tests/integration/ or src/test/integration/ makes it clear these are slower and more thorough.
• Gate them behind CI, not your editor: split the runner commands (npm run test:unit vs npm run test:integration) so the fast loop stays fast.
• Use random ports and isolated databases: never run integration tests against your development or production database.
• Limit scope: one integration point per test. A test that hits the database, the HTTP layer and a queue at once tells you something broke, but not where.

Common Mistakes

• Pointing integration tests at production databases: one bad test can wipe or corrupt real data, and depending on the industry it can breach compliance rules too.
• Not cleaning up after tests: leftover data causes flaky tests that pass in isolation but fail in a suite.
• Mocking everything in an integration test: if you mock the database and HTTP layer, you’re writing an elaborate unit test, not an integration test.
• Using hard-coded ports: port conflicts cause flaky tests. Let the runtime assign ports: getMappedPort(5432) in Java or get_connection_url() in Python return whatever free port Docker picked.
• Testing too much in one test: when a broad integration test fails, you spend more time debugging which layer broke than writing the fix.

Troubleshooting

• Container fails to start on CI: the runner needs Docker access. On GitHub Actions the hosted runners provide it; on self-hosted runners verify DOCKER_HOST and that the Docker daemon is running. Without Docker, use service containers (GitHub Actions services:) as a fallback.
• First run is slow: pulling postgres:16 takes a minute. Cache the image in CI or keep the suite small enough that startup amortizes over it.
• “Connection refused” right after container start: the container is up before the database accepts connections. Testcontainers handles this with wait strategies; if you manage containers yourself, poll a health check instead of sleeping a fixed time.
• Tests pass locally, fail on CI: usually resource limits or leftover state. Check that cleanup runs per test and that CI workers have enough memory for the containers.
• Order-dependent flakes: rows or queue messages from one test leak into the next. Truncating tables between tests, or rolling back a transaction per test, removes that shared state.

See Also

• Testcontainers documentation
• Supertest on GitHub
• JUnit 5 User Guide
• Martin Fowler — The Practical Test Pyramid
• Test Strategy Document Template — where these CI gates and coverage targets get written down