Who cleans up after test preparation fails? unittest and XAT

An API test may acquire a connection, create test data, and then prepare a request. If the third step fails, resources created by the first two still need releasing. Whether a single cleanup function at the end is enough depends on which lifecycle stages the test framework calls after preparation fails.

Follow unittest’s call order first: which functions still run after setup fails, and what happens to the remaining resources if cleanup itself raises? The standard library supplies addCleanup for this lifecycle. XAT builds on it by retaining failures from each phase.

What a setUp failure skips

The normal sequence is setUp → test → tearDown → cleanups. If setUp raises, the test method and tearDown do not run, but registered cleanups still do. Releasing everything only in tearDown therefore misses partial preparation followed by failure.

This standalone Python example uses a list in place of external resources to expose the call order:

import io
import unittest

trace = []

class Example(unittest.TestCase):
    def setUp(self):
        trace.append("acquire A")
        self.addCleanup(trace.append, "release A")
        trace.append("acquire B")
        self.addCleanup(trace.append, "release B")
        raise RuntimeError("prepare C failed")

    def test_body(self):
        trace.append("test")

    def tearDown(self):
        trace.append("teardown")

result = unittest.TextTestRunner(stream=io.StringIO()).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(Example)
)
assert trace == ["acquire A", "acquire B", "release B", "release A"]
assert len(result.errors) == 1
print(trace)

The output shows B and A being released in reverse order even though C failed. Each cleanup was registered immediately after its resource was acquired. Had registration waited until all setup finished, C would have failed before A and B were registered. Releasing B before A also fits a dependency where B relies on A.

When cleanup itself fails

A handwritten finally block can perform cleanup, but its first exception can prevent later statements from running. unittest collects exceptions from individual cleanups separately, so one failing cleanup does not discard the remaining stack. The standard-library documentation specifies this ordering and behavior after setup failure.

The next example intentionally fails an assertion and a cleanup. Both problems should remain in the result, while another cleanup still runs:

import io
import unittest

released = []

class TwoFailures(unittest.TestCase):
    def test_body(self):
        self.addCleanup(released.append, "connection closed")
        self.addCleanup(lambda: 1 / 0)
        self.assertEqual(1, 2, "business assertion")

result = unittest.TextTestRunner(stream=io.StringIO()).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(TwoFailures)
)
assert released == ["connection closed"]
assert len(result.failures) == 1
assert len(result.errors) == 1

XAT preserves these lifecycle semantics. Phase wrappers provide logging, while the result collector keeps each issue. A method can contain both an assertion failure and a cleanup exception: its primary state is SCRIPT_ERROR, but the assertion remains in its issues. Keeping only the final exception loses the original test outcome; keeping only the assertion loses a clue about leftover resources.

Resource lifetime depends on the execution unit

XAT’s default method mode starts an interpreter for each method. Connections and Sessions should be acquired and closed by that execution lifecycle; class fixtures in one file are not shared across all method workers. File mode shares class and module fixtures within its suite. Their resources need corresponding class or module cleanup facilities; method cleanups do not replace every scope.

Forced termination creates another boundary. When the parent kills a worker after a deadline, Python cannot continue with tearDown or cleanups. The operating system reclaims process resources, but external orders, leases, and account state do not thereby reset. Business-specific expiry or reclamation is required, beyond unittest’s guarantee.

The examples therefore cover only one part of cleanup verification: call order while Python can still run. A separate termination test must establish whether external resources have an expiry or reclamation path. Successful addCleanup checks cannot answer that question. See method and file execution for scheduling scope and result reporting for multiple errors in one method.