How do you assert thrown errors in Swift Testing, and what do withKnownIssue and Issue.record add?

22.8.3 · L3 · Interview

Predict first

What do the typed, valued and returned throw checks do, and what happens to a known issue that no longer fails?

// === Package.swift ===
// swift-tools-version:6.0
import PackageDescription
let package = Package(name: "Lib", targets: [.target(name: "Lib"), .testTarget(name: "LibTests", dependencies: ["Lib"])])
// === Sources/Lib/Lib.swift ===
public func square(_ x: Int) -> Int { x * x }
// === Tests/LibTests/T.swift ===
import Testing
@testable import Lib
struct Boom: Error, Equatable { let code: Int }
func fail(_ c: Int) throws { throw Boom(code: c) }

@Test func typeOnly() { #expect(throws: Boom.self) { try fail(1) } }
@Test func valueMatch() { #expect(throws: Boom(code: 2)) { try fail(2) } }
@Test func returned() throws {
    let e = try #require(throws: Boom.self) { try fail(3) }
    print("RETURNED_CODE", e.code)
}
@Test func wrongValue() { #expect(throws: Boom(code: 9)) { try fail(4) } }
@Test func neverThrows() { #expect(throws: Never.self) { try fail(5) } }
@Test func knownIssue() { withKnownIssue("bug 12") { #expect(square(2) == 5) } }
@Test func knownIssueFixed() { withKnownIssue("bug 13") { #expect(square(2) == 4) } }
  • Typed and valued pass, the returned code is 3, a wrong value names both errors, the fixed known issue fails with “not recorded”
  • All checks pass
  • The known issue always passes

Mental model. A known issue is a “wet paint” sign: it explains the failure, and gets flagged for removal as soon as the paint is dry.

#expect(throws: SomeError.self) { try work() } passes only if the closure throws an error of that type; #expect(throws: Boom(code: 2)) (an Equatable error) checks the exact value, and a mismatch tells you what arrived (“expected error Boom(code: 9), but Boom(code: 4) was thrown”). #expect(throws: Never.self) asserts that nothing is thrown, and a thrown error fails with the error shown. try #require(throws: Boom.self) { … } stops the test on failure and returns the error, so you can assert on its fields afterwards (the check reads code from the returned error). There is also an overload with a closure to inspect the error. This replaces XCTAssertThrowsError with its trailing closure.

withKnownIssue("reason") { … } wraps code that currently fails: its failure is recorded as a known issue and the test passes (“with 1 known issue”), but if the code stops failing, the wrapper itself reports “Known issue was not recorded” as a failure, which is the point: it reminds you to delete the wrapper once the bug is fixed. That is better than .disabled for a bug you still want exercised. Issue.record("message") reports a failure from helper code or branches that should be unreachable, without an expectation, and is how a custom assertion helper fails a test.

Throw checks compare type or value, #require(throws:) returns the error, and a known issue that stops failing is itself a failure.

Common traps

  • Catching the error with do/catch and forgetting to fail when nothing is thrown. Use #expect(throws:), which fails when no error occurs.
  • Leaving a withKnownIssue in place after the fix. The test starts failing with “Known issue was not recorded”; remove the wrapper.

They might follow up with

  • Can a known issue be limited to some conditions? Yes, there is an isIntermittent option and a condition for when the issue is expected.
  • How do I report a failure from a shared helper? Issue.record, with the source location of the helper’s caller if you pass it.

Related questions

  • 22.8.2 What are traits in Swift Testing, and how do you skip, tag, bound and annotate tests with them?
  • 22.8.4 How does confirmation count events in Swift Testing, and what can it not do?
  • 22.2.1 How do XCTest and Swift Testing differ, and when do you still use XCTest?

← What are traits in Swift Testing, and how do you skip, tag, bound and…   How does confirmation count events in Swift Testing, and what can it… →