Why can a CI job pass when the build failed, and how do pipes, pipefail, set -e and PIPESTATUS change that?

25.8.1 · L3 · Interview

Predict first

Different failure-handling options are tried on a pipeline whose first stage fails. What status does each report?

// === run.sh ===
bash -c 'false | cat; echo "plain pipeline status: $?"'
bash -c 'set -o pipefail; false | cat; echo "pipefail status: $?"'
bash -c 'true | false | true; echo "PIPESTATUS: ${PIPESTATUS[@]}"'
bash -c 'set -e; false | cat; echo "set -e alone: kept going"'
bash -c 'set -eo pipefail; false | cat; echo "unreachable"'; echo "set -eo pipefail: exit $?"
sh -c 'xcodebuild -project Missing.xcodeproj -scheme X build 2>&1 | tail -1; echo "xcodebuild | tail status: $?"'
bash -c 'set -o pipefail; xcodebuild -project Missing.xcodeproj -scheme X build 2>&1 | tail -1; echo "with pipefail: $?"' 
  • plain 0, pipefail 1, PIPESTATUS 0 1 0, set -e alone continues, set -eo pipefail exits 1, xcodebuild | tail reports 0 but with pipefail non-zero
  • The plain pipeline reports 1
  • set -e alone stops the script

Mental model. A relay team where only the last runner’s time is reported: if the second runner dropped the baton, the result still says “finished”.

In a shell pipeline the exit status is by default the status of the last command. The classic iOS case is xcodebuild … | xcpretty or | tail: the formatter succeeds, so the step reports success even though the build failed. The check shows it with a real xcodebuild pointed at a missing project: with | tail -1 the status printed is 0, with set -o pipefail it is non-zero. pipefail makes the pipeline return the status of the rightmost failing command; PIPESTATUS (bash) holds every stage’s status (0 1 0 in the check); and set -e on its own does not stop the script for a failing earlier stage of a pipeline, while set -eo pipefail does (the last line of the check never runs).

The standard header for a release script is set -euo pipefail: exit on a failed command, on an unset variable and on a failed pipeline stage. Some CI systems run each step with sh -e only, without pipefail, so put the header in your own script, written in bash, and keep logic out of inline YAML one-liners (Q480). Also: log the exit status of the tool, not of the formatter (formatters often accept a flag to preserve the result), save the raw log as an artefact so the formatted one cannot hide the failure, and add a step that checks the artefact exists (an archive, an ipa) instead of trusting the previous step. Beware the opposite problem too: grep that finds nothing returns 1, so a pipeline that legitimately finds no matches fails under pipefail; handle it explicitly. Checked with bash and the system sh on macOS; other shells differ in the details of PIPESTATUS.

Only pipefail makes the pipeline carry the failure of an earlier stage.

Common traps

  • Piping the build through a formatter without pipefail. The job is green and the artefact does not exist.
  • Relying on set -e alone. It ignores failures inside a pipeline unless pipefail is on.

They might follow up with

  • Does pipefail apply to sh? It is a bash/zsh option; macOS sh accepts it today, but write scripts for bash explicitly.
  • How do I keep the formatted log and the real status? Write the raw output to a file with tee, format from the file, and use the tool’s status.

Related questions

  • 25.2.1 What are the main ways to run iOS CI/CD (Xcode Cloud, GitHub Actions, Fastlane, others), and what do they all…
  • 25.2.3 How do you design a release pipeline: hermetic builds, secrets, caching, artefact promotion and the order of…
  • 23.7.1 What does an Xcode scheme control, how do test plans and derived data fit in, and how do you use xcodebuild…

Can you use the git commit count as the build number, and what breaks… →