Grep Exits Zero When It Finds the Word FAILED

The gate that let a red build through

Here's a shell-scripting trap that's bitten a lot of people, and it bit me too: a CI step was chained behind a command that filtered test output through grep for readability --

python -m unittest ... | grep -E "^Ran |^OK|^FAILED"

The idea was reasonable enough: run the suite, pipe the noisy output through grep so only the summary lines survive, and let the next step in the pipeline decide pass or fail based on what came through.

The problem is what grep's own exit code actually means. Per GNU grep's Exit Status documentation and the POSIX specification for grep it implements, grep exits 0 when it finds a match -- any match -- and exits 1 only when it finds nothing at all. It has no concept of "the thing I matched was bad news." So when the test suite failed and printed a line starting with FAILED, grep matched that line, found what it was looking for, and happily reported success. The suite was red. The grep succeeded. The commit landed.

Why this is so easy to write by accident

Nothing about the command looks wrong. You're filtering output for readability, which is a completely normal thing to do in a pipeline. The bug is purely in what "success" means to two different programs at the same moment -- the test runner and grep are answering two different questions, and only one of them is the question you actually care about.

The fix is to stop asking grep to make the pass/fail call at all. Either check the actual exit code of the test runner directly (before piping it anywhere), or use grep's own inverted-match trick deliberately -- exit 0 only when the bad pattern is absent, not merely when a pattern is present. The rule that generalizes: gate on exit codes, never on what a search happened to find.

Where else this shows up

The same shape of bug appears anywhere a pipeline's final exit code isn't obviously the one you think it is -- a command piped into another command usually reports the last stage's exit status, not the first's, which is its own classic trap and worth knowing separately. Any time a check "passed" and the result still looks wrong, the first thing worth asking is: which command's exit code actually decided that?