Automation must be zero-failure

All automated tests run as part of the CI pipeline for every change (pull request and mainline).
The pipeline is a quality gate: builds are not considered “green” unless test automation completes with a 0% failure rate.
Teams do not ship with known test failures (e.g., “10% failing is fine”) and do not rely on manual triage to decide which failures matter.
Flaky tests are treated as defects in the delivery system and are fixed/removed from the gating suite immediately.

A non-zero failure rate normalises broken feedback loops and makes CI signals unreliable.
If teams have to review failures to decide what is important, the pipeline is no longer a gate—it’s noise.
A zero-failure standard preserves confidence, reduces rework, and prevents degraded quality from reaching customers.

CI runs the full gating automated suite on every PR and blocks merges on any failure. If not on PR, then at least as part of the path to production
The team maintains a clear, owned process to eliminate flakiness quickly (e.g., same-day fix or quarantine policy with strict time limit).
Test results are visible and actionable (Show in Azure, link to logs, failures are treated as incidents when they block delivery).
The gating suite stays fast enough to be used consistently; longer-running suites can exist, but they do not replace the zero-failure gate.
Following Playwright standards/best practices to prevent flaky tests - build the quality from the start.

A project meets this principle when:

The CI pipeline includes automated test execution as a required step for merge/release.
The expected failure rate for the gating suite is 0% (no accepted baseline of failing tests).
Any test failure blocks shipping until resolved.
There is no routine manual “failure review” to decide whether a failed test can be ignored.