Functional UI tests check that the right things are on the page: the price is £39.99, the button is enabled, the error message appears. They say nothing about whether the price is overlapping the product name, whether the button has turned white on white, or whether a CSS change on another page pushed this card’s layout out of shape. Visual regression tests fill that gap. They take a screenshot, compare it with a previously approved image, and fail when the pixels differ.
The idea is simple. Nearly all the work is making the screenshot come out identical whenever nothing has changed, because a visual test that fails at random is worse than none. This article walks through a Playwright screenshot test from the first baseline to a stable comparison, using real runs against Chrome on macOS.
How a screenshot comparison works
The first time a visual test runs, there’s nothing to compare against, so the tool saves the screenshot as the baseline. Someone reviews that image and commits it. On later runs, the tool takes a new screenshot (the actual), compares it pixel by pixel with the baseline, and fails if more pixels differ than the configured tolerance. On failure it writes three images: expected, actual and a diff highlighting what changed. When a change is intended, you regenerate the baseline and the new image goes through code review like any other change.
Playwright Test builds this into expect(...).toHaveScreenshot(). Percy, Chromatic, BackstopJS and Loki follow the same flow.
A product card and its first baseline
The component under test is a product card: a badge, a name, a price, and a line saying when the price was last checked, filled in by a script using new Date(). The Playwright config pins the things that affect rendering:
export default defineConfig({
use: {
channel: 'chrome',
viewport: { width: 480, height: 360 },
deviceScaleFactor: 1,
locale: 'en-GB',
timezoneId: 'Europe/London',
},
});And the test screenshots just the card, not the whole page:
test('product card', async ({ page }) => {
await page.setContent(cardHtml());
await expect(page.locator('.card')).toHaveScreenshot('product-card.png');
});The first run fails by design:
Error: A snapshot doesn't exist at card.spec.js-snapshots/product-card-darwin.png, writing actual.Notice -darwin in the file name. Playwright stores baselines per platform, because the same page renders slightly differently on macOS, Linux and Windows: different font rasterisation, different anti-aliasing. A baseline made on a Mac will fail on a Linux CI runner. The usual fix is to generate and compare baselines inside the same Docker image everywhere (Playwright publishes one), rather than keeping a set per developer machine.
The clock in the corner
Run the test again a couple of seconds later:
Error: expect(locator).toHaveScreenshot(expected) failed
Locator: locator('.card')
46 pixels (ratio 0.01 of all image pixels) are different.
Call log:
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 46 pixels (ratio 0.01 of all image pixels) are different.Nothing about the card changed except “Price checked 09:30:14” becoming “09:30:16”. (An earlier run differed by 32 pixels; the count depends on which digits changed.) The call log is also worth reading for what it shows Playwright doing on its own: it disables CSS animations and transitions, waits for web fonts to finish loading, and retries until two consecutive screenshots match. Those defaults remove three common sources of instability. It can’t know that your content depends on the clock.
Playwright can control the page’s clock. Add one line before loading the content:
await page.clock.setFixedTime(new Date('2026-10-11T09:30:00Z'));Regenerate the baseline with npx playwright test --update-snapshots, and repeated runs now pass:
✓ 1 card.spec.js:4:1 › product card (340ms)
✓ 1 card.spec.js:4:1 › product card (359ms)The locale and timezoneId settings in the config matter for the same reason. Without them, toLocaleTimeString formats the time according to whatever machine runs the test, so a developer in New York and a CI server in UTC would produce different screenshots from identical code.
With the content stable, a real change shows up immediately. Increasing the price’s top padding from 8px to 12px:
Expected an image 334px by 134px, received 334px by 138px. 745 pixels (ratio 0.02 of all image pixels) are different.Even a 1px change (8px to 9px) fails, with the image one pixel taller and 205 pixels different.
Why loosening the threshold is the wrong fix
When a visual test flakes, the tempting fix is a tolerance. Playwright accepts maxDiffPixels or maxDiffPixelRatio, so you could let up to 2% of pixels differ and the clock problem disappears. With the clock not fixed and maxDiffPixelRatio: 0.02, the test passes. Problem solved?
Now change the price from £39.99 to £38.99, keeping the clock fixed. With the strict default, the test fails:
6 pixels (ratio 0.01 of all image pixels) are different.With maxDiffPixelRatio: 0.02, the same wrong price passes. Changing one digit of a price alters only 6 pixels, far less than the 46 the timestamp noise produced. Any tolerance large enough to swallow the noise swallows this too. (The 1px padding change still failed with the tolerance, but only because it changed the image’s height, and Playwright treats a size mismatch as a failure.)
Two lessons follow. First, stabilise the content rather than loosening the comparison. Fix the clock, seed the data, and for regions you truly can’t control, such as a third-party ad or a live map tile, use Playwright’s mask option to cover that element with a solid box in both images. Masking removes one specific area from the comparison; a tolerance removes a fraction of every area. Second, don’t rely on screenshots to check text that matters. The price should have a normal assertion, await expect(page.locator('.price')).toHaveText('£39.99'), which fails with a clear message whatever the tolerance. Visual tests are for layout, spacing, colour and overlap, the things that ordinary assertions can’t express.
A small tolerance does have a legitimate use: absorbing anti-aliasing differences of a few pixels when you can’t run everything in one container. Keep it as small as possible, and prefer the absolute maxDiffPixels over a ratio, because a ratio scales with screenshot size and grows looser the bigger the image.
Other sources of unstable renders
The clock is one source of noise among several. A checklist for a stable visual suite:
- Data: fixed fixtures, not whatever is in the staging database today. Product names of varying length change wrapping.
- Viewport and scale: a fixed viewport and
deviceScaleFactor. A Retina screenshot has four times the pixels of a standard one. - Fonts: bundled web fonts rather than system fonts, which differ between machines. Playwright waits for web fonts, but can’t make Arial on Linux look like Arial on macOS.
- Animations and media: CSS animations are disabled automatically; JavaScript animations, carousels, videos and blinking cursors aren’t, so pause or hide them.
- Lazy content: images and components that load on scroll or after a delay. Wait for them explicitly, or the screenshot races them.
- Random and live content: generated avatars, “trending now” lists, A/B test variants, relative times like “3 minutes ago”.
Keeping the suite reviewable
Every baseline update is a code change, so make it reviewable: commit baselines in the same pull request as the CSS change, and look at the diff images, not just the green tick after updating. A habit of running --update-snapshots whenever a visual test fails, without looking, turns the suite into an elaborate way of approving whatever the page looks like now.
Screenshot components and key states rather than every full page. A full-page screenshot fails for any change anywhere on the page, so every unrelated tweak means re-approving dozens of images, and reviewers stop looking. A card, a form in its error state, the site header at mobile width: each small enough that a diff shows what changed at a glance.
Interview practice
Your team’s visual test suite has 300 full-page screenshots. About 10% fail on every CI run, the team has raised the tolerance to 5%, and engineers routinely update all baselines when tests fail. A customer then reports that the checkout button is invisible in dark mode, and nobody’s visual test caught it. What would you change?
Answer
The suite has lost its value in three compounding ways. The random failures mean people no longer trust it. The 5% tolerance means it couldn’t catch small but important changes even when it runs cleanly: a button losing its contrast changes a few thousand pixels at most, well within 5% of a full page. And blanket baseline updates mean any real regression that did fail was approved into the baseline. It’s also likely dark mode was never in the suite at all.
I’d start by finding and removing the sources of instability: run tests in one pinned container image, fix the clock, locale and time zone, use fixture data, and mask or stub third-party content. Then I’d bring the tolerance back to zero or a handful of pixels, so the stability comes from deterministic renders rather than from the comparison ignoring things. I’d replace most full-page shots with component-level screenshots of the states that matter, explicitly including dark mode and the mobile breakpoints, since a checkout button in dark mode is exactly the kind of state nobody screenshots by accident. Finally, I’d make baseline updates part of review, with diff images attached to the pull request, and stop the practice of updating everything when tests fail. I’d also add a plain functional assertion that the checkout button is visible and enabled, since that’s too important to leave to pixel comparison alone.