What manual cross-browser testing really costs agencies

Web agencies rarely live off a single project. The normal case is a portfolio: twenty, thirty, fifty client websites that are looked after, updated and kept running. Part of that care is a piece of work that appears as its own line item in hardly any quote and yet comes up regularly: checking after every change that the site still looks everywhere the way it should. This invisible work has a very visible price — you just have to calculate it once.

A worked example, deliberately set conservatively: a client website has eight page types that cover the layout risk — home page, two content templates, services overview, contact form, blog article, legal pages, error page. Checking is done in three browsers and three device classes. That gives 72 views per website and per pass. If you need only half a minute per view — open, scroll, look — you are busy for over half an hour. Per website. Per pass.

Now this figure multiplies in two directions. First across the portfolio: with thirty websites under management, the half-hour turns into a whole working day of nothing but looking. Second over time: content management systems and their plugins want updating regularly, often monthly, and for security reasons usually without delay. Every one of these updates can shift the layout — and strictly speaking ought to trigger a check pass. Add the two together and you quickly arrive at several person-days a month spent solely on repeated visual inspection.

In reality this bill is rarely paid — and that is precisely the problem. Very few maintenance contracts list browser testing as a line item. So the work either lands unpaid in the agency's margin, or it is quietly cut back: you check only the one browser, only the home page, only when there is time. Both are business decisions, even if they are seldom called that. The first costs profit, the second buys risk.

You can set up the same calculation from the budget side: how much checking actually fits into a typical maintenance contract? An allowance of two hours a month is usually exhausted by updates, small text changes and occasional support before the first browser check has even begun. Visual inspection is therefore not competing with idle time but with other, more visible work — and it loses that competition structurally, because its absence is not noticed by anyone at first. That is exactly why appealing to discipline helps little. A check that is meant to happen reliably must either stand in the contract as its own paid item or become so small in effort that it happens in passing. In practice it usually takes both: the line item so that the work is paid for, and the automation so that it fits inside the line item.

On top of that comes a factor that hourly rates do not capture: repeated visual inspection is tiring, and tired eyes miss things. Anyone looking at the same home page for the fortieth time sees what they expect to see — not the three pixels of shift in the header or the button that has recently started wrapping on narrow screens. The reliability of the check declines precisely where its scope grows. People are excellent at judging a layout critically once, and poor at judging the same layout critically, month after month, unchanged.

And what does it cost the other way round, not to check? The most expensive variant is the most common: the client spots the fault first. A broken contact form on the iPhone, an overlapping menu in Firefox — unnoticed for weeks, until an irritated email arrives. The actual damage is rarely the repair effort but the loss of trust: the maintenance contract that was supposed to prevent exactly such cases has visibly failed. On top of that come firefighting jobs at the worst possible moment, which displace planned work and generate follow-on costs of their own.

This is where automation changes the arithmetic — not because software looks more thoroughly than a human, but because it parallelises the creation of the views and takes over the comparison. Screenshot services produce the complete set of browser and device views from a URL in a short time; with ScanU such a pass takes around 30 seconds and covers Chrome, Firefox and Safari in mobile, tablet and desktop views. ScanU explains the course of such a check run on its website. Half a day of clicking through becomes a quarter of an hour of reviewing reports.

The decisive lever is not the individual screenshot but the comparison. If a reference state is saved after every approved change, nobody has to assess 72 views at the next update — only the places where something has changed. The machine answers the question “is anything different from before?”, the human the question “is that bad?”. That is a considerably better division of labour than the previous one, in which the human had to answer both questions at once, view by view.

An honest calculation includes the tool costs. ScanU is tiered by credits, projects and device count: the free tier covers 500 credits a month with one project, the Pro tier 3,000 credits and five projects for 19 euros, Pro+ 10,000 credits and ten projects for 29 euros, the Max tier 50,000 credits and twenty projects for 49 euros a month — the details are listed in ScanU's pricing overview. Whether that pays off is something every agency can work out with its own hourly rate: at most agencies the monthly cost of the largest tier amounts to less than a single hour of billable work.

The project and credit tiering enforces a useful discipline in the process: which client projects need close-meshed checking, and which get by with a monthly pass? An online shop under active development justifies more check runs than a static business-card website that is touched twice a year. Anyone who sorts their portfolio this way once has, as a by-product, a basis for the next maintenance contract negotiation — the scope of checking can suddenly be quantified and stated as a service rather than as a silent matter of course.

How an introduction across an entire portfolio can concretely proceed can be sketched in four steps. First, take stock: all the websites under management into a list, noting for each project how often something actually changes there and which page types exist. Second, classify: projects with frequent changes or ordering flows get a close-meshed check profile, quiet legacy sites a monthly one. Third, create baselines — and specifically on the basis of a state that somebody has checked and approved, not simply the current state, which may already contain an unnoticed fault. Fourth, hook the check into the existing update routine, so that after every plugin or CMS update new views are produced automatically and compared against the baseline. Realistically that stretches over a few weeks if the portfolio comprises thirty projects — not because the technology is slow, but because creating clean reference states for each project demands a short but genuine quality decision that nobody should wave through in bulk.

One side effect is documentary in nature: shareable check reports can be built into client communication. Instead of assuring the client that “we tested everything”, the agency can show what was checked, when and in which browsers. That does not replace a relationship of trust, but it makes the invisible work visible for the first time — and visible work is easier to get paid for than invisible work.

What remains open in many agencies is the question of who actually looks at the reports. For runs with a specific trigger the answer is simple: whoever installed the update assesses the reported deviations — that person can classify them fastest. For the regular runs without a specific trigger, a rotating duty has proved itself, of the kind many teams know from the support inbox: one named person per week, with the mandate to record anything conspicuous directly as a task rather than merely taking note of it. What matters is less the specific arrangement than its existence. Reports that belong to nobody become background noise within a few weeks — and then the entire automation is ineffective, however reliably it runs technically.

It should be just as clear what automation does not achieve. Screenshot comparisons check presentation, not function: whether the form actually sends, whether the payment goes through, whether the search finds anything — only functional tests or manual work clarify that. Assessing the reported deviations also remains human work, and false alarms caused by advertising banners, carousels or changing content cost time of their own until the test environments are cleanly set up. Anyone introducing automation trades routine work for set-up and maintenance work — the trade is usually worth it, but it is not free.

Realistic planning therefore includes this: the first weeks are the most strenuous. Portfolios are heterogeneous — one client site carries a news carousel, the next changing advertising banners, the third a booking widget with daily dates. Each of these sources initially produces false alarms until the check settings for each project are right. Anyone who does not budget for that experiences the introduction as a disappointment and breaks it off before the benefit becomes visible. It makes more sense to treat the changeover as a small internal project: with someone responsible, a time budget and the declared goal of having a calm, trustworthy check signal for every project after a few weeks. The provider collects answers to typical set-up questions in the frequently asked questions about ScanU.

And the arithmetic is not the same for every agency. Anyone looking after two websites that rarely change can check by hand in an hour what automation would barely do faster. The mathematics tips with volume: with every additional project, every additional update cycle, every additional browser in the test list. There is a point at which manual work is the more honest choice — and one beyond which it only looks that way.

A controlled experiment suggests itself as a way in: map a single, changeable client project onto the free tier, run it for a month in parallel with existing practice and then compare — deviations found, time consumed, false alarms. An English-language overview of the service is given by ScanU's home page. After that month there is a figure of your own on the table instead of an estimate.

In the end it is a simple decision about the form of the costs. Checking has to happen one way or another — the only question is whether an agency pays recurring working hours whose reliability declines with repetition, or builds structure once and bears ongoing tool costs. Both are legitimate. Only the third variant, quietly letting the checking lapse, is not a saving. It is a loan against the clients' trust, and it falls due eventually.