Why Your Shopify Speed Test Score Changes Depending on Which Tool You Run

By Milan Dhameliya · · 10 min read

Run the same Shopify store through PageSpeed Insights, GTmetrix, and Shopify's own admin, and you can get three different numbers — sometimes 40 points apart — without anything being broken. Each tool measures something different: a synthetic run on a fixed connection, or real visitor data collected over weeks. Only one of those is what Google actually uses to rank you, and it usually isn't the one merchants stare at first.

Your Shopify speed test score changes depending on which tool you run because each one is measuring a different thing: a synthetic test on a fixed, throttled connection, or real data collected from actual visitors over days or weeks. They are not disagreeing by accident, and neither number is wrong. But only one of them is what Google actually uses when it decides whether your store's performance helps or hurts your rankings, and it is usually not the number merchants check first.

If you have ever run PageSpeed Insights, GTmetrix, and Shopify's own admin against the same product page and gotten three scores that don't agree — sometimes 30 or 40 points apart — this is why, and which one to trust depends on what you're trying to decide.

The four numbers you might be looking at, and what each one actually is

"Shopify speed test" isn't one measurement. Depending on where you look, you're reading one of these:

Four tools, and really only two underlying methods: a controlled, repeatable lab test, or a rolling sample of what actually happened to real people. The first two Shopify reports map onto that same split — the older Themes score is Shopify's lab test, the newer Web performance report is Shopify's field data. Merchants who've noticed their two Shopify reports don't agree with each other aren't imagining it; Shopify is running both systems in parallel, and nothing forces them to move together.

Why lab and field data disagree, mechanically

A Lighthouse lab test — whether it runs inside PageSpeed Insights, GTmetrix, or Shopify's Themes score — loads your page once, from one server location, on a fixed simulated connection (a typical mobile lab run throttles to roughly 1.6 Mbps with 150ms of round-trip latency), with an empty cache. It's deliberately artificial, and that's the point: it's repeatable, so you can change one thing on your theme and see the number move.

Field data is the opposite of controlled. It's a distribution built from real sessions: a customer in Sydney on a train's spotty 4G, a customer in Toronto on fiber with your homepage already cached from yesterday, a customer whose ad blocker strips half your tracking scripts before they ever load. CrUX and Shopify's Web performance report both report the 75th percentile of that distribution — the point where three-quarters of real visits were faster than this — precisely because an average would get flattened by all that variance and stop meaning anything.

The two also diverge for reasons that have nothing to do with network speed. Lab tests use a cold cache every run; most real visitors have your CSS and fonts already cached from a previous page. Lab tests load one fixed viewport; field data mixes phone, tablet, and desktop, and the element that counts as your Largest Contentful Paint can be a different element entirely depending on screen size. And if you run any A/B test, personalization app, or third-party script that behaves differently per visitor — common on Shopify themes carrying five or six apps — field data captures all of that variance and a lab test captures none of it, because a lab test only ever sees one configuration.

Which one Google actually uses to rank you

For Core Web Vitals as a ranking signal, Google uses field data from CrUX — specifically the 75th percentile over its trailing 28-day window — not the lab score from Lighthouse or PageSpeed Insights. A page can score 55 in a PageSpeed lab run and still pass Core Web Vitals in Search Console, or the reverse, because the lab run and the ranking signal are measuring different populations of visits under different conditions. If you're optimizing toward a lab number because it's the one displayed most prominently, you can spend real effort moving a score that has no direct bearing on how Google treats your rankings.

That doesn't make lab data useless — it's the better tool for a different job. Field data tells you there's a problem and roughly how bad it is for real visitors; it can't tell you why, because a p75 LCP of 4.1 seconds doesn't come with a stack trace. Lab data, run against a specific page under fixed conditions, is what you use to actually debug: disable an app's script, rerun the lab test, see if the number moves. Field data is the scoreboard. Lab data is the workbench.

Why new or low-traffic stores often see no score at all

Both CrUX and Shopify's Web performance report require enough real sessions in the sampling window to produce a statistically meaningful percentile. A brand-new store, a password-protected one, or a low-traffic collection page frequently shows "not enough data" instead of a Good/Moderate/Poor rating — that's not a bug, it's the dataset correctly declining to report a number it can't support. For stores in that position, a lab test is temporarily the only speed signal available, which is one of the few situations where leaning on Lighthouse over field data is the right call rather than a compromise.

A workflow that actually resolves the disagreement

Rather than treating the four numbers as competing opinions, use them for what each is actually good at:

This is also why our own free store scanner pulls Core Web Vitals from CrUX field data rather than running its own synthetic Lighthouse pass: a merchant can reasonably argue with a lab score generated under conditions they don't control, but they can't argue with what their own real visitors experienced. When a store doesn't have enough CrUX traffic yet, the scanner says so explicitly instead of guessing from a lab number, for the same reason Shopify's own Web performance report does.

Where this fits with the rest of your performance work

Reading the right number is the first step, not the whole job. If your field data is pointing at a specific page type, our CLS/LCP triage guide for app-heavy themes walks through isolating which Liquid section or app script is actually responsible. If you're earlier than that — trying to build a repeatable process rather than chase one bad score — our full store speed optimization guide and the 2026 Core Web Vitals remediation checklist cover the theme-level and app-level changes that move field data specifically, not just a lab number.

If you'd rather have someone read the four numbers for you and tell you which page is actually costing you rankings or conversions, that's what I do for Shopify stores — get in touch and I'll tell you honestly whether it's a five-minute fix or a bigger job.

The short version

Lab tools — Lighthouse, PageSpeed Insights' top score, GTmetrix — run one simulated visit under fixed conditions and are best for debugging a specific page. Field data — CrUX, PageSpeed Insights' bottom panel, and Shopify's Web performance report — is a rolling sample of real visitors and is what Google actually uses for Core Web Vitals ranking. Shopify's older Themes speed score is a lab number wearing a Shopify-branded interface; treat it the same way you'd treat GTmetrix. When the numbers disagree, believe the field data, and use the lab tools only to find out why.

Sources reviewed