Zero flicker in our A/B testing benchmark.
Cascayd recorded 0 ms of flicker on desktop and throttled mobile. Compare the full speed and rendering results with Intelligems and Visually.
First-party results from Cascayd. These runs used styling-only Cascayd variants, not every type of experiment.
Read the methodology0 ms
Desktop median flicker
0 ms
Mobile median flicker
Desktop comparison
Compare paint, flicker and variant-application timings across the three tested tools.
| Metric | Cascayd | Intelligems | Visually |
|---|---|---|---|
| Variant loads | 58 | 29 | 40 |
| First paint | 790 ms | 910 ms | 746 ms |
| Largest contentful paint (LCP) | 870 ms | 970 ms | 842 ms |
| Median flicker | 0 ms | 22 ms | 103 ms |
| Variant loads with flicker over 16 ms | 0 % | 62 % | 83 % |
| Worst flicker | 0 ms | 95 ms | 401 ms |
| Variant applied | 344 ms | 944 ms | 831 ms |
Timing values are in milliseconds. Lower is better for timing and flicker; variant-load counts show sample size.
Mobile comparison
Network and CPU throttling simulated extremely constrained mobile conditions. These timings are not typical mobile browsing speeds.
| Metric | Cascayd | Intelligems | Visually |
|---|---|---|---|
| Variant loads | 40 | 17 | 40 |
| First paint | 1668 ms | 3256 ms | 1576 ms |
| Largest contentful paint (LCP) | 2524 ms | 4200 ms | 2486 ms |
| Median flicker | 0 ms | 317 ms | 53 ms |
| Variant loads with flicker over 16 ms | 0 % | 100 % | 90 % |
| Worst flicker | 0 ms | 1067 ms | 491 ms |
| Variant applied | 708 ms | 3626 ms | 1651 ms |
Timing values are in milliseconds. Lower is better for timing and flicker; variant-load counts show sample size.
What these results can tell you.
Use this comparison to assess storefront rendering, then validate the same experiment on your own store.
In these results, Cascayd has the lowest reported variant-application timing and zero recorded median and worst flicker in both environments. Visually has slightly lower first-paint and LCP timings in both.
No no-script/control baseline was supplied, so script overhead or Core Web Vitals improvement cannot be inferred.
Why the experiment scope matters
The Cascayd team reports that these variants used CSS styling changes only. The zero-flicker observation belongs to those tested variants, not to every possible experiment.
How to read this benchmark
Source: Cascayd research, with first-party desktop and throttled-mobile results supplied by the Cascayd team. Variant-load counts differ across tools and environments, and mobile network and CPU throttling simulated extremely constrained conditions.
Exact throttle settings, device, browser, hardware, tool versions, test date, URLs, raw runs, full configurations and timing aggregation method were not supplied. Competitor settings and equivalent variant changes were not supplied, so this is not proof of matched feature scope or statistical significance.
- Variant loads
- Observed variant-load count.
- First paint
- Runner-reported first-paint timing, not relabeled first contentful paint.
- LCP
- Largest contentful paint.
- Flicker
- Runner-reported flicker; detection method was not supplied. Median and worst are kept distinct, and the >16 ms threshold is reported as supplied.
- Variant applied
- Runner-reported timing with its start point not supplied. Do not add it to paint timings.
Related reading: Shopify A/B testing guide and Cascayd’s testing service.
Benchmark questions, answered.
What these observations mean, and where their limits are.
Which tool applied variants fastest in this benchmark?
Cascayd had the lowest reported variant-application timing in both environments: 344 ms on desktop and 708 ms on mobile. This describes the tested runs, not every store or experiment.
Does zero measured flicker mean Cascayd can never flicker?
No. In these desktop and mobile runs, Cascayd recorded 0 ms median and worst flicker and 0% of variant loads with flicker over 16 ms; different stores, scripts, networks and experiments can behave differently.
Was Cascayd fastest on every page-speed metric?
No. Visually had slightly lower first-paint and largest contentful paint (LCP) timings in both environments; Cascayd’s earlier variant application is a different metric.
Why did Cascayd use CSS styling changes?
The Cascayd variants in this benchmark were styling-only changes made with CSS. That scope matters when interpreting the flicker results: CSS changes alone are not a guarantee of zero flicker, and these results do not cover every experiment type.
How was mobile tested?
Mobile was tested with network and CPU throttling to simulate extremely constrained conditions. The exact throttle settings and device details were not supplied, so these timings should not be treated as typical mobile browsing speeds.
Is this an independent or fully reproducible benchmark?
These are first-party results supplied by Cascayd, without a claim of independent verification, and with unequal variant-load counts. Raw runs, test date, URLs, versions, browser, hardware, full test configuration and timing aggregation details were not published, so treat these as directional observations, not statistical proof or a guarantee.
See what we would test first.
Get a complimentary Shopify friction audit and see the first experiments we would run for your store.