Checkout query specs

Checkout query specs

The query definitions for every path. Quick answer runs one or two of these; Full analysis runs the whole battery (Step 1 → 4); an Investigate click reuses a Step 4 breakdown and goes deeper. Principles 1 and 2, the Setup notes, and the Session-tool capabilities live in SKILL.md and apply throughout.

Signal model — what earns a priority card

Absolute levels are misleading in checkout: a high cart→checkout drop and mobile converting below desktop (or vice-versa) are normal. So the board flags change, not level:

  • Change-based signals (most of them) — funnel step drops, device conversion, market/country conversion, discount patterns. Flagged only when the metric regressed vs the prior comparable window — never because the absolute level looks bad or differs from another segment or the site average.
  • Qualified active errors (the one absolute trigger) — a checkout error earns a card when it's active and genuinely fixable (see Q5), regardless of trend. Everything else must be a regression.

The comparison window

The prior comparable window is the equal-length window immediately before the current one (last 30 days → the 30 days before that). All change-based signals compare against it.

  • A regression worth flagging = the rate dropped by ≥ 3 pp absolute or ≥ 10% relative vs prior, and the segment clears the volume threshold (Step 4). Keep the current absolute numbers for context/labels, but the trigger is the drop vs prior.
  • No / low-volume prior window (new store, sparse history): change-based signals can't be computed — say so briefly and lean on the qualified error cards + the Overview. Do not fall back to absolute-level flags (that's the false-positive trap we're avoiding).
  • Seasonality caveat: the prior window is calendar-adjacent, so a window spanning a seasonal shift (e.g. a sale period) can move rates for benign reasons — note it when relevant.

Step 1 — Broad overview (current + prior windows)

  • First tell the user what you're doing, e.g. "Starting with a broad look across your checkout, comparing the last 30 days to the prior 30."
  • Fire the current-window battery (Q1–Q7) and, in parallel, the prior-window counterparts for the change-detection metrics (Q1, Q2, Q6, Q7). Q3/Q4 (payment/delivery, descriptive) and Q5 (errors, recency-based) need no prior copy.
  • No minimum session thresholds at this stage. Principle 1: skip any empty/null slice silently.

Q1 — Full funnel by depth (current + prior). Session tool, group by funnel depth, measure session count, order by sessions desc.

  • One row per depth: 0/null = no action, 1 = ATC, 2 = checkout started, 3 = payment submitted, 4 = completed. Compute drop-off in post-processing.
  • If depth group-by is unavailable: one all-sessions query measuring total sessions, per-stage event counts, and the predefined conversion rate — those intermediate counts are event proxies, so lean on total → completed.
  • If intermediate buckets are near-zero while completions are healthy → Principle 2.
  • Express checkout inflates depth-4 vs depth-3. Apple Pay, Shop Pay, etc. bypass the payment-info page, so depth-4 > depth-3 is normal — explain it, don't treat it as an error.

Q2 — Cart & order value baseline (current + prior). Session tool, filter to funnel depth ≥ 2, group by discount-applied, measure session count + median completed order value + median product quantity, order by sessions desc.

  • Depth ≥ 2 avoids null discounts on pre-checkout sessions. Gives discount rate among checkout-entering sessions and whether discounted orders differ in size.
  • Use the completion-time discount value, not checkout-start. Checkout-start discount fields significantly undercount usage.
  • Discount survivorship bias. Only checkout-reaching sessions can have a discount, so this discount rate is among checkout-entering sessions only.

Q3 — Payment method mix (current only). Only if a payment-method dimension exists (see Session tool note). Filter to completed checkouts, group by payment method names (array join, limit 20), measure session count + median completed order value, order by sessions desc. If unavailable, skip (Principle 1).

  • Completed-orders only (applies to Q4 too). Payment and delivery fields populate only on completed sessions — never imply a method was rejected because of low volume; it may just not have been chosen by completers.

Q4 — Delivery method mix (current only). Only if a delivery/shipping dimension exists. Filter to completed checkouts, group by delivery method names (array join, limit 20), measure session count + median completed order value + median shipping cost, order by sessions desc. If unavailable, skip (Principle 1).

Q5 — Active fixable checkout errors. Priority errors tool, priority-type issues on URLs containing "checkout". Then qualify each candidate — surface only the genuinely fixable ones (this mirrors tech-diagnosis's "fixable" bucket). Fetch stack trace + error-type detail for the top candidates by occurrence, and keep an error only if all hold:

  • Active — last seen within 48 hours (drop anything older, even if still flagged priority).
  • Has a stack trace — no stack trace → skip (no path to a fix).
  • Not a 4XX HTTP type — any 4XX is platform-class; skip (the HTTP type overrides frame analysis).
  • Not pure third-party — if every frame is a third-party domain, the merchant can't action it → skip; keep only errors with merchant-owned (first-party) frames.
  • Not a script error, even if first-party — exclude script-origin errors (third-party tags/pixels/widgets and opaque cross-origin "Script error.") regardless of frame ownership.

What remains = active, fixable checkout errors. These earn priority cards (with Investigate + Ignore) regardless of trend. If none qualify, there's no error card — that's fine.

Q6 — Device conversion split (current + prior). Session tool, group by device, measure session count + conversion rate, order by sessions desc. Feeds the Overview Mobile/Desktop stat and the device regression check.

Q7 — Country conversion split (current + prior). Session tool, group by country code (limit 25), measure session count + conversion rate, order by sessions desc. Feeds the market regression check.

Step 2 — Cross-reference + period-over-period

Compute current rates and the change vs the prior window, for metrics whose inputs are populated (Principle 1). depth-N = sessions reaching at least that stage.

Current rates (from Q1):

  • ATC rate = depth-1+ ÷ total · Checkout start = depth-2+ ÷ total · Checkout → payment = depth-3+ ÷ depth-2+ · Payment → completion = depth-4 ÷ depth-3+ · Overall completion = depth-4 ÷ depth-2+ (or the predefined conversion rate).
  • If intermediate events are uninstrumented (Principle 2), use cart → order (depth-4 ÷ depth-1+). depth-4 > depth-3 is expected (express checkout — see the Q1 note).

Period-over-period deltas (current − prior), for each populated metric: every funnel step rate, overall completion, each device's CVR (Q6), each market's CVR (Q7), the discount rate and discounted-vs-full conversion (Q2). A metric is a candidate signal only if its delta is a regression past the threshold above.

Cart profile (from Q2): discount rate, median order value (discounted vs full-price), median products per order — context only; not a signal unless the discount rate or discounted conversion regressed.

Step 3 — Pick signals (change-based, top 3)

From the Step 2 deltas, pick the top 3 regressions by sessions affected (fewer is fine). A normal-but-stable level is not a signal — only a drop vs the prior window is.

If a metric regressed vs the prior window… Consider this follow-up
Overall checkout completion fell Device split (Q6 current vs prior) to locate the new friction; cart-exit journeys if the cart→checkout step drove it
Checkout → payment or payment → completion fell Device or payment-method breakdown — find where the new drop concentrates
A device's conversion fell vs its own prior That device's funnel — isolate the step that moved
A market's conversion fell vs its own prior Country cross-tab — shipping/payment/localization change
Discount rate or discounted conversion moved materially Promo mix — did a campaign change the picture
  • Markets are relative. Flag a country only when its conversion regressed vs its prior. A chronically-low or chronically-0% market that didn't change is not a signal (it's likely a market they don't serve). A market with no prior baseline can't be assessed — don't flag it.
  • Discount-value gap is benign. "Discounted orders complete at a lower value" is normal coupon behaviour — never a standalone card. Only surface discounts if the rate or discounted conversion regressed.
  • The only non-regression card is a qualified active error (Q5).

Step 4 — Deeper follow-ups (confirming a regression / Investigate)

Device and country baselines are already in Step 1; these go deeper on a flagged regression or when an Investigate click needs more.

Traffic thresholds, calibrated to store volume (also gate which regressions are worth flagging):

  • High (>500K sessions/mo): segment ≥ ~0.1–0.2% of total · Mid (50K–500K): ~0.3–0.5% · Low (<50K): keep very low or skip.

  • Device funnel breakdown — filter to depth ≥ 1 (or ≥ 2 when checkout-start is reliable), group by device, measure session + completion (+ intermediate where populated). Find the step where the regression concentrates.

  • Country cross-tab — for a market whose CVR regressed, break the funnel down to see whether it's a shipping/payment block vs a UX change.

  • Payment method completion — when payment → completion regressed and payment data is populated: depth ≥ 3, group by payment method, completion per method.

  • Cart-page exit journeys — when cart → checkout regressed: user journeys tool anchored to /cart, loose mode, forward paths only, max depth 5. Forward-nav back to product = purchase uncertainty; exits to external URLs = distraction/price comparison.

  • Error detail — Q5 already qualified the fixable errors; deeper root-cause and the fix route through the tech-diagnosis handoff (see SKILL.md Investigate click).