The widget as a product — not its answer accuracy (covered in amber.html / adversarial.html). Run 2026-07-14, live via claude-in-chrome + JS inspection, across LMT · GitHub · Ryanair · CSDD · e-veselība · DataMed. Test matrix designed by a 7-lens panel (41→~20 checks).
Objectively verified via JS: on "how do I create a new issue?", exactly one page element carried the highlight — outline: rgb(99,102,241) solid 2px; outline-offset: 2px — and it was the BUTTON "New issue" the answer named (not an ancestor wrapper, not a wrong element). The outline is inline on the target element itself, so it tracks scroll for free. Across the session it also correctly boxed the right nav section on LMT/CSDD/e-veselība. (One earlier miss: it boxed a public support number as "yours" — that was a PII-classification error, not a positioning error.)
Legible dark bubbles; LV/RU/EN + mixed script all render correctly. Good scope handling ("this is a telecom provider, it doesn't book doctor's appointments"). The "Walk me through it" multi-step mode works — advances 1/N → Next and highlights a real, present element each step (seen on DataMed, e-veselība, LMT). Caveat (untested): the matrix flags a "the page keeps changing under me" cancel path that could mis-fire on dynamic SPAs.
Now instrumented (in-widget MutationObserver, method-unbiased): submit→answer-rendered measured at 6.3s and 5.5s for two simple GitHub queries — millisecond timestamps, independent of my polling (validated: the ~28s gap between the two queries was my own MCP overhead and is correctly excluded). So real user-perceived latency is ~5.5–6.3s for simple queries. The earlier coarse "~5–8s / up to 14s" was method-inflated by my screenshot cadence but broadly consistent — the 14s was a heavier/longer generation, not sampled here. A "thinking.." state + stop control show throughout, so it never looks frozen. A Playwright+CDP run could further split network-vs-render latency if that decomposition is ever wanted.
Inconsistent, and that's the problem. GitHub's and Ryanair's widgets each showed only their own conversation — but earlier a query asked on Latvija.lv appeared in the widgets on e-veselība and EDS. So it's neither cleanly per-site nor cleanly global. Cross-site history bleed means one site's questions can surface on another site's widget — a real data-hygiene / privacy wrinkle on top of the main PII finding. Needs the harness to pin down the exact state key.
The panel repositions unpredictably across sites and on scroll (centre, left, right), and regularly overlaps page content and its own highlight tooltips overlap targets. There's a drag handle, but default placement is inconsistent enough to be distracting.
Corrected finding: the widget must be turned on per-tab (extension icon → Allow) on every new tab — it's off by default. Not a CSP bug (my earlier claim was wrong). But enable-per-tab-per-site is real funnel friction: until you enable it, a new tab silently has no widget, with no prompt inviting you to.
Arguably too deterministic — Ryanair returned byte-identical "Click 'Manage my bookings'…" answers for three different questions. Consistent, but it reads as templated rather than genuinely reasoning about each query.
A screenshot capture timed out once on the heavy e-veselība portal. ⚠️ Caveat: this was an MCP capture timeout — it may be a measurement artifact of Claude driving Chrome, not a real widget/page freeze. Unconfirmed; do not treat as a widget defect. Separately (and independent of measurement): long-lived portal sessions (LMT, gov) expire underneath the widget — not the widget's fault, but it keeps answering against a logged-out page context, so guidance can drift stale.
The widget runs fully isolated — shadow DOM + its API call originates from the extension context — so its network, console, and DOM are invisible to page-level inspection. Good: zero conflict with or pollution of the host page. Trade-off: the data egress isn't page-auditable or page-CSP-governed, and reliability can't be instrumented from the page (hence the Playwright+CDP harness).
Not verifiable with claude-in-chrome: resize_window resized the OS window but the screenshot capture resolution is decoupled, so mobile/tablet reflow couldn't be observed; and keyboard/focus/ARIA live inside the widget's (likely closed) shadow root. Both are first-class jobs for the Playwright harness (setViewportSize + CDP a11y tree).
1. Cross-tab state hygiene — make conversation state deterministically per-site (or explicitly global with the user's knowledge); stop one site's history surfacing on another.
2. Latency — get typical answers under ~3s or stream tokens; 5–14s undercuts the "quick help" promise.
3. Positioning — a stable, predictable default anchor that doesn't overlap content or its own tooltips.
4. Onboarding — a first-run nudge so users discover the per-tab enable instead of landing on a silent no-widget tab.
5. Heavy-page resilience — the freeze on e-veselība (a flagship target) needs a repro + a capture/time budget.
Method note: highlight accuracy is JS-proven; latency/positioning/cross-tab/determinism/robustness are live-observed across many queries this session; responsive + a11y + byte-level payload + precise latency are deferred to Openwalk_Reliability_Harness (Playwright+CDP, scaffolded, needs the extension build from Nick). Companions: amber.html · adversarial.html · Latvia targets.