← Run 20260904-150307-2b65f658
Reconcile conflicting Meta and GA4 performance numbers
task analytics-source-conflictSep 4, 2026

Answers & verdicts

Gemini 3.8 Flash high effort

Task passed
Task the exact prompt the model was given — assignment, source material, and skill guidance
A founder asks: 'Meta says our ads printed money last month, our analyst says GA4 disagrees. What is actually true, and what should we do?' Using only the supplied reports and tracking notes, deliver: (1) the Meta totals and the blended CPA and ROAS under each source, arithmetic shown; (2) an explanation of why the two sources disagree, grounded strictly in the documented causes; (3) what can honestly be concluded about Meta-attributed purchases, including the defensible range; and (4) a recommendation that separates conclusions robust to the measurement disagreement from conclusions that depend on which source you believe, plus the next measurement step. Do not average the sources into a single number, do not invent causes the notes do not document, and do not treat either source's attribution as proof of causation. --- ## Supplied source: meta-platform-report.csv source,campaign,window,spend_usd,reported_conversions,reported_revenue_usd,attribution_setting meta_ads_manager,ARC-Prospecting,Oct 1-28,1500.00,38,3420.00,"7-day click, 1-day view" meta_ads_manager,ARC-Retargeting,Oct 1-28,900.00,22,1980.00,"7-day click, 1-day view" --- ## Supplied source: ga4-report.csv channel,window,sessions,purchases,revenue_usd,attribution_model Paid Social (Meta),Oct 1-28,1840,36,3240.00,"last non-direct click, 30-day" Organic Search,Oct 1-28,2210,41,3690.00,"last non-direct click, 30-day" Direct,Oct 1-28,960,35,3150.00,"last non-direct click, 30-day" --- ## Supplied source: tracking-notes.md # Tracking notes for the October reconciliation (analyst handoff) - Meta Ads Manager counts view-through conversions under its "1-day view" setting; GA4 never counts view-through under any setting. - Attribution differs by construction: Meta claims a conversion on a 7-day click / 1-day view window against its own ads; GA4 assigns each purchase to one channel by last non-direct click with a 30-day lookback. - Auto-tagging was broken on Oct 12-13; an estimated 8 Meta-referred sessions landed without UTMs during the outage and were classified as Direct in GA4. - The store platform (source of truth for orders) recorded 118 completed purchases in the window. GA4 recorded 112 site-wide. Summing every ad platform's self-reported conversions (Meta plus the search and affiliate platforms) gives 149 claimed conversions. - No other diagnostics are available for this window.
Answer the model's full response$0.026 · 42.8s · 6.9k tok
### 1. Meta Totals, Blended CPA, and ROAS Under Each Source #### Meta Ads Manager (Self-Reported) * **Total Spend:** $\$1,500.00 + \$900.00 = \$2,400.00$ * **Total Conversions:** $38 + 22 = 60$ * **Total Revenue:** $\$3,420.00 + \$1,980.00 = \$5,400.00$ * **Blended CPA:** $$\text{CPA} = \frac{\text{Total Spend}}{\text{Total Conversions}} = \frac{\$2,400.00}{60} = \$40.00$$ * **Blended ROAS:** $$\text{ROAS} = \frac{\text{Total Revenue}}{\text{Total Spend}} = \frac{\$5,400.00}{\$2,400.00} = 2.25\times \text{ (or } 225\%)$$ #### Google Analytics 4 (GA4 Paid Social) * **Total Spend:** $\$2,400.00$ (incurred across Meta campaigns) * **Attributed Purchases:** $36$ * **Attributed Revenue:** $\$3,240.00$ * **Blended CPA:** $$\text{CPA} = \frac{\text{Total Spend}}{\text{Attributed Purchases}} = \frac{\$2,400.00}{36} \approx \$66.67$$ * **Blended ROAS:** $$\text{ROAS} = \frac{\text{Attributed Revenue}}{\text{Total Spend}} = \frac{\$3,240.00}{\$2,400.00} = 1.35\times \text{ (or } 135\%)$$ --- ### 2. Documented Causes for the Disagreement Based strictly on the tracking notes, the two sources diverge due to four specific structural and operational factors: 1. **View-Through Conversion Tracking:** Meta Ads Manager includes view-through conversions via its "1-day view" window. GA4 never tracks or credits view-through conversions under any configuration. 2. **Attribution Logic and Lookback Windows:** Meta uses a siloed model claiming credit if a user interacted within a 7-day click or 1-day view window, evaluating conversions only against its own ads. GA4 uses a cross-channel "last non-direct click" model with a 30-day lookback window that awards 100% of the conversion to only one channel. If a buyer clicked a Meta ad but later clicked an Organic Search result, GA4 credits Organic Search, whereas Meta still claims the purchase. 3. **Broken Auto-Tagging Outage (Oct 12–13):** Auto-tagging failed for two days, causing an estimated 8 Meta-referred sessions to arrive without UTM parameters. These were misclassified in GA4 as **Direct** instead of **Paid Social (Meta)**. 4. **Platform Double-Counting vs. Site-Wide Measurement Gaps:** Ad platforms collectively claim overlapping orders: summing all platform reports (Meta, search, affiliate) totals 149 claimed conversions, even though the store platform recorded only 118 real orders. Furthermore, GA4 recorded 112 total site-wide purchases, meaning it missed 6 completed store orders entirely. --- ### 3. Defensible Range and Honest Conclusions on Meta Attribution * **Defensible Range of Attributed Purchases:** **36 to 60 purchases** (associated with **$\$3,240.00$ to $\$5,400.00$** in attributed revenue). * **Floor (36 purchases):** Documented by GA4 under strict last non-direct click attribution. If any of the 8 untagged Meta sessions on Oct 12–13 resulted in a purchase, the true last-click floor lies between **36 and 44 purchases**. * **Ceiling (60 purchases):** Documented by Meta Ads Manager under 7-day click / 1-day view attribution. * **Attribution vs. Causation:** Neither number represents *incrementality* (purchases that occurred strictly *because* of the ads): * Meta’s figure of 60 includes view-throughs and retargeting (22 conversions) that likely capture shoppers who were already intending to buy. * GA4’s figure of 36 strips credit from Meta whenever another non-direct channel was the final click, obscuring Meta’s role in opening or assisting the path to purchase. --- ### 4. Recommendations and Next Steps #### Robust Conclusions (True Regardless of Measurement Model) * Meta generated substantial genuine interest: 1,840 tracked sessions and at least 36 confirmed last-click purchases. * Meta generated more top-line revenue than the $\$2,400.00$ spent under both reporting lenses ($1.35\times$ in GA4; $2.25\times$ in Meta). * Platform self-reporting is inherently non-deduplicated across vendors: the 149 platform-claimed conversions exceed the store’s actual 118 orders by 31 orders. Meta’s figures cannot be evaluated in a silo without acknowledging shared touchpoints. * Site-level tracking leaks exist: GA4 dropped 6 actual store orders (112 vs. 118), and the Oct 12–13 outage artificially deflated GA4’s Paid Social session and conversion counts. #### Source-Dependent Conclusions (Belief Required) * **Whether the ads "printed money":** If you believe Meta ($2.25\times$ ROAS, $\$40.00$ CPA), the campaign was highly profitable. If you believe GA4 ($1.35\times$ ROAS, $\$66.67$ CPA), the ads produced modest returns that may be near or below break-even once product margins and overhead are factored in. * **The value of ad views:** Believing Meta assumes 1-day view impressions actively drove purchasing decisions; believing GA4 assumes ad impressions that did not receive a click had zero commercial impact. * **Retargeting efficiency:** Meta attributes 22 purchases to retargeting; GA4's cross-channel view likely reallocates a portion of those to Organic Search or Direct if those channels captured the final non-direct click. #### Next Measurement Step 1. **Infrastructure Remediation:** Reconcile the discrepancy between the store’s 118 orders and GA4’s 112 orders, and audit auto-tagging/UTM pipelines to ensure Meta traffic does not fall into Direct. 2. **Causal Incrementality Testing:** Because neither observational attribution model measures true cause-and-effect, run a **controlled incrementality test** (such as a geo-lift test or conversion holdout test on Meta). This will measure whether turning Meta ads on or off causes a net change in total store orders (out of the 118 baseline), rather than debating which tracking tool gets the credit.
Score the judge's verdict, criterion by criterion9/9 criteria passed
RequirementVerdictEvidence
platform_oracle
Meta platform totals are correct: $2,400 spend, 60 reported conversions, $5,400 reported revenue, blended CPA of $40.00, and ROAS of 2.25.
✓Response computes $2,400 spend, 60 conversions, $5,400 revenue, $40.00 CPA, 2.25x ROAS exactly as required.
ga4_oracle
GA4 figures for Meta are correct: 36 purchases, $3,240 revenue, CPA of about $66.67, and ROAS of 1.35 against the same $2,400 spend.
✓Response states 36 purchases, $3,240 revenue, CPA ~$66.67, ROAS 1.35x against $2,400 spend.
no_single_truth
The response never declares one reconciled 'true' Meta conversion count and never averages or splits the difference between 36 and 60 to produce one.
✓The response presents a range (36-60) and explicitly separates 'robust' vs 'source-dependent' conclusions rather than averaging or declaring one true number.
defensible_range
Meta-attributed purchases are bounded between 36 and 60 (optionally noting the 8 lost-UTM sessions push GA4's floor upward), presented as a range rather than a point estimate.
✓It states 'Defensible Range of Attributed Purchases: 36 to 60 purchases' and notes the 8 lost-UTM sessions could push the floor to 36-44.
documented_causes_only
The disagreement is explained using only the documented causes — view-through counting, the differing attribution windows/models, the Oct 12-13 lost-UTM sessions classified as Direct, and cross-platform over-claiming (149 claimed versus 118 store orders) — with no invented causes such as privacy signal loss, bot traffic, or tracking-pixel bugs.
✓The four causes listed (view-through, attribution windows, Oct 12-13 UTM outage, 149 vs 118 over-claiming) are all documented in the notes; no invented causes like bots or pixel bugs appear.
store_anchor
The store platform's 118 completed purchases is treated as the order-count source of truth, and the 149 summed platform claims are identified as over-claiming against it.
✓The response states 'summing all platform reports... totals 149 claimed conversions, even though the store platform recorded only 118 real orders' and later calls this 'non-deduplicated' over-claiming exceeding orders 'by 31 orders.'
no_causal_overclaim
Neither source's attribution is presented as proof that Meta ads caused the purchases; incrementality is acknowledged as unmeasured.
✓The response explicitly states 'Neither number represents *incrementality*... purchases that occurred strictly because of the ads' and recommends a causal test since 'neither observational attribution model measures true cause-and-effect.'
robust_vs_dependent
The recommendation explicitly separates at least one conclusion that holds under both sources from at least one that depends on the source chosen, and any spend decision is framed conditionally rather than as settled.
✓The response has explicit 'Robust Conclusions' and 'Source-Dependent Conclusions' sections, e.g. 'Whether the ads printed money' is framed conditionally on which source you believe.
grounded_next_step
At least one concrete next measurement step comes from the notes (repairing auto-tagging/UTMs, reconciling platform claims against store orders, or aligning attribution settings) rather than from invented tooling.
✓It recommends 'Reconcile the discrepancy between the store's 118 orders and GA4's 112 orders, and audit auto-tagging/UTM pipelines,' both grounded in the notes.