📋 The Brief
The Problem: Competing Blind on Price
An established e-commerce brand in the sporting goods space was noticing a consistent pattern: traffic was healthy, but conversion was declining. Their CRO team had optimised the site extensively. The product quality was strong. Something else was wrong.
When they finally did manual price checks across 20 competitors, they found the answer: on 30% of their top-selling SKUs, they were being undercut by 10–20% without knowing it. Customers were adding to cart, then leaving to buy the same product cheaper elsewhere. The brand had no pricing intelligence — just a spreadsheet someone updated every few weeks.
They needed automated, daily visibility into exactly where they stood on price across every major competitor — plus actionable alerts they could act on same day, not same week.
❌ Challenge
Manual price checks across 200+ sites — 3 hours of analyst time weekly, and still always out of date
✓ Solution
Fully automated daily scrape of all 200+ sites — zero manual effort, always current
❌ Challenge
No product matching — same item listed under 8 different names across competitors, impossible to compare
✓ Solution
AI-powered product matching using title, EAN/UPC, image hash and spec similarity — 94% match accuracy
❌ Challenge
Price changes happening intraday — by the time the team responded, the window had passed
✓ Solution
Real-time alert system fires Slack + email within 15 minutes of a significant competitor price movement
📦 What Was Delivered
Six Deliverables, One Intelligence System
The project delivered a complete price intelligence suite — not just a scraper, but a full decision-support system the pricing team uses daily.
🕷️
Daily Price Crawler
Automated scraper covering 200+ competitor sites and marketplaces — runs every night, ready by 7am.
🔗
AI Product Matching
Matches your SKUs to competitor equivalents using EAN codes, titles and image similarity — 94% accuracy.
🔔
Real-Time Alerts
Fires Slack and email alerts within 15 minutes when a tracked competitor drops below a configured threshold.
📊
Pricing Dashboard
Visual dashboard showing your position vs market for every SKU — sortable by gap, category and competitor.
📈
Historical Trend Charts
90-day price history for every product and competitor — identify seasonal patterns and competitive behaviour.
📤
Daily Digest Report
Automated morning email digest: overnight movers, undercut alerts and repricing recommendations.
🏗️ Architecture
How the Pipeline Works
The system runs as a scheduled pipeline overnight and as an on-demand real-time monitor for high-priority SKUs throughout the day.
🌐
200+ Sites
Retailers & marketplaces
→
🤖
Scrapy + Selenium
Dynamic sites handled
→
🔗
Product Matcher
EAN + AI similarity
→
📊
Price DB
PostgreSQL time series
→
🔔
Alert Engine
Slack + email <15min
Product matching was the hardest problem. The same running shoe can be listed as "Nike Air Zoom Pegasus 40" on one site and "Air Zoom Peg 40 Men's Road Shoe" on another. We built a multi-signal matching system — first trying EAN/UPC barcode match, then falling back to title fuzzy match + image perceptual hash + specification comparison. The 94% match accuracy was validated against a manually-curated ground truth dataset of 5,000 product pairs.
⚙️ Tech Stack
Technologies Used
Python 3.11
Scrapy
Selenium / Playwright
PostgreSQL (TimescaleDB)
Redis
OpenAI (product matching)
Apache Airflow
React (dashboard)
Recharts
Slack API
AWS EC2 + S3
SendGrid
📅 Timeline
Live in 6 Weeks
1
Week 1
Source Audit & SKU Mapping
Audited all 200+ competitor sites for scrapeability. Received the client's full SKU catalogue (50K+ products). Defined priority tiers by sales volume for phased rollout.
2
Week 2–3
Crawlers + Product Matcher Built
Core scrapers for top 50 sites built and running. Product matching pipeline live — EAN lookup first, AI fallback for unmatched. First 10K product pairs validated at 91% accuracy.
3
Week 4
Alert Engine & Dashboard
Alert thresholds configured with the pricing team. Slack and email delivery live. React dashboard deployed — pricing team could see their market position for the first time.
4
Week 5–6
Full Scale & Historical Backfill
All 200+ sites running. 90-day historical backfill completed for trend charts. Daily digest email live. Match accuracy tuned to 94% after 2 weeks of feedback from the pricing team.
📈 Results
Repricing Strategy Changed Within the First Week
Within 3 days of the dashboard going live, the pricing team had already identified 47 SKUs where they were materially overpriced versus the market. They repriced 23 of them that same week. The following Monday they saw conversion on those products jump by an average of 18%.
200+
Competitor sites tracked automatically, every day
18%
Average conversion uplift on repriced products in week one
3hrs
Weekly analyst time saved — now zero manual price checking
The most valuable outcome wasn't just the repricing — it was the margin opportunities the system surfaced that they'd been leaving on the table. On several premium products, they were the market low by a significant margin. They raised prices on 11 SKUs without losing conversion, recovering margin they hadn't realised they were giving away.