CDP Ozon Auto-Surfer Collector

Uses the Cloud Bridge CDP tunnel to control a real Chrome browser open on Ozon.ru, automatically hopping between product pages and extracting data.

Architecture

Agent → HTTP :9229 → WS :9228 → Desktop Electron → Extension WS → chrome.debugger → Ozon page

API Base: http://127.0.0.1:9229
API Key: Bing2026Cao$$$
Key endpoints: /cdp/attach, /cdp/detach, /cdp/evaluate, /cdp/navigate, /cdp/send

Script Location

/home/ubuntu/ozon_collector.py

How It Works

Finds Ozon page targets via Target.getTargets
Attaches via CDP, extracts product data from DOM
Finds all /product/ links on the current page
Picks a random unvisited link and navigates to it
If no fresh links → searches a random keyword from a built-in list
Repeats with 30-90s random delays between cycles

Data Extraction (JS expression in `collect_page()`)

Ozon uses obfuscated CSS classes (.a9y6, .kp6, etc.) that change frequently. Do NOT rely on class selectors. Instead:

Price: Parse document.body.innerText lines containing ₽ (U+8381). Skip promo line "Товары за 1₽", take 2nd match as card price.
Rating: Regex (\d+\.\d+)\s*[•·]\s*(\d+\s*отзыв) from body text
Seller: Regex Магазин[\s\S]*?\n([\w\s]+?)\n.*О магазине or fallback to a[href*="seller"]
SKU: Extract from URL pattern /product/...-(\d+)/
Stock: Regex (\d+[\s\d]*\s*(?:шт|единиц)\s*осталось)
Images: Filter img[src*="cdn"] containing "ozon" or "wbcdn", exclude "payments-cdn" and "marketing-api"
Delivery: Regex Доставим\s*(с\s*\d+[^\n]*)

Output Format

JSONL file at /home/ubuntu/ozon_collection.jsonl:

{
  "timestamp": "2026-05-19T06:51:36.123456",
  "cycle": 42,
  "run_id": "20260519_012342",
  "title": "...",
  "url": "https://www.ozon.ru/product/...",
  "h1": "Product name",
  "price": "85 ₽",
  "allPrices": ["Товары за 1₽", "85 ₽", "86 ₽", "2 146 ₽"],
  "rating": "4.8 • 746 отзыв",
  "ratingValue": "4.8",
  "reviewCount": "746 отзыв",
  "seller": "",
  "sku": "4267896148",
  "imgs": ["https://..."],
  "stock": "99 единиц осталось",
  "delivery": "с 4 июня"
}

Pitfalls & Lessons Learned

No class selectors for Ozon — They obfuscate and rotate CSS classes. Always use text-based regex parsing from body.innerText.
Price parsing order matters — First ₽ line is often promo "Товары за 1₽". Real price is 2nd+ line.
Duplicate tabs — Target.getTargets returns both the Ozon homepage and product sub-tab. Deduplicate by URL (strip query params).
CDP attach follows openerId — When attaching to a child tab, CDP may return the parent's targetId. Use /cdp/navigate to go directly to the desired URL instead.
Page load timing — After navigate, wait 3-6 seconds (randomized) before extracting data. Too short → empty h1/price.
Seller extraction unreliable — Ozon renders seller info via React hydration; regex may miss it. The a[href*="seller"] fallback also often fails.
Images need filtering — Raw img selector picks up payment icons and marketing banners. Filter out payments-cdn and marketing-api URLs.
Python output buffering — Run with python3 -u for unbuffered output in background processes.

Running

# Start
python3 -u /home/ubuntu/ozon_collector.py &

# Check status
ps aux | grep ozon_collector | grep -v grep
wc -l /home/ubuntu/ozon_collection.jsonl

# Stop
pkill -f ozon_collector.py

Monitoring Queries

# Count unique SKUs
grep -oP '"sku": "\K[^"]+' /home/ubuntu/ozon_collection.jsonl | sort -u | wc -l

# Latest 5 records
tail -5 /home/ubuntu/ozon_collection.jsonl | python3 -c "
import sys,json
for l in sys.stdin:
    d=json.loads(l)
    print(f\"{d['timestamp'][:19]} | SKU={d.get('sku','')} | {d.get('h1','')[:55]} | {d.get('price','')} | ⭐{d.get('ratingValue','')}\")"

4.3 KiB Raw Blame History Unescape Escape