--- name: atomk-api-export category: cross-border-ecommerce description: Extract product and listing data from AtomK via its undocumented REST API (JWT auth), then import into Miaoshou ERP collection box or other platforms. --- # AtomK API Export & Integration Pipeline ## Overview AtomK (`atomlisting.com`) exposes an internal REST API that can be called directly with a JWT Bearer token. This is dramatically more efficient than browser-scraping for bulk export of claimed products and AI-generated listings. ## Authentication The API uses JWT Bearer tokens stored in browser `localStorage` after login. There is a second authentication method specifically for ERP integration, but agent nodes should only use Method 1 (JWT token via login). ### Extract Token (DrissionPage) ```python token = page.run_js('return localStorage.getItem("atomk_token");') headers = { 'Authorization': f'Bearer {token}', 'Content-Type': 'application/json', 'Accept': 'application/json', } ``` **Token payload fields**: `user_id`, `username`, `email`, `role`, `system_code`, `exp` ## Complete API Endpoint List (v1 — 2025-05 Upgrade) ### 认证 Auth | 方法 | 端点 | 说明 | |------|------|------| | POST | /api/v1/auth/login | 登录获取JWT | | POST | /api/v1/auth/refresh | 刷新token | | GET | /api/v1/auth/verify-token | 验证token有效性 | | GET | /api/v1/auth/me | 获取当前用户信息 | ### 产品 Products | 方法 | 端点 | 说明 | |------|------|------| | GET | /api/v1/products/remote | 远程产品池列表 | | GET | /api/v1/products/remote/random | 随机获取远程产品 | | GET | /api/v1/products/remote/by-code/:code | 按产品编码查询 | | GET | /api/v1/products/remote/categories | 远程产品分类 | | GET | /api/v1/products/remote/:id | 远程产品详情 | | POST | /api/v1/products/claim | 认领产品 | | GET | /api/v1/products/claimed | 已认领产品列表 | | GET | /api/v1/products/claimed/count | 已认领产品数量 | | GET | /api/v1/products/claimed/categories | 已认领产品分类 | | GET | /api/v1/products/claimed/:id | 已认领产品详情 | | PUT | /api/v1/products/claimed/:id | 更新已认领产品 | | DELETE | /api/v1/products/claimed/:id | 删除已认领产品 | | POST | /api/v1/products/images/from-url | 从URL上传图片 | | POST | /api/v1/products/images/from-file | 从文件上传图片 | | DELETE | /api/v1/products/images/:id | 删除图片 | ### Listing 上架 | 方法 | 端点 | 说明 | |------|------|------| | GET | /api/v1/listings | 列表(分页,max 100/page) | | GET | /api/v1/listings/count | 数量统计 | | POST | /api/v1/listings | 创建listing | | GET | /api/v1/listings/:id | 详情 | | PUT | /api/v1/listings/:id | 更新(含platform_fields) | | DELETE | /api/v1/listings/:id | 删除 | | POST | /api/v1/listings/generate | AI生成listing | | POST | /api/v1/listings/:id/publish | 发布到平台 | ### 社交帖子 Social | 方法 | 端点 | 说明 | |------|------|------| | GET | /api/v1/social/posts | 社交帖子列表 | | POST | /api/v1/social/posts | 创建帖子 | | GET | /api/v1/social/posts/:id | 帖子详情 | | PUT | /api/v1/social/posts/:id | 更新帖子 | | DELETE | /api/v1/social/posts/:id | 删除帖子 | | POST | /api/v1/social/posts/generate | AI生成帖子 | | POST | /api/v1/social/posts/:id/publish | 发布到社交平台 | | GET | /api/v1/social/listings | 社交关联listing | ### 店铺 Stores (16 平台) | 方法 | 端点 | 说明 | |------|------|------| | GET | /api/v1/stores | 所有店铺列表 | | POST | /api/v1/stores | 创建店铺 | | GET | /api/v1/stores/:id | 店铺详情 | | PUT | /api/v1/stores/:id | 更新店铺 | | DELETE | /api/v1/stores/:id | 删除店铺 | | POST | /api/v1/stores/:id/test-connection | 测试连接 | | GET | /api/v1/stores/:id/categories | 店铺分类 | ### 设置 Settings | 方法 | 端点 | 说明 | |------|------|------| | GET | /api/v1/settings/accounts | 聚合:stores+social+erp+email | | GET | /api/v1/settings/system-code | 公开,无需认证 | | GET/POST/PUT/DELETE | /api/v1/settings/social-accounts | 社交账号 CRUD | | GET/POST/PUT/DELETE | /api/v1/settings/erp-accounts | ERP账号 CRUD(10系统) | | GET/POST/PUT/DELETE | /api/v1/settings/email-accounts | 邮箱账号 CRUD(10服务商) | ### 同步 Sync | 方法 | 端点 | 说明 | |------|------|------| | POST | /api/v1/settings/sync/center-to-local | 中心→本地同步(需admin API key) | | GET | /api/v1/settings/sync/status | 同步状态查询 | --- ## Key Data Structures ### Claimed Product Fields | Field | Description | |-------|-------------| | `id` | Internal AtomK product ID | | `remote_product_code` | Product code (e.g., `CNVGX8F4`) | | `local_sku` | Local SKU | | `product_name` | English name | | `product_name_zh` | Chinese name | | `category` | Category (e.g., `家居百货`) | | `source_url` | Original 1688/source URL | | `images` | List of COS image objects | | `claimed_at` | ISO timestamp | **Image structure**: ```python { "id": 8230, "image_url": "https://9websclub-1251422183.cos.ap-hongkong.myqcloud.com/AK-101/users/4/products/CNVGX8F4/0_0d3a9223.jpeg", "cos_key": "AK-101/users/4/products/CNVGX8F4/0_0d3a9223.jpeg", "is_primary": True, "sort_order": 0 } ``` ### Listing Fields | Field | Description | |-------|-------------| | `id` | Listing ID | | `claimed_product_id` | Links to product `id` | | `platform` | `temu`, `ozon`, `aliexpress`, etc. | | `title` | AI-generated title | | `description` | AI-generated description | | `price` | Price | | `currency` | Currency code | | `keywords` | Comma-separated keywords | | `status` | `generated` or other | | `ai_model` | `hunyuan`, `deepseek`, etc. | | `csv_url` | **Public COS URL** to Temu-standard CSV | | `html_url` | **Public COS URL** to formatted HTML product page | | `platform_fields` | Nested dict with SKU, UPC, GTIN, EAN, package dims, image URLs, etc. | | `created_at` / `updated_at` | ISO timestamps | **Platform fields** (Temu example): ```python { "SKU": "RC-MACARON-CANDLES-BOX360", "ean": "7477596853977", "upc": "747759886757", "gtin": "0747759886757", "price": 10.28, "price_cny": 69.9, "quantity": "360", "exchange_rate": 6.8, "package_width": "10", "package_height": "2", "package_length": "15", "package_weight": "0.1", "image_urls": "url1;url2;url3;...", # semicolon-separated COS URLs "category_path": "Home & Garden > Party Supplies" } ``` ### CSV Export (Pre-generated) The `csv_url` is a **publicly accessible** COS link. No auth required to download: ```python import requests csv_resp = requests.get(listing['csv_url'], timeout=30) # Returns UTF-8 BOM CSV with Temu-standard columns ``` ### HTML Export (Pre-generated) The `html_url` is also public. Self-contained, styled HTML product detail page: ```python html_resp = requests.get(listing['html_url'], timeout=30) ``` ## Integration with Miaoshou ERP 采集箱 ### Miaoshou Import Collection Box URL ``` https://erp.91miaoshou.com/common_collect_box/index?fetchType=importCopy ``` ### Import Methods 1. **导入链接采集** (Import Link Collection) — requires xls with source URLs 2. **Excel表格导入** (Excel Import) — for bulk data 3. **本地素材包导入** (Local Material Package Import) ### Import Link Collection Template Downloaded template: `导入产品链接模板.xls` **Columns**: | Column | Field | Required | |--------|-------|----------| | A | 链接地址(必填) | ✅ Yes | | B | 产品标题 | Optional | | C | 价格(RMB) | Optional | | D | 促销价(RMB) | Optional (Lazada only) | **How Miaoshou processes it**: For each link, Miaoshou's scraper visits the URL and extracts product info (images, title, description). Optional columns override scraped values. ### Workflow: AtomK → Miaoshou (Link Import) — Validated ✅ **CRITICAL**: Miaoshou's scraper only supports **1688/Taobao/Pinduoduo** URLs. AtomK's `html_url` (COS-hosted HTML) will **always fail** with "解析失败". You must use the **original `source_url`** from `/api/v1/products/claimed`. ```python import requests import xlwt requests.packages.urllib3.disable_warnings() # 1. Login AtomK login_resp = requests.post('https://www.atomlisting.com/api/v1/auth/login', json={'username': 'admincao', 'password': 'Tt123456!'}, timeout=10, verify=False) token = login_resp.json()['token'] headers = {'Authorization': f'Bearer {token}', 'Content-Type': 'application/json'} # 2. Get claimed products (for source_url) AND listings (for title/price) products = requests.get('https://www.atomlisting.com/api/v1/products/claimed?page=1&limit=100', headers=headers, timeout=30, verify=False).json() listings = requests.get('https://www.atomlisting.com/api/v1/listings', headers=headers, timeout=30, verify=False).json() # 3. Match listing → product via claimed_product_id to get 1688 source_url matched = [] for listing in listings: if not listing.get('html_url'): # skip incomplete continue product = next((p for p in products if p['id'] == listing['claimed_product_id']), None) if product and product.get('source_url'): matched.append({'listing': listing, 'product': product}) # 4. Build Miaoshou import xls using OFFICIAL template format workbook = xlwt.Workbook() sheet = workbook.add_sheet('Worksheet') # Official columns (verified by downloading template from Miaoshou UI): sheet.write(0, 0, '链接地址(必填)') sheet.write(0, 1, '产品标题') sheet.write(0, 2, '价格(RMB)') sheet.write(0, 3, '促销价(RMB)') sheet.write(0, 4, '提示:促销价仅可用于采集到Lazada') for idx, item in enumerate(matched, start=1): listing = item['listing'] product = item['product'] pf = listing.get('platform_fields', {}) sheet.write(idx, 0, product['source_url']) # ✅ 1688 link, NOT html_url sheet.write(idx, 1, listing['title']) sheet.write(idx, 2, pf.get('price_cny', '')) sheet.write(idx, 3, '') sheet.write(idx, 4, '') workbook.save('/tmp/miaoshou_import.xls') ``` **5. Upload to Miaoshou via browser automation:** ```python # Prerequisites: logged into Miaoshou via DrissionPage (see captcha-auto-login skill) page.get('https://erp.91miaoshou.com/common_collect_box/index?fetchType=importCopy') time.sleep(8) # Step 1: Close notification popups (critical — they block interactions) for _ in range(5): page.run_js(''' var btns = document.querySelectorAll("button, .jx-dialog__close, .jx-dialog__headerbtn"); for (var i = 0; i < btns.length; i++) { var text = btns[i].textContent.trim(); var ariaLabel = btns[i].getAttribute('aria-label') || ''; var cls = btns[i].className || ''; if (text === "关闭" || text === "我知道了" || ariaLabel.includes("关闭") || cls.includes('close') || cls.includes('headerbtn')) { btns[i].click(); } } ''') time.sleep(2) # Step 2: Open import dialog page.run_js(''' var btns = document.querySelectorAll("button"); for (var i = 0; i < btns.length; i++) { if (btns[i].textContent.trim().includes("导入链接采集")) { var evt = new MouseEvent("click", { bubbles: true, cancelable: true, view: window }); btns[i].dispatchEvent(evt); return true; } } return false; ''') time.sleep(8) # Step 3: Upload file via hidden input file_input = page.ele('css:.jx-upload__input', timeout=5) file_input.input('/tmp/miaoshou_import.xls') time.sleep(5) # Step 4: Click confirm page.run_js(''' var modals = document.querySelectorAll(".jx-dialog"); for (var i = 0; i < modals.length; i++) { var style = window.getComputedStyle(modals[i]); if (style.display !== "none") { var btns = modals[i].querySelectorAll("button"); for (var j = 0; j < btns.length; j++) { if (btns[j].textContent.trim() === "确认") { btns[j].click(); return true; } } } } return false; ''') # Step 5: Wait for parsing (1-5 minutes) time.sleep(180) # Status will show: 等待解析 → 解析成功 (X/Y/Z) or 解析失败 ``` **Expected result**: `解析成功 3/0/3` (all succeeded) — NOT `解析失败`. **If it shows "等待解析"**: Wait 1-5 minutes then refresh. If it shows "解析失败 解析文件数据错误", the template format is wrong. If it shows "导入文件未识别到:【产品主编号】表头", you're using the wrong import method or an outdated template. ### Alternative: Full Data Injection (Bypass Scraping) If link scraping fails, use the **complete data** from `platform_fields` to directly create products in Miaoshou or target platforms: - Title, description, price, quantity, SKU, UPC, GTIN, EAN - Package dimensions & weight - Image URLs (semicolon-separated COS links) - Keywords This bypasses Miaoshou's collection box entirely and goes straight to platform-specific CSV generation. ## Pitfalls - **✅ CRITICAL: `html_url` does NOT work with Miaoshou** — Miaoshou's scraper only supports 1688/Taobao/Pinduoduo. You MUST use `source_url` from `/api/v1/products/claimed` (original 1688 link). Using `html_url` produces `解析失败 0/3/3` 100% of the time. - **Wrong template = "产品主编号" error** — If you see "导入文件未识别到:【产品主编号】表头", you are either using the wrong import method (e.g., "Excel表格导入" instead of "导入链接采集") or an outdated template. The correct "导入链接采集" template has only: `链接地址(必填), 产品标题, 价格(RMB), 促销价(RMB)`. - **Notification popups block UI** — Miaoshou shows "我知道了" notification popups that MUST be closed before clicking "导入链接采集". Failure to close them causes the click to do nothing. - **Sub-account permission limits** — The account `xiaochaoren2026` is a Miaoshou sub-account. It can access `采集箱/导入采集` but shows "暂无当前模块权限" on product publish/edit pages. Use a main account for full publishing. - **店小秖 (Dianxiaomi) requires slider CAPTCHA** — Login page has "拖动下方拼图完成验证" which is extremely difficult to bypass automatically. Prefer Miaoshou for automation. - **API is partially documented** — 2025-05 upgrade added many endpoints. Some new endpoints may have undocumented body fields or response shapes. Monitor 404/500 responses. - **Token expiry** — JWT has `exp` field. Use `POST /api/v1/auth/refresh` with refresh token if needed. - **Image format mix** — AtomK stores both `.jpeg` and `.webp` images. Some platforms prefer `.jpeg` only. Use `POST /api/v1/products/images/from-url` or `from-file` for image management. - **COS URLs are public** — but tied to AtomK's Tencent Cloud account. If AtomK changes storage policy, URLs may break. - **No batch export API** — `/api/v1/export` does not exist. Export is done product-by-product via pre-generated `csv_url`/`html_url`. - **ERP accounts should NOT be stored in Stores** — use `/api/v1/settings/erp-accounts` instead. The old workaround of `platform=woocommerce` + `extra_data.real_platform=miaoshou_erp` is deprecated. - **`POST /settings/erp-accounts` has 500 bug (2025-05)** — all creation attempts fail regardless of platform. Store ERP credentials in local file (`~/.hermes/credentials/accounts.md`) until backend is fixed. - **`password` in `extra_data` auto-filtered to `***`** — always use `pwd` field for any account type (stores, ERP, email). - **Silent UI Failures for Account Linking** — `POST /api/v1/stores` may return 502 (telnumber DB constraint), `POST /api/v1/settings/social-accounts` may return 500 for unsupported platforms. Intercept requests to debug. - **Social platform whitelist** — only `facebook`, `tiktok`, `twitter`, `wordpress` work. `instagram`, `pinterest` return 500. - **Sync API needs API key** — `GET /settings/sync/status` and `POST /settings/sync/center-to-local` require `x-api-key` header, not JWT. - **`status="imported"` triggers 500** — always use `status="generated"` when creating listings. - **`platform_fields` is silently ignored in POST** — you must use PUT to set it after creation. - **`claimed_product_id` must be unique** — assigning the same product ID to multiple listings creates duplicates. - **API returns max 100 listings per page** — use `page` parameter to paginate. - **502 Bad Gateway on rapid fire** — add 50ms delay between requests, or implement retry with backoff. - **Consecutive 500s mean a bad product** — if you get 10 consecutive 500s, skip that product. The product data itself is likely invalid. - **Deduplicate before creating** — always check existing listings first. Re-running a sync script without dedup creates duplicates. - **`POST /api/v1/settings/sync/center-to-local`** requires admin API key, not regular JWT. - **Ozon Seller API keys expire** — both Hzqjone(client_id=3098640) and Hzqjtwo(client_id=3103005) keys returned "Invalid Api-Key" as of 2025-05. Must regenerate from seller.ozon.ru (requires Russian IP). Ozon product import uses `/v3/product/import` (not `/v1/`). - **Ozon publish not supported** — `POST /listings/:id/publish` returns 400 for Ozon. Must use Ozon Seller API directly to create products. ### Stores API Details **Working platform values**: `walmart`, `temu`, `ozon`, `aliexpress`, `amazon`, `ebay`, `woocommerce`, `lazada`, `shopee` **Store fields**: | Field | Description | |-------|-------------| | `id` | Store ID | | `platform` | E-commerce platform (strict validation) | | `store_name` | Human-readable name | | `store_url` | Platform URL | | `extra_data` | Arbitrary JSON (API keys, credentials, etc.) | | `is_active` | Boolean | **Pitfalls for Stores API**: - **`password` in `extra_data` auto-filtered to `***`** — use `pwd` field instead. - **Platform validation is strict** — `miaoshou`, `shopify`, `other`, `erp` all return 500. Only e-commerce platforms accepted. - **ERP accounts do NOT belong in Stores** — use `/api/v1/settings/erp-accounts` instead. - **`POST /stores/:id/test-connection`** — validates store credentials (API key, URL, etc.) - **`GET /stores/:id/categories`** — fetches platform category tree (not all platforms support this; walmart returns 400) ### Settings API Details **`GET /api/v1/settings/accounts`** — 聚合所有账号信息(stores + social + erp + email),一站式获取。 **`GET /api/v1/settings/system-code`** — 公开端点,无需认证。返回 `{"system_code": "AK-101"}`。 **ERP Accounts CRUD** (`/api/v1/settings/erp-accounts`): - **必填字段**: `erp_name`, `account_name` - **可选字段**: `platform`, `extra_data`, `is_active` - ⚠️ **POST currently returns 500 (backend bug, 2025-05)** — all platform values fail. Workaround: store ERP credentials in `/home/ubuntu/.hermes/credentials/accounts.md` until backend is fixed. - When fixed, expected usage: ```python erp = { "erp_name": "妙手ERP", "platform": "miaoshou", "account_name": "17762501033", "extra_data": {"phone": "17762501033", "pwd": "Tt123456!"}, "is_active": True } requests.post(f'{BASE}/settings/erp-accounts', json=erp, headers=headers) ``` **Email Accounts CRUD** (`/api/v1/settings/email-accounts`): - **必填字段**: `email_provider`, `email_address`, `account_name` - **可选字段**: `smtp_host`, `smtp_port`, `smtp_username`, `use_tls`, `extra_data`, `is_active`, `linked_store_id` - 密码存 `extra_data.pwd`(不要用 `password`) ```python email = { "email_provider": "gmail", "email_address": "admin@9webs.cn", "account_name": "9webs Admin", "smtp_host": "smtp.gmail.com", "smtp_port": 587, "smtp_username": "admin@9webs.cn", "use_tls": True, "extra_data": {"pwd": "xxx"}, "is_active": True } ``` **Social Accounts CRUD** (`/api/v1/settings/social-accounts`): - **字段**: `platform`, `account_name`, `account_handle`, `linked_store_id`, `is_active` - **可用platform**: `facebook`, `tiktok`, `twitter`, `wordpress` - ⚠️ `instagram`, `pinterest` 返回 500 (backend bug) ### Social Posts API Social模块管理社交帖子(Facebook、Twitter等),结构类似Listings: - **`GET /social/posts`** — 帖子列表 - **`POST /social/posts/generate`** — AI生成社交帖子内容 - **`POST /social/posts/:id/publish`** — 发布到社交平台 - **`GET /social/listings`** — 获取与社交帖子关联的listing数据 **Social Post fields**: | Field | Description | |-------|-------------| | `id` | Post ID | | `platform` | Social platform | | `content` | Post content text | | `images` | Image URLs | | `status` | `draft` / `generated` / `published` | | `linked_listing_id` | Optional link to a listing | ### Sync API Details **`POST /api/v1/settings/sync/center-to-local`**: - 从中心数据库同步到本地实例 - ⚠️ 需要 admin API key(`x-api-key` header),非JWT token。普通JWT返回401。 - 用于多实例部署场景 **`GET /api/v1/settings/sync/status`**: - 查询当前同步状态 - 同样需要 API key ### Listings Generate & Publish **`POST /api/v1/listings/generate`** — AI生成listing数据: - 请求体: `{"claimed_product_id": 123, "platform": "ozon", "store_id": 12}` - 自动生成: 目标语言标题/描述/关键词(Ozon=俄语)、EAN/UPC/GTIN条码、定价(汇率6.8换算)、包裹尺寸、分类路径 - 耗时约30-60秒/个,需设timeout=120+ - 返回完整listing对象(含id/title/description/platform_fields/csv_url/html_url等) - 批量生成时需逐个调用,不能并行。每个generate是同步的(不是async) **`POST /api/v1/listings/:id/publish`** — 推送到目标平台: - 前提: store已配置且 `test-connection` 通过 - ⚠️ **Ozon不支持** — 返回400 "Platform ozon publishing not supported yet" - Walmart/Temu/AliExpress可能支持(未测试) - Ozon上架需走Ozon Seller API (`/v3/product/import`) 直接推 **批量Generate模式** (推荐): ```python # 逐个generate,带timeout和重试 import requests, time s = requests.Session() s.verify = False # ... login ... # 获取未使用的claimed products products = [] # 从 /products/claimed 获取 used = set() # 从 /listings 获取已用的claimed_product_id unused = [p for p in products if p['id'] not in used] for i, prod in enumerate(unused[:N]): try: r = s.post(f'{BASE}/listings/generate', json={ 'claimed_product_id': prod['id'], 'platform': 'ozon', 'store_id': 12, }, timeout=120) if r.status_code == 200: listing = r.json() print(f"[{i+1}] OK id={listing['id']} title={listing.get('title','')[:40]}") else: print(f"[{i+1}] FAIL {r.status_code}: {r.text[:100]}") except requests.exceptions.ReadTimeout: print(f"[{i+1}] TIMEOUT (product {prod['id']} may be too complex)") time.sleep(0.5) ``` ## Verification Steps 1. Confirm `api/v1/listings` returns expected data shape and `claimed_product_id` matches `api/v1/products/claimed` items. 2. Confirm `source_url` exists on claimed products and points to `detail.1688.com`. 3. Generate a 1-row test XLS with official template format (`链接地址(必填), 产品标题, 价格(RMB)`) using `source_url`. 4. Upload to Miaoshou via "导入链接采集". Wait 1-5 minutes. Expected: `解析成功 1/0/1`. 5. If you see `解析失败`, check: (a) URL is 1688 not AtomK HTML, (b) template downloaded from Miaoshou UI is current, (c) popup notifications were closed before upload. ## Listings API — Write Side (Create / Update / Delete) ### Create Listing ``` POST https://www.atomlisting.com/api/v1/listings ``` **Required body fields**: ```python { "claimed_product_id": 123, # must be a valid, unassigned claimed product ID "platform": "ozon", # platform name "store_id": 12, # store ID from /api/v1/stores "title": "Product Title", "description": "Description", "price": 9.99, "currency": "CNY", "status": "generated" # NOT "imported" — that causes 500! } ``` **Pitfalls for Create**: - **`status="imported"` triggers 500** — always use `status="generated"`. - **`platform_fields` is silently ignored in POST** — you must use PUT to set it after creation. - **`claimed_product_id` must be unique** — assigning the same product ID to multiple listings creates duplicates. Track which product IDs are already used. - **API pagination limit is 100** — `GET /listings?limit=100&page=N`. `limit>100` returns empty items. ### Update Listing (Set platform_fields) ``` PUT https://www.atomlisting.com/api/v1/listings/{id} ``` ```python { "platform_fields": { "product_id": "123456789", "offer_id": "SKU-001", "store_name": "Hzqjone", "url": "https://www.ozon.ru/product/123456789" } } ``` ### Delete Listing ``` DELETE https://www.atomlisting.com/api/v1/listings/{id} ``` ### Bulk Sync Pattern (Ozon → AtomK) When syncing external platform listings (Ozon, Temu, etc.) into AtomK, use this pattern: ```python import requests, json, time from urllib3.exceptions import InsecureRequestWarning requests.packages.urllib3.disable_warnings(InsecureRequestWarning) BASE = 'https://www.atomlisting.com/api/v1' s = requests.Session() s.verify = False # 1. Login r = s.post(f'{BASE}/auth/login', json={'username': 'admincao', 'password': 'Tt123456!'}, timeout=10) s.headers.update({'Authorization': f'Bearer {r.json()["token"]}'}) # 2. Fetch ALL existing listings with pagination (max 100 per page) all_listings = [] page = 1 while True: r = s.get(f'{BASE}/listings', params={'page': page, 'limit': 100}, timeout=15) d = r.json() items = d if isinstance(d, list) else d.get('items', []) all_listings.extend(items) if len(items) < 100: break page += 1 # 3. Build index: product_id -> [listing_ids] for dedup ozon_listings = [l for l in all_listings if l.get('platform') == 'ozon'] by_pid = {} for l in ozon_listings: pf = l.get('platform_fields') or {} pid = pf.get('product_id') if pid: by_pid.setdefault(pid, []).append(l) # 4. Delete duplicates (keep first, delete rest) for pid, lst in by_pid.items(): for l in lst[1:]: s.delete(f'{BASE}/listings/{l["id"]}', timeout=10) time.sleep(0.05) # 5. Get unused claimed_product_ids products = [] page = 1 while True: r = s.get(f'{BASE}/products/claimed', params={'page': page, 'limit': 100}, timeout=15) items = r.json() if isinstance(r.json(), list) else r.json().get('items', []) products.extend(items) if len(items) < 100: break page += 1 used_pids = {l['claimed_product_id'] for l in ozon_listings} unused = [p['id'] for p in products if p['id'] not in used_pids] # 6. Create missing listings existing_pids = set(by_pid.keys()) for i, product_data in enumerate(external_products): if product_data['product_id'] in existing_pids: continue if not unused: print("No more unused claimed_product_ids!") break cp_id = unused.pop(0) # POST to create (without platform_fields) r = s.post(f'{BASE}/listings', json={ 'claimed_product_id': cp_id, 'platform': 'ozon', 'store_id': store_id, 'title': product_data.get('name', f'Ozon-{product_data["product_id"]}'), 'description': '', 'price': product_data.get('price', 0), 'currency': 'CNY', 'status': 'generated', }, timeout=10) if r.status_code == 201: listing_id = r.json()['id'] # PUT to set platform_fields s.put(f'{BASE}/listings/{listing_id}', json={ 'platform_fields': { 'product_id': str(product_data['product_id']), 'offer_id': product_data.get('offer_id', ''), 'store_name': store_name, 'url': f'https://www.ozon.ru/product/{product_data["product_id"]}', } }, timeout=10) time.sleep(0.05) # rate limit protection ``` **Critical Pitfalls for Bulk Sync**: - **Pagination is mandatory** — `/listings` returns max 100 items. Always paginate or you'll create hundreds of duplicates. - **Two-step creation** — POST creates the listing, but `platform_fields` must be set via a separate PUT. The POST silently ignores `platform_fields`. - **`status="imported"` = 500** — use `"generated"`. - **502 Bad Gateway on rapid fire** — add 50ms delay between requests, or implement retry with backoff. - **Consecutive 500s mean a bad product, not rate limiting** — if you get 10 consecutive 500s, skip that product. The product data itself is likely invalid (e.g., null fields the backend can't handle). - **Deduplicate before creating** — always check existing listings first. Re-running a sync script without dedup creates 2x, 3x duplicates. ## References - `references/backend-development.md` — Full guide for adding new API endpoints: model creation, router setup, Alembic migration, production deployment (no CI/CD, manual SSH), plus a debug pattern for Desktop-backend API mismatch errors. - `references/billionmail-api.md` — BillionMail (SG4) email server API reference: mailbox creation, SMTP relay, and default domain (`9webs.site`). ## Related Skills - `atomk-temu-batch-listing` — browser-based batch claiming and AI generation - `miaoshou-collect-box` — Miaoshou ERP collection box navigation - `captcha-auto-login` — automated ERP login with CAPTCHA solving