Add archived/tongtool-temu-shipping-calc

This commit is contained in:
2026-07-10 16:10:52 +08:00
parent ac930ec63e
commit f1cde885a8
@@ -0,0 +1,559 @@
---
name: tongtool-temu-shipping-calc
description: Playwright automation for Tongtool ERP TEMU semi-managed orders — filter pre-print orders WITHOUT existing remarks, login with captcha retry, iterate through each order, open 仓储物流 → 邮寄方式解析 → TEMU平台接口试算, click 查询, find cheapest shipping price, write remark to order (or error remark on failure), and support scheduled looping.
---
# Tongtool TEMU Shipping Cost Calculator (v2)
End-to-end automation: **find pre-print (未完成通途打印) TEMU half-managed orders without remarks** → compute cheapest TEMU API shipping price → write remark back to order → repeat on schedule.
## Core Requirements (User Spec)
1. **筛选目标**:未完成通途打印步骤 **且** 没有备注 的 TEMU 半托管订单
2. **逐个处理**:订单详情 → 仓储物流 → 邮寄方式解析 → **TEMU平台接口试算**(⚠️ 不是运费模板试算)→ 点击查询 → 找出最低价
3. **写入备注**
- 成功:格式如 `GOFO,Standard 预估$2.50`
- 失败:写入 `出错没找到最低价`
4. **定时循环**:支持 cron/systemd 定时重复执行
## Prerequisites
- `playwright` + Chromium at `/home/ubuntu/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome`
- `ddddocr` for captcha OCR
- `requests` for captcha image download
- Tongtool account: `ozonezsjgw@163.com` / `111111`
## 1. Login with Captcha Retry
**Critical fix**: `ddddocr` may return CJK symbols/non-printable chars. Filter to ASCII alphanumeric before character substitution.
```python
import asyncio, re, subprocess, requests
from playwright.async_api import async_playwright
USERNAME = 'ozonezsjgw@163.com'
PASSWORD = '111111'
BROWSER_PATH = '/home/ubuntu/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome'
def correct_ocr(raw):
cleaned = ''.join(c for c in raw if c.isascii() and c.isalnum())
return cleaned.translate(str.maketrans('oliOSsZz', '01105522'))
async def do_login(page, context):
await page.goto('https://passport.tongtool.com/')
await asyncio.sleep(3)
await page.fill('input[name="username"]', USERNAME)
await page.fill('input[name="password"]', PASSWORD)
for attempt in range(5):
img = page.locator('img#captcha')
src = await img.get_attribute('src')
if src.startswith('/'):
src = 'https://passport.tongtool.com' + src
cookies = await context.cookies()
cookie_dict = {c['name']: c['value'] for c in cookies}
resp = requests.get(src, cookies=cookie_dict, timeout=10)
raw = subprocess.check_output(['python3', '-c', f"""
import ddddocr
print(ddddocr.DdddOcr(show_ad=False).classification({resp.content!r}))
"""]).decode().strip()
code = correct_ocr(raw)
if len(code) < 4:
await img.click()
await asyncio.sleep(2)
continue
await page.fill('input[name="captcha"]', code)
await page.click('button.btn-primary')
await asyncio.sleep(8)
if 'member' in page.url:
return True
return False
```
## 2. Navigate + Select Warehouse
```python
async def goto_wait_stock(page):
await page.goto('https://yijiety.tongtool.com/warehouse/assignStockSelfMaintenance/index.htm')
await asyncio.sleep(30)
# Open warehouse dropdown
await page.evaluate("""
var spans = document.querySelectorAll('span');
for (var s of spans) {
if (s.textContent.trim() === '请选择仓库:') {
s.parentElement.click(); break;
}
}
""")
await asyncio.sleep(3)
await page.evaluate("document.querySelector('td[title=\"武汉仓库\"]').click()")
await asyncio.sleep(15)
# Click 查询
await page.evaluate("""
var all = document.querySelectorAll('a, button, span');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '查询') { all[i].click(); break; }
}
""")
await asyncio.sleep(10)
```
## 3. ⭐ Filter: Pre-Print TEMU Half-Managed Orders WITHOUT Remarks
Extract each row's: order_no, print status, existing remark. Keep only rows where **Tongtool print step is NOT completed AND remark is empty**.
**Critical Update**: Do not rely purely on UI scraping (which often loops over the same first page of 50 already-skipped items). Instead, fetch a fresh list of target un-remarked orders using the Tongtool OpenAPI backend, then feed that list into the UI script to force processing of unseen pages/orders.
```python
async def find_target_orders(page):
"""Returns list of target order numbers.
- Matches TEMU order pattern (-PO-)
- Relies on idempotency (skips if already remarked)
"""
targets = await page.evaluate("""() => {
var rows = document.querySelectorAll('tr, div.grid-row, div.x-grid3-row');
var results = [];
for (var row of rows) {
var orderNo = '';
// Remove whitespace from textContent to prevent regex breaking on visually wrapped long IDs
for (var cell of row.querySelectorAll('td, div.cell')) {
var txt = (cell.textContent || '').replace(/\\s+/g, '');
var m = txt.match(/([A-Za-z0-9]+-PO-\\d+-\\d+)/i);
if (m) { orderNo = m[1]; break; }
}
if (!orderNo) continue;
// Idempotency: skip if already processed
var rowText = row.textContent || '';
var hasRemark = rowText.includes('[TEMU试算]') || rowText.includes('出错没找到最低价');
if (hasRemark) continue;
results.push(orderNo);
}
return results;
}""")
# Deduplicate
seen = set()
unique = []
for o in targets:
if o not in seen:
seen.add(o)
unique.append(o)
print(f'Found {len(unique)} pre-print, un-remarked TEMU half-managed orders')
return unique
```
> **Note**: Exact selectors for order status and remark columns may vary. If the default logic misses, use DevTools to find the specific column class/index and adjust the `rowText` check to read targeted `tds[idx]` instead.
## 4. Open 邮寄方式解析 Popup (per order)
**Critical**: 编辑 button requires exact text match `=== '编辑'` + visibility check. `includes('编辑')` matches invisible nodes.
```python
async def open_shipping_popup(page, order_no):
# Close residual popups
await page.evaluate("""
var closeBtns = document.querySelectorAll('.jclose-blue, .close, [title="关闭"], .layui-layer-close');
for (var i=0; i<closeBtns.length; i++) closeBtns[i].click();
""")
await asyncio.sleep(2)
# Click order number link
clicked = await page.evaluate(f"""() => {{
var links = document.querySelectorAll('a');
for (var i=0; i<links.length; i++) {{
if (links[i].textContent.includes('{order_no}')) {{
links[i].click(); return true;
}}
}}
return false;
}}""")
if not clicked:
return False
await asyncio.sleep(12)
# Click 仓储物流
await page.evaluate("""() => {
var all = document.querySelectorAll('*');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '仓储物流' && all[i].offsetHeight > 0) {
all[i].click(); return;
}
}
}""")
await asyncio.sleep(8)
# Click 编辑 — exact match + visibility
await page.evaluate("""() => {
var all = document.querySelectorAll('a, button, span');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '编辑') {
var rect = all[i].getBoundingClientRect();
if (rect.width > 0 && rect.height > 0) {
all[i].click(); return;
}
}
}
}""")
await asyncio.sleep(8)
# Click 邮寄方式解析
opened = await page.evaluate("""() => {
var all = document.querySelectorAll('a');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '邮寄方式解析') {
all[i].click(); return true;
}
}
return false;
}""")
await asyncio.sleep(12)
return opened
```
## 5. ⭐ Switch to TEMU平台接口试算 Tab + Click 查询 + Extract Cheapest
**Critical**:
- Popup is a `div[windowid]` with `.jbox` class (NOT iframe)
- Tab panel has `id="tabpanelDiv"`, tabs are `<li tabtitle="...">`
- **Must click `TEMU平台接口试算` tab** (NOT `运费模板试算`)
- After switching tab, **must click 查询** button inside `#temuShippingMethod` to trigger API call
- Prices come back as e.g. `GOFO` (carrier) / `Standard` (method) / `$2.50` (USD price)
```python
async def extract_temu_cheapest(page):
"""Returns string like 'GOFO,Standard 预估$2.50' or None on failure."""
# Step 1: Switch to TEMU tab
switched = await page.evaluate("""() => {
var popups = document.querySelectorAll('div[windowid]');
var popup = null;
for (var p of popups) {
if (p.querySelector('#tabpanelDiv')) { popup = p; break; }
}
if (!popup) return false;
var tab = popup.querySelector('li[tabtitle="TEMU平台接口试算"]');
if (tab) { tab.click(); return true; }
return false;
}""")
if not switched:
return None
await asyncio.sleep(10)
# Step 2: Click 查询 button inside #temuShippingMethod
await page.evaluate("""() => {
var popups = document.querySelectorAll('div[windowid]');
var popup = null;
for (var p of popups) {
if (p.querySelector('#tabpanelDiv')) { popup = p; break; }
}
if (!popup) return;
var temuDiv = popup.querySelector('#temuShippingMethod');
if (temuDiv && !temuDiv.classList.contains('hide')) {
var btns = temuDiv.querySelectorAll('a, button');
for (var b of btns) {
if (b.textContent.includes('查询') || b.textContent.includes('试算')) {
b.click(); return;
}
}
}
}""")
await asyncio.sleep(12)
# Step 3: Extract all carrier/method/price triples and find cheapest
prices = await page.evaluate("""() => {
var popups = document.querySelectorAll('div[windowid]');
var popup = null;
for (var p of popups) {
if (p.querySelector('#tabpanelDiv')) { popup = p; break; }
}
if (!popup) return [];
var temuDiv = popup.querySelector('#temuShippingMethod');
if (!temuDiv) return [];
var results = [];
var rows = temuDiv.querySelectorAll('tr');
for (var row of rows) {
var tds = row.querySelectorAll('td');
if (tds.length < 3) continue;
// Try to find USD price ($x.xx) in the row
var rowText = row.textContent;
var currency = '$';
var priceMatch = rowText.match(/\\$\\s*(\\d+\\.\\d+)/);
if (!priceMatch) {
priceMatch = rowText.match(/[¥¥]\\s*(\\d+\\.\\d+)/);
currency = '¥';
}
if (!priceMatch) continue;
// Skip disabled options ("未启用") entirely
if (rowText.includes('未启用')) continue;
// carrier usually in td[1] or td[0], method in td[2]
var carrier = '', method = '';
for (var td of tds) {
var t = td.textContent.trim();
if (!t || t === priceMatch[0]) continue;
if (!carrier) { carrier = t; continue; }
if (!method) { method = t; break; }
}
if (!carrier && !method) continue;
results.push({
carrier: carrier,
method: method,
currency: currency,
price: parseFloat(priceMatch[1])
});
}
return results;
}""")
if not prices:
return None
cheapest = min(prices, key=lambda x: x['price'])
# Format: 'GOFO,Standard 预估$2.50'
# Fallback to handle missing carrier
c = (cheapest.get('carrier') or '').strip()
m = (cheapest.get('method') or '').strip()
label = f"{c}>>{m}" if c and m else (c or m)
return f"{label} {cheapest['currency']}{cheapest['price']:.2f}"
```
## 6. Add Order Remark (success OR error)
```python
async def add_remark(page, remark_text):
"""Add remark text (success price OR error message) to the open order."""
clicked = await page.evaluate("""() => {
var btn = document.querySelector('#addOrderRemark');
if (btn) {
var a = btn.querySelector('a');
if (a) { a.click(); return true; }
}
return false;
}""")
if not clicked:
return False
await asyncio.sleep(5)
# Fill textarea (escape single quotes in remark_text)
escaped = remark_text.replace("'", "\\'").replace('"', '\\"')
await page.evaluate(f"""() => {{
var ta = document.querySelector('textarea');
if (ta) {{
ta.value = '{escaped}';
ta.dispatchEvent(new Event('input', {{bubbles:true}}));
ta.dispatchEvent(new Event('change', {{bubbles:true}}));
}}
}}""")
await asyncio.sleep(1)
# Click 保存
saved = await page.evaluate("""() => {
var links = document.querySelectorAll('a, button');
for (var i=0; i<links.length; i++) {
if (links[i].textContent.trim() === '保存') {
links[i].click(); return true;
}
}
return false;
}""")
await asyncio.sleep(3)
return saved
```
## 7. Close Popup (between orders)
```python
async def close_popup(page):
await page.evaluate("""
var closeBtns = document.querySelectorAll('.jclose-blue, .close, [title="关闭"], .layui-layer-close');
for (var i=0; i<closeBtns.length; i++) closeBtns[i].click();
""")
await asyncio.sleep(2)
```
## 8. Full Workflow (One Pass)
```python
ERROR_REMARK = '出错没找到最低价'
async def process_one_order(page, order_no):
"""Returns dict: {order, remark, success}"""
try:
opened = await open_shipping_popup(page, order_no)
if not opened:
return {'order': order_no, 'remark': ERROR_REMARK, 'success': False,
'reason': 'popup_failed'}
remark = await extract_temu_cheapest(page)
if not remark:
remark = ERROR_REMARK
success = False
else:
success = True
await add_remark(page, remark)
await close_popup(page)
return {'order': order_no, 'remark': remark, 'success': success}
except Exception as e:
try:
await add_remark(page, ERROR_REMARK)
await close_popup(page)
except Exception:
pass
return {'order': order_no, 'remark': ERROR_REMARK, 'success': False,
'reason': str(e)}
async def run_once():
async with async_playwright() as p:
browser = await p.chromium.launch(
executable_path=BROWSER_PATH,
headless=True,
args=['--no-sandbox', '--disable-dev-shm-usage', '--disable-gpu']
)
context = await browser.new_context()
page = await context.new_page()
if not await do_login(page, context):
await browser.close()
return []
await goto_wait_stock(page)
targets = await find_target_orders(page)
results = []
for order_no in targets:
r = await process_one_order(page, order_no)
print(r)
results.append(r)
await asyncio.sleep(3)
await browser.close()
return results
```
## 9. ⭐ Scheduled Loop
### Option A: Python in-process loop
```python
import time
INTERVAL_MINUTES = 30
async def scheduled_loop():
while True:
print(f'=== Run started at {datetime.now()} ===')
try:
results = await run_once()
success = sum(1 for r in results if r['success'])
print(f'Done: {success}/{len(results)} succeeded')
except Exception as e:
print(f'Run failed: {e}')
await asyncio.sleep(INTERVAL_MINUTES * 60)
if __name__ == '__main__':
asyncio.run(scheduled_loop())
```
### Option B: systemd timer (recommended for production)
Create `/etc/systemd/system/temu-shipping.service`:
```ini
[Unit]
Description=Tongtool TEMU Shipping Cost Auto-Remark
[Service]
Type=oneshot
User=ubuntu
WorkingDirectory=/home/ubuntu
ExecStart=/home/ubuntu/.hermes/hermes-agent/venv/bin/python3 /home/ubuntu/temu_shipping.py
```
Create `/etc/systemd/system/temu-shipping.timer`:
```ini
[Unit]
Description=Run TEMU shipping cost calc every 30 min
[Timer]
OnBootSec=5min
OnUnitActiveSec=30min
Persistent=true
[Install]
WantedBy=timers.target
```
Enable:
```bash
sudo systemctl daemon-reload
sudo systemctl enable --now temu-shipping.timer
```
### Option C: Hermes cronjob tool
```python
cronjob(
action='create',
name='tongtool-temu-shipping-auto',
schedule='*/30 * * * *', # every 30 min
prompt='Load skill tongtool-temu-shipping-calc and run run_once().',
skills=['tongtool-temu-shipping-calc'],
enabled_toolsets=['terminal', 'file', 'browser'],
deliver='local'
)
```
## Pitfalls & Lessons Learned
**Read `references/pitfalls_and_discoveries.md`** before writing new automation! Contains critical solutions for bypassing the infinite-skip pagination trap using OpenAPI, and preventing the selection of "未启用" (disabled) shipping methods during DOM scraping.
1. **Use TEMU平台接口试算, NOT 运费模板试算**: Default tab is 运费模板试算; must explicitly click `li[tabtitle="TEMU平台接口试算"]`. Running extraction without sorting returns wrong (template-based) data.
2. **Price is USD ($), not CNY (¥)**: TEMU API试算 returns US dollar prices formatted `$2.50`. Regex: `/\$\s*(\d+\.\d+)/`. Remark format: `GOFO,Standard 预估$2.50`.
3. **Error-recovery remark**: On ANY failure (popup timeout, empty price list, exception) → write `出错没找到最低价` so next run's filter (`hasRemark`) skips this order and doesn't retry infinitely.
4. **Filter must exclude already-remarked orders**: This is what makes scheduled looping idempotent. If row text contains `$x.xx`, `预估$`, or `出错没找到最低价` → already processed, skip.
5. **Captcha OCR filter**: `c.isascii() and c.isalnum()` before char substitution, else ddddocr CJK noise breaks login.
6. **"编辑" button**: exact match `=== '编辑'` + `getBoundingClientRect().width > 0`. Naive `includes('编辑')` hits invisible DOM.
7. **Popup is `div[windowid]`, not iframe**: Random `windowid` attribute, lives in main page DOM. Search by `#tabpanelDiv` structure, not by hardcoded ID.
8. **查询 button inside #temuShippingMethod must be clicked after tab switch**: Tab switch alone doesn't trigger API call.
9. **Dispatch input/change events after setting textarea value**: React/Vue components won't detect `.value = ...` assignment without events. Use `dispatchEvent(new Event('input'))`.
10. **Warehouse selection**: dropdown is custom component — click label parent, then `td[title="武汉仓库"]`.
11. **Generous waits between actions**: SPA uses ExtJS-style rendering; use 815s between clicks for reliable popup/tab rendering.
12. **Session 2026-04-28 test**: Successfully logged in and processed 2 TEMU half-managed orders out of 50 total orders.
13. **2026-04-28 单轮实测(limit=3**: 登录一次过(captcha='35cn2'),目标订单 1 个 `ALuckhalf-PO-211-17213720356471367`,成功写入备注 `[TEMU试算]USPS,Ground Advantage $3.32`(原始文本 `预估$3.32; 2-5 个工作日送达`)。脚本:`/home/ubuntu/scripts/tongtool_temu_once.py`(用法 `python3 scripts/tongtool_temu_once.py [limit]`)。
14. **carrier 列可能为空**: 某些 TEMU 平台返回的表只有 method 列而 carrier 列空,格式化备注时要做兜底 `label = f"{c}>>{m}" if c and m else (c or m)`,否则备注变成 `>>USPS,...` 这种首字段空的难看格式。
15. **避免 ddddocr subprocess 启动**: 用 subprocess 把图片字节 `{resp.content!r}` 作为参数拼进 python -c 命令会传超大二进制,极易失败。应在主进程直接 `import ddddocr; ocr.classification(bytes)`。venv `/home/ubuntu/.hermes/hermes-agent/venv/bin/python3` 已装 ddddocr。
16. **运行结果持久化**: 每轮输出 `/home/ubuntu/tongtool_temu_runs/run_<ts>.json`,便于事后审计和诊断。
17. **Order Number Extraction**: Use `textContent` and `.replace(/\s+/g, '')` instead of `innerText`. Tongtool inserts visual line breaks in long order numbers (e.g., `2ALuckhalf...726`), which breaks regex matching if whitespace isn't stripped.
18. **Order Number Pattern**: TEMU half-managed orders do NOT always contain the word `half` (e.g., `88NewGrou-PO-...`). Use `([A-Za-z0-9]+-PO-\d+-\d+)` to match all of them.
19. **Status Filter**: Removed DOM-based status checks per user preference. Rely purely on idempotency (checking for `[TEMU试算]` or `出错没找到最低价`) to determine if an order needs processing.
20. **Tongtool 订单状态生命周期**: 订单状态依次流转为:`订单生成``付款``检查``配货``物流商下单``通途打印``通途发货``物流商发货``签收`。筛选时,若要处理发货前的订单,需排查包含 `通途打印` 及后续状态的关键字,而不能仅依赖 `已发货`,因为“通途打印”标志着仓库作业已实质开始。
21. **⚠️ PASSWORD 被占位符替换导致登录失败**: 脚本文件(如 `tongtool_temu_once_v2.py`)中的 `PASSWORD='***'` 是脱敏占位符,实际运行时会被拒绝。若 OCR 返回的验证码看起来正常(如 `g72fg``55m38d`)但登录反复失败,先用 `grep -n "PASSWORD"` 检查是否为 `'***'``'111111'`。可通过以下命令批量检查目录下所有脚本:`grep -rn "PASSWORD.*='\\*\\*\\*'" /home/ubuntu/scripts/`
22. **浏览器启动参数需加 `ignore_https_errors=True`**: `context = await browser.new_context(ignore_https_errors=True)` 避免 Tongtool 子域(如 `yijiety.tongtool.com`)的证书问题阻断页面加载。
23. **`goto_wait_stock` 需等待 30 秒**: 待配货页 (`assignStockSelfMaintenance/index.htm`) 是重度 SPA,首次加载后 ExtJS 组件渲染耗时约 20-30 秒。`await asyncio.sleep(30)` 后再操作仓库下拉框,否则选择器可能匹配不到 DOM。
24. **单脚本文件路径**: 当前稳定版本为 `/home/ubuntu/scripts/tongtool_temu_once_v2.py`,用法:`python3 scripts/tongtool_temu_once_v2.py [limit]`。运行结果自动写入 `/home/ubuntu/tongtool_temu_runs/run_<ts>.json`
25. **"未启用" (Disabled) Shipping Options**: TEMU API often returns shipping methods that are disabled ("未启用") and completely lack carrier/method text, causing them to be parsed with empty labels (e.g., `>>undefined`) but sometimes containing valid prices. The DOM scraper must explicitly check `rowText.includes('未启用')` and skip the row, and also skip saving entries where both `carrier` and `method` are empty.
26. **OpenAPI Pagination Bypass**: To process batches of orders without getting stuck on paginated UI containing already-handled orders, query `open.tongtool.com/apiv3-service/openapi/tongtool/ordersQuery` (using `merchantId`, `pageNo`, and `pageSize`) to get a clean list of orders first (filtering out ones with `[TEMU试算]` in remark), sort them by the last 4 digits of order ID, and feed this target list to the UI automation script.