Files
atomk-hermes-skills/skills/archived/tongtool-temu-shipping-calc/SKILL.md
T

559 lines
23 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
name: tongtool-temu-shipping-calc
description: Playwright automation for Tongtool ERP TEMU semi-managed orders — filter pre-print orders WITHOUT existing remarks, login with captcha retry, iterate through each order, open 仓储物流 → 邮寄方式解析 → TEMU平台接口试算, click 查询, find cheapest shipping price, write remark to order (or error remark on failure), and support scheduled looping.
---
# Tongtool TEMU Shipping Cost Calculator (v2)
End-to-end automation: **find pre-print (未完成通途打印) TEMU half-managed orders without remarks** → compute cheapest TEMU API shipping price → write remark back to order → repeat on schedule.
## Core Requirements (User Spec)
1. **筛选目标**:未完成通途打印步骤 **且** 没有备注 的 TEMU 半托管订单
2. **逐个处理**:订单详情 → 仓储物流 → 邮寄方式解析 → **TEMU平台接口试算**(⚠️ 不是运费模板试算)→ 点击查询 → 找出最低价
3. **写入备注**
- 成功:格式如 `GOFO,Standard 预估$2.50`
- 失败:写入 `出错没找到最低价`
4. **定时循环**:支持 cron/systemd 定时重复执行
## Prerequisites
- `playwright` + Chromium at `/home/ubuntu/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome`
- `ddddocr` for captcha OCR
- `requests` for captcha image download
- Tongtool account: `ozonezsjgw@163.com` / `111111`
## 1. Login with Captcha Retry
**Critical fix**: `ddddocr` may return CJK symbols/non-printable chars. Filter to ASCII alphanumeric before character substitution.
```python
import asyncio, re, subprocess, requests
from playwright.async_api import async_playwright
USERNAME = 'ozonezsjgw@163.com'
PASSWORD = '111111'
BROWSER_PATH = '/home/ubuntu/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome'
def correct_ocr(raw):
cleaned = ''.join(c for c in raw if c.isascii() and c.isalnum())
return cleaned.translate(str.maketrans('oliOSsZz', '01105522'))
async def do_login(page, context):
await page.goto('https://passport.tongtool.com/')
await asyncio.sleep(3)
await page.fill('input[name="username"]', USERNAME)
await page.fill('input[name="password"]', PASSWORD)
for attempt in range(5):
img = page.locator('img#captcha')
src = await img.get_attribute('src')
if src.startswith('/'):
src = 'https://passport.tongtool.com' + src
cookies = await context.cookies()
cookie_dict = {c['name']: c['value'] for c in cookies}
resp = requests.get(src, cookies=cookie_dict, timeout=10)
raw = subprocess.check_output(['python3', '-c', f"""
import ddddocr
print(ddddocr.DdddOcr(show_ad=False).classification({resp.content!r}))
"""]).decode().strip()
code = correct_ocr(raw)
if len(code) < 4:
await img.click()
await asyncio.sleep(2)
continue
await page.fill('input[name="captcha"]', code)
await page.click('button.btn-primary')
await asyncio.sleep(8)
if 'member' in page.url:
return True
return False
```
## 2. Navigate + Select Warehouse
```python
async def goto_wait_stock(page):
await page.goto('https://yijiety.tongtool.com/warehouse/assignStockSelfMaintenance/index.htm')
await asyncio.sleep(30)
# Open warehouse dropdown
await page.evaluate("""
var spans = document.querySelectorAll('span');
for (var s of spans) {
if (s.textContent.trim() === '请选择仓库:') {
s.parentElement.click(); break;
}
}
""")
await asyncio.sleep(3)
await page.evaluate("document.querySelector('td[title=\"武汉仓库\"]').click()")
await asyncio.sleep(15)
# Click 查询
await page.evaluate("""
var all = document.querySelectorAll('a, button, span');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '查询') { all[i].click(); break; }
}
""")
await asyncio.sleep(10)
```
## 3. ⭐ Filter: Pre-Print TEMU Half-Managed Orders WITHOUT Remarks
Extract each row's: order_no, print status, existing remark. Keep only rows where **Tongtool print step is NOT completed AND remark is empty**.
**Critical Update**: Do not rely purely on UI scraping (which often loops over the same first page of 50 already-skipped items). Instead, fetch a fresh list of target un-remarked orders using the Tongtool OpenAPI backend, then feed that list into the UI script to force processing of unseen pages/orders.
```python
async def find_target_orders(page):
"""Returns list of target order numbers.
- Matches TEMU order pattern (-PO-)
- Relies on idempotency (skips if already remarked)
"""
targets = await page.evaluate("""() => {
var rows = document.querySelectorAll('tr, div.grid-row, div.x-grid3-row');
var results = [];
for (var row of rows) {
var orderNo = '';
// Remove whitespace from textContent to prevent regex breaking on visually wrapped long IDs
for (var cell of row.querySelectorAll('td, div.cell')) {
var txt = (cell.textContent || '').replace(/\\s+/g, '');
var m = txt.match(/([A-Za-z0-9]+-PO-\\d+-\\d+)/i);
if (m) { orderNo = m[1]; break; }
}
if (!orderNo) continue;
// Idempotency: skip if already processed
var rowText = row.textContent || '';
var hasRemark = rowText.includes('[TEMU试算]') || rowText.includes('出错没找到最低价');
if (hasRemark) continue;
results.push(orderNo);
}
return results;
}""")
# Deduplicate
seen = set()
unique = []
for o in targets:
if o not in seen:
seen.add(o)
unique.append(o)
print(f'Found {len(unique)} pre-print, un-remarked TEMU half-managed orders')
return unique
```
> **Note**: Exact selectors for order status and remark columns may vary. If the default logic misses, use DevTools to find the specific column class/index and adjust the `rowText` check to read targeted `tds[idx]` instead.
## 4. Open 邮寄方式解析 Popup (per order)
**Critical**: 编辑 button requires exact text match `=== '编辑'` + visibility check. `includes('编辑')` matches invisible nodes.
```python
async def open_shipping_popup(page, order_no):
# Close residual popups
await page.evaluate("""
var closeBtns = document.querySelectorAll('.jclose-blue, .close, [title="关闭"], .layui-layer-close');
for (var i=0; i<closeBtns.length; i++) closeBtns[i].click();
""")
await asyncio.sleep(2)
# Click order number link
clicked = await page.evaluate(f"""() => {{
var links = document.querySelectorAll('a');
for (var i=0; i<links.length; i++) {{
if (links[i].textContent.includes('{order_no}')) {{
links[i].click(); return true;
}}
}}
return false;
}}""")
if not clicked:
return False
await asyncio.sleep(12)
# Click 仓储物流
await page.evaluate("""() => {
var all = document.querySelectorAll('*');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '仓储物流' && all[i].offsetHeight > 0) {
all[i].click(); return;
}
}
}""")
await asyncio.sleep(8)
# Click 编辑 — exact match + visibility
await page.evaluate("""() => {
var all = document.querySelectorAll('a, button, span');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '编辑') {
var rect = all[i].getBoundingClientRect();
if (rect.width > 0 && rect.height > 0) {
all[i].click(); return;
}
}
}
}""")
await asyncio.sleep(8)
# Click 邮寄方式解析
opened = await page.evaluate("""() => {
var all = document.querySelectorAll('a');
for (var i=0; i<all.length; i++) {
if (all[i].textContent.trim() === '邮寄方式解析') {
all[i].click(); return true;
}
}
return false;
}""")
await asyncio.sleep(12)
return opened
```
## 5. ⭐ Switch to TEMU平台接口试算 Tab + Click 查询 + Extract Cheapest
**Critical**:
- Popup is a `div[windowid]` with `.jbox` class (NOT iframe)
- Tab panel has `id="tabpanelDiv"`, tabs are `<li tabtitle="...">`
- **Must click `TEMU平台接口试算` tab** (NOT `运费模板试算`)
- After switching tab, **must click 查询** button inside `#temuShippingMethod` to trigger API call
- Prices come back as e.g. `GOFO` (carrier) / `Standard` (method) / `$2.50` (USD price)
```python
async def extract_temu_cheapest(page):
"""Returns string like 'GOFO,Standard 预估$2.50' or None on failure."""
# Step 1: Switch to TEMU tab
switched = await page.evaluate("""() => {
var popups = document.querySelectorAll('div[windowid]');
var popup = null;
for (var p of popups) {
if (p.querySelector('#tabpanelDiv')) { popup = p; break; }
}
if (!popup) return false;
var tab = popup.querySelector('li[tabtitle="TEMU平台接口试算"]');
if (tab) { tab.click(); return true; }
return false;
}""")
if not switched:
return None
await asyncio.sleep(10)
# Step 2: Click 查询 button inside #temuShippingMethod
await page.evaluate("""() => {
var popups = document.querySelectorAll('div[windowid]');
var popup = null;
for (var p of popups) {
if (p.querySelector('#tabpanelDiv')) { popup = p; break; }
}
if (!popup) return;
var temuDiv = popup.querySelector('#temuShippingMethod');
if (temuDiv && !temuDiv.classList.contains('hide')) {
var btns = temuDiv.querySelectorAll('a, button');
for (var b of btns) {
if (b.textContent.includes('查询') || b.textContent.includes('试算')) {
b.click(); return;
}
}
}
}""")
await asyncio.sleep(12)
# Step 3: Extract all carrier/method/price triples and find cheapest
prices = await page.evaluate("""() => {
var popups = document.querySelectorAll('div[windowid]');
var popup = null;
for (var p of popups) {
if (p.querySelector('#tabpanelDiv')) { popup = p; break; }
}
if (!popup) return [];
var temuDiv = popup.querySelector('#temuShippingMethod');
if (!temuDiv) return [];
var results = [];
var rows = temuDiv.querySelectorAll('tr');
for (var row of rows) {
var tds = row.querySelectorAll('td');
if (tds.length < 3) continue;
// Try to find USD price ($x.xx) in the row
var rowText = row.textContent;
var currency = '$';
var priceMatch = rowText.match(/\\$\\s*(\\d+\\.\\d+)/);
if (!priceMatch) {
priceMatch = rowText.match(/[¥¥]\\s*(\\d+\\.\\d+)/);
currency = '¥';
}
if (!priceMatch) continue;
// Skip disabled options ("未启用") entirely
if (rowText.includes('未启用')) continue;
// carrier usually in td[1] or td[0], method in td[2]
var carrier = '', method = '';
for (var td of tds) {
var t = td.textContent.trim();
if (!t || t === priceMatch[0]) continue;
if (!carrier) { carrier = t; continue; }
if (!method) { method = t; break; }
}
if (!carrier && !method) continue;
results.push({
carrier: carrier,
method: method,
currency: currency,
price: parseFloat(priceMatch[1])
});
}
return results;
}""")
if not prices:
return None
cheapest = min(prices, key=lambda x: x['price'])
# Format: 'GOFO,Standard 预估$2.50'
# Fallback to handle missing carrier
c = (cheapest.get('carrier') or '').strip()
m = (cheapest.get('method') or '').strip()
label = f"{c}>>{m}" if c and m else (c or m)
return f"{label} {cheapest['currency']}{cheapest['price']:.2f}"
```
## 6. Add Order Remark (success OR error)
```python
async def add_remark(page, remark_text):
"""Add remark text (success price OR error message) to the open order."""
clicked = await page.evaluate("""() => {
var btn = document.querySelector('#addOrderRemark');
if (btn) {
var a = btn.querySelector('a');
if (a) { a.click(); return true; }
}
return false;
}""")
if not clicked:
return False
await asyncio.sleep(5)
# Fill textarea (escape single quotes in remark_text)
escaped = remark_text.replace("'", "\\'").replace('"', '\\"')
await page.evaluate(f"""() => {{
var ta = document.querySelector('textarea');
if (ta) {{
ta.value = '{escaped}';
ta.dispatchEvent(new Event('input', {{bubbles:true}}));
ta.dispatchEvent(new Event('change', {{bubbles:true}}));
}}
}}""")
await asyncio.sleep(1)
# Click 保存
saved = await page.evaluate("""() => {
var links = document.querySelectorAll('a, button');
for (var i=0; i<links.length; i++) {
if (links[i].textContent.trim() === '保存') {
links[i].click(); return true;
}
}
return false;
}""")
await asyncio.sleep(3)
return saved
```
## 7. Close Popup (between orders)
```python
async def close_popup(page):
await page.evaluate("""
var closeBtns = document.querySelectorAll('.jclose-blue, .close, [title="关闭"], .layui-layer-close');
for (var i=0; i<closeBtns.length; i++) closeBtns[i].click();
""")
await asyncio.sleep(2)
```
## 8. Full Workflow (One Pass)
```python
ERROR_REMARK = '出错没找到最低价'
async def process_one_order(page, order_no):
"""Returns dict: {order, remark, success}"""
try:
opened = await open_shipping_popup(page, order_no)
if not opened:
return {'order': order_no, 'remark': ERROR_REMARK, 'success': False,
'reason': 'popup_failed'}
remark = await extract_temu_cheapest(page)
if not remark:
remark = ERROR_REMARK
success = False
else:
success = True
await add_remark(page, remark)
await close_popup(page)
return {'order': order_no, 'remark': remark, 'success': success}
except Exception as e:
try:
await add_remark(page, ERROR_REMARK)
await close_popup(page)
except Exception:
pass
return {'order': order_no, 'remark': ERROR_REMARK, 'success': False,
'reason': str(e)}
async def run_once():
async with async_playwright() as p:
browser = await p.chromium.launch(
executable_path=BROWSER_PATH,
headless=True,
args=['--no-sandbox', '--disable-dev-shm-usage', '--disable-gpu']
)
context = await browser.new_context()
page = await context.new_page()
if not await do_login(page, context):
await browser.close()
return []
await goto_wait_stock(page)
targets = await find_target_orders(page)
results = []
for order_no in targets:
r = await process_one_order(page, order_no)
print(r)
results.append(r)
await asyncio.sleep(3)
await browser.close()
return results
```
## 9. ⭐ Scheduled Loop
### Option A: Python in-process loop
```python
import time
INTERVAL_MINUTES = 30
async def scheduled_loop():
while True:
print(f'=== Run started at {datetime.now()} ===')
try:
results = await run_once()
success = sum(1 for r in results if r['success'])
print(f'Done: {success}/{len(results)} succeeded')
except Exception as e:
print(f'Run failed: {e}')
await asyncio.sleep(INTERVAL_MINUTES * 60)
if __name__ == '__main__':
asyncio.run(scheduled_loop())
```
### Option B: systemd timer (recommended for production)
Create `/etc/systemd/system/temu-shipping.service`:
```ini
[Unit]
Description=Tongtool TEMU Shipping Cost Auto-Remark
[Service]
Type=oneshot
User=ubuntu
WorkingDirectory=/home/ubuntu
ExecStart=/home/ubuntu/.hermes/hermes-agent/venv/bin/python3 /home/ubuntu/temu_shipping.py
```
Create `/etc/systemd/system/temu-shipping.timer`:
```ini
[Unit]
Description=Run TEMU shipping cost calc every 30 min
[Timer]
OnBootSec=5min
OnUnitActiveSec=30min
Persistent=true
[Install]
WantedBy=timers.target
```
Enable:
```bash
sudo systemctl daemon-reload
sudo systemctl enable --now temu-shipping.timer
```
### Option C: Hermes cronjob tool
```python
cronjob(
action='create',
name='tongtool-temu-shipping-auto',
schedule='*/30 * * * *', # every 30 min
prompt='Load skill tongtool-temu-shipping-calc and run run_once().',
skills=['tongtool-temu-shipping-calc'],
enabled_toolsets=['terminal', 'file', 'browser'],
deliver='local'
)
```
## Pitfalls & Lessons Learned
**Read `references/pitfalls_and_discoveries.md`** before writing new automation! Contains critical solutions for bypassing the infinite-skip pagination trap using OpenAPI, and preventing the selection of "未启用" (disabled) shipping methods during DOM scraping.
1. **Use TEMU平台接口试算, NOT 运费模板试算**: Default tab is 运费模板试算; must explicitly click `li[tabtitle="TEMU平台接口试算"]`. Running extraction without sorting returns wrong (template-based) data.
2. **Price is USD ($), not CNY (¥)**: TEMU API试算 returns US dollar prices formatted `$2.50`. Regex: `/\$\s*(\d+\.\d+)/`. Remark format: `GOFO,Standard 预估$2.50`.
3. **Error-recovery remark**: On ANY failure (popup timeout, empty price list, exception) → write `出错没找到最低价` so next run's filter (`hasRemark`) skips this order and doesn't retry infinitely.
4. **Filter must exclude already-remarked orders**: This is what makes scheduled looping idempotent. If row text contains `$x.xx`, `预估$`, or `出错没找到最低价` → already processed, skip.
5. **Captcha OCR filter**: `c.isascii() and c.isalnum()` before char substitution, else ddddocr CJK noise breaks login.
6. **"编辑" button**: exact match `=== '编辑'` + `getBoundingClientRect().width > 0`. Naive `includes('编辑')` hits invisible DOM.
7. **Popup is `div[windowid]`, not iframe**: Random `windowid` attribute, lives in main page DOM. Search by `#tabpanelDiv` structure, not by hardcoded ID.
8. **查询 button inside #temuShippingMethod must be clicked after tab switch**: Tab switch alone doesn't trigger API call.
9. **Dispatch input/change events after setting textarea value**: React/Vue components won't detect `.value = ...` assignment without events. Use `dispatchEvent(new Event('input'))`.
10. **Warehouse selection**: dropdown is custom component — click label parent, then `td[title="武汉仓库"]`.
11. **Generous waits between actions**: SPA uses ExtJS-style rendering; use 815s between clicks for reliable popup/tab rendering.
12. **Session 2026-04-28 test**: Successfully logged in and processed 2 TEMU half-managed orders out of 50 total orders.
13. **2026-04-28 单轮实测(limit=3**: 登录一次过(captcha='35cn2'),目标订单 1 个 `ALuckhalf-PO-211-17213720356471367`,成功写入备注 `[TEMU试算]USPS,Ground Advantage $3.32`(原始文本 `预估$3.32; 2-5 个工作日送达`)。脚本:`/home/ubuntu/scripts/tongtool_temu_once.py`(用法 `python3 scripts/tongtool_temu_once.py [limit]`)。
14. **carrier 列可能为空**: 某些 TEMU 平台返回的表只有 method 列而 carrier 列空,格式化备注时要做兜底 `label = f"{c}>>{m}" if c and m else (c or m)`,否则备注变成 `>>USPS,...` 这种首字段空的难看格式。
15. **避免 ddddocr subprocess 启动**: 用 subprocess 把图片字节 `{resp.content!r}` 作为参数拼进 python -c 命令会传超大二进制,极易失败。应在主进程直接 `import ddddocr; ocr.classification(bytes)`。venv `/home/ubuntu/.hermes/hermes-agent/venv/bin/python3` 已装 ddddocr。
16. **运行结果持久化**: 每轮输出 `/home/ubuntu/tongtool_temu_runs/run_<ts>.json`,便于事后审计和诊断。
17. **Order Number Extraction**: Use `textContent` and `.replace(/\s+/g, '')` instead of `innerText`. Tongtool inserts visual line breaks in long order numbers (e.g., `2ALuckhalf...726`), which breaks regex matching if whitespace isn't stripped.
18. **Order Number Pattern**: TEMU half-managed orders do NOT always contain the word `half` (e.g., `88NewGrou-PO-...`). Use `([A-Za-z0-9]+-PO-\d+-\d+)` to match all of them.
19. **Status Filter**: Removed DOM-based status checks per user preference. Rely purely on idempotency (checking for `[TEMU试算]` or `出错没找到最低价`) to determine if an order needs processing.
20. **Tongtool 订单状态生命周期**: 订单状态依次流转为:`订单生成``付款``检查``配货``物流商下单``通途打印``通途发货``物流商发货``签收`。筛选时,若要处理发货前的订单,需排查包含 `通途打印` 及后续状态的关键字,而不能仅依赖 `已发货`,因为“通途打印”标志着仓库作业已实质开始。
21. **⚠️ PASSWORD 被占位符替换导致登录失败**: 脚本文件(如 `tongtool_temu_once_v2.py`)中的 `PASSWORD='***'` 是脱敏占位符,实际运行时会被拒绝。若 OCR 返回的验证码看起来正常(如 `g72fg``55m38d`)但登录反复失败,先用 `grep -n "PASSWORD"` 检查是否为 `'***'``'111111'`。可通过以下命令批量检查目录下所有脚本:`grep -rn "PASSWORD.*='\\*\\*\\*'" /home/ubuntu/scripts/`
22. **浏览器启动参数需加 `ignore_https_errors=True`**: `context = await browser.new_context(ignore_https_errors=True)` 避免 Tongtool 子域(如 `yijiety.tongtool.com`)的证书问题阻断页面加载。
23. **`goto_wait_stock` 需等待 30 秒**: 待配货页 (`assignStockSelfMaintenance/index.htm`) 是重度 SPA,首次加载后 ExtJS 组件渲染耗时约 20-30 秒。`await asyncio.sleep(30)` 后再操作仓库下拉框,否则选择器可能匹配不到 DOM。
24. **单脚本文件路径**: 当前稳定版本为 `/home/ubuntu/scripts/tongtool_temu_once_v2.py`,用法:`python3 scripts/tongtool_temu_once_v2.py [limit]`。运行结果自动写入 `/home/ubuntu/tongtool_temu_runs/run_<ts>.json`
25. **"未启用" (Disabled) Shipping Options**: TEMU API often returns shipping methods that are disabled ("未启用") and completely lack carrier/method text, causing them to be parsed with empty labels (e.g., `>>undefined`) but sometimes containing valid prices. The DOM scraper must explicitly check `rowText.includes('未启用')` and skip the row, and also skip saving entries where both `carrier` and `method` are empty.
26. **OpenAPI Pagination Bypass**: To process batches of orders without getting stuck on paginated UI containing already-handled orders, query `open.tongtool.com/apiv3-service/openapi/tongtool/ordersQuery` (using `merchantId`, `pageNo`, and `pageSize`) to get a clean list of orders first (filtering out ones with `[TEMU试算]` in remark), sort them by the last 4 digits of order ID, and feed this target list to the UI automation script.