Add archived/tongtool-temu-y2-cheapest-scraper
This commit is contained in:
@@ -0,0 +1,243 @@
|
|||||||
|
---
|
||||||
|
name: tongtool-temu-y2-cheapest-scraper
|
||||||
|
skill_name: tongtool-temu-y2-cheapest-scraper
|
||||||
|
category: cross-border-ecommerce
|
||||||
|
description: >
|
||||||
|
全流程自动化:登录Tongtool(通途)ERP → 筛选产品 → 抓取TEMU Y2 cheapest价格数据 → 添加订单备注。
|
||||||
|
适用于从通途ERP批量采集TEMU最低价并同步记录订单备注信息。
|
||||||
|
tags: [tongtool, temu, y2, scraping, e-commerce, price-monitoring, 通途, 订单备注]
|
||||||
|
---
|
||||||
|
|
||||||
|
# 通途 ERP → TEMU Y2 cheapest 抓取 + 订单备注流程
|
||||||
|
|
||||||
|
## 前置条件
|
||||||
|
- 通途账号:`ozonezsjgw@163.com`(Cookie 已保存,通常可自动登录)
|
||||||
|
- 通途 Cookie 文件:`/home/ubuntu/.hermes/cookies/tongtool_cookies.json`
|
||||||
|
- 工具:Playwright(Node.js)或 DrissionPage
|
||||||
|
- 页面特征:通途ERP TEMU → Y2版本 → cheapest列、订单详情页备注框
|
||||||
|
|
||||||
|
## 步骤 1:登录通途 ERP
|
||||||
|
|
||||||
|
### 1.1 加载 Cookie 免密登录(推荐)
|
||||||
|
```python
|
||||||
|
import json
|
||||||
|
from playwright.sync_api import sync_playwright
|
||||||
|
|
||||||
|
with sync_playwright() as p:
|
||||||
|
browser = p.chromium.launch(headless=True)
|
||||||
|
context = browser.new_context(ignore_https_errors=True)
|
||||||
|
|
||||||
|
# 加载已保存的 Cookie
|
||||||
|
with open('/home/ubuntu/.hermes/cookies/tongtool_cookies.json', 'r') as f:
|
||||||
|
cookies = json.load(f)
|
||||||
|
context.add_cookies(cookies)
|
||||||
|
|
||||||
|
page = context.new_page()
|
||||||
|
page.goto("https://erp.tongtool.com")
|
||||||
|
# Cookie有效则自动进入工作台;若失效则走账号密码重登
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 账号密码登录(备用)
|
||||||
|
- 账号:`ozonezsjgw@163.com`
|
||||||
|
- 密码:根据实际凭证
|
||||||
|
- 注意:通途可能有滑动验证码,参照 `captcha-auto-login` skill
|
||||||
|
|
||||||
|
## 步骤 2:导航至 TEMU Y2 页面
|
||||||
|
|
||||||
|
### 2.1 进入 TEMU 模块
|
||||||
|
```python
|
||||||
|
page.click('text=TEMU 半托管') # 或对应菜单文字
|
||||||
|
page.wait_for_load_state('networkidle')
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.2 切换到 Y2 视图
|
||||||
|
```python
|
||||||
|
page.click('text=Y2') # 页面上的 Y2 标签/按钮
|
||||||
|
page.wait_for_timeout(1000)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 确认页面定位成功
|
||||||
|
```python
|
||||||
|
assert page.locator('text=Y2 cheapest, nth=0').is_visible() or \
|
||||||
|
page.locator('th:has-text("Y2 cheapest")').count() > 0
|
||||||
|
```
|
||||||
|
|
||||||
|
## 步骤 3:筛选目标产品
|
||||||
|
|
||||||
|
常用筛选条件:
|
||||||
|
- 品类/类目
|
||||||
|
- 店铺
|
||||||
|
- 价格区间
|
||||||
|
- 上架状态
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 示例:展开筛选面板并设置条件
|
||||||
|
page.click('button:has-text("筛选")')
|
||||||
|
page.select_option('select[name="category"]', '家居百货') # 根据实际控件调整
|
||||||
|
page.click('button:has-text("查询")')
|
||||||
|
page.wait_for_timeout(1500)
|
||||||
|
```
|
||||||
|
|
||||||
|
等待列表加载:
|
||||||
|
```python
|
||||||
|
page.wait_for_selector('table tbody tr')
|
||||||
|
```
|
||||||
|
|
||||||
|
## 步骤 4:抓取 Y2 cheapest 数据
|
||||||
|
|
||||||
|
### 4.1 定位 Y2 cheapest 列
|
||||||
|
- 表头关键词:`Y2 cheapest`
|
||||||
|
- 值特征:数字,可能带 `$` 或 `USD` 前缀,颜色可能标识低价优势
|
||||||
|
|
||||||
|
### 4.2 单页抓取
|
||||||
|
```python
|
||||||
|
import json
|
||||||
|
|
||||||
|
data = []
|
||||||
|
rows = page.query_selector_all('table tbody tr')
|
||||||
|
|
||||||
|
for row in rows:
|
||||||
|
cells = row.query_selector_all('td')
|
||||||
|
product_name = cells[1].inner_text().strip() if len(cells) > 1 else ""
|
||||||
|
y2_cheapest = cells[-2].inner_text().strip().replace('$', '').replace(',', '') if len(cells) > 2 else ""
|
||||||
|
|
||||||
|
data.append({
|
||||||
|
"product_name": product_name,
|
||||||
|
"y2_cheapest": float(y2_cheapest) if y2_cheapest else None
|
||||||
|
})
|
||||||
|
|
||||||
|
with open('temu_y2_cheapest.json', 'w', encoding='utf-8') as f:
|
||||||
|
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.3 分页抓取
|
||||||
|
```python
|
||||||
|
while True:
|
||||||
|
scrape_page()
|
||||||
|
next_btn = page.query_selector('button.next:enabled, a:has-text("下一页"):not(.disabled)')
|
||||||
|
if not next_btn or "disabled" in (next_btn.get_attribute('class') or ''):
|
||||||
|
break
|
||||||
|
next_btn.click()
|
||||||
|
page.wait_for_timeout(2000)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 步骤 5:添加订单备注
|
||||||
|
|
||||||
|
在抓取 Y2 cheapest 后,为对应订单添加备注以便后续运营跟踪。
|
||||||
|
|
||||||
|
### 5.1 打开订单详情页
|
||||||
|
列表页中点击目标订单的「订单号」或「详情」按钮:
|
||||||
|
```python
|
||||||
|
# 点击第一行的"查看详情"或订单号链接(假设操作列在末尾)
|
||||||
|
first_row = page.query_selector('table tbody tr')
|
||||||
|
detail_btn = first_row.query_selector('a:has-text("详情"), button:has-text("详情")')
|
||||||
|
if detail_btn:
|
||||||
|
detail_btn.click()
|
||||||
|
page.wait_for_timeout(1500)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 打开备注/留言输入区域
|
||||||
|
备注入口通常位于:
|
||||||
|
- 订单详情页的 **"订单备注"** 标签页
|
||||||
|
- 右下角弹出的备注图标
|
||||||
|
- 页面底部的备注输入框
|
||||||
|
|
||||||
|
```python
|
||||||
|
# 方法 A:点击备注标签
|
||||||
|
remark_tab = page.locator('.el-tabs__item:has-text("订单备注"), text=订单备注')
|
||||||
|
if remark_tab.count() > 0:
|
||||||
|
remark_tab.click()
|
||||||
|
page.wait_for_timeout(500)
|
||||||
|
|
||||||
|
# 方法 B:直接定位备注文本域
|
||||||
|
remark_box = page.locator('textarea[placeholder*="备注"], textarea[placeholder*="留言"], .remark-input')
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 填写并保存备注
|
||||||
|
```python
|
||||||
|
remark_text = f"Y2 cheapest 采集价: ${price_val} — 采集时间: {datetime.now().strftime('%Y-%m-%d %H:%M')}"
|
||||||
|
|
||||||
|
# 填入备注框
|
||||||
|
page.fill('textarea[placeholder*="备注"]', remark_text)
|
||||||
|
|
||||||
|
# 点击保存/提交
|
||||||
|
page.click('button:has-text("保存备注"), button:has-text("提交"), button:has-text("确定")')
|
||||||
|
page.wait_for_timeout(800)
|
||||||
|
|
||||||
|
# 可选:等待保存成功提示
|
||||||
|
page.wait_for_selector('.el-notification:has-text("成功"), .el-message:has-text("保存成功")')
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.4 快捷:列表页批量添加备注(如支持)
|
||||||
|
部分版本支持在列表行内直接展开备注:
|
||||||
|
```python
|
||||||
|
# 点击备注图标展开编辑
|
||||||
|
remark_icon = row.query_selector('i.icon-remark, span:has-text("备注")')
|
||||||
|
if remark_icon:
|
||||||
|
remark_icon.click()
|
||||||
|
page.wait_for_timeout(300)
|
||||||
|
page.fill('.inline-remark-input', remark_text)
|
||||||
|
page.press('.inline-remark-input', 'Enter')
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.5 批量备注完整示例
|
||||||
|
结合抓取与备注:
|
||||||
|
```python
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
rows = page.query_selector_all('table tbody tr')
|
||||||
|
for idx, row in enumerate(rows):
|
||||||
|
cells = row.query_selector_all('td')
|
||||||
|
if len(cells) < 3:
|
||||||
|
continue
|
||||||
|
|
||||||
|
order_no = cells[0].inner_text().strip() # 订单编号
|
||||||
|
y2_price = cells[-2].inner_text().strip().replace('$', '')
|
||||||
|
|
||||||
|
remark = f"TEMU Y2 cheapest: {y2_price} | 采集: {datetime.now().strftime('%Y-%m-%d')}"
|
||||||
|
|
||||||
|
# 进入详情写备注
|
||||||
|
detail = row.query_selector('a:has-text("详情")')
|
||||||
|
if detail:
|
||||||
|
detail.click()
|
||||||
|
page.wait_for_timeout(1000)
|
||||||
|
|
||||||
|
page.fill('textarea[placeholder*="备注"]', remark)
|
||||||
|
page.click('button:has-text("保存")')
|
||||||
|
page.wait_for_timeout(500)
|
||||||
|
|
||||||
|
# 返回列表
|
||||||
|
page.click('button:has-text("返回"), .back-btn')
|
||||||
|
page.wait_for_timeout(500)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 步骤 6(可选):导出 CSV
|
||||||
|
```python
|
||||||
|
import csv
|
||||||
|
|
||||||
|
with open('temu_y2_cheapest.csv', 'w', newline='', encoding='utf-8-sig') as f:
|
||||||
|
writer = csv.DictWriter(f, fieldnames=["product_name", "y2_cheapest"])
|
||||||
|
writer.writeheader()
|
||||||
|
writer.writerows(data)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 常见问题与排查
|
||||||
|
|
||||||
|
| 问题 | 原因 | 解决 |
|
||||||
|
|------|------|------|
|
||||||
|
| Cookie 登录失效 | Cookie过期或被注销 | 需重新手动登录并导出Cookie |
|
||||||
|
| 页面无 Y2 标签 | 账号权限或菜单结构不同 | 确认通途账号有TEMU Y2权限 |
|
||||||
|
| Y2 cheapest 列值空 | 数据未加载或 lazily rendered | 滚动页面或等 `networkidle` 再抓 |
|
||||||
|
| 备注保存失败 | 输入框未聚焦或页面未正确渲染 | 先 `click()` 再 `fill()` 备注框 |
|
||||||
|
| 抓取频率受限 | 触发风控 | 增加随机延时 `random.uniform(2,5)` |
|
||||||
|
|
||||||
|
## 相关技能
|
||||||
|
- `tongtool-erp-automation` — 通途ERP通用自动化
|
||||||
|
- `tongtool-temu-shipping-calc` — 通途TEMU运费计算
|
||||||
|
- `captcha-auto-login` — 验证码自动识别登录
|
||||||
|
- `node-playwright-web-automation` — Node.js Playwright 自动化指南
|
||||||
|
|
||||||
|
## 安全提示
|
||||||
|
- Cookie 文件含敏感凭证,勿上传公开仓库
|
||||||
|
- 建议对 `tongtool_cookies.json` 设置 `chmod 600`
|
||||||
|
- 遵守通途平台服务条款,避免高频自动化导致封号
|
||||||
Reference in New Issue
Block a user