Add web/proxy-management

2026-07-10 16:10:24 +08:00
parent cd82d9e410
commit 001106b6be
1 changed files with 99 additions and 0 deletions
@@ -0,0 +1,99 @@
+---
+name: proxy-management
+description: "Proxy acquisition, testing, and management — free proxy lists, Bright Data commercial proxies, and browser cookie injection."
+version: 1.0.0
+author: Hermes Agent
+license: MIT
+metadata:
+  hermes:
+    tags: [proxy, bright-data, residential-proxy, socks5, scraping, free-proxy, commercial-proxy]
+    related_skills: [web-scraping-toolkit, camoufox-google-2fa-login]
+---
+
+# Proxy Management
+
+Comprehensive guide for acquiring, testing, and using proxies — from free lists to commercial Bright Data residential/datacenter proxies.
+
+## Choosing a Proxy Source
+
+| Need | Source | Cost |
+|------|--------|------|
+| Quick one-off scraping | Free proxy lists (proxyscrape.com) | Free |
+| Country/city targeting | Bright Data residential/DC | Paid |
+| High reliability | Bright Data datacenter zones | Paid |
+| Anti-bot bypass | Bright Data residential + browser | Paid |
+
+## Section 1: Free Proxy Lists
+
+### proxyscrape.com (recommended, no API key)
+
+```bash
+# SOCKS5 Russian proxy
+curl -s "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=socks5&timeout=5000&country=ru"
+
+# HTTP proxies
+curl -s "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=http&timeout=5000&country=us"
+```
+
+Parameters: `protocol` (http/https/socks5/socks4/all), `timeout` (ms), `country` (ISO code), `anonymity` (elite/anonymous/transparent)
+
+### Other Free Sources
+
+- geonode API: `https://proxylist.geonode.com/api/proxy-list?country=RU&limit=50`
+- free-proxy-list.net requires JS rendering (use DrissionPage or Playwright)
+
+### Pitfalls
+
+- Free proxies are unreliable — test before use, retry on failure
+- cn.proxy-tools.com is behind reCAPTCHA + paywall — use proxyscrape.com instead
+
+**See:** `references/proxy-testing.md` for full testing procedures and Playwright cookie injection.
+
+## Section 2: Bright Data — Commercial Proxies
+
+Bright Data provides residential, datacenter, ISP, and mobile proxy networks via API or control panel.
+
+**Key endpoints:**
+- Proxy: `brd.superproxy.io:22225` (non-SSL) or `brd.superproxy.io:33335` (SSL)
+- API: `https://api.brightdata.com`
+- Control panel: `https://brightdata.com/cp/zones`
+
+### API Authentication
+
+```python
+import urllib.request, json
+AK = "your-api-key"
+
+def api_get(endpoint):
+    req = urllib.request.Request("https://api.brightdata.com" + endpoint)
+    req.add_header("Authorization", "Bearer " + AK)
+    resp = urllib.request.urlopen(req, timeout=15)
+    return json.loads(resp.read().decode())
+```
+
+### Zone Management
+
+```bash
+# List all zones
+curl -s -H "Authorization: Bearer $AK" https://api.brightdata.com/zone | jq
+
+# Enable/disable zones
+curl -X PUT -H "Authorization: Bearer $AK" -H "Content-Type: application/json" \
+  -d '{"zone":"ZONE_NAME","enable":1}' https://api.brightdata.com/zone/set_enable
+```
+
+### Proxy Authentication Format
+
+```
+brd.superproxy.io:22225:zone-ZONE_NAME:PASSWORD
+```
+
+Country targeting: `-country-us` suffix in username. City targeting: `-city-us-newyork`.
+
+### Pitfalls
+
+- Zone passwords are separate from API key — retrieve via API or dashboard
+- SSL endpoint (33335) required for HTTPS targets
+- Residential proxies are metered — check GB usage before large scrapes
+
+**See:** `references/brightdata-proxy-setup.md` for full API reference, zone management, and password rotation.