3.3 KiB
3.3 KiB
name, description, version, author, license, metadata
| name | description | version | author | license | metadata | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| proxy-management | Proxy acquisition, testing, and management — free proxy lists, Bright Data commercial proxies, and browser cookie injection. | 1.0.0 | Hermes Agent | MIT |
|
Proxy Management
Comprehensive guide for acquiring, testing, and using proxies — from free lists to commercial Bright Data residential/datacenter proxies.
Choosing a Proxy Source
| Need | Source | Cost |
|---|---|---|
| Quick one-off scraping | Free proxy lists (proxyscrape.com) | Free |
| Country/city targeting | Bright Data residential/DC | Paid |
| High reliability | Bright Data datacenter zones | Paid |
| Anti-bot bypass | Bright Data residential + browser | Paid |
Section 1: Free Proxy Lists
proxyscrape.com (recommended, no API key)
# SOCKS5 Russian proxy
curl -s "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=socks5&timeout=5000&country=ru"
# HTTP proxies
curl -s "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=http&timeout=5000&country=us"
Parameters: protocol (http/https/socks5/socks4/all), timeout (ms), country (ISO code), anonymity (elite/anonymous/transparent)
Other Free Sources
- geonode API:
https://proxylist.geonode.com/api/proxy-list?country=RU&limit=50 - free-proxy-list.net requires JS rendering (use DrissionPage or Playwright)
Pitfalls
- Free proxies are unreliable — test before use, retry on failure
- cn.proxy-tools.com is behind reCAPTCHA + paywall — use proxyscrape.com instead
See: references/proxy-testing.md for full testing procedures and Playwright cookie injection.
Section 2: Bright Data — Commercial Proxies
Bright Data provides residential, datacenter, ISP, and mobile proxy networks via API or control panel.
Key endpoints:
- Proxy:
brd.superproxy.io:22225(non-SSL) orbrd.superproxy.io:33335(SSL) - API:
https://api.brightdata.com - Control panel:
https://brightdata.com/cp/zones
API Authentication
import urllib.request, json
AK = "your-api-key"
def api_get(endpoint):
req = urllib.request.Request("https://api.brightdata.com" + endpoint)
req.add_header("Authorization", "Bearer " + AK)
resp = urllib.request.urlopen(req, timeout=15)
return json.loads(resp.read().decode())
Zone Management
# List all zones
curl -s -H "Authorization: Bearer $AK" https://api.brightdata.com/zone | jq
# Enable/disable zones
curl -X PUT -H "Authorization: Bearer $AK" -H "Content-Type: application/json" \
-d '{"zone":"ZONE_NAME","enable":1}' https://api.brightdata.com/zone/set_enable
Proxy Authentication Format
brd.superproxy.io:22225:zone-ZONE_NAME:PASSWORD
Country targeting: -country-us suffix in username. City targeting: -city-us-newyork.
Pitfalls
- Zone passwords are separate from API key — retrieve via API or dashboard
- SSL endpoint (33335) required for HTTPS targets
- Residential proxies are metered — check GB usage before large scrapes
See: references/brightdata-proxy-setup.md for full API reference, zone management, and password rotation.