--- name: proxy-management description: "Proxy acquisition, testing, and management — free proxy lists, Bright Data commercial proxies, and browser cookie injection." version: 1.0.0 author: Hermes Agent license: MIT metadata: hermes: tags: [proxy, bright-data, residential-proxy, socks5, scraping, free-proxy, commercial-proxy] related_skills: [web-scraping-toolkit, camoufox-google-2fa-login] --- # Proxy Management Comprehensive guide for acquiring, testing, and using proxies — from free lists to commercial Bright Data residential/datacenter proxies. ## Choosing a Proxy Source | Need | Source | Cost | |------|--------|------| | Quick one-off scraping | Free proxy lists (proxyscrape.com) | Free | | Country/city targeting | Bright Data residential/DC | Paid | | High reliability | Bright Data datacenter zones | Paid | | Anti-bot bypass | Bright Data residential + browser | Paid | ## Section 1: Free Proxy Lists ### proxyscrape.com (recommended, no API key) ```bash # SOCKS5 Russian proxy curl -s "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=socks5&timeout=5000&country=ru" # HTTP proxies curl -s "https://api.proxyscrape.com/v2/?request=displayproxies&protocol=http&timeout=5000&country=us" ``` Parameters: `protocol` (http/https/socks5/socks4/all), `timeout` (ms), `country` (ISO code), `anonymity` (elite/anonymous/transparent) ### Other Free Sources - geonode API: `https://proxylist.geonode.com/api/proxy-list?country=RU&limit=50` - free-proxy-list.net requires JS rendering (use DrissionPage or Playwright) ### Pitfalls - Free proxies are unreliable — test before use, retry on failure - cn.proxy-tools.com is behind reCAPTCHA + paywall — use proxyscrape.com instead **See:** `references/proxy-testing.md` for full testing procedures and Playwright cookie injection. ## Section 2: Bright Data — Commercial Proxies Bright Data provides residential, datacenter, ISP, and mobile proxy networks via API or control panel. **Key endpoints:** - Proxy: `brd.superproxy.io:22225` (non-SSL) or `brd.superproxy.io:33335` (SSL) - API: `https://api.brightdata.com` - Control panel: `https://brightdata.com/cp/zones` ### API Authentication ```python import urllib.request, json AK = "your-api-key" def api_get(endpoint): req = urllib.request.Request("https://api.brightdata.com" + endpoint) req.add_header("Authorization", "Bearer " + AK) resp = urllib.request.urlopen(req, timeout=15) return json.loads(resp.read().decode()) ``` ### Zone Management ```bash # List all zones curl -s -H "Authorization: Bearer $AK" https://api.brightdata.com/zone | jq # Enable/disable zones curl -X PUT -H "Authorization: Bearer $AK" -H "Content-Type: application/json" \ -d '{"zone":"ZONE_NAME","enable":1}' https://api.brightdata.com/zone/set_enable ``` ### Proxy Authentication Format ``` brd.superproxy.io:22225:zone-ZONE_NAME:PASSWORD ``` Country targeting: `-country-us` suffix in username. City targeting: `-city-us-newyork`. ### Pitfalls - Zone passwords are separate from API key — retrieve via API or dashboard - SSL endpoint (33335) required for HTTPS targets - Residential proxies are metered — check GB usage before large scrapes **See:** `references/brightdata-proxy-setup.md` for full API reference, zone management, and password rotation.