Hi everyone,
I wanted to share a small scripting pattern and ask how others handle upstream socket resets when running automated batch telemetry and headless browser tasks (Playwright/Selenium) over rotating 4G/5G mobile gateways on Debian/Siduction.
When cascading requests through cellular proxy endpoints, mobile network operators periodically rotate bearer IPs behind Carrier-Grade NAT (CGNAT). Under IETF RFC 7230 Section 6.3 persistent connection standard (https://datatracker.ietf.org/doc/html/rfc7230), clients expect persistent TCP connections, but when the carrier IP renews, active sockets in the local connection pool receive unexpected
ConnectionResetError: [Errno 104] Connection reset by peer rather than an orderly termination.
To avoid failing long-running jobs or crashing browser contexts, we implemented an explicit backoff and socket recycling wrapper using urllib3 and requests adapters:
import time
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
class ResilientProxySession(requests.Session):
def __init__(self, proxy_url, max_retries=3):
super().__init__()
self.proxies = {'http': proxy_url, 'https': proxy_url}
retries = Retry(
total=max_retries,
backoff_factor=1.5,
status_forcelist=[502, 503, 504],
raise_on_status=False
)
adapter = HTTPAdapter(max_retries=retries, pool_connections=10, pool_maxsize=10)
self.mount('http://', adapter)
self.mount('https://', adapter)
For headless browser orchestration, similar connection handling and proxy server routing strategies are outlined in the Playwright Python network proxy documentation (https://playwright.dev/python/docs/network) and comparative benchmarks are detailed in this Selenium and Playwright mobile proxy automation architecture (https://pxm2.io/use-cases/web-scraping/mobile-proxy-selenium-playwright/) guide. Standard Python socket options can also be referenced in the Python urllib socket documentation (https://docs.python.org/3/library/urllib.request.html).
Do you typically wrap socket renewal at the application layer with backoff adapters like this, or do you handle gateway renegotiations via local systemd watchdog / redsocks routing tables?
Curious to hear what patterns people find most reliable for long-running network workers on Siduction.