I have a python script running a headless Selenium Webdriver, looking up many individual records from a site. Most of the time it runs fine but every 10 minutes or so it starts hitting 403 errors. From trying different delayed retries, I've found that the 403s consistently happen for about 45-60 seconds. So the best I can do as a workaround is sleep for 60 seconds once I hit a 403, then resume normal requests. I've tried setting a non-headless user agent, namely Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36. This didn't help. This is also consistent whether I'm running from my local machine or from an EC2 instance.

What else can I try?