Implement rate limiting for an external API
At work I maintain a NestJS microservice which is used by many other engineers. One of the features depends on a third party API for which I need to implement rate limit:
- The API has a soft limit of 20 req/s
- There is no way to programmatically monitor this limit on their end
- Surpassing the limit means that when someone looks at the dashboard then they will manually disable us, and we have to start another manual process to unblock us
- This API is owned by the government, so we can't ask or hope for changes
How would you implement rate limit for this external dependency? Here's what I thought:
- Have a token bucket limiter inside each service instance, but then scaling
- Store the above token bucket in a database, like mongo or dynamo, but it would be very inefficient
- Use redis, but I would have to spin up and maintain an additional dependency just for this feature
Can you think of a better approach?