Hi,
I am having some issues with X-RateLimit-Remaining. It seems to be very erratic and is dropping randomly. I am doing a cooldown code which reacts to X-RateLimit-Remaining, the closer I am to zero to more cooldown I do. However, somtimes after a few calls I get the value drop below 15, one call later it goes below 10. I am doing a lot of cooldowns which, when added together give me way less than 40calls in 10sec window, so thats a problem.
I am trying to get an understanding of how rate limiter works, is it tied to the size of the result as well?
Thanks, /m
Can't find a movie or TV show? Login to create it.
Want to rate or add this item to a list?
Not a member?
Reply by mmitrovic@gmail.com
on November 7, 2018 at 4:05 AM
For example:
you see, the call count does not correspond to values I get from headers back. I am definitely the only one on this IP. Could this be an issue with your IP address grouping in limit calculation?
Reply by mmitrovic@gmail.com
on November 8, 2018 at 7:12 AM
Any news on this? This broken limiter is a real issue , with faulty remaining rates it took us 2h to process something it would take 30mins with 40calls per 10sec.
Reply by Travis Bell
on November 8, 2018 at 7:27 PM
Hi,
I'm not sure if I understand the issue you're describing 100%. So let's walk through this. When you make your first request, you'll get these 3 headers back:
The
X-RateLimit-Reset
header is saying when the current bucket will expire, and reset your count to 40. This is a rolling 10 second window that starts at the time you make the first request. So by looking at both of those headers you should know when to pause and resume your requests. Here's some mock Ruby code I threw together, and the subsequent run I let go for a minute or so. I am adding a 1 second buffer to my sleep since even though my computer and the server should be in sync, they could be slightly off.Does that help clear it up at all?
Reply by mmitrovic@gmail.com
on November 9, 2018 at 7:15 AM
Ah, ok, this acatually explains a lot. let me work this into y solution and see how it works now. I guess I was under the impression that the 10sec window was a rolling one.
Reply by mmitrovic@gmail.com
on November 9, 2018 at 7:56 AM
Ok, works now! Thanks.
Reply by Travis Bell
on November 9, 2018 at 10:01 AM
Perfect, happy to help!