@merricat The other thing to mention is that even though our API servers are supposed to be in sync, I have seen one or two of them be a few seconds ahead or behind (our API fleet uses auto scaling so we add and remove new instances all the time--as in 10+ times a day). Since we don't use sticky sessions at the load balancer, you could be hitting any one of them at any time while you're rifling through requests.

What that could possibly mean is that depending on the server you hit, your window could, in theory, be different. There's a few different ways to solve this problem, but I don't have time to look into any of that any time soon.

@ProIcons - thanks for the suggestion. However, is there a difference between the ResetTime and the Retry-After header? Won't they just give me the same info? If there's a real advantage I will definitely look into changing my code to fit it in.

@travisbell - that would explain the behavior. Although I'm surprised if I'm the first one to notice it/bring it up, as it pretty much happens every time I send a batch of requests (whether it's 8 requests or 40). Would chaining requests (where each requests waits for a callback before sending out the next one) get rid of the odd ResetTimes? Or is it just random?

In my program, I locally keep track of 40 available requests. Once that number reaches 0 (i.e. I sent out 40 requests), I pause the sending of any further requests for either 10 seconds or until "CurrentTime minus the highest ResetTime received". Every time I receive a callback, I increase my number of available requests by 1, and update my local ResetTime only if it's higher than its previous value.

I have seen one or two of them be a few seconds ahead or behind.

Because I grab the highest ResetTime, my method has worked for me so far (if it's a few secs ahead, it's ignored; if it's behind, it uses its value, which only means that the requests on the "permanent instances" already have a new bucket, so it's really erring on the side of caution).

On a side note, in the case of sending 41 requests, to me it just seems awkward to wait 10 whole seconds just to receive that 41st request. I guess there's no real way around that, other than possibly staggering (or chaining) the sending of requests, or at least staggering the display of data from the callbacks. @travisbell - do you know if chaining is the standard way your users deal with this? do you think in this case it could benefit me?

Thank you guys for the input

Retry After shows on a failed request due to exceeding the limit.

Exactly, so the only difference is that instead of preventing a failed request (which is what I'm currently doing), your interceptor idea waits for a request to fail before handling any further requests. Plus, I'm pretty sure that the Retry-After and ResetTime headers are generated from the same data, and based on what Travis said regarding the instances not being synced, the end result would be the same (Retry-After would be a few seconds off).

Till now never happened. Tmdb java runs over 150 tests. None failed

I was just trying to understand the difference between our two approaches, but I don't think we're understanding each other. This is what I understood from your approach: your "interceptor" catches failed requests and waits for X seconds (given by the failed request's Retry-After header) before sending out any more requests. One of your requests has to fail in order to wait X amount of seconds. If none of your requests fail, then your tests are not sending more than 40 requests in the same 10 second window.

The way interceptor is build, is as follows. We make a Query to the Service, intercept the body and status code. Is it ok to proceed? Proceed and send the data back to controller, Has failed? Block The main thread for x seconds, make again the query to the service (without controller's intervention) and return back the data to controller.

public static Response handleIntercept(Chain chain, Tmdb tmdb) throws IOException {

...

       if (!response.isSuccessful()) {
            String retryAfter = response.header("Retry-After");
            if (retryAfter != null) {
                try {
                    Integer retry = Integer.parseInt(retryAfter);
                    Thread.sleep((int) ((retry + 0.5) * 1000));
                    return handleIntercept(chain,tmdb);
                } catch (Exception exc) {

                }
            }
        }
        handleErrors(response, tmdb);

        return response;
}

this way, even if there's the unsync state as Travis mentioned, the next time it will make the query if it fails again it will wait again the amount of time service wants. So it's solid and stable.

Got it! Ok so it's pretty much the same system as I have, except I make my own "wait for x seconds" by subtracting ResetTime from Now.Time(). Now, another question for you. Do you chain your requests, or send them all as you have them?

This is a library for java. Not a project making multiple queries at one. The developer will choose how he will use this library. This is just a toolset for easy access on the tmdb.

It's far from Same as yours. You are trying to predict when it will fail, while i'm waiting for it to fail. Prediction will result to a fail which it might not be handled. Mine it won't fail. Even it will seem like it is failing, the controller will get it's data no matter what, without knowing that query failed one or more times. And this is the optimal solution considering the problem Travis mentioned.

So without having the Interceptor the deal would go like this

Controller -> Request to Service -> Response -> Controller -> (Failed?Succeed?=>All Good)

With interceptor things are changing.

Controller -> Request to Service -> Interceptor -> Response -> Interceptor -> (Failed?=> try again)(Succeed?=> Controller)

The Default approach is being shown on this image

Your approach is being shown on this image

While my approach is being shown on this image

Got it, thanks!