The Movie Database Подршка

Hi, I'm totally confused on the header responses regarding the rate limiting. So far, I understand that "X-RateLimit-Remaining" shows me how many requests I can still do before the next "X-RateLimit-Reset" (by converting this number to a date). And "Date" is the the exact moment in which your server received my request. Am I correct so far? Because I noticed throughout my tests that "X-RateLimit-Reset" has the same value between tests. For instance, I ran my first test (120 requests/ spaced out by 10seconds, getting "X-RateLimit-Reset" with a value of 1496955001 for the first 40 requests, then 1496955016 for the second batch, and random values like 1496955206, 1496955216 and 1496954985 for the last). The second time I ran the test, say 20 minutes later, I get the same values in pretty much the same order, so I'm really confused as to what this Reset time is supposed to be. Also, I converted it to a date, and it's always in the past, it's not less than 10s in the future, so I'm confused. Having said so, I'm having a hard time coming with a solution as to when to "reset locally" (aka as to when to send my next batch of requests). Is there anyone here who has a solid grasp? I'm using this in an iOS app by the way. Thanks

26 одговора (на страни 1 од 2)

Jump to last post

Следећа страницаПоследња страница

X-RateLimit-Reset is the time at which your current 10 second bucket counter expires. The current counter starts on the first request you make that isn't part of an existing counter. The new "bucket" and will reset (or expire) 10 seconds in the future.

travisbell$ curl -I -L "https://api.themoviedb.org/3/movie/550?api_key=###"
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: ETag, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After
Cache-Control: public, max-age=28800
Content-Length: 1351
Content-Type: application/json;charset=utf-8
Date: Thu, 08 Jun 2017 22:44:39 GMT
ETag: "9b2b58a196bd9f1c90bff5c190361fcb"
Server: openresty
Vary: Accept-Encoding
X-Memc: HIT
X-Memc-Age: 21529
X-Memc-Expires: 7271
X-Memc-Key: a7a8be33582080c3e162cfcb93607b73
X-RateLimit-Limit: 40
X-RateLimit-Remaining: 39
X-RateLimit-Reset: 1496961889
Connection: keep-alive

Parsing the X-RateLimit-Reset header to a human readable time:

2.3.3 :005 > Time.at(1496961889)
 => 2017-06-08 16:44:49 -0600

2017-06-08 16:44:49 -0600 is exactly 10 seconds in the future from the time of my first request. But remember, it's only a counter that gets set on the first request and will automatically expire after 10 seconds. The loop goes on an on every request you make every 10 seconds.

Thanks, so I understood it right all along. Although how do we explain: (1) I'm getting times that are in the past, (2) I'm getting the same values minutes apart, (3) sometimes I'm getting values that are 1 second apart, like 1496955016 and 1496955017. I just ran my code, and I'm currently getting 1496955001 for my first 9 requests. That's the same value I've been getting all along. Something is wrong sweat

I get new values in every test I run. Are you caching the response or something along those lines?

Hmm, how would go about finding that out? I'm running these tests on an iOS app. This is how I'm building my request in swift 3: URLRequest(url: ###, cachePolicy: .useProtocolCachePolicy, timeoutInterval: 10.0)

@merricat said:

Hmm, how would go about finding that out? I'm running these tests on an iOS app. This is how I'm building my request in swift 3: URLRequest(url: ###, cachePolicy: .useProtocolCachePolicy, timeoutInterval: 10.0)

Seems like you do

You can disable the caching by using

**``` .reloadIgnoringLocalCacheData

Awesome, that seems to have done the trick! Thanks for the input guys, it now works. After a bit of testing, I'm still getting some ResetTimes to be slightly different. Two examples: the first, I sent 9 requests, the first came back with a ResetTime like xxxxxx87, the second was xxxxxx76, and the rest were back to xxxxxx87. For the second example I sent 15 requests, the first one came back with a ResponseTime of 1497046462, while the remaining 14 were 1497046463. Any ideas? Thanks!!

Are you 100% sure your requests are being executed (and returned) in order? My gut tells me that the request that had xxxxxx76 as an expires was your first request, then every subsequent request was the next bucket. You can add you your own timestamp at the time when you make the request (not when you get the response back, but when you issue the request).

You could also pair the X-RateLimit-Reset with the X-RateLimit-Remaining header. You should be able to make more sense of what is happening by looking at those together.

Yeah I'm definitely not getting responses in the same order. I thought this was normal though, as some requests may need more time to execute? Here, I just ran my code and this is what I got. The date in the middle is the "Date" that comes with your response.

RemainingRequests: 36 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 38 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 37 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 35 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 32 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0
RemainingRequests: 39 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198317.0
RemainingRequests: 31 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0
RemainingRequests: 33 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0
RemainingRequests: 34 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0

How do I get the responses back in order? Thanks.

I don't have any experience with iOS so I can't answer that. For language or platform specific questions like that, you'll probably have better luck posting your question on something like Stack Overflow.

Travis, shouldn't this be an issue server-side? Because I'm printing the responses as I get them, and they're not in order. Plus, if you look at the logs I pasted in the answer above, you can see that the first request (RemainingRequests: 39) has a Reset time of xxxx17, then requests 38 through 35 have xxxx14 (why smaller???), then 34-31 have a xxxx14. Does this look normal to you? Because I'm assuming those values (Reset, Date, etc.) are generated on your side.

Hey @merricat No, it won't be anything server side. It's a client issue. There's never going to be a guarantee with orders of HTTP responses if they are being called asynchronously, which I assume is what is happening here. There could also be many different problems with any one of your single requests between you issuing the request and then receiving the response.

If order of responses is important (it won't always be, but looks like it is in this case) you need to co-ordinate this logic with your HTTP library.

I did a quick Google search for you and came across these 2 posts which might be useful:

Ok thanks for the info. The order isn't too important honestly, I think I can live without it. But regardless of the order in which I receive the responses, are the Reset/RemainingRequest pairs (in the log I posted) making sense to you ? Or do you think it's because I'm sending the request asynchronously that your server is handling it weird and sending back weird Reset times?

I'm late. Are you familiar with Asynchronous programming? In an Asynchronous environment, you are working in parallel. Meaning there's no more Serial execution. What your computer does is to send these requests simultaneously on the service, and when each is complete to take back the result and return it. Now TMDb Service, as each service depending on the details you ask may take a bit longer to respond. So while a request sent and server mapped it as #13th request, it may be finished later than the #18th request. (data fetching, processing, transporting). So therefore you get this result.

If for example Server was responding with 10ms latency to each of your requests, and your async requests were not scrambled, you would end up having serial RemainingRequests, But that's won't gonna happen. In every single request to the same service, with the same query you will end up getting different latency everytime. That's happening because, the Service is running into a machine that is doing other things (consuming cpu and i/o times), or handling other requests at the same time with yours, but also because your Link to the Service is not 100% Stable. Your Link to the Internet is not 100% Stable. And the fact that your internet may be have already some traffic. All these factors affects the latency you get from your terminal to a random endpoint on the internet.

Thanks @ProIcons . I now understand a bit better what's happening, but I'm still don't know where the problem is. If we re-order the responses I get from TMDB in my example above, this is what we get:

RemainingRequests: 39 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198317.0
RemainingRequests: 38 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 37 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 36 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 35 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198314.0
RemainingRequests: 34 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0
RemainingRequests: 33 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0
RemainingRequests: 32 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0
RemainingRequests: 31 | Date: 2017-06-11 16:25:14 +0000 | Reset: 1497198324.0

why do requests 38 through 35 reset at an earlier time than the first request (#39). Or I guess the better question is "why is the first one resetting at xxxx17 instead of being either 14 or 24 (the 10 second minimum window)" ?.
On my end, it seems like I'm sending my requests in order, since my print statements are all in order when I send a request. The following might be too much info, but maybe you guys can see if there's a problem with what I'm doing (all in swift 3):

I'm using a custom Queue structure containing closures as nodes. These closures get executed on dequeue( ). When executed, each closure creates a URLRequest, a URLSession, and calls the following

session.dataTask(with: request) { data, response, error in
     OperationQueue.main.addOperation {
          //callback code
     }
}.resume()

I just ran more tests since the last time I posted on this page. FYI no one else can have my API key. I sent 8 requests at once, I got all 8 callbacks after about 0.5 seconds. The first request came back with a ResetTime of xxxx6, and the 7 remaining were xxx16. This makes no sense to me: should they not all have the same value, especially since I hadn't sent any requests previously (for a couple of months)? On the last test I ran, my last request also came back with a weird ResetTime (outside of the 10 second minimum window), even though it was in the same "bucket" as a couple other requests. So the odd ones out seem to be the first and last requests. This seems to be consistent. Any ideas?

You can add you your own timestamp at the time when you make the request (not when you get the response back, but when you issue the request).

@travisbell - you mean I can add my own timestamp, so that when I get the callback I can check it again? how do I go about doing that?

Thanks guys, appreciate the help wink

You can put aside the ResetTime header. I recommend you to build an interceptor to Intercept the Retry-After header. You pause all the async requests till Retry After time passes you retry the request that failed and you will be good to go

You can see how i've done it using java's okhttp3 in tmdb-java lib

https://github.com/ProIcons/tmdb-java/blob/master/src/main/java/com/uwetrottmann/tmdb2/TmdbInterceptor.java

Не можеш да нађеш филм или серију? Пријави се да додаш.

Глобално

s фокусирај траку за претрагу
p отворите мени профила
esc затворити отворени прозор
? отвори прозор пречице на тастатури

На страницама медија

b вратити се назад (или родитељу кад је то применљиво)
e иди на страницу за уређивање

На страницама ТВ сезона

(стрелица надесно) пређите на следећу сезону
(лева стрелица) иди на претходну сезону

На страницама ТВ епизода

(стрелица надесно) пређите на следећу епизоду
(лева стрелица) иди на претходну епизоду

На свим страницама са сликама

a отвори прозор за додавање слике

На свим страницама за уређивање

t отвори селектор превода
ctrl+ s пошаљи образац

На страницама за дискусију

n отвори нову дискусију
w мења статус гледања
p мења јавно/приватно
c мења отвори/затвори
a отвори активност
r одговор на дискусију
l иди на последњи одговор
ctrl+ enter пошаљи поруку
(стрелица надесно) следећа страница
(лева стрелица) претходна страница

Подешавања

Желите ли да оцените или додате ову ставку у листу?

Пријави се