The Movie Database Support Forum

According to these docs there are no request rate limits.

I don't encounter this in production, but because my local environment is faster I get the following on large updates to my database

status: 429,
statusText: 'Too Many Requests',

Is there a hidden request rate limit I can be sure to throttle by to avoid this?

21 Antworten (Seite 1 von 2)

Jump to last post

Nächste SeiteLetzte Seite

@Kravek said:
Is there a hidden request rate limit?
According to these docs there are no request rate limits.
I don't encounter this in production, but because my local environment is faster I get the following on large updates to my database

status: 429,
statusText: 'Too Many Requests',

Is there a hidden request rate limit I can be sure to throttle by to avoid this?

As far as I know, no.
I didn't find this error in the TMDb error listing.
https://www.themoviedb.org/documentation/api/status-codes
Are you sure this error message was sent by the TMDb server?

I only found a 429, but it's not the same message.

Code    HTTP    Message
25      429     Your request count (#) is over the allowed limit of (40).

I assume that if a DDoS attack occurs, some restriction is created by the Administrator.
Can you inform the country of origin of your IP or service ?

One of our CDN providers enforces some base level rate limiting to help prevent DDOS attacks. I believe it's a maximum of 50 requests per second and 20 connections per IP.

Thank you @ticao2 and @travisbell 👍

Great timing for this recent thread; have looking for something pertaining to this. Follow-up question [for @travisbell ]:

I want to make sure the product currently in development is not pulling in too many requests at once (v3 API key should be linked to this account), especially for images. I found a thread that @travisbell stated per API policy that it's okay to locally cache data as long as we follow appropriate take-down requests (currently, titles are set to update daily or weekly depending on past/future release date logic).

The product being developed needs up to 100's of TMDB titles loaded on each page request. Right now, it's loading in the images from TMDB directly, with one or two per title (posters and backdrop images). I'm looking for ways to store them on the server or JS LocalStorage, but my main concern is hoping that image requests are currently backed by the aforementioned CDN, and server cost is relatively low until that can be resolved.

I'd rather ask than be hit with a sudden block of service.

@SynergeticWebSolutions said:

Great timing for this recent thread; have looking for something pertaining to this. Follow-up question [for @travisbell ]:

I want to make sure the product currently in development is not pulling in too many requests at once (v3 API key should be linked to this account), especially for images. I found a thread that @travisbell stated per API policy that it's okay to locally cache data as long as we follow appropriate take-down requests (currently, titles are set to update daily or weekly depending on past/future release date logic).

The product being developed needs up to 100's of TMDB titles loaded on each page request. Right now, it's loading in the images from TMDB directly, with one or two per title (posters and backdrop images). I'm looking for ways to store them on the server or JS LocalStorage, but my main concern is hoping that image requests are currently backed by the aforementioned CDN, and server cost is relatively low untill that can be resolved.

I'd rather ask than be hit with a sudden block of service.

I have the same question.

I have the problem with the error 429 as well. Getting 20 items from 1 page works great but then I want to get at least 40 per page. So I did call the second page but when the results show the movie posters are not loaded (randomly), getting the error 429. After refreshing the site they are loaded. I tried different ways, calling with different functions, concating arrays but every time the error occurs. My question is, how I am reaching the limit for number of requests 40, am I not doing only 2 requests, 1 for every page? Using JavaScript and axios for the calls. Any ideas would be appreciated since I am out of ideas to get more than 20 items on a single page.

Something definitely has changed in the last few weeks in terms of image asset rate limiting. I also have been running into this. The problem is this is done on a per-ip basis, so regardless of where you're pulling images from, whether via a image proxy, or directly by the client browser, they are going to run into this if loading too many images at once.

One solution would be rate limiting how many images are downloaded in a certain period of n seconds on the client. This could be done with lazy loading images.

Another solution is to build a way of exponential retries for loading images.

You could also cache posters on your own server and serve them via your own image cache proxy if you know what posters may need to be loaded before a user loads them.

Use multiple external ips on an image proxy and spread them out.

The limit is only a simultaneous connection limit on image.tmdb.org. So if you’re doing something server side, simply make sure to use a keep alive connection and reuse your connections. It should be pretty easy to keep this under 20.

If you’re talking about browser requests, using HTTP/2, your browser is likely only going to open a single connection so you should never run into an issue. Read more about that here.

UPDATE

For our use case, the default poster/backdrop/still is uploaded to our app's S3 bucket / CDN on a title's creation/update. This way it easily complies with takedown requests and reduces the 50 requests/sec limit. We're also (alternatively?) looking into using JS's IndexedDB (since LocalStorage has a 5MB limitation) to locally store the (full-sized) image blobs on a user's browser, drastically reducing [local] server/CDN costs and user data consumption. Still developing out that proof of concept to even know if it's viable/useful (we don't want to make a user store 10+ GB of images, especially on a mobile device).

For web pages using images directly from TMDB CDN I've solved this by adding property load=lazy to my IMG tags, it's a standard feature supported by all modern browsers now, the browser will load only images that about to appear on the screen, it works really well and it took just 1 line of code. Of course I understand it may not solve everyone's problem but probably will be a solution for many. Cheers.

img loading="lazy"

Hi, do you have any news on this subject?

I've been stuck on it for a few days now, I can't find a solution! Even with the Lazy loading img... Have a nice day

So do I, please help someone if there is a solution? @travisbell

Apologies in advance for the complex response, but hopefully this helps from client side if nothing can be done from the CDN side...

Here's our IndexedDB beta solution utilizing Jake Archibald's IDB library. It stores the blob images locally in the user's browser. Obviously, if you don't wish to use this, that's understandable; see Cons -> #4 below for alternative.

PROS:

  1. Lowers users' data consumption after initial load.
  2. Lowers CDN costs after initial load.
  3. Automatically replaces the blob if the (see below) data-image-src="" is different than what is stored.
  4. Automatically buffers initial loads with a setTimeout to prevent CDN issues.

CONS:

  1. Increases users' storage data. However, we're confident that even with 1,000's of titles with two images each, it's within 1-2GB. Most users aren't likely to routinely exceed this expectation. A future version might include a way to manage / clear images after after exceeding a maximum length and/or with a last accessed timestamp older than a certain duration (e.g., 6 months).
  2. Increases users' CPU usage during page load.
  3. Images aren't loaded in immediately, and take extra time if it's the initial load with 100+ images.
  4. Doesn't work in (some) incognito/private browser modes when IndexedDB is disabled. It will alternatively utilize a onNoIDB callback, which essentially cycles through images with a setTimeout and sets the src="" to data-image-src.

NOTE: utilizing jQuery, but can be changed to vanilla JS, etc., as needed.

HTML | JS module

We have this setup within the same module to run by default. There are times when we want to skip automatic run (e.g., prompt user first due to large data consumption), so we exclude the data-image-key and rerun it later.

This has gotten worse for me lately. I am developing an application and often load many images during development and active personal use of it. I don't know if I triggered some sort of ban, but I can barely load images now without getting my IP blocked. (I wasn't loading anything crazy; forty images? Maybe sixty on a page?)

I suddenly won't get any response from the CDN for images, and all connections will time out. If I hop on a VPN, everything starts working as expected. I understanding wanting to protect against DDOS attacks, but I should be far from triggering a block like this, I imagine?

I had even implemented local caching, but when browsing through many titles and their images, it's easy to hit the request limit again. (And subsequently, whatever IP ban is now happening to me.) This started a few days ago. (The rate limiting has been a problem for longer, but the full on time outs are very recent)

Edit: i now seem to be permanently blocked. Barely have made any requests in the last few days.

@sctsnipe said: I suddenly won't get any response from the CDN for images, and all connections will time out. If I hop on a VPN, everything starts working as expected. I understanding wanting to protect against DDOS attacks, but I should be far from triggering a block like this, I imagine?

Interesting...found the opposite to be true (disabling VPN fixes the problem). Wondering if it's the ISP/VPN that makes the difference? Also, sometimes the tmdb.org vs themoviedb.org domains will alternatively work (find an image URL provided on this website vs the API's configuration URL), but that's probably not recommended switching for the sake of TMDB's primary servers. Either way, it's doubtful this is something that is within @travisbell 's (no pun intended) domain to fix. This is likely something automatically determined by the CDN.

Es fehlt ein Film oder eine Serie? Logge dich ein zum Ergänzen.

Allgemein

s Fokus auf Suchfeld
p Profil öffnen
esc Fenster schließen
? Tastenkürzel anzeigen

Videos

b Zurück
e Bearbeiten

Staffeln

Nächste Staffel
Vorherige Staffel

Episoden

Nächste Episode
Vorherige Episode

Bilder

a Poster oder Hintergrundbild hinzufügen

Editieren

t Sprachauswahl öffnen
ctrl+ s Speichern

Diskussionen

n Neue Diskussion erstellen
w Beobachten an / aus
p Diskussion öffentlich / privat
c Diskussion öffnen / schließen
a Diskussionsverlauf anzeigen
r Auf Diskussion antworten
l Letzte Antwort anzeigen
ctrl+ enter Senden
Nächste Seite
Vorherige Seite

Einstellungen

Diesen Eintrag bewerten oder zu einer Liste hinzufügen?

Anmelden