The Movie Database Support

I'm developing a system that heavily accesses the content of tmdb and was hoping there was some support for downloading the entire database and then being able to query for changes to avoid stressing the API services?

I've thought about just iterating over every possible id slowly and pulling the entire database out, and that would work, but then I wouldn't know if anything has changed without iterating over everything periodically.

12 replies (on page 1 of 1)

Jump to last post

Hi nabotv,

Unfortunately we don't provide any kind of downloadable dump. We are fine with caching and storing the data locally though, just be sure to attribute TMDb as the source. Best you can do is loop through IDs and grab the data that is important to you.

Thanks for the response Travis. Is there any way to easily get updates without having to iterate over everything in the database?

Check the API documents for the changes methods: http://docs.themoviedb.apiary.io/reference/changes

Edited to add that the new documents location is https://developers.themoviedb.org/3/changes

Thanks, that's exactly what I needed. :)

Such a pity that you don't provide all the IDs

@faresar What do you mean by not providing all of the IDs? Our daily ID exports provide all of the IDs that are valid on TMDb. You can use them to iterate over, knowing in advance which IDs are valid and which are not.

@travisbell I am in exact same situation where I would like to have a local copy of entire TMDB (movies/tv-shows/people etc). I would also like to keep the data updated on daily basis. I understand that "Daily ID exports" provide the list of IDs that have changed in last 24 hours. But I am wondering how can I do the entire database Import for the first time?

Also my guess is if an Object (movie, tv-show, person etc) has not changed, than daily id exports isn't going to have ID for that correct?

If a new Object is added to TMDB, is Daily ID Exports going to have id for that? is there any flag in daily id exports that indicates if the Object is Added or Modified?

@nafr1 said:

@travisbell I am in exact same situation where I would like to have a local copy of entire TMDB (movies/tv-shows/people etc). I would also like to keep the data updated on daily basis. I understand that "Daily ID exports" provide the list of IDs that have changed in last 24 hours. But I am wondering how can I do the entire database Import for the first time?

Not quite, the the ID export does not represent the changed IDs, it's an export of all of the valid IDs that exist at the time of the export. In order to make an initial import, you will have to iterate over the IDs and use the API.

Also my guess is if an Object (movie, tv-show, person etc) has not changed, than daily id exports isn't going to have ID for that correct?

As mentioned above, not quite. The daily ID export has nothing to do with changes.

If a new Object is added to TMDB, is Daily ID Exports going to have id for that? is there any flag in daily id exports that indicates if the Object is Added or Modified?

See my comment above.

If you want to track changes, you will want to use the /changes end point(s).

@travisbell

Thanks very much for prompt response. So my algorithm should be

1) Go through "Daily ID Exports" everyday, check if they exist in my local database and only Import the Objects to my Local Database that doesn't already exist (that means that this is new ID).

2) Go through all the IDs in my database (once in a while, maybe once a week) and use the "/changes" endpoint to get the changes and update my local database.

I guess that way I will always have updated database? Please let me know if I am missing anything?

Thanks

@nafr1 said:

1) Go through "Daily ID Exports" everyday, check if they exist in my local database and only Import the Objects to my Local Database that doesn't already exist (that means that this is new ID).

Yes, that would be one way. The other is to just ping the /changes method every day and if you don't have an ID that is returned, you can assume it's new (or not tracked by you yet).

2) Go through all the IDs in my database (once in a while, maybe once a week) and use the "/changes" endpoint to get the changes and update my local database.

You don't need to go through all of your local IDs, it's easier to just use the changes endpoint to track what has been changed every n days. It will return the usual few hundred/thousand IDs that get changed in your queried time period and you can make your own choice about what to do with them. You can query up to the past 14 days so this is completely configurable by you.

@travisbell said:

@nafr1 said:

1) Go through "Daily ID Exports" everyday, check if they exist in my local database and only Import the Objects to my Local Database that doesn't already exist (that means that this is new ID).

Yes, that would be one way. The other is to just ping the /changes method every day and if you don't have an ID that is returned, you can assume it's new (or not tracked by you yet).

2) Go through all the IDs in my database (once in a while, maybe once a week) and use the "/changes" endpoint to get the changes and update my local database.

You don't need to go through all of your local IDs, it's easier to just use the changes endpoint to track what has been changed every n days. It will return the usual few hundred/thousand IDs that get changed in your queried time period and you can make your own choice about what to do with them. You can query up to the past 14 days so this is completely configurable by you.

The Changes endpoint also gives the ids of the Objects whose Popularity changes on daily basis?

No, popularity changes on every object every day, so there's no real need to track it separately.

If all you are wanting is to track popularity changes, you'd be better off just pulled that data from the daily ID exports and not touching the changes end points.

Can't find a movie or TV show? Login to create it.

Global

s focus the search bar
p open profile menu
esc close an open window
? open keyboard shortcut window

On media pages

b go back (or to parent when applicable)
e go to edit page

On TV season pages

(right arrow) go to next season
(left arrow) go to previous season

On TV episode pages

(right arrow) go to next episode
(left arrow) go to previous episode

On all image pages

a open add image window

On all edit pages

t open translation selector
ctrl+ s submit form

On discussion pages

n create new discussion
w toggle watching status
p toggle public/private
c toggle close/open
a open activity
r reply to discussion
l go to last reply
ctrl+ enter submit your message
(right arrow) next page
(left arrow) previous page

Want to rate or add this item to a list?

Login