Hey Dominic,

HEAD requests do indeed count towards the rate limiting. The intended workflow would be after you've grabbed some data and cached it locally (make sure to store the ETag), you can periodically check the ETag to determine if you need to re-download the entire data.

What I would suggest you do in order to save the additional request is look at making a "conditional GET" request. The primary difference is sending a If-None-Match header with your request which will automatically return a 304 Not Modified if the Etag hasn't changed but if it has changed, return the whole response as a regular GET.

An example cURL request for this looks like (assuming I had previously cached the ETag):

curl -v --header 'If-None-Match: "d47ca324270b1b5a86ea4d706c19fefa"' "http://api.themoviedb.org/3/movie/550?api_key=###"

The response to this request is:

* About to connect() to api.themoviedb.org port 80 (#0)
*   Trying 23.23.108.225...
* connected
* Connected to api.themoviedb.org (23.23.108.225) port 80 (#0)
> GET /3/movie/550?api_key=### HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: api.themoviedb.org
> Accept: */*
> If-None-Match: "d47ca324270b1b5a86ea4d706c19fefa"
>
< HTTP/1.1 304 Not Modified
< Cache-Control: public, max-age=172800
< Date: Mon, 08 Apr 2013 16:21:32 GMT
< ETag: "d47ca324270b1b5a86ea4d706c19fefa"
< Last-Modified: Mon, 08 Apr 2013 16:21:32 GMT
< Server: nginx
< Via: 1.1 varnish
< X-Cache: MISS
< X-Varnish: 926925187
< X-Varnish-Sid: tm-v-p-1200
< Connection: keep-alive
<
* Connection #0 to host api.themoviedb.org left intact
* Closing connection #0

Cheers.

That clears up a lot. I guess in a distributed app that would be worth it but I would only be making one set of requests as it's server side so it would seem that getting back a "304 Not Modified" as opposed to a page of JSON data wouldn't be much of a performance advantage. I mean a few ms or even sec delay per request would not be an issue for me and JSON data is only a few bytes/kb. Or am I missing something?

Also what is the advantage of using Etag as opposed to comparing the date modified (Last-Modified: Mon, 08 Apr 2013 16:21:32 GMT) ?

Or am I missing something?

Nope, probably not but it all depends on your usage and desired interval to check for updates. At the end of the day, conditional GETs benefit everyone since they:

1. Save your server (or an individual app) the time spent downloading data that hasn't changed
2. Saves our server time spent on a connection
3. Saves both our servers (or end user) the actual data being sent over the wire

Is it going to be a dramatic difference? Probably not, but it's still good design.

Implementations is quite simple so I think I will use version check. Can you elaborate why Etag is used and not last modified date? It looks to me like a date would be be human friendly that Etag. Storing Etag is a database has only one purpose, last modified date could have other uses too.

Currently, Last-Modified is not maintained on our end. Things like language and locale params all play into the data that gets served which is used to generate the unique ETag. This is not true for Last-Modified.