Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Scaling - a proposal for a more scalable thingtracker #13
The current approach does not scale in the slightest. I think an approach that invalidates a minimum amount of information at a time would be better.
All files that are described by this specification SHOULD follow HTTP caching specifications.
An object list
An object description
is something we've already kind of specified
At first I didn't get what you were trying to say, but now I see it, and your suggestion of dereferencing each level of the tracker is a good one that I think should be adopted.
Your concerns about scaling are well founded - a tracker with even only several hundred objects could result in several MB of data, and even with gzip compression this would be barely usable. My initial thoughts on tacking this was my means of an API - using offset and limit, or lastUpdated, parameters to limit the number of objects returned. Implementing an API is a greater undertaking than simply serving up a document, and your suggestion for dereferencing the data may mitigate the need for one, or at least minimise it.
Microdata is also a great idea, and coincides with my recent reading on Linked Data, which I am realising covers pretty much what we are trying to achieve. I am starting to think we should at least consider directing the efforts of the project towards being a vocabulary and implementation of Linked Data. Your suggestions fit in very well with such an approach.
Are you on Google+? I'd like to write about your suggestions to the TTN community there and would obviously like to have your input too.