API for history management

[josephg]

getOps should be exposed in the REST and socket.io interfaces, so the client can request historical operations for timeline views and things like that.

[aribn]

This would be fantastic.

[wmertens]

I guess this means #62 and #63 are dupes and should be closed?

[wmertens]

Any proposed API?

The MongoDB discussion made me wonder if keeping the ops is such a great way to keep history. Wouldn't it be better to keep some snapshots of historical versions and remove ops between snapshots?

Alternatively, have an OT optimizer comb through ancient ops and combine them intelligently. (e.g. ops submitted by a single client in a short period of time could be combined into a single diff op)

[wmertens]

@rileydutton said:

It would be especially useful if along with each version I could get the date/time it was created and the person who modified it (based on the new agent_id information), although those are certainly not necessary.

[luto]

So I guess it is currently not possible to a snapshot of a document at a given version? :-/
How would one do this internally? Calling getOps to get everything from revision 0 to x and then applying all the ops seems to be stupid from a performance point of view (or the other way around if x > currentVersion / 2).

My use case is as follows:
A client wants submit an op, it depends on the state of the document whether they are allowed to do so. A op is not yet transformed when auth is called, so I need to get the document at the version action.v.

[wmertens]

In your case that's exactly how I would do it, but I'd keep a snapshot of a while ago to use as a starting point in the auth function.

On Jun 2, 2013, at 22:09 , mluto notifications@github.com wrote:

So I guess it is currently not possible to a snapshot of a document at a given version? :-/
How would one do this internally? Calling getOps to get everything from revision 0 to x and then applying all the ops seems to be stupid from a performance point of view (or the other way around if x > currentVersion / 2).

My use case is as follows:
A client wants submit an op, it depends on the state of the document whether they are allowed to do so. A op is not yet transformed when auth is called, so I need to get the document at the version action.v.

—
Reply to this email directly or view it on GitHub.

[mizzao]

I'm using ShareJS for a social science study and need to view past revisions of a document.

Even without a history API, what's the easiest way to replay a sequence of ops to view a past revision of a document? Are there any demo apps where one can just pull a slider and view the document over time? Is it possible to apply ops "backwards"?

[luto]

https://github.com/shownotes/show-pad/blob/asyncnoter/src/documenttypes/asyncnoter.js#L258
My dirty solution. There is no demo and it won't run without the rest of the app, though. Feel free to copy.

[mizzao]

@luto That's super useful! Thanks a million times.

[luto]

Let me know when you built something nice :)

[mizzao]

I've been integrating ShareJS into Meteor: http://documents.meteor.com/.

If you haven't heard of Meteor yet, it's a crazy efficient framework for making real-time web apps. However it doesn't support OT yet so I needed to cook something together so people could do real-time docs. I also need to view the history of those docs later.

My package https://github.com/mizzao/meteor-sharejs currently supports ShareJS editors in meteor apps and will support history scrubbing as well once I get your code integrated!

[josephg]

Thats cool!

Yeah what @luto did is about right - just grab the operations and apply them one at a time from the start. (Or in his case, invert them and apply them in reverse order).

[mizzao]

@josephg and @luto, I'm not that familiar with the ShareJS code so maybe you can answer a quick question for me. If I am using redis as a datastore for ShareJS, does it store all the ops starting from version 0, or does it combine ops into snapshots for faster playback later?

The latter would clearly be faster from an implementation perspective but would cause history to be lost and not be viewable. I guess the optimal implementation would be to store occasional snapshots along with all ops which would allow any revision to be accessed in O(log n) time and O(log n) space...but not sure if that's what you guys did.

[josephg]

It stores every op starting at version 0. And yeah - if you want to be able to efficiently see any version, keyframes is the right answer. But if you don't want them, they will blow out your database size significantly.

Also please ask questions like that on the mailing list.

[curran]

Related discussion on the mailing list: accessing version history (for read-only timeslider functionality).