Tailing files

How to implement?

We need to keep track of the last file location. When document is updated, we read from that position to end. How do we know if it should be a new row or not? First version: Assume that all new data comes as a new row.

Where to put solution. Tail document view caches everything. And only reads from a given row when it receives a notify message. This is nice, but to perform good tailing, we needs to look at the file pointer's end location in the file rather than files. A new view hence needs to be implemented. This file should be able to speak the document views' language, ie. it should support returning specific rows to a calling client. A way to do this is to index the read file with a file row positions. Then assume that the read file position aren't changed. This should be sufficient for a first version. If this solution is integrated with the tail document view, we should have a good solution for tailing files.

When do we perform the re-index? You have to actively re-index the file if you want to redo. If you read rows from an old location, the index is update for all that is read. This way, the file indexing will be self-sustaining.

How to define the fromline when analyzing a document.

1. We don't need to keep track of both start and end position of all lines, that would give redundant information.
2. We should let the line count start with index = 0. So that it is like any normal array.
3. Method that are affected by the line numbering are: indexDocument, readFromLine.

Thought for future: We want tail to be able to know more kinds of streams, not only a document? What kinds ?

• Multiple documents at one time.
• An input stream (is it possible to generalize the solution to support this? That would be able to integrate very nicely into sending filtered log data out on the network, to bind together multiple log files.