cmd/syncthing, lib/db, lib/model, lib/protocol: Implement delta indexes (fixes #438)

[calmh]

Purpose

This is the delta index implementation. There are a few main parts to it:

• The "IndexID" stuff which is to ensure that we can detect when a given LocalVersion counter has reset or doesn't mean what we think it means. It also tells us whether the other side actually implements this part of the protocol or not (IndexID==0 meaning no support).
• A tweak around how we handle incrementing LocalVersion in the database. I think this is simpler, and it's now just a regular incrementing counter per folder.
• How we send indexes is changed, and it's now triggered from inside the code that takes care of receiving the ClusterConfig. There are a number of checks for the announced IndexID and MaxLocalVersion and if it checks out we start sending index updates only from that MaxLocalVersion and onwards.

Testing

I've ran the integration tests that are suitable, including the various "restart during transfer" tests and it checks out. I'm running it myself on my device since a week or so, and it works as far as I can tell.

Docs

Spec update: syncthing/docs#208. It's a backwards compatible change.

[Zillode]

s.localVersion is never set?

[calmh]

Hm? I set it on the line below your comment. :)

[calmh]

Ah you mean initialized perhaps? It's initialized to zero as we allocate the "s" struct above.

[AudriusButkevicius]

@st-review merge

[AudriusButkevicius]

Nothing to pick on, clear and understandable.

🎉

[st-review]

👌 Merged as 47fa4b0. Thanks, @calmh!

[calmh]

Lets hope it works

[rumpelsepp]

Wow! That's really exciting! Updating now...