Initial implementation of sync-v2 server #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yrliou
force-pushed
the
master
branch
4 times, most recently
from
ba2d630
to
3ee7eaa
Compare
yrliou
force-pushed
the
master
branch
3 times, most recently
from
e02f90c
to
f4f4690
Compare
yrliou
force-pushed
the
master
branch
3 times, most recently
from
d932894
to
801f4cb
Compare
yrliou
force-pushed
the
master
branch
2 times, most recently
from
7671d1e
to
e8238fb
Compare
1. Timestamp handler will reply the current unix time in milliseconds to sync clients to used in auth requests. 2. Auth handler authenicates the client by verifying its signature and the freshness of the timestamp. 3. Authorize function queries the database to check if the access token in the client's request is valid or not and returns the client ID associated with the access token.
yrliou
force-pushed
the
master
branch
2 times, most recently
from
06ecb0c
to
7b5d680
Compare
yrliou
changed the title
yrliou
changed the title
Wrong assumption was made that Deleted and Folder values will not be nil in client messages, this PR fixed the bug by setting default values for new entities and don't update their values when client does not include the field in the message.
diracdeltas
reviewed
May 21, 2020
husobee
reviewed
May 22, 2020
…tly. In this commit, auth endpoint is removed and sync clients are expected to generate valid access tokens from the timestamp responses directly to use in future sync requests, server no longer save the states of token and clientID.
It seems the bat-go setting of the RateLimiter is not applicable in our case. It's easy to hit the limit bat-go is using for sync server case.
|
Kudos, SonarCloud Quality Gate passed!
|
husobee
reviewed
May 29, 2020
husobee
reviewed
May 29, 2020
husobee
approved these changes
May 29, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Spec of sync-v2 is currently an internal google document, link could be found in our internal issue: https://github.com/brave/security/issues/154.
Endpoints
GET /v2/timestampendpoint returns a UNIX timestamp in milliseconds and expected time for a token to expire in JSON format. Sync clients are responsible to create valid access tokens using timestamps returned by the server. The response is defined inschema/json/timestamp.go.POST /v2/command/endpoint handles Commit and GetUpdates requests from sync clients and return corresponding responses both in protobuf format. Detailed of requests and their corresponding responses are defined inschema/protobuf/sync_pb/sync.proto. Previous granted access token should be passed in the request's Authorization header.Endpoint 1 above is implementing what we specified in the Authenication section in the sync-v2 spec. Client will generate access token from received timestamp to use in requests to
v2/command, and the verification of the token is implemented inauth/auth.go.Endpoint2 is the main functionality of this sync server, which
command/command.gocould be a good starting point, it handles client's sync requests, processes the messages and calls out to datastore module for DB operations, then send the result responses to clients.The protocol between client and server is specified in https://source.chromium.org/chromium/chromium/src/+/master:components/sync/protocol/sync.proto, see
GetUpdatesMessage,GetUpdatesResponse,CommitMessage, andCommitResponse.Datastore
Currently we use dynamoDB as the datastore, the schema could be found in
schema/dynamodb/table.json, all DB operations are located in the database module.Sync Protocol
The sync protocol in
schema/protobuf/sync_pbfolder is defined by chromium sync client, in this first implementation, we manually copied the proto files from chromium (*.protoundercomponents/sync/protocol/), and domake protobufto generate codes in golang.This manually process should be feasible for the short term because the main protocol buf file we cared about in our server is
sync.protoand currently what we processed are critical fields which should not be changed frequently and also chromium will ensure backward compatibility when updating the protocol.We might want to setup a better process on updating the protocol at server side but it is out of the scope of our initial implementation.