Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new sync #1093

Merged
merged 23 commits into from Jul 14, 2023
Merged

Add new sync #1093

merged 23 commits into from Jul 14, 2023

Conversation

ellie
Copy link
Member

@ellie ellie commented Jul 9, 2023

So this has taken me a while, combined planning/implementation time

Improvements:

  1. We always sync a linked list. It is always append-only, so we never insert into the past. This makes syncing things easy
  2. We try and make the bare-min requests in order to sync. Get an index, compare with the local index, then do operations to get the state to match
  3. We can sync... anything. Yeah so this actually just syncs the kv store, but would be easy to map history or other use cases in here

So yeah! Faster, more flexible, and far less wasteful.

This has not yet been fully tested, and may contain issues. Until released, migrations may be removed or deleted. Run at your own risk.

@vercel
Copy link

vercel bot commented Jul 9, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
atuin ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 12, 2023 1:17pm

atuin-client/src/api_client.rs Outdated Show resolved Hide resolved
atuin-client/src/record/sqlite_store.rs Outdated Show resolved Hide resolved
atuin-client/src/record/sqlite_store.rs Outdated Show resolved Hide resolved
atuin-common/src/record.rs Outdated Show resolved Hide resolved
atuin-server-database/src/lib.rs Outdated Show resolved Hide resolved
atuin-server-postgres/src/lib.rs Outdated Show resolved Hide resolved
* use some fancy types

* fmt
Co-authored-by: Conrad Ludgate <conradludgate@gmail.com>
@ellie ellie marked this pull request as ready for review July 13, 2023 18:53
@ellie ellie merged commit 97e24d0 into main Jul 14, 2023
9 checks passed
@ellie ellie deleted the ellie/recordindex-api branch July 14, 2023 19:44
ealap pushed a commit to ealap/atuin that referenced this pull request Jul 18, 2023
* Add record migration

* Add database functions for inserting history

No real tests yet :( I would like to avoid running postgres lol

* Add index handler, use UUIDs not strings

* Fix a bunch of tests, remove Option<Uuid>

* Add tests, all passing

* Working upload sync

* Record downloading works

* Sync download works

* Don't waste requests

* Use a page size for uploads, make it variable later

* Aaaaaand they're encrypted now too

* Add cek

* Allow reading tail across hosts

* Revert "Allow reading tail across hosts"

Not like that

This reverts commit 7b0c72e.

* Handle multiple shards properly

* format

* Format and make clippy happy

* use some fancy types (atuinsh#1098)

* use some fancy types

* fmt

* Goodbye horrible tuple

* Update atuin-server-postgres/migrations/20230623070418_records.sql

Co-authored-by: Conrad Ludgate <conradludgate@gmail.com>

* fmt

* Sort tests too because time sucks

* fix features

---------

Co-authored-by: Conrad Ludgate <conradludgate@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants