Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AST-based synchronization #8237

Closed
kazcw opened this issue Nov 7, 2023 · 20 comments · Fixed by #8893
Closed

AST-based synchronization #8237

kazcw opened this issue Nov 7, 2023 · 20 comments · Fixed by #8893
Assignees
Labels
-gui d-hard Difficulty: significant prior knowledge required p-medium Should be completed in the next few sprints

Comments

@kazcw
Copy link
Contributor

kazcw commented Nov 7, 2023

To be done after #8367

Use new AST implemented in #8236 as the representation synchronized through Yjs.

@kazcw kazcw self-assigned this Nov 7, 2023
@farmaazon farmaazon added d-hard Difficulty: significant prior knowledge required p-medium Should be completed in the next few sprints labels Nov 8, 2023
@farmaazon
Copy link
Contributor

Implementation notes:

  • This includes adding Yjs data structures and mapping between it and the new Frontent AST.
    • Any changes to the module shall be done on Yjs representation, it will be then synchronized with frontend data structure.

@kazcw kazcw mentioned this issue Dec 18, 2023
5 tasks
mergify bot pushed a commit that referenced this issue Dec 19, 2023
- The module is edited by changing the AST, not the text representation.
- `IdMap`s no longer need to be maintained in parallel with the module content; they are snapshots produced as needed from the ASTs.
- Simplistic synchronization is in place until #8237: Edits are never merged; if two edits are started from the same state, one will be overwritten.
@enso-bot
Copy link

enso-bot bot commented Jan 4, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-02):

Progress: Designing Y.Js data model, details of synchronization dataflow. It should be finished by 2024-01-08.

Next Day: Next day I will be working on the #8237 task. Finish design, start implementing.

@enso-bot
Copy link

enso-bot bot commented Jan 5, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-03):

Progress: Finished designing Y.Js data model and synchronization process. It should be finished by 2024-01-08.

Next Day: Next day I will be working on the #8237 task. Implementing.

@enso-bot
Copy link

enso-bot bot commented Jan 12, 2024

Keziah Wesley reports a new 🔴 DELAY for yesterday (2024-01-10):

Summary: There is 7 days delay in implementation of the AST-based synchronization (#8237) task.
It will cause 0 days delay for the delivery of this weekly plan.

Delay Cause: Fixing ID-stability issues, and sick day.

@enso-bot
Copy link

enso-bot bot commented Jan 12, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-10):

Progress: Fixed a replaceRef bug, working on sync design. It should be finished by 2024-01-15.

Next Day: Next day I will be working on the #8237 task. Implement translation to sync data model

@enso-bot
Copy link

enso-bot bot commented Jan 13, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-11):

Progress: Implemented translation to sync data model It should be finished by 2024-01-15.

Next Day: Next day I will be working on the #8237 task. Move smart printing logic to concreteChildren, implement diff / delta application on YJs format.

@enso-bot
Copy link

enso-bot bot commented Jan 16, 2024

Keziah Wesley reports a new STANDUP for the last Friday (2024-01-12):

Progress: Refactored smart printing. Working on reconciling new YJs metadata with COW modules. It should be finished by 2024-01-15.

Next Day: Next day I will be working on the #8237 task. Implement diff / delta application on YJs format, new metadata APIs.

@enso-bot
Copy link

enso-bot bot commented Jan 17, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-15):

Progress: Designing AST changes to use YJs for edit staging. It should be finished by 2024-01-15.

Next Day: Next day I will be working on the #8237 task. Implementing YJs-backed ASTs.

@enso-bot
Copy link

enso-bot bot commented Jan 18, 2024

Keziah Wesley reports a new 🔴 DELAY for yesterday (2024-01-16):

Summary: There is 4 days delay in implementation of the AST-based synchronization (#8237) task.
It will cause 0 days delay for the delivery of this weekly plan.

Delay Cause: Scope has expanded to include supporting direct edits of Y.Js types associated with Asts

@enso-bot
Copy link

enso-bot bot commented Jan 18, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-16):

Progress: Ported the first Ast type to YJs storage as a prototype; implemented most of the supporting APIs. It should be finished by 2024-01-19.

Next Day: Next day I will be working on the #8237 task. Port the rest of the AST to YJs storage.

@enso-bot
Copy link

enso-bot bot commented Jan 18, 2024

Keziah Wesley reports a new STANDUP for today (2024-01-17):

Progress: Ported Ast and Module definitions to YJs storage. It should be finished by 2024-01-19.

Next Day: Next day I will be working on the #8237 task. Update translation, implement new staging/sync.

@enso-bot
Copy link

enso-bot bot commented Jan 19, 2024

Keziah Wesley reports a new STANDUP for today (2024-01-18):

Progress: Updaed translation. Implemented mutability APIs, as I found this should be done before new sync. It should be finished by 2024-01-19.

Next Day: Next day I will be working on the #8237 task. Implement new staging/sync.

@enso-bot
Copy link

enso-bot bot commented Jan 23, 2024

Keziah Wesley reports a new STANDUP for the last Friday (2024-01-19):

Progress: Ported match/replace to new APIs. Wrote Vue reactive impl of Module interface. Implemented main sync operation. It should be finished by 2024-01-19.

Next Day: Next day I will be working on the #8237 task. Update yjsmodel and port AST editing logic to new APIs

@enso-bot
Copy link

enso-bot bot commented Jan 23, 2024

Keziah Wesley reports a new 🔴 DELAY for today (2024-01-22):

Summary: There is 4 days delay in implementation of the AST-based synchronization (#8237) task.
It will cause 0 days delay for the delivery of this weekly plan.

Delay Cause: Decoupling sync identities from ExprIds because it will be easier to do now than separately

@enso-bot
Copy link

enso-bot bot commented Jan 23, 2024

Keziah Wesley reports a new STANDUP for today (2024-01-22):

Progress: Updated yjsmodel. Ported most AST editing logic to new APIs. It should be finished by 2024-01-23.

Next Day: Next day I will be working on the #8237 task. Finish porting API users.

@enso-bot
Copy link

enso-bot bot commented Jan 25, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-23):

Progress: Implemented new ID distinction, and opened a PR for all the refactoring existing code. It should be finished by 2024-01-23.

Next Day: Next day I will be working on the #8237 task. PR the main API changes.

@enso-bot
Copy link

enso-bot bot commented Jan 26, 2024

Keziah Wesley reports a new 🔴 DELAY for yesterday (2024-01-24):

Summary: There is 3 days delay in implementation of the AST-based synchronization (#8237) task.
It will cause 0 days delay for the delivery of this weekly plan.

Delay Cause: Added several improvements that are easier to do at the same time.

@enso-bot
Copy link

enso-bot bot commented Jan 26, 2024

Keziah Wesley reports a new STANDUP for yesterday (2024-01-24):

Progress: Finished PR for the new AST APIs. It should be finished by 2024-01-26.

Next Day: Next day I will be working on the #8237 task. Debugging and respond to review.

@enso-bot
Copy link

enso-bot bot commented Jan 26, 2024

Keziah Wesley reports a new STANDUP for today (2024-01-25):

Progress: Debugging and respond to review. It should be finished by 2024-01-26.

Next Day: Next day I will be working on the #8237 task. Switch to new synchronization.

@enso-bot
Copy link

enso-bot bot commented Jan 30, 2024

Keziah Wesley reports a new STANDUP for the last Friday (2024-01-26):

Progress: Implemented new synchronization. It should be finished by 2024-01-26.

Next Day: Next day I will be working on the #8804 task. Metadata in ASTs.

@kazcw kazcw mentioned this issue Jan 30, 2024
5 tasks
mwu-tow pushed a commit that referenced this issue Feb 2, 2024
)

- Synchronize Y.Js clients by AST (implements #8237).
- Before committing an edit, insert any parentheses-nodes needed for the concrete syntax to reflect tree structure (fixes #8884).
- Move `externalId` and all node metadata into a Y.Map owned by each `Ast`. This allows including metadata changes in an edit, enables Y.Js merging of changes to different metadata fields, and will enable the use of Y.Js objects in metadata. (Implements #8804.)

### Important Notes

- Metadata is now set and retrieved through accessors on the `Ast` objects.
- Since some metadata edits need to take effect in real time (e.g. node dragging), new lower-overhead APIs (`commitDirect`, `skipTreeRepair`) are provided for careful use in certain cases.
- The client is now bundled as ESM.
- The build script cleans up git-untracked generated files in an outdated location, which fixes lint errors related to `src/generated` that may occur when switching branches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
-gui d-hard Difficulty: significant prior knowledge required p-medium Should be completed in the next few sprints
Projects
Archived in project
2 participants