[WIP] Project Roots #5041

ChrisPenner · 2024-06-03T16:37:42Z

Overview

Major changes:

Loose code is gone from UCM, any code you had in your local loose-code will be automatically migrated to a project called legacy (or legacy1, legacy2, etc. if any name is taken). Note: you can still pull from loose code (for now).
The single global root is gone. Every project branch is loaded by its own root. Only one project-branch is loaded at a time.
Absolute paths now resolve to the root of the project-branch
Some commands can now accept paths/names from other branches of the form /branch:.path.in.project
You shouldn't EVER see another project UUID in a path 🎉
New reflog commands!
- reflog, branch.reflog or reflog.branch shows the changes for the current branch head
- project.reflog or reflog.project shows changes to any branches in the current project
- reflog.global shows changes across all project-branches in any project.
reset will correctly set only the current project-branch head.
undo rewinds one node in the history of the current project-branch, and will no longer cause any corruption or weirdness to your projects

Implementation notes

Adds new ProjectPath data type which is a fully-qualified path containing a project, branch, and absolute path within it. This is used pretty much everywhere.
There's no longer an in-memory Branch in Loop State, it was primarily removed to make it easier to pre-load other projects and keep them cached to speed up project-switching, but as a nice side-effect it's now impossible to get your memory branch out of sync with Sqlite from a ctrl-c.
I kept the old reflog around behind a deprecated.root-reflog command, but it's not scoped to projects, so probably isn't terribly useful.
- the old reflog is never written to anymore, but supposing you upgraded your codebase from an earlier UCM than this PR, and still wanted to reference some hashes that had been in its reflog, you can still read them here.
New Project-based reflog allows listing the reflog by project-branch, project, or global, and shows the hash for the head of the branch rather than the global root.
Removes "mostRecentNamespace" in favour of "currentProjectPath" in sqlite, but this otherwise behaves the same.
I had to refactor the migration module because this migration requires disabling foreign keys (temporarily, and I do a foreign key check after re-enabling), but unfortunately you can't do this inside a transaction, so I had to refactor the migrations to take a connection instead.
Moves createSchema from U.Codebase.Sqlite.Queries to Unison.Codebase.SqliteCodebase.Operations to prevent cyclic imports
Adds a new codebase migration which:
- Adds the new project reflog table
- Adds the project-branch head as a causal-hash-id column on the project-branch table, and populates it from all the projects in the project root.
- Makes a new legacy project from the existing root branch (minus .__projects)
- Adds a new scratch/main project
- Adds a currentProjectPath table to replace the most-recent-path functionality.

Interesting/controversial decisions

I ended up changing the branch relative syntax from /branch:path.to.thing into /branch:.path.to.thing because it seemed more consistent now that absolute paths are project relative, we can talk about changing it back if we like :)
transcripts run by transcripts (not ucm transcript) won't print timestamps in the reflog output

Test coverage

I ported most of the transcripts to projects on trunk, but obviously there are still a LOT of transcript changes to reflect the new behaviours, so please read through the transcript changes and transcript output.
Added an undo transcript, we didn't seem to have one

Loose ends

names.global, view.global, find.global are currently disabled until we decide how to address them, but we do plan to bring them back (and hopefully faster than before?)
Maybe improve caching of multiple branches so you can swap back-and-forth between them quickly
It's now pretty easy to add branch-relative-path support to more commands, so we should do a pass on that, not as part of this PR though.

More loose ends:

actual root namespace table still exists
Ensure newly updated/put branches get into the branch cache.

Add projectPath parser

ChrisPenner · 2024-07-10T17:40:50Z

unison-src/transcripts/cycle-update-5.md

This test is now inexpressible since we can't cd into subnamespaces in projects with the transcript parser, and cd'ing is deprecated anyways.

ChrisPenner · 2024-07-10T17:42:40Z

unison-src/transcripts/delete-namespace.output.md

+
+
+
+  □ 1. #sg60bvjo91 (start of history)


Notably you can no longer have an actually "empty" history, the project-branch-head has to be set to something, in this case the empty branch.

We could possibly check for equivalence to the empty hash and call it out in the output if we want.

ChrisPenner · 2024-07-10T17:45:18Z

unison-src/transcripts/diff-namespace.output.md

-    Original            Changes
-    1. ns2.fromJust  ┐  2. ns2.fromJust' (removed)
-    3. ns2.fromJust' │  
-    4. ns2.yoohoo    │  
-    5. ns3.fromJust  │  
-    6. ns3.fromJust' ┘  


We're running the delete from inside a specific branch now, so it doesn't show Originals from the others.

ChrisPenner · 2024-07-10T17:47:09Z

unison-src/transcripts/diff-namespace.output.md

-  b#aapqletas7 : ##Nat
-  b#aapqletas7 = ##Nat.+ a#vrs8gtkl2t 1
+  a#r3msrbpp1v : Nat
+  a#r3msrbpp1v = 777


I use the same original value for creating conflicts for both a and b now (777) because we can't yet alias terms from one branch into another.

ChrisPenner · 2024-07-10T17:47:45Z

unison-src/transcripts/empty-namespaces.output.md

+
+
+
+  □ 1. #sg60bvjo91 (start of history)


Same note as above, every branch must have a head, so there's no "empty namespace" anymore.

ChrisPenner · 2024-07-10T17:50:28Z

unison-src/transcripts/fix2254.output.md

@@ -86,30 +86,33 @@ scratch/a2> view A NeedsA f f2 f3 g
    = NeedsA (A a b Nat Nat)
    | Zoink Text

-  f : A Nat Nat Nat Nat -> Nat
+  f : #re3rf9cedk Nat Nat Nat Nat -> Nat


This needs looking into 🤔

ChrisPenner · 2024-07-10T17:56:02Z

unison-src/transcripts/names.output.md

-- We can search by suffix and find all definitions in the codebase named 'x', and each of their aliases respectively.
-.some> names.global x
-
-  Terms
-  Hash:   #gjmq673r1v
-  Names:  .some.otherplace.y .some.place.x .somewhere.z
-
-  Hash:   #pi25gcdv0o
-  Names:  .some.otherplace.x
+TODO: swap this back to a 'ucm' block when names.global is re-implemented

+```
+-- We can search from a different branch and find all names in the codebase named 'x', and each of their aliases respectively.
+scratch/other> names.global x
 -- We can search by hash, and see all aliases of that hash in the codebase
-.some> names.global #gjmq673r1v
-
-  Term
-  Hash:   #gjmq673r1v
-  Names:  .some.otherplace.y .some.place.x .somewhere.z
-
+scratch/other> names.global #gjmq673r1v


Temporarily disabled.

ChrisPenner · 2024-07-10T17:57:11Z

unison-src/transcripts/propagate.output.md

@@ -175,93 +175,3 @@ scratch/main> view preserve.otherTerm
  preserve.otherTerm y = someTerm y

 ```
-### Propagation only applies to the local branch


This is inexpressable now that we can't cd into a subnamespace of a project branch.

ChrisPenner · 2024-07-10T17:58:18Z

unison-src/transcripts/reflog.output.md

This transcript got mostly rewritten

ChrisPenner · 2024-07-10T17:58:32Z

unison-src/transcripts/reset.output.md

This transcript got mostly rewritten

ChrisPenner · 2024-07-10T17:59:30Z

unison-src/transcripts/undo.output.md

Brand new transcript

unison-src/transcripts/upgrade-sad-path.output.md

aryairani · 2024-07-10T18:34:01Z

codebase2/codebase-sqlite/sql/012-add-current-project-path-table.sql

+  project_id INTEGER NOT NULL,
+  branch_id INTEGER NOT NULL,
+  -- A json array like ["foo", "bar"]; the root namespace is represented by the empty array
+  path TEXT PRIMARY KEY NOT NULL,


the current path in the current project (might go away if cd goes away)

aryairani · 2024-07-10T19:13:25Z

unison-src/transcripts-round-trip/main.output.md

@@ -771,7 +771,7 @@ a |> f = f a
 This diff should be empty if the two namespaces are equivalent. If it's nonempty, the diff will show us the hashes that differ.

 ```ucm
-.> diff.namespace a1 a2
+scratch/main> diff.namespace /a1:. /a2:.


Would like to be able to diff.namespace /a1 /a2; would that already work?

unison-src/transcripts-round-trip/main.output.md

aryairani · 2024-07-10T19:28:22Z

unison-src/transcripts/diff-namespace.output.md


  Done.

 ```
 ```ucm
-.> diff.namespace b1 b2
+scratch/main> diff.namespace /b1: /b2:


Seems like this works too, which is nice

aryairani · 2024-07-10T19:40:42Z

unison-src/transcripts/diff-namespace.output.md

@@ -189,26 +184,26 @@ unique type Y a b = Y a b
  Added definitions:

    8.  type Y a b
-    9.  Y.Y : a -> b -> Y a b
+    9.  Y.Y : a -> b -> #md85ksgqel a b


unison-src/transcripts/diff-namespace.output.md

unison-src/transcripts/reflog.output.md

# Conflicts: # unison-cli-integration/integration-tests/IntegrationTests/transcript.output.md # unison-src/transcripts-round-trip/main.output.md # unison-src/transcripts/add-run.output.md # unison-src/transcripts/bug-strange-closure.output.md # unison-src/transcripts/cycle-update-5.output.md # unison-src/transcripts/delete.output.md # unison-src/transcripts/diff-namespace.output.md # unison-src/transcripts/move-namespace.output.md # unison-src/transcripts/name-selection.output.md # unison-src/transcripts/names.output.md # unison-src/transcripts/namespace-dependencies.output.md # unison-src/transcripts/propagate.output.md # unison-src/transcripts/reflog.output.md # unison-src/transcripts/reset.output.md # unison-src/transcripts/tab-completion.output.md # unison-src/transcripts/transcript-parser-commands.output.md

ChrisPenner added 23 commits May 21, 2024 09:32

Add Location pt 2

73145db

Remove direct access to root branch

fe751e1

Add migration to move project branches to sqlite

12b3107

Propagate root branch accessors

39f065e

Checkpoint

2c64c6a

Set currentProjectPath in SQLite

1a15c3f

Add getShallowProjectRootByNames

dca8431

Rewrite BranchRelativePaths machinery

2b50419

Resolve merge conflicts

5a11d23

Checkpoint

29fd307

Checkpoint

2c98ad1

Checkpoint

467dee1

Allow updating branch heads

457ca14

Merge deletion of git commands into project-root

9de134f

WIP

9756ac8

merge trunk

8377f68

Merge trunk

965f36f

Fix weird added file

33c42db

Remove root branch accessors from codebase

0d80992

Use onCreate hook to initialize projects

216967f

Clean up uses of root in Codebase, kill the root branch cache

1d906b3

Fix up uses of root branch in Backend

3361010

Propagate lack of root branch combinators to local Codebase Server

2e754a8

ChrisPenner force-pushed the cp/project-root branch from e32bfda to 34eeea4 Compare June 3, 2024 21:00

ChrisPenner added 6 commits June 3, 2024 14:12

Fix argparse for running 'main' within project

2773153

Add projectPath parser

Fix LSP to work with projects

33458a8

Merge trunk

77af994

Fix up some more root branch gets

4e2ccf2

Fix up input patterns for project branch args

138c6e9

Fix up Merge2

3267a3e