Skip to content
This repository has been archived by the owner on Aug 1, 2022. It is now read-only.

feat: thank you next #1463

Merged
merged 1 commit into from
Mar 26, 2021
Merged

feat: thank you next #1463

merged 1 commit into from
Mar 26, 2021

Conversation

xla
Copy link
Contributor

@xla xla commented Dec 5, 2020

Closes #1236
Closes #1226

Edit: Closes #1163
Edit: Closes #1078

@xla xla added proxy feature Something that doesn't exist yet labels Dec 5, 2020
@xla xla self-assigned this Dec 5, 2020
@NunoAlexandre
Copy link
Contributor

NunoAlexandre commented Jan 12, 2021

@FintanH @xla given the size of this PR, to ease the review, I would like to know if you made any technical decisions worth noting or what kind of interesting problems you've faced :)

@FintanH
Copy link
Contributor

FintanH commented Jan 12, 2021

given the size of this PR, to ease the review, I would like to know if you made any technical decisions worth noting or what kind of interesting problems you've faced :)

Good thinking, @NunoAlexandre. So I would sum up the majority of this PR as being mechanical changes. Mostly revolving around the new data types Project, Person, and LocalIdentity which replace the old entity data types Project and User.

Some interesting changes that needed to be made were involving the working copy creation (checkout and create). The interactions with git -- known as local transport because they are local interactions between the monorepo and a working copy -- are all done through Remotes now. This removes the need for registering local transports and waiting on results.

The removal of cypress tests is due to some interactions being idempotent (taking the same action twice has no effect). So creating the same project or person twice does nothing, and the same goes for checking out a project.

One notable change in the subroutine code was that @xla added an error variant for the default owner having to be set. This is because a lot of interactions need a default owner. Aside, we'd like to explore a more correct-by-construction approach to that post PR.

That's all I can think of for now, but if anything comes up I'll add it in a follow-up comment.

@NunoAlexandre
Copy link
Contributor

given the size of this PR, to ease the review, I would like to know if you made any technical decisions worth noting or what kind of interesting problems you've faced :)

Good thinking, @NunoAlexandre. So I would sum up the majority of this PR as being mechanical changes. Mostly revolving around the new data types Project, Person, and LocalIdentity which replace the old entity data types Project and User.

Some interesting changes that needed to be made were involving the working copy creation (checkout and create). The interactions with git -- known as local transport because they are local interactions between the monorepo and a working copy -- are all done through Remotes now. This removes the need for registering local transports and waiting on results.

The removal of cypress tests is due to some interactions being idempotent (taking the same action twice has no effect). So creating the same project or person twice does nothing, and the same goes for checking out a project.

One notable change in the subroutine code was that @xla added an error variant for the default owner having to be set. This is because a lot of interactions need a default owner. Aside, we'd like to explore a more correct-by-construction approach to that post PR.

That's all I can think of for now, but if anything comes up I'll add it in a follow-up comment.

Thank you, Fintan! Will give it a review tomorrow morning if that works for you.

@rudolfs
Copy link
Member

rudolfs commented Jan 13, 2021

Looking real good!

Gave it a spin already to familiarise myself with the new backend and noticed a couple of things:

  • we should probably remove the default seed node address from settings hynewpywqj6x4mxgj7sojhue3erucyexiyhobxx4du9w66hxhbfqbw@seedling.radicle.xyz:12345 so it doesn't try to connect to it, because seedling still runs the old backend code
  • something is off with checkouts, steps to repro:
    1. create a new project
    2. via the UI check out the project source to a different folder
    3. via the terminal cd into the project
    4. expected to be able to do git log and see the initial commit, instead got fatal: your current branch 'master' does not have any commits yet -- it seems like the project is created with main as the default branch, but the checkout defaults to master
  • there's an issue with showing a replicated project, steps to repro:
    1. set up 2 machines on a local network, let's call them machine A and machine B
    2. add A's peer address to B's seed nodes in B's settings (PEER_ID@IP:PORT), wait until the network icon show a 1 connection
    3. create a new project on machine A and note the project URN
    4. on machine B search for the URN and click "Follow"
    5. once the project is replicated on machine B, navigate to it
    6. expected to see the project contents, instead an error message is shown; if we select the maintainer's view in the repo selector, we can see the repo contents just fine
{
"code": "RemoteStoreError",
"message": "Incorrect input",
"source": {
"code": "UnknownException",
"message": "Incorrect input",
"stack": "Error: Incorrect input\n    at http (file:///Users/rudolfs/work/radicle-upstream-2/public/bundle.js:3457:19)\n    at async Promise.all (index 0)\n    at async fetch$6 (file:///Users/rudolfs/work/radicle-upstream-2/public/bundle.js:58889:31)",
"details": {
"response": {},
"name": "Error"}}}

own
maintainers-view

Copy link
Contributor

@NunoAlexandre NunoAlexandre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid overall!

Some questions/thoughts:

The removal of cypress tests is due to some interactions being idempotent (taking the same action twice has no effect). So creating the same project or person twice does nothing, and the same goes for checking out a project.

  • This is great to have correct by design! However, I'm wondering if it wouldn't be better to have the tests in place to validate such property?

  • What is the rationale behind the name LocalIdentity? I see that wraps a VerifiedPerson, which seems rather easier and less obscure to understand. Understanding it might help me build a better mental model.

proxy/api/src/http.rs Show resolved Hide resolved
proxy/api/src/http/error.rs Outdated Show resolved Hide resolved
),
create::validation::Error::Remote(_) => (
StatusCode::INTERNAL_SERVER_ERROR,
"NO_REMOTE",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these string representations of the errors are suffixed with _ERROR, others not. Is that intentional?

proxy/coco/tests/working_copy.rs Show resolved Hide resolved
@kim
Copy link
Contributor

kim commented Jan 13, 2021

* What is the rationale behind the name `LocalIdentity`? I see that wraps a `VerifiedPerson`, which seems rather easier and less obscure to understand

A LocalIdentity is not just any person, but one that was created using the local keypair, or has a verifiable delegation to the local key, and has been selected as the identity to use for a certain operation (you can have any number of identities associated with the local keypair).

@NunoAlexandre
Copy link
Contributor

A LocalIdentity is not just any person, but one that was created using the local keypair, or has a verifiable delegation to the local key, and has been selected as the identity to use for a certain operation

That makes sense reading, but I am specifically talking about the type definition:

pub struct LocalIdentity(VerifiedPerson);

// where
pub type VerifiedPerson = VerifiedIdentity<PersonDoc>;

(you can have any number of identities associated with the local keypair).

Having the type definitions in mind, I am not sure how you mean this or how the two come together, as I'd assume from that sentence that LocalIdentity would be a union type, not exclusively a VerifiedPerson.

@kim
Copy link
Contributor

kim commented Jan 13, 2021

@NunoAlexandre LocalIdentity is a newtype with a "smart constructor", if you will. You can only construct it such that the aforementioned invariants hold (one of which is the verification status, which is captured in the type system), or by unsafe coercion. What's unclear about that?

[...] as I'd assume from that sentence that LocalIdentity would be a union type, not exclusively a VerifiedPerson.

Why is that? A union of what? Look:

From the local perspective, there is one and only one keypair to use with radicle-link. Associating an "identity" (that is, a blob of data containing some metadata) with that key is an explicit interaction -- it is not even required that you do this, it is perfectly fine to work with an "anonymous" key.

Likewise, there is nothing which prevents you (or anyone else) from creating multiple identity documents and adding that key to it, for example on another device -- to prevent someone else from impersonating as you, there is the additional constraint that the LocalIdentity must be signed by the local key, which I forgot to mention.

It is not that you gain anything by doing this -- after all, anyone can correlate them by the key, so it doesn't make sense to e.g. create "nuno" and "nuno-work" with the expectation that this would be privacy-preserving. However, if we were to enforce on the protocol level that any given public key can appear in one and only one identity, every peer would have to keep track of the mapping key -> identity, and if a duplicate is detected... do what? Stop replicating anything signed by that key?

You see that this would create additional complexity with dubious benefits -- for example, how do you bubble up to the user that such a conflict was detected, and therefore some data which they'd expect to be there is not there? How do you recover from such a situation?

Since it is not harmful per-se to have multiple personalities (it may only confuse the people you interact with), we thus allow key -> identity to have a 0..* cardinality.

@NunoAlexandre
Copy link
Contributor

NunoAlexandre commented Jan 13, 2021

@NunoAlexandre LocalIdentity is a newtype with a "smart constructor", if you will. You can only construct it such that the aforementioned invariants hold (one of which is the verification status, which is captured in the type system), or by unsafe coercion. What's unclear about that?

[...] as I'd assume from that sentence that LocalIdentity would be a union type, not exclusively a VerifiedPerson.

Why is that? A union of what? Look:

From the local perspective, there is one and only one keypair to use with radicle-link. Associating an "identity" (that is, a blob of data containing some metadata) with that key is an explicit interaction -- it is not even required that you do this, it is perfectly fine to work with an "anonymous" key.

Likewise, there is nothing which prevents you (or anyone else) from creating multiple identity documents and adding that key to it, for example on another device -- to prevent someone else from impersonating as you, there is the additional constraint that the LocalIdentity must be signed by the local key, which I forgot to mention.

It is not that you gain anything by doing this -- after all, anyone can correlate them by the key, so it doesn't make sense to e.g. create "nuno" and "nuno-work" with the expectation that this would be privacy-preserving. However, if we were to enforce on the protocol level that any given public key can appear in one and only one identity, every peer would have to keep track of the mapping key -> identity, and if a duplicate is detected... do what? Stop replicating anything signed by that key?

You see that this would create additional complexity with dubious benefits -- for example, how do you bubble up to the user that such a conflict was detected, and therefore some data which they'd expect to be there is not there? How do you recover from such a situation?

Since it is not harmful per-se to have multiple personalities (it may only confuse the people you interact with), we thus allow key -> identity to have a 0..* cardinality.

We are discussing this at different levels.

My confusion comes from a place were, given that "Identity", in Link, can take the form of a Person, a Project, etc, calling LocalIdentity to a VerifiedPerson seems confusing. So I was trying to understand if there was a reason for that broad scope in the naming when, at the type level, and as you said, it can only hold a "Person" that meets the mentioned criteria.

Using an analogy to ease this out, say that we'd have the notion of Shape = Circle Float | Rectangle Float Float (standing for "Identities") and we'd have SpecialRectangle (standing for VerifiedPerson) that represents a Rectangle whose length == π*width. Now, we'd have LocalShape(SpecialRectangle). That's where my confusion emerged: if this can only be a SpecialRectangle, why the broad naming.

Edit: This is out of scope for this PR, sorry for the noise, happy to have it removed.

@kim
Copy link
Contributor

kim commented Jan 13, 2021

Shape = Circle Float | Rectangle Float Float

Your confusion seems to be that VerifiedPerson has the broader meaning. There is a partial isomorphism between LocalIdentity and VerifiedPerson: a LocalIdentity can always be converted to a VerifiedPerson, but not the other way round. In your example, circles and rectangles are of the same type, but verified and local identities are not -- you must prove that the identity you got is a local one (it's a refinement type in that sense).

Persons and Projects unify elsewhere, but there is no need for unifying VerifiedPerson and LocalIdentity (because of the prism).

@kim
Copy link
Contributor

kim commented Jan 13, 2021

Oh I get it: what throws you off is "identity". If the type was called LocalPerson, it would check out with you. I think at that level, I would just recommend import aliasing 🤷

@NunoAlexandre
Copy link
Contributor

NunoAlexandre commented Jan 13, 2021

Oh I get it: what throws you off is "identity". If the type was called LocalPerson, it would check out with you. I think at that level, I would just recommend import aliasing 🤷

Exactly :) it's not a big deal, I just wanted to understand if there was more to it that I should learn about. And there was :) thanks for bearing!

@FintanH FintanH added this to Next in Thank You Next Jan 14, 2021
}

#[tokio::test]
async fn can_checkout() -> Result<(), Box<dyn std::error::Error>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NunoAlexandre you mentioned validating the properties, here is the relevant one for radicle-upstream. For project and person creation, these are properties in librad and I believe are tested there.

@FintanH
Copy link
Contributor

FintanH commented Jan 15, 2021

we should probably remove the default seed node address from settings hynewpywqj6x4mxgj7sojhue3erucyexiyhobxx4du9w66hxhbfqbw@seedling.radicle.xyz:12345 so it doesn't try to connect to it, because seedling still runs the old backend code

Good shout 👍

something is off with checkouts, steps to repro:

So the default HEAD value always seems to be refs/heads/master. It's a simple fix to set the head to our default branch.

there's an issue with showing a replicated project, steps to repro:

We weren't going far enough when checking that a peer is a contributor. We were only checking that rad/signed_refs exists, however, we should have been checking that the heads in there were empty if rad/signed_refs did exist.

All fixes for those issues have been pushed now :)

@FintanH FintanH mentioned this pull request Jan 20, 2021
.await
.ok();

match state.clone_project(url.clone(), None).await {
match state
.clone_project(urn.clone(), remote_peer, None, None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xla: I opted for defaulting the fetch::Limit -- introduced here ddb6dae -- but it would be nice to be able to change this at runtime as well.

Any thoughts on how this could be injected into the subroutines at runtime? Another channel perhaps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you expand on what potential scenarios look like that would constitute runtime configuration changes? The reason I'm asking is that neither this comment or the original commit message reason about the why such a capability is desried.

Any thoughts on how this could be injected into the subroutines at runtime? Another channel perhaps?

Yeah usually configuration changes over time would come through some form of channel. For the configured seeds we use a watch channel to provide last value semantics. Would need to look deeper into this use-case to give proper recommendation. There might be a general need for a latest configuration made to be known to the internals of the peer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So a use case would be the following scenario.

We ask to clone the Linux project, but as we're doing so the fetcher throws an error and says we've exceeded the configured limits. We're sure that this really is the Linux project and not a bunch of spammy data, so we configure our limits to go higher, try the replication again, and once we've got the project we can reset the limit.

Does that help shed some light on the question here?

Copy link
Contributor Author

@xla xla Feb 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat. Will the limit go down again after the linux kernel has been replicated from pc-torvalds01. By the sound of it this has to do with trust and spam protection. Which seems odd to be bound to runtime configuration changes. As in where would the signal come from that the fetch limit can be increased. I don't want to challenge the scenario too specifically, just curious to see how the concerns align.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat. Will the limit go down again after the linux kernel has been replicated from pc-torvalds01.

Ya you would want to turn it down again.

By the sound of it this has to do with trust and spam protection.

Exactly, it's exactly a trust mechanism.

Which seems odd to be bound to runtime configuration changes. As in where would the signal come from that the fetch limit can be increased.

As @kim and I deliberated on it, we landed on "sane" default values and should be configurable depending on the scenario -- giving the malkovich the final say when using replicate.

How are you thinking about it?

@@ -239,13 +259,15 @@ impl Repository {
let path = path.join(name);
log::debug!("Setting up new repository @ '{}'", path.display());
let repo = Self::initialise(path, description, &default_branch)?;
log::debug!("REPO PATH: {}", repo.path().display());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

@FintanH
Copy link
Contributor

FintanH commented Feb 3, 2021

Note to self:

  • Tracking and finding a user didn't add them to the include file
  • When creating a project there's a message saying the upstream is gone

@FintanH
Copy link
Contributor

FintanH commented Feb 3, 2021

When creating a project there's a message saying the upstream is gone

Fixed in bcad1d9

@rudolfs
Copy link
Member

rudolfs commented Feb 4, 2021

Did a small test run with 2 laptops and setzling and bumped into some issues.
I'm not sure whether they're known issues or something new, thought it would be worth mentioning.

Steps:

  • onboarded both machines
  • created a project on machine A
  • saw the project appear on setzling
  • then replicated the project on machine B and did a local checkout via the Fork button
  • created a new commit on machine A and pushed it to the rad remote
  • commit appeared in the project on machine A
  • waited around 5 mins, but the commit didn't appear on machine B
  • added machine B as a remote to the project on machine A, waited 5 min, the remote was still in the waiting area
  • restarted Upstream on both machines, waited 5 min, but the commit still didn't appear on machine B, however the remote showed up as added on machine A
  • on machine A selected machine B in the project's peer selector and got an error

Machine A device id: hydddxdbqoj6azkq39637eso6inuxw99tpo9h8aqgnkracfaufy9jn
Machine B device id: hydk51kw7tisdgz5dukrxyrcuakmx8s8co48souqoj3jr3ac75skj1

Error messages

UI:

{
  "code": "RemoteStoreError",
  "message": "Incorrect input",
  "source": {
    "code": "UnknownException",
    "message": "Incorrect input",
    "stack": "Error: Incorrect input\n    at http (file:///Users/rudolfs/work/radicle-upstream/public/bundle.js:1991:19)\n    at async Promise.all (index 0)\n    at async fetch$6 (file:///Users/rudolfs/work/radicle-upstream/public/bundle.js:143163:31)",
    "details": {
      "response": {},
      "name": "Error"
    }
  }
}

Proxy:

 ERROR api::http::error         > Rejection(State(MissingRef { reference: Reference { remote: Some(PeerId(PublicKey(VerificationKeyBytes("d5b92a9d8d6c335f639a88f01193c296f3d8ec868f684dd04e524ce19ddd9499")))), category: Heads, name: RefLike("master"), namespace: Some(Namespace(Urn { id: Oid(783793840e886b19fc8d301df4be190bc9ce96dc), path: None })) } }))
Screenshot 2021-02-04 at 09 26 38
Machine A monorepo:
Application Support/xyz.radicle.radicle-link » tree
.
├── active_profile
└── d9a6c997-c481-4e0b-888f-6b0e30792458
    ├── git
    │   ├── FETCH_HEAD
    │   ├── HEAD
    │   ├── config
    │   ├── description
    │   ├── hooks
    │   │   └── README.sample
    │   ├── info
    │   │   └── exclude
    │   ├── objects
    │   │   ├── 04
    │   │   │   └── 612ea1862481ac2b936ca2d6f482709859ef72
    │   │   ├── 1a
    │   │   │   └── 7f34ee9e09d604630292973a41e09a581f99d7
    │   │   ├── 24
    │   │   │   ├── 25d9839bbaac941047038c6b2e00d8d90e4461
    │   │   │   └── 2ba95a2c8b60144e7d27d1628e127b042b12fe
    │   │   ├── 29
    │   │   │   └── ae94c5cb1201b85ae6e054f8f7fbe4b92dd67a
    │   │   ├── 32
    │   │   │   └── fe621707c650f6f6e302f90c6911653f87e4a7
    │   │   ├── 33
    │   │   │   └── f428753e20bf09b16a72bc27ff162385d1572f
    │   │   ├── 3e
    │   │   │   └── 640a3d6cb7d114e532d20f08999185dd61eebd
    │   │   ├── 45
    │   │   │   └── 9113266b6690a9086c929f8cedf3f34baa48fe
    │   │   ├── 68
    │   │   │   └── 64c20f006de8f2ee3f8129de8d68690549565a
    │   │   ├── 69
    │   │   │   └── 304ea4b1b500d8ceccccb96ac23e2b12bb87eb
    │   │   ├── 6f
    │   │   │   └── bea447d0c202a3153c612b0cc6bd8c7a1d50bb
    │   │   ├── 78
    │   │   │   └── 3793840e886b19fc8d301df4be190bc9ce96dc
    │   │   ├── 7e
    │   │   │   └── d378e2fdfeebc3e684ad31b7d46cc7423047fb
    │   │   ├── 88
    │   │   │   └── 3b84e17bd280fa83c4e06ac80a787b3c6ef4ac
    │   │   ├── 8b
    │   │   │   └── d6bd0bdca19938d09936cf0ec94549e4ed47b2
    │   │   ├── 8c
    │   │   │   └── f95616a22dc7e90777d5730487d15d669a0459
    │   │   ├── 90
    │   │   │   └── 8d891149eb1ad2e288bbb7259b5f2f3e02951b
    │   │   ├── 97
    │   │   │   ├── 70fc68874662b0709fc43ddf6d21477865c7dc
    │   │   │   └── d0a8d8cf2491889e5a142f71f42ae2fbd15cdd
    │   │   ├── a3
    │   │   │   └── 949341992a6853e93eeec8c677e03f23f1b5c8
    │   │   ├── ad
    │   │   │   └── 74a118d941897401e596ceb9b095a49d34e155
    │   │   ├── cb
    │   │   │   └── 2438b389488201a3030e33dc7403923cb8a95a
    │   │   ├── cf
    │   │   │   └── e4ff04675474f90498cf30933148ff9b0fc60b
    │   │   ├── d1
    │   │   │   └── 9e047c534404193de7514263fcdfde58fbe6cb
    │   │   ├── db
    │   │   │   └── a885d983566603f023f51ec4cb865785d6e4d2
    │   │   ├── dd
    │   │   │   └── ba3c73d2889f9324af5b254ce96fd89e934bc9
    │   │   ├── e9
    │   │   │   └── 023890699d385c232e4aaa5e5b485dc2328c92
    │   │   ├── ee
    │   │   │   └── b010f312eabfabfe80b390d66abe0a39f930b9
    │   │   ├── f3
    │   │   │   └── 3b283de52fd955fcd48368d562934a5fce0991
    │   │   ├── info
    │   │   └── pack
    │   │       ├── pack-1f080b0f30be160399a894e8d87639d3d54780c5.idx
    │   │       ├── pack-1f080b0f30be160399a894e8d87639d3d54780c5.pack
    │   │       ├── pack-c8a58d53531d768a77c318b233acd3099cb501ea.idx
    │   │       └── pack-c8a58d53531d768a77c318b233acd3099cb501ea.pack
    │   └── refs
    │       ├── heads
    │       ├── namespaces
    │       │   ├── hnrk8ophuoo8eo4a391guy8xwzacoz1qq15qy
    │       │   │   └── refs
    │       │   │       ├── heads
    │       │   │       │   └── master
    │       │   │       ├── rad
    │       │   │       │   ├── id
    │       │   │       │   ├── ids
    │       │   │       │   │   └── hnrknejq3oqp5imrwnbdo8ddmfayptseqetoo
    │       │   │       │   ├── self
    │       │   │       │   └── signed_refs
    │       │   │       └── remotes
    │       │   │           ├── hydddxdbqoj6azkq39637eso6inuxw99tpo9h8aqgnkracfaufy9jn
    │       │   │           │   └── rad
    │       │   │           │       ├── id
    │       │   │           │       ├── ids
    │       │   │           │       │   └── hnrknejq3oqp5imrwnbdo8ddmfayptseqetoo
    │       │   │           │       ├── self
    │       │   │           │       └── signed_refs
    │       │   │           └── hydk51kw7tisdgz5dukrxyrcuakmx8s8co48souqoj3jr3ac75skj1
    │       │   │               └── rad
    │       │   │                   ├── id
    │       │   │                   ├── ids
    │       │   │                   │   └── hnrknejq3oqp5imrwnbdo8ddmfayptseqetoo
    │       │   │                   ├── self
    │       │   │                   └── signed_refs
    │       │   └── hnrknejq3oqp5imrwnbdo8ddmfayptseqetoo
    │       │       └── refs
    │       │           └── rad
    │       │               ├── id
    │       │               ├── self
    │       │               └── signed_refs
    │       └── tags
    ├── git-includes
    │   └── hnrk8ophuoo8eo4a391guy8xwzacoz1qq15qy.inc
    └── keys
        └── librad.key

56 directories, 59 files
Machine B monorepo state:
Application Support/xyz.radicle.radicle-link » tree
.
├── 59652cfc-8de1-4928-a4b5-78ad406d1dd5
│   ├── git
│   │   ├── FETCH_HEAD
│   │   ├── HEAD
│   │   ├── config
│   │   ├── description
│   │   ├── hooks
│   │   │   └── README.sample
│   │   ├── info
│   │   │   └── exclude
│   │   ├── objects
│   │   │   ├── 27
│   │   │   │   └── 94615c11b607b294e5ec78e9c8e6f44f6d2cf2
│   │   │   ├── 2c
│   │   │   │   └── 2b685f5af48b9c55dc19d2f5b83f1560d9f5dc
│   │   │   ├── 2d
│   │   │   │   └── 81d98333ba4677beb41d24dcce6235a396d32f
│   │   │   ├── 35
│   │   │   │   └── c241d67118d01dad47d188ea196b0751883b32
│   │   │   ├── 48
│   │   │   │   └── 0109e51ce02f9e21fc68e430e80df27e7172e4
│   │   │   ├── 4f
│   │   │   │   └── 2f0fe8786382cba3b849e1e92faeebf75907c8
│   │   │   ├── 64
│   │   │   │   └── dd30923ff40e49afcde6cc5cd326af9c526379
│   │   │   ├── 67
│   │   │   │   └── d03ae07efd4c844ea0ce41e4b59c0e252cd17b
│   │   │   ├── 80
│   │   │   │   └── 78c1aa2bacc9629f1da091bb6883fe658b1240
│   │   │   ├── 93
│   │   │   │   └── 51c27897f8032d4f339461933da8df7aa57937
│   │   │   ├── 9a
│   │   │   │   └── 652cb6b54a72a953d4cdecb3a73921d416a160
│   │   │   ├── a4
│   │   │   │   └── 8d107dc0eac5a3f91aefe1a36b321200e88eb6
│   │   │   ├── ab
│   │   │   │   └── bd01223a0db5daf819fcf4ed1909dfcd4c4d6e
│   │   │   ├── ad
│   │   │   │   └── 3af707da2a02437007c59877356e88bad5e1c9
│   │   │   ├── af
│   │   │   │   └── b7a993211ab388e4c05e9d40fdf4c56d8ae851
│   │   │   ├── b3
│   │   │   │   └── f0ffb9e2b475cd5056383b0fadd96f18a89629
│   │   │   ├── e0
│   │   │   │   └── 1b3b8422d66c713f42bbde050c8b09cbea799c
│   │   │   ├── e3
│   │   │   │   └── 8fc677d89fc5fa4e715e3ab9b50607f116ea19
│   │   │   ├── info
│   │   │   └── pack
│   │   │       ├── pack-8a0c7cf50ac057dc54f417ff5a3b945692d5cc08.idx
│   │   │       ├── pack-8a0c7cf50ac057dc54f417ff5a3b945692d5cc08.pack
│   │   │       ├── pack-a7e92c4dd37bf2459956377e47b6c6f40eeed56e.idx
│   │   │       ├── pack-a7e92c4dd37bf2459956377e47b6c6f40eeed56e.pack
│   │   │       ├── pack-c0a22e4f510e6368ea8abdf8e31c2485a35aaf84.idx
│   │   │       └── pack-c0a22e4f510e6368ea8abdf8e31c2485a35aaf84.pack
│   │   └── refs
│   │       ├── heads
│   │       ├── namespaces
│   │       │   ├── hnrk8ophuoo8eo4a391guy8xwzacoz1qq15qy
│   │       │   │   └── refs
│   │       │   │       ├── heads
│   │       │   │       │   └── master
│   │       │   │       ├── rad
│   │       │   │       │   ├── id
│   │       │   │       │   ├── ids
│   │       │   │       │   │   └── hnrknejq3oqp5imrwnbdo8ddmfayptseqetoo
│   │       │   │       │   ├── self
│   │       │   │       │   └── signed_refs
│   │       │   │       └── remotes
│   │       │   │           ├── hybz9gfgtd9d4pd14a6r66j5hz6f77fed4jdu7pana4fxaxbt369kg
│   │       │   │           │   └── rad
│   │       │   │           │       └── ids
│   │       │   │           └── hydddxdbqoj6azkq39637eso6inuxw99tpo9h8aqgnkracfaufy9jn
│   │       │   │               ├── heads
│   │       │   │               │   └── master
│   │       │   │               └── rad
│   │       │   │                   ├── id
│   │       │   │                   ├── self
│   │       │   │                   └── signed_refs
│   │       │   ├── hnrkey6gbiei431mnuhq4brp5pnb9h3cmnjyy
│   │       │   │   └── refs
│   │       │   │       └── rad
│   │       │   │           ├── id
│   │       │   │           ├── self
│   │       │   │           └── signed_refs
│   │       │   └── hnrknejq3oqp5imrwnbdo8ddmfayptseqetoo
│   │       │       └── refs
│   │       │           └── rad
│   │       │               └── id
│   │       └── tags
│   ├── git-includes
│   │   └── hnrk8ophuoo8eo4a391guy8xwzacoz1qq15qy.inc
│   └── keys
│       └── librad.key
└── active_profile

49 directories, 46 files

@FintanH
Copy link
Contributor

FintanH commented Feb 4, 2021

Seems like the seed isn't replicating heads when someone forks the project. This would explain @rudolfs error messages above.

@xla xla changed the title feat(proxy): thank you next feat: thank you next Feb 9, 2021
coco::blob(&mut browser, Some(revision.clone()), path, None)
})
.await?;
let arrows = "text/arrows.txt";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The renaming here was clippy giving out about name shadowing of path.

Copy link
Contributor

@FintanH FintanH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@geigerzaehler @MeBrei @rudolfs I've left my inline summary of portions of the code that are probably worth more attention. Thank you and good luck 🙇

proxy/api/src/context.rs Outdated Show resolved Hide resolved
/// Peer [`librad::net::peer::RunLoop`] to advance the network protocol.
run_loop: RunLoop,
/// Underlying state that is passed to subroutines.
state: State,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

net::peer::Peer is the new State. It's essentially what was the PeerApi, but a simplified and less footgun-y version

@@ -74,21 +84,27 @@ impl MaybeFrom<&Input> for Event {
Input::Announce(input::Announce::Succeeded(updates)) => {
Some(Self::Announced(updates.clone()))
},
Input::Peer(event) => match event {
PeerEvent::GossipFetch(FetchInfo {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synced: usize,
/// Number of synchronisation underway.
syncs: usize,
/// The set of failed syncs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now keep track of the PeerIds we've interacted with for the syncing process on startup

@@ -140,9 +158,10 @@ pub struct RunState {
/// `Connected(Peer1) -> Connected(Peer1) -> Disconnecting(Peer1)`
//
// FIXME(xla): Use a `Option<NonEmpty>` here to express the invariance.
connected_peers: HashMap<PeerId, usize>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer get messages about Connected and Disconnected so we don't need this HashMap. Instead, we can poll for the set of connected peers

/// Current internal status.
pub status: Status,
stats: net::protocol::event::downstream::Stats,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stats is what we can poll for Peer information, such as the connected peers

/// Handle [`input::Sync`]s.
fn handle_peer_sync(&mut self, input: &input::Sync) -> Vec<Command> {
if let Status::Syncing { synced, syncs } = self.status {
if let Status::Syncing {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic is changing here because of the move from usize to HashSet<PeerId>

match (&self.status, event) {
// Go from [`Status::Stopped`] to [`Status::Started`] once we are listening.
(Status::Stopped { .. }, ProtocolEvent::Listening(_addr)) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Listening doesn't exist anymore and it's now Endpoint::Up, with Endpoint::Down being its inverse

self.status = Status::Started;
self.status_since = SystemTime::now();

vec![]
},
(state, ProtocolEvent::Connected(peer_id)) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -1,799 +1,878 @@
//! Utility to work with the peer api of librad.

use std::{convert::TryFrom as _, net::SocketAddr, path::PathBuf, sync::Arc, time::Duration};
use std::{convert::TryFrom as _, net::SocketAddr, path::PathBuf};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file easily has the worst diff to review. We removed the State struct and made all the functions top-level. They now take &net::peer::Peer to interact with the underlying Storage. The changes were mechanical and the functions should be on par with what was there.

@geigerzaehler
Copy link
Contributor

geigerzaehler commented Mar 3, 2021

@MeBrei, @rudolfs, I’m taking a look at everything in proxy/coco.

@@ -106,25 +106,6 @@ context("project checkout", () => {
expect(result.stdout).to.equal(`rad`);
}
);

// Make sure we can't check out a project to the same directory twice.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this removed because the behavior changed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user facing behaviour has changed, and I think we should test this new behaviour. It's fine if we address this in a follow-up issue.

However, when testing the new behaviour manually, I bumped into some errors.

In a nutshell: when I create a project, and then re-add the same project again but with a different description, I get an error. Afterwards the project list screen also shows an error on every load.

Screen.Recording.2021-03-03.at.16.00.51.mov
{
  "code": "ProjectCreationFailure",
  "message": "Could not create project: the `rad` remote was found but the url field does not match the provided url, found: 'rad://hnrkps7zycckatezfsndcynur3f7jt5x94rdy.git' expected: 'rad://hnrkm8gpgi1koebhaaq965guh1193wysberto.git'",
  "source": {
    "response": {}
  }
}

{
  "code": "ProjectRequestFailure",
  "message": "The project new could not be loaded",
  "details": {
    "type": "defaultBranch",
    "urn": "rad:git:hnrkm8gpgi1koebhaaq965guh1193wysberto",
    "shareableEntityIdentifier": "%rad:git:hnrkm8gpgi1koebhaaq965guh1193wysberto",
    "metadata": {
      "name": "new",
      "description": "hola",
      "defaultBranch": "main",
      "maintainers": [
        "rad:git:hnrkmndyrdnf17aspdu4ikm1s3adqwa16stby"
      ]
    },
    "stats": null
  }
}

I also wonder if the new behaviour where adding an existing project just opens the project makes sense from a user's perspective. When re-adding the project, the notification says "Project created" although the project already exists, whereas before the UI would just show a validation error telling the user that the project already exists and blocking further action. \cc @juliendonck

Screenshot 2021-03-03 at 16 14 08

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thoroughness @rudolfs 🙇 So what's happening is that the project with the same name and description has a different URN to the same name and no description. This is because they have different hash outputs. So, with this in mind, it's probably best that we don't allow to use the same working copy for checkout and guard against it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this is a bit more sinister than I thought. Before we were able to tell if an Entity ended up with the same URN. But we can only know the URN once we create the project, which is idempotent. So we can't easily tell if the project was created or just didn't do anything. Need to think about that more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we were able to tell if an Entity ended up with the same URN.

Wait what, how is this different now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to determine the Urn before calling create? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, was just looking at that 😅 I'll create a proof for theory tomorrow

@@ -378,28 +378,6 @@ context("project creation", () => {
.contains(`Project ${repoName} successfully created`)
.should("exist");
commands.pick("notification").contains("Close").click();

// Make sure we can't add the same project twice.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same question as above

Copy link
Member

@rudolfs rudolfs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I built the release dmg package on my macOS Big Sur machine via yarn dist and it seems to be broken due to some code signing issues in libraries linked in proxy. When I build the package on master, it just works, so something must have changed in our dependencies. 🤷

I don't have a Linux machine, but somebody should test our Linux AppImage package on this branch as well!

Screenshot 2021-03-03 at 17 01 23

dyld: Library not loaded: /usr/local/opt/openssl@1.1/lib/libssl.1.1.dylib
  Referenced from: /Applications/Radicle Upstream.app/Contents/Resources/radicle-proxy
  Reason: no suitable image found.  Did find:
	/usr/local/opt/openssl@1.1/lib/libssl.1.1.dylib: code signature in (/usr/local/opt/openssl@1.1/lib/libssl.1.1.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

@@ -106,25 +106,6 @@ context("project checkout", () => {
expect(result.stdout).to.equal(`rad`);
}
);

// Make sure we can't check out a project to the same directory twice.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user facing behaviour has changed, and I think we should test this new behaviour. It's fine if we address this in a follow-up issue.

However, when testing the new behaviour manually, I bumped into some errors.

In a nutshell: when I create a project, and then re-add the same project again but with a different description, I get an error. Afterwards the project list screen also shows an error on every load.

Screen.Recording.2021-03-03.at.16.00.51.mov
{
  "code": "ProjectCreationFailure",
  "message": "Could not create project: the `rad` remote was found but the url field does not match the provided url, found: 'rad://hnrkps7zycckatezfsndcynur3f7jt5x94rdy.git' expected: 'rad://hnrkm8gpgi1koebhaaq965guh1193wysberto.git'",
  "source": {
    "response": {}
  }
}

{
  "code": "ProjectRequestFailure",
  "message": "The project new could not be loaded",
  "details": {
    "type": "defaultBranch",
    "urn": "rad:git:hnrkm8gpgi1koebhaaq965guh1193wysberto",
    "shareableEntityIdentifier": "%rad:git:hnrkm8gpgi1koebhaaq965guh1193wysberto",
    "metadata": {
      "name": "new",
      "description": "hola",
      "defaultBranch": "main",
      "maintainers": [
        "rad:git:hnrkmndyrdnf17aspdu4ikm1s3adqwa16stby"
      ]
    },
    "stats": null
  }
}

I also wonder if the new behaviour where adding an existing project just opens the project makes sense from a user's perspective. When re-adding the project, the notification says "Project created" although the project already exists, whereas before the UI would just show a validation error telling the user that the project already exists and blocking further action. \cc @juliendonck

Screenshot 2021-03-03 at 16 14 08

native/main.ts Outdated
"--default-seed",
"hynewpywqj6x4mxgj7sojhue3erucyexiyhobxx4du9w66hxhbfqbw@seedling.radicle.xyz:12345",
];
proxyArgs = [];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a follow-up issue about setting up a production #next seed and updating the value here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in my brain stack

Copy link
Contributor

@geigerzaehler geigerzaehler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m about 70% through. Here are my (minor) comments so far.

proxy/coco/src/peer/subroutines.rs Outdated Show resolved Hide resolved
proxy/coco/src/peer/subroutines.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@geigerzaehler geigerzaehler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more things I discovered

proxy/coco/tests/common.rs Show resolved Hide resolved
proxy/coco/tests/gossip.rs Outdated Show resolved Hide resolved
proxy/coco/tests/working_copy.rs Outdated Show resolved Hide resolved
proxy/coco/tests/working_copy.rs Outdated Show resolved Hide resolved
proxy/coco/tests/replication.rs Outdated Show resolved Hide resolved
proxy/coco/tests/replication.rs Outdated Show resolved Hide resolved
proxy/coco/tests/replication.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@geigerzaehler geigerzaehler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple more questions and suggestions.

proxy/coco/src/project.rs Show resolved Hide resolved
/// # Errors
/// * if initialisation of the repository fails
/// * if branch or remote manipulation fails
pub fn clone<F>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not used outside the module

Suggested change
pub fn clone<F>(
fn clone<F>(

proxy/coco/src/project/checkout.rs Show resolved Hide resolved
proxy/coco/src/project/checkout.rs Show resolved Hide resolved
proxy/coco/src/project/checkout.rs Show resolved Hide resolved
proxy/coco/src/project/checkout.rs Show resolved Hide resolved
There were some major changes in the radicle-link project that required
radicle-upstream to update to in order for it to move forward on upcoming
features. This patch introduces those changes and a short list of those changes
are outlined below.

The [Identity
specification](https://github.com/radicle-dev/radicle-link/blob/a8f79e4b557030c07e64f7cfb299c000c754c751/docs/spec/sections/002-identities/index.md)
was introduced and implemented. This replaces the Entity code, using Project,
Person, and LocalIdentity. This is a breaking change that means the Radicle IDs
for all projects and persons are different to the previous Entity Radicle IDs.

The local transport became safer and we can use an API supplied by radicle-link
for safely interacting with working copies.

There was also an overhaul of the networking stack in radicle-link. This
required some of the protocol messages to be removed and/or changed. In turn
radicle-upstream had to adapt to these changes. A bonus of this part of the
upgrade was that the project could bump the version of the tokio dependency to
the latest major version >1.0.

New features were reliant on radicle-link's latest changes. Combining this with
the long life of this branch, some other changes are included in this patch. We
list these changes here as well.

Profile support was added. This means that a person can have multiple
subdirectories under their XDG_* directories. These are keyed by a UUID and
allows the person to switch to different profiles manually.

The first step in Ethereum attestations was added. This allows a radicle-link Person
document to be extended with an Eth address that was claimed on the blockchain.

Signed-off-by: Alexander Simmerl <a.simmerl@gmail.com>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Signed-off-by: Igor Zuk <igor.zuk@protonmail.com>
Signed-off-by: Merle Breitkreuz <merle@monadic.xyz>
Signed-off-by: Rūdolfs Ošiņš <rudolfs@osins.org>
Signed-off-by: Thomas Scholtes <thomas@monadic.xyz>
@FintanH FintanH merged commit f0081b3 into master Mar 26, 2021
Thank You Next automation moved this from Next to ... Mar 26, 2021
@FintanH FintanH deleted the xla/1236-thank-you-next branch March 26, 2021 18:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature Something that doesn't exist yet
Projects
No open projects
8 participants