[vcpkg registries] Registries: Take 2 #13590

strega-nil · 2020-09-17T17:15:17Z

This RFC is the new state of the world with regard to our plan for registries; it is very different from the old RFC. It includes registry databases, which are how one describes the set of versions of ports that one has access to. It also includes the specification for git registries, and updates the specification for filesystem registries with these new registry databases.

Since this RFC completely changes how registries work, the existing implementation will be broken.

Rendered

BillyONeal

Thanks for your great work!

docs/specifications/git-registries.md

docs/specifications/registries-2.md

klalumiere · 2020-10-16T22:54:30Z

The RFC is super interesting! 🤓

Something it doesn't talk about is triplets. Will it be possible to store them in the registry alongside the ports?

A little more details about the context/motivations of my question might help. Right now, in our prototype integration of vcpkg, we do something like vcpkg install mypackage:x64-linux-dynamic --overlay-ports ../vcpkg-overlays/ports --overlay-triplets ../vcpkg-overlays/triplets. When available, registries and vcpkg-configuration.json will "manage" the --overlay-ports ../vcpkg-overlays/ports for us. Actually it will do much more: pulling the repository for us, locking the SHA, etc. Will we be able to get rid of the --overlay-triplets ../vcpkg-overlays/tripplets part too?

strega-nil · 2020-10-18T21:54:55Z

@klalumiere we may consider adding triplet registries at some point, but the plan at this point is to just add overlay triplets to the configuration file.

vicroms · 2020-10-20T00:24:47Z

docs/specifications/registries-2.md

+[
+  {
+    "version-string": "1.12.7",
+    "git-tree": "466e96fd2e17dd2453aa31dc0bc61bdcf53e7f61",
+  },
+  {
+    "version-string": "1.12.1",
+    "port-version": 1,
+    "git-tree": "0de81b4f7e0ec24966e929c2ea64e16c15e71d5e",
+  },
+  ...
+]


Database generator implemented according to spec in PR #13777

strega-nil · 2020-10-30T21:32:30Z

I'm calling for an FCP. Please post all comments this next week; assuming no new blocking issues, this specification will be merged at 2020-11-06T17:00-0800 (2020-11-07T01:00Z) (i.e., at 5 pm next friday redmond time)

docs/specifications/registries-2.md

strega-nil · 2020-11-16T23:07:26Z

I agree with @Neumann-A's assessment that the registry-path: "/foo/bar" is a bit of unfortunate magic. However, after discussing this with the rest of the team, we think that there are two real options for fixing this:

We could use @Neumann-A's suggestion outright; this is pretty simple, but it would require figuring out what we should do with paths that are "weird", for example, "${base}foo, ${base}/../foo, or paths that aren't rooted at ${base}. As a part of this, base might also not be the best option.
@BillyONeal suggested "$ as root", i.e., $/foo, and requiring paths to be rooted there. This fixes all of the issues with "weird paths" from 1, since we'd just disallow all of those kinds of paths.

I'd like to take input from the community. If you think that option 1 is better, please respond to this message with 🚀 ; if you think that option 2 is better, please respond to this message with 🎉 . If you have more complex thoughts, please respond in a reply :)

klalumiere · 2020-11-17T23:17:31Z

docs/specifications/registries-2.md

+We should first copy the existing implementation to a new folder:
+
+```sh
+$ cp -r ports/a-/asmjit/2020-07-22 ports/a-/asmjit/2020-10-08


Do you think it would be a good idea to add an option to the vcpkg add-port command so that this part is done "automatically" for us? Something like

vcpkg add-port ./ports/a-/asmjit/2020-10-08 --base ports/a-/asmjit/2020-07-22 --port-type registry-path

Make the edits required to ports/a-/asmjit/2020-10-08 to update to latest.

This would remove a manual manipulation, and it prevents repeating ports/a-/asmjit/2020-10-08 twice, removing at the same time an opportunity to make a typo. 🙂

unfortunately without all the history because I really screwed up the git commands This RFC is the new state of the world with regard to our plan for registries; it is very different from the old RFC. It includes registry databases, which are how one describes the set of versions of ports that one has access to. It also includes the specification for git registries, and updates the specification for filesystem registries with these new registry databases. Since this RFC completely changes how registries work, the existing implementation will be broken.

vicroms · 2020-12-11T08:57:51Z

docs/specifications/registries-2.md

+    "abseil": { "version-string": "2020-03-03" },
+    "zlib": { "version-string": "1.2.11", "port-version": 9 }


I think we should make this a generic version field that does not imply any kind of scheme.

I would suggest "version" but it is kind of unfortunate that it overlaps with the "Relaxed" scheme.

"abseil": { "version": "2020-03-03" }, "zlib": { "version": "1.2.11", "port-version": 9 }

Another suggestion would be "version-text"

"abseil": { "version-text": "2020-03-03" } "zlib": { "version-text": "1.2.11", "port-version": 9 }

Or we could change the spec for versioning so that "version" does not imply relaxed and instead make it explicit "version-relaxed".

vicroms · 2020-12-11T08:59:20Z

docs/specifications/registries-2.md

+    "version-string": "1.12.7",
+    "git-tree": "466e96fd2e17dd2453aa31dc0bc61bdcf53e7f61",


Same as with baselines use a generic version field.

{ "version": "1.12.7", "git-tree": "466e96fd2e17dd2453aa31dc0bc61bdcf53e7f61" }

This definitely shouldn't use a generic field; there's no other place to grab the version scheme from.

This PR merges the Registries changes and the versioning changes, so that one can use both at the same time. There is one major difference between this PR and the RFC (microsoft#13590), which is that instead of version files looking like: ```json [ ... ] ``` version files look like: ``` { "versions": [ ... ] } ``` this is to support interop between this PR and existing demos and the like; fixing this, along with perhaps renaming `port_versions` to `port-versions` should be done after this is merged, should be a trivial change.

* [vcpkg registries] support versions This PR merges the Registries changes and the versioning changes, so that one can use both at the same time. There is one major difference between this PR and the RFC (#13590), which is that instead of version files looking like: ```json [ ... ] ``` version files look like: ``` { "versions": [ ... ] } ``` this is to support interop between this PR and existing demos and the like; fixing this, along with perhaps renaming `port_versions` to `port-versions` should be done after this is merged, should be a trivial change.

* [vcpkg registries] support versions This PR merges the Registries changes and the versioning changes, so that one can use both at the same time. There is one major difference between this PR and the RFC (microsoft#13590), which is that instead of version files looking like: ```json [ ... ] ``` version files look like: ``` { "versions": [ ... ] } ``` this is to support interop between this PR and existing demos and the like; fixing this, along with perhaps renaming `port_versions` to `port-versions` should be done after this is merged, should be a trivial change.

BillyONeal · 2021-02-03T19:14:43Z

@strega-nil Given that this feature has now landed, shouldn't we merge this RFC?

strega-nil · 2021-02-03T22:44:40Z

@BillyONeal yes.

* [vcpkg registries] support versions This PR merges the Registries changes and the versioning changes, so that one can use both at the same time. There is one major difference between this PR and the RFC (microsoft#13590), which is that instead of version files looking like: ```json [ ... ] ``` version files look like: ``` { "versions": [ ... ] } ``` this is to support interop between this PR and existing demos and the like; fixing this, along with perhaps renaming `port_versions` to `port-versions` should be done after this is merged, should be a trivial change.

BillyONeal approved these changes Sep 17, 2020

View reviewed changes

docs/specifications/git-registries.md Outdated Show resolved Hide resolved

ras0219-msft reviewed Sep 17, 2020

View reviewed changes

JackBoosY assigned PhoebeHui Sep 18, 2020

JackBoosY added category:vcpkg-feature The issue is a new capability of the tool that doesn’t already exist and we haven’t committed info:internal This PR or Issue was filed by the vcpkg team. labels Sep 18, 2020

ras0219 mentioned this pull request Sep 29, 2020

How do I host a custom vcpkg feed #13789

Closed

strega-nil changed the title ~~[vcpkg registries] RFC for Git Registries~~ [vcpkg registries] Registries: Take 2 Oct 9, 2020

vicroms reviewed Oct 9, 2020

View reviewed changes

docs/specifications/registries-2.md Outdated Show resolved Hide resolved

vicroms reviewed Oct 9, 2020

View reviewed changes

docs/specifications/registries-2.md Outdated Show resolved Hide resolved

BillyONeal approved these changes Oct 12, 2020

View reviewed changes

vicroms reviewed Oct 20, 2020

View reviewed changes

vicroms approved these changes Oct 21, 2020

View reviewed changes

strega-nil mentioned this pull request Oct 21, 2020

[vcpkg] Initial Registries: Part 2 MVP #14153

Merged

BillyONeal approved these changes Oct 30, 2020

View reviewed changes

Neumann-A reviewed Oct 30, 2020

View reviewed changes

docs/specifications/registries-2.md Outdated Show resolved Hide resolved

strega-nil mentioned this pull request Nov 4, 2020

[vcpkg] RFC: Versioning #11758

Merged

strega-nil added the requires:discussion label Nov 12, 2020

klalumiere reviewed Nov 17, 2020

View reviewed changes

vicroms mentioned this pull request Dec 4, 2020

Is it possible to build a port against a different fork/branch/source-tweaks? #14939

Closed

BillyONeal removed the requires:discussion label Dec 10, 2020

strega-nil force-pushed the git-registries branch from 2a416ae to 0095a9b Compare December 10, 2020 23:52

strega-nil mentioned this pull request Dec 11, 2020

[vcpkg registries] Add git registries #15054

Merged

vicroms reviewed Dec 11, 2020

View reviewed changes

strega-nil mentioned this pull request Dec 14, 2020

[vcpkg registries] support versions #15114

Merged

Jimmy-Hu mentioned this pull request Dec 22, 2020

[vcpkg registries] support versions (#15114) Jimmy-Hu/vcpkg#452

Merged

strega-nil added 2 commits January 19, 2021 13:32

update the registries RFC for modern design

0cfedfb

remove bikeshedding

0e72fa9

strega-nil merged commit f226416 into microsoft:master Feb 3, 2021

strega-nil deleted the git-registries branch February 11, 2021 21:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vcpkg registries] Registries: Take 2 #13590

[vcpkg registries] Registries: Take 2 #13590

strega-nil commented Sep 17, 2020 •

edited

BillyONeal left a comment

klalumiere commented Oct 16, 2020

strega-nil commented Oct 18, 2020

vicroms Oct 20, 2020

strega-nil commented Oct 30, 2020 •

edited

strega-nil commented Nov 16, 2020

klalumiere Nov 17, 2020

vicroms Dec 11, 2020

vicroms Dec 11, 2020

strega-nil Dec 11, 2020

BillyONeal commented Feb 3, 2021

strega-nil commented Feb 3, 2021

		"abseil": { "version-string": "2020-03-03" },
		"zlib": { "version-string": "1.2.11", "port-version": 9 }

		"version-string": "1.12.7",
		"git-tree": "466e96fd2e17dd2453aa31dc0bc61bdcf53e7f61",

[vcpkg registries] Registries: Take 2 #13590

[vcpkg registries] Registries: Take 2 #13590

Conversation

strega-nil commented Sep 17, 2020 • edited

BillyONeal left a comment

Choose a reason for hiding this comment

klalumiere commented Oct 16, 2020

strega-nil commented Oct 18, 2020

vicroms Oct 20, 2020

Choose a reason for hiding this comment

strega-nil commented Oct 30, 2020 • edited

strega-nil commented Nov 16, 2020

klalumiere Nov 17, 2020

Choose a reason for hiding this comment

vicroms Dec 11, 2020

Choose a reason for hiding this comment

vicroms Dec 11, 2020

Choose a reason for hiding this comment

strega-nil Dec 11, 2020

Choose a reason for hiding this comment

BillyONeal commented Feb 3, 2021

strega-nil commented Feb 3, 2021

strega-nil commented Sep 17, 2020 •

edited

strega-nil commented Oct 30, 2020 •

edited