-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[darc/maestro++] Darc should be buildable from source #2578
Comments
I agree. Does darc even need to be buildable from source? What darc functionality would be used in the source-build scenario? |
source-build will use darc for dotnet/arcade#427 "Create a product source build layout from a repo sha" but even if that's done "before" the build from source, I'm pretty sure we'll be using darc to flow dependencies between repos. |
@dagood I know you mentioned that you'll try building Arcade from source in the following two weeks, I assume this includes darc. Would you consider any potential issues in the process to be preview (12/14) blockers? |
Doing a "proper" build from source (no prebuilts) isn't a blocker for 3.0 preview, so no. That said, the sooner we can build Darc from source rather than downloading it, the better, to avoid building up debt that ends up delaying the delivery of a fully source-built .NET Core. (And yeah, Darc is part of the required toolset, so it's included.) When I try building Arcade from source I'm planning to file issues to get this on the radar. |
With Darc in particular, I don't see any source-buildability issues yet beyond the dependency stuff I already mentioned in the issue description, so I just added some more info and more detail on the design suggestion. |
@dagood - do you feel this requirement is fulfilled yet? If not, what do you see is missing? |
AFAIK none of the work discussed in this thread has been done. This was discussed in a email thread ~11/29 "Source-buildability of Darc's local capabilities" with @ChadNedzlek, @mmitche, and @jcagme, so the stuff to do is still known. |
@mmitche - what's needed for Preview II? |
From the source-build perspective, this is not needed for preview 2. We'll need it to be able to deliver source-build on new distros without prebuilts, which I don't think is tied to any specific 3.0.0 milestones. |
Is "darc" necessary to do anything more than fetching the correct commits/source blobs? Presumably that could be run on something other than the target computer (since it's not actually "the build"... it's just getting the source code), then that thing that's created would be either shipped off to distros like RedHat or built on the target computer... but at that point nothing in darc would be necessary. If there is no prebuilt stuff, darc won't be able to run anyway, so it doesn't help in that scenario either (since it requires a full functioning dotnet core to run). Darc is a pretty intense mess of dependencies and pulling it apart is going to be painful and awkward and difficult to maintain. I'm hoping we can find a more sustainable solution that doesn't drag the web service into some dependency cycle. |
Yes, when I mentioned "I'm pretty sure we'll be using darc to flow dependencies between repos." I'm referring to this:
Currently we use the Repo API (specifically Package Version Props) to do this, and from the above it sounds like using Package Version Props won't be supported. (And we might still need Darc for a standard way to flow zips through?)
That's correct, to me the reliance on Microsoft-prebuilt bits to do the trivial-seeming thing of gathering the source makes community maintenance less sustainable, but I don't have justification for this put together right now.
This isn't the case: the N-1 SDK and Runtime aren't prebuilts. When there is no N-1, first-time bootstrapping has specific exceptions, but we can only use stuff we can build, not e.g. Octokit and Azure packages. If we can bring those into source-build, sweet, but that seems like a lot more maintenance cost than cutting them out of the build. If you're interested, EngineeringPlanLinuxDistroRepos.md should clarify the requirements and N-1/bootstrap process. (It looks like it hasn't been updated for the switch from
From what I've seen so far, I think the concerns are a bit overstated, but I think it's reasonable to wait for me to give it a try or something. Keep in mind that we aren't asking for this for preview2 or RTM. |
@dagood I don't think this is required for preview 2, right? |
Correct, not needed for preview2. |
@dagood is this required for .net 5 rtm? |
Nope. (I think we (source-build) should actually consider extracting the subset of |
Thanks @dagood. The idea seems interesting. Let me know what you decide to see if we can close this or if we should re-examine for future releases. |
I noticed this when reading this thread back and I think I should give an update. It turns out that this didn't happen: we kept on using Package Version Props, and that infra has been maintained. We only use
I think we can close it when we no longer depend on the prebuilt |
I noticed issues about using Octokit #2580 and VSTS C# libs #2579 in Darc. This sounds like a great idea, but I want to make sure it's implemented in a way that leaves Darc able to build from source.
If Octokit and VSTS APIs can build from source, ok, but it would be preferable not to build them at all. The builds take time, and the capabilities won't be used in source-build.
/cc @alexperovich @jcagme
I haven't looked into every one, but these are unknowns when it comes to source-build:
An easy way I see to do this is to split code that needs to use these dependencies into separate csproj's, condition the dependency, and condition some small part of the code to link in (or not) web client capability. (
Microsoft.DotNet.Maestro.Client
is an example of something that's already split off!) The propertyExcludeFromSourceBuild
is used to skip a project during source-build.I can imagine something fancier being done, where there's a real extensibility point on Darc. The user could choose to download some extra packages (or something) to enable web client capabilities.
The text was updated successfully, but these errors were encountered: