Building from source

[dpavicic]

Hey Tom,
Here's the report about building from the source I promised. I'll write down the steps and issues I encountered when cloning a repo:

Report

1. Opened ModularPipelines.Examples.sln
2. All projects in the solution have missing dependencies
   

• ModularPipelines project has missing dependencies to ModularPipelines.Analyzers.Package
• ModularPipelines.Examples project has missing dependencies to: ModularPipelines.DotNet, ModularPipelines.Git, ModularPipelines.Slack

3. When I add all missing dependencies, now ModularPipeline.Analyzers.Package reports missing dependencies to ModularPipelines.Analyzers and ModularPipelines.Analyzers.CodeFixes
4. When I add the rest of the missing dependencies, it now reports 106 errors (Elements should be documented, Enumeration items should be documented, etc.). The stylecop is set too hard [report warnings as errors] and reports errors for missing XML documentation comments, but there are not many doc comments so it produces a ton of them. These settings should be toned down as Rider's Solution-Wide Analysis is always in red and working non-stop.
5. When I comment on the addition of FailedModule (I thought the build failed because of that module), it seems that the build crashes with timeout exceptions on all added modules.
6. When I comment out all modules and leave just 'SuccessModule', it finally builds and passes:

collection.AddModule<SuccessModule>();

Observations

• Maybe to avoid setting 'FailedModule' in the example. Somebody could make the wrong conclusion that the project is not working as expected.. but it works as expected.
• Example to be easy to load and run. It seems, now it's broken.
• Tone down stylecop since it's quite rigorous. Except for missing xml doc comments, it throws concrete errors preventing building for small stuff like:
  -- Program.cs(36, 9): [SA1515] Single-line comment should be preceded by blank line (https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/master/documentation/SA1515.md)
  -- Program.cs(36, 9): [SA1005] Single line comment should begin with a space (https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/master/documentation/SA1005.md)

if I just comment out one module as in this example:

[thomhurst]

I've tidied up the examples project now. I hope it builds better for you!

[dpavicic]

Thank Tom!
I tried, and everything works ok. There are no missing dependencies and builds with .NET 6, 7, and 8.

The only thing I noticed that's weird in the Example app (It is not related to dependencies and stylecop errors but to the example itself), is that all modules timeout except SuccessModule.
To have a successful build and passable pipeline without throwing exceptions, you have to remove all modules except SuccessModule:
collection.AddModule<SuccessModule>();

I didn't investigate why. Anyways, it opens and builds without errors now. Thanks, again.

[thomhurst]

Does the results table show which one failed first? I imagine it's throwing for some reason and then cancelling the pipeline

[dpavicic]

It doesn't get to the results table as it crashes before actually starting the modules in the pipeline. It lists the module dependency chain and, after a few seconds, goes kaboom.

It throws the timeout exception on a module, for example:

ModularPipelines.Exceptions.ModuleTimeoutException: LogSecretModule has timed out after 30m & 0s
   at ModularPipelines.Modules.Module`1.<>c__DisplayClass39_0.<ExecuteInternal>b__2(Task t) in C:\Users\User\Work\repositories\ModularPipelines\src\ModularPipelines\Modules\Module.cs:line 338
   at System.Threading.Tasks.ContinuationTaskFromTask.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)

It crashes on LogSecretModule, SubmodulesModule, and GitLastCommitModule modules. With the rest of them, it works fine, and the pipeline finishes as it should.

Also, it reports it timed out after 30 minutes, but it timed out after a few seconds (right after GitVersion calculates the version, I guess [as it takes some time to do it]).

Also, I noticed that GitVersion in ModularPipelines is somewhat slower than in Nuke. I've read their recent roadmap about speeding it up (as it gets really slow with a large number of commits). I thought Modular Pipelines was using some older version. but now I'm looking at the GitVersioning class and see that it just downloads the latest CLI tool (GitVersion.Tool), so it is something else or my imagination.

[thomhurst]

Hmm... That timeout issue is odd. Leave that with me and I'll see if I can dig into what's causing that.
Unfortunately it doesn't do that on my machine, so might be hard to debug!

If you've got any ideas on speeding up GitVersion, then I'm open to suggestions and/or PRs 😄

[dpavicic]

Heh, not really, as it is up to the guys from GitVersion. For local dev, maybe cache the tool instead of downloading it on every build.
I have to rewrite the whole Nuke codebase I have to Modular Pipelines and make the GitHub workflows generation part also. I'll try to deal with tool caching after that.