Source file pre-processing

[ericstj]

I've found myself numerous times needing to do some simple source preprocessing.

I'll often resort to readlinesfromfile|File.readalltext/writelinestofile with a few replacements. This type of hack is an abomination. We should have a general purpose text pre-processing task at the least. Ideally we do that and define some item/target semantics for doing something with the preprocessed item.

/cc @weshaggard

[alexperovich]

Are you thinking something like a simple key/value pair string replace of tokens like __<key>__ in the file? Or something more complete like a handlebars/mustache template?

[ericstj]

Simple key/value replacement is min bar but we could go bigger (like T4).

It needs to be simple to use, and ideally make sense when in VS as well for both template editing and output viewing.

It needs to be dynamic at build.

It needs to be batchable.

I think if you search for writelinestofile in a few of our repos you can come up with some scenarios.

[bricelam]

+1 We use this in EF to inject the PowerShell module version:

@{
    ModuleVersion = '%version%'

    # ...
}

[bricelam]

Example workaround:

<WriteLinesToFile
    File="$(TargetDir)EntityFrameworkCore.psd1"
    Lines="$([System.IO.File]::ReadAllText('EntityFrameworkCore.psd1.in').Replace('%version%', $(AssemblyVersion)))" />

[ericstj]

Yeah we have the same sort of hacks in CoreFx / core-setup and that's why I opened this issue.

[natemcmaster]

I generally just use WriteLinesToFile when I can, but have found that it is problematic when the file content contains \, /, or ;.

We wrote a simple task for pre-processing files in ASP.NET Core. I'd be happy to contribute it.

[natemcmaster]

See also dotnet/msbuild#4018

[dougbu]

Let's make this a specific ask 😺: Copy GenerateFileFromTemplate, FileHelpers and MSBuildListSplitter into https://github.com/dotnet/arcade/tree/main/src/Microsoft.DotNet.Arcade.Sdk/src and a bit of Internal.AspNetCore.BuiltTasks.props into https://github.com/dotnet/arcade/tree/main/src/Microsoft.DotNet.Arcade.Sdk/tools. Of course, rename as appropriate and skip FileHelpers and / or MSBuildListSplitter if equivalent helpers already exist in Arcade.

Ideal would be to do this in 'main', 'release/3.x', and 'release/5.0'.

I may be able to help w/ but time is tight at the moment. Will wait for triage in any case.

[dougbu]

See also #135 and dotnet/msbuild#4018

[dougbu]

@alexperovich @markwilkie we need a decision here urgently because using the task from aspnet/BuildTools is incompatible w/ Arcade-powered source build (see dotnet/aspnetcore#32492). If GenerateFileFromTemplate won't move to dotnet/arcade in the next week or so, we must stash it in dotnet/aspnetcore.

/cc @dotnet/aspnet-build @dseefeld

[alexperovich]

I don't see an issue with adding this to arcade. I think it shouldn't be part of arcade.sdk directly though, I would be more in favor of a separate package that enables template functionality with Template items or something.

[markwilkie]

Yea, let's do it! (and Alex is correct, it shouldn't be part of the Arcade SDK)

[dougbu]

The MSBuild Task is currently wrapped in its own package (Internal.AspNetCore.BuildTasks) though the only Task it contains is GenerateFileFromTemplate. I suggest we pick a name like Microsoft.DotNet.Build.Tasks.Templating or add it to an existing package. Preferences❔

Any objection to postponing new features such as using item groups for later❔

In any case, @JunTaoLuo let me know if you start working on this before I get to it. Unless of course @markwilkie volunteers someone first😀

[markwilkie]

No preferences on package naming from me.... (and sorry @dougbu, we're full up right now)

[JunTaoLuo]

Reopening, @dougbu we want this in 3.1 and 5.0 as well right?

[dougbu]

@dougbu we want this in 3.1 and 5.0 as well right?

It would be amazing to remove our last aspnet/BuildTools dependency in those branches😀 But, back-porting is up to @markwilkie

[markwilkie]

Seems like a good thing to do to backport. Thoughts @mmitche ?

[mmitche]

Fine by me if it helps get rid of some legacy infra.

[akoeplinger]

Do we still care about backporting this to 5.0/3.1?

[mmitche]

No