Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Post-Processing feature. Related issue #144 #148

Merged
merged 14 commits into from May 7, 2020
Merged

Adds Post-Processing feature. Related issue #144 #148

merged 14 commits into from May 7, 2020

Conversation

3F
Copy link
Owner

@3F 3F commented Apr 28, 2020

Closes #144
Continues direction of PR #146

Explanation and details:

This also updates MvsSln (core of this PR) to 2.5.2 since current PR was based on features that were updated in 2.5.2:

@3F 3F added this to the 1.7.1 milestone Apr 28, 2020
@3F 3F marked this pull request as draft April 28, 2020 13:21
…ironment:

Format: $(SolutionPath);$(MSBuildThisFileFullPath);...populated property names...;....

```
<Target Name="DllExportPostProc">

    <!-- Now we can access the following properties and items:

    $(DllExport)     - version
    $(DllExportSln)  - full path to .sln which controls current project
    $(DllExportPrj)  - full path to current project where processed .NET DllExport

    @(DllExportDirX64)    - $(TargetDir)x64\*.*
    @(DllExportDirX86)    - $(TargetDir)x86\*.*
    @(DllExportDirBefore) - $(TargetDir)Before\*.*
    @(DllExportDirAfter)  - $(TargetDir)After\*.*

    @(DllExportDependents + populated property name)
       - each populated properties from DllExportInvokedPoint, e.g. DllExportDependentsTargetDir

    @(DllExportDependencies + populated property name)
      - each populated properties from DllExportInvokedPoint, e.g. DllExportDependenciesTargetDir

    -->

</Target>
```
@3F
Copy link
Owner Author

3F commented Apr 30, 2020

I implemented backend logic with configurable environment. ANY properties can be populated like for parent projects (depend on) and/or current dependencies where processed.

Currently no GUI but it's planned. Later.

How to play with this today:

  1. We need to define DllExportProcEnv property:

Format: $(SolutionPath);$(MSBuildThisFileFullPath);...populated property names...;....

For example:

<PropertyGroup>
  <DllExportProcEnv>
    $(SolutionPath);$(MSBuildThisFileFullPath);
    TargetDir;
    AssemblyName;
    TargetPath;
    ...
  </DllExportProcEnv>
</PropertyGroup>
  1. We need callable target, named as DllExportPostProc:
<Target Name="DllExportPostProc">

    <!-- Now we can access the following properties and items:
    
    $(DllExport)     - version
    $(DllExportSln)  - full path to .sln which controls current project
    $(DllExportPrj)  - full path to current project where processed .NET DllExport
    
    @(DllExportDirX64)    - $(TargetDir)x64\*.*
    @(DllExportDirX86)    - $(TargetDir)x86\*.*
    @(DllExportDirBefore) - $(TargetDir)Before\*.*
    @(DllExportDirAfter)  - $(TargetDir)After\*.*
    
    @(DllExportDependents + populated property name) 
       - each populated properties from DllExportProcEnv, 
          e.g. DllExportDependentsTargetDir
       
    @(DllExportDependencies + populated property name) 
      - each populated properties from DllExportProcEnv, 
          e.g. DllExportDependenciesTargetDir

    @(DllExportSeqDependents + populated property name) 
       - each populated properties from DllExportProcEnv, 
          e.g. DllExportSeqDependentsTargetDir
      
    -->

</Target>
  1. Enjoy.

@NickAcPT

Can you confirm this solution for your case? For example, for the case with #144 we need like this:

<Target Name="DllExportPostProc">

  <Copy SourceFiles="@(DllExportDirX64)" OverwriteReadOnlyFiles="true"
      DestinationFolder="@(DllExportDependentsTargetDir->'%(Identity)x64\')" />

  <Copy SourceFiles="@(DllExportDirX86)" OverwriteReadOnlyFiles="true"
      DestinationFolder="@(DllExportDependentsTargetDir->'%(Identity)x86\')" />

</Target>

Let me know if any bugs.


update: DllExportInvokedPoint renamed as DllExportProcEnv [?]

update2: new DllExportSeqDependents... items [?]

update3: future updates will be on the new wiki page: https://github.com/3F/DllExport/wiki/PostProc#post-processing

@3F 3F marked this pull request as ready for review April 30, 2020 21:12
@NickAcPT
Copy link

@3F Thanks for your work so far.

I've been having some trouble reproducing your results on my side, but it appears to be because I have something misconfigured.

I'm a bit unsure on which project I need to put the DllExportPostProc target into, and what is the best configuration for the DllExportInvokedPoint property.

I'd appreciate for you to explain a bit more on your answer.

Here's my project structure:

  • ProjectA (.NET Framework project; uses DllExport; x86+x64)
  • ProjectB (.NET Core 3.1 project; depends on ProjectA)

@3F
Copy link
Owner Author

3F commented Apr 30, 2020

@NickAcPT

Any modification should be only where DllExport is installed.

That is, for your case:

  1. DllExportPostProc should be placed only for ProjectA (where DllExport)
  2. After, MvsSln will calculate required data (that was mentioned in Question regarding depending on the DllExported dlls on another project #144) and populate it inside DllExportPostProc target.
  3. Now DllExportPostProc can be used for many cases including yours. Bonus: Both directions.

and what is the best configuration for the DllExportInvokedPoint property.

Minimal for the case when @(DllExportDependentsTargetDir->'%(Identity)x64\') is $(SolutionPath);$(MSBuildThisFileFullPath);TargetDir

@3F
Copy link
Owner Author

3F commented May 2, 2020

Please note: DllExportInvokedPoint renamed as DllExportProcEnv. I also updated my example above

@3F
Copy link
Owner Author

3F commented May 2, 2020

@NickAcPT I added basic GUI support for predefined options, please check this out. It covers at least your case.

That is, today some Post-Proc features are not yet available in GUI. Only manual configuring with msbuild as I said before.

Since this PR is mainly for adding an Post-processing support, full GUI will be considered through a new one, and seems like not for 1.7.1. Of course anybody can open PR to get this ASAP. Welcome.

@NickAcPT
Copy link

NickAcPT commented May 3, 2020

@3F Sorry for the huge delay. Haven't had much time to mess with this for a while.

Thanks a lot for the work you've done. I've ran across a problem when using Fody.Costura (haven't tried yet with your ILMerge post processing step).

Exception
DllExportMod:
  Found method: WinTabby.Hooks.ManagedHook..method public hidebysig static native int 'CaptionPaintBlockHandler'(int32 'nCode', native int 'wParam', native int 'lParam') cil managed
  	exporting as CaptionPaintBlockHandler and index 1
D:\Visual Studio - Projects\WinTabby\packages\DllExport.1.7.0\tools\net.r_eg.DllExport.targets(76,5): error : The given key was not present in the dictionary. [D:\Visual Studio - Projects\WinTabby\WinTabby.Hooks\WinTabby.Hooks.csproj]
     at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
     at net.r_eg.MvsSln.Core.ProjectReferences.<>c__DisplayClass15_0.<InitMap>b__0(Item i) in C:\projects\dllexport-ix27o\MvsSln\MvsSln\Core\ProjectReferences.cs:line 114
     at System.Collections.Generic.List`1.ForEach(Action`1 action)
     at net.r_eg.MvsSln.Core.ProjectReferences.InitMap() in C:\projects\dllexport-ix27o\MvsSln\MvsSln\Core\ProjectReferences.cs:line 114
     at net.r_eg.MvsSln.Core.ProjectReferences..ctor(ISlnPDManager slndep, IEnumerable`1 xprojects) in C:\projects\dllexport-ix27o\MvsSln\MvsSln\Core\ProjectReferences.cs:line 90
     at net.r_eg.MvsSln.Core.SlnParser.Parse(StreamReader reader, SlnItems type) in C:\projects\dllexport-ix27o\MvsSln\MvsSln\Core\SlnParser.cs:line 159
     at net.r_eg.MvsSln.Core.SlnParser.Parse(String sln, SlnItems type) in C:\projects\dllexport-ix27o\MvsSln\MvsSln\Core\SlnParser.cs:line 106
     at net.r_eg.MvsSln.Sln..ctor(String file, SlnItems type) in C:\projects\dllexport-ix27o\MvsSln\MvsSln\Sln.cs:line 55
     at RGiesecke.DllExport.MSBuild.PostProc..ctor(String raw, TaskLoggingHelper log) in C:\projects\dllexport-ix27o\RGiesecke.DllExport.MSBuild\PostProc.cs:line 82
     at RGiesecke.DllExport.MSBuild.ExportTaskImplementation`1.ExecutePostProc(Boolean ret, TaskLoggingHelper log) in C:\projects\dllexport-ix27o\RGiesecke.DllExport.MSBuild\RGiesecke.DllExport.MSBuild\ExportTaskImplementation.cs:line 508
     at RGiesecke.DllExport.MSBuild.ExportTaskImplementation`1.Execute() in C:\projects\dllexport-ix27o\RGiesecke.DllExport.MSBuild\RGiesecke.DllExport.MSBuild\ExportTaskImplementation.cs:line 482

I'll update this comment if I find anything else.

@3F
Copy link
Owner Author

3F commented May 3, 2020

@NickAcPT Thank you for the important information!

What build or commit have you tried? Is this a new error after #148 (comment) ?

Can you please use /v:diag to show more details after DllExportMod: Found method: lines?
Or better, can you attach WinTabby.Hooks.csproj file?

@3F
Copy link
Owner Author

3F commented May 3, 2020

Actually I'm watching ProjectReferences problem in your stack trace, ie. more like MvsSln part. I need some details from your .csproj related to ProjectReferences for reproducing on my laptop.

Or it could be a problem with new flags in MvsSln: SlnItems.ProjectDependenciesXml | SlnItems.LoadMinimalDefaultData

Because I tested only this:

@NickAcPT
Copy link

NickAcPT commented May 3, 2020

@3F

I need some details from your .csproj related to ProjectReferences for reproducing on my laptop.

Sure, which details do you need?

@3F
Copy link
Owner Author

3F commented May 3, 2020

@NickAcPT

I need the following:

  1. Full definitions of the all ProjectReference from WinTabby.Hooks.csproj, such as:
    <ProjectReference Include="ClassLibrary2.csproj">
      <Project>{64ad76ca-2c85-4039-b0b3-734cf02b2999}</Project>
      <Name>ClassLibrary2</Name>
    </ProjectReference>
  1. I need definitions of the all ProjectGuid properties (if used) from in WinTabby.Hooks.csproj, such as:
<ProjectGuid>{6CE57BB1-4A6D-4714-B775-74A3637EC992}</ProjectGuid>
  1. I need Project sections from your .sln, such as:
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ClassLibrary2", "ClassLibrary2.csproj", "{64AD76CA-2C85-4039-B0B3-734CF02B2999}"
EndProject
  1. Also ProjectSection(ProjectDependencies) from .sln (if used) should be inside Project sections above.

Thanks,

3F added a commit to 3F/MvsSln that referenced this pull request May 3, 2020
@3F
Copy link
Owner Author

3F commented May 3, 2020

@NickAcPT I found and fixed two related errors in MvsSln project. Please try with 1bd7675

@NickAcPT
Copy link

NickAcPT commented May 3, 2020

@NickAcPT I found and fixed two related errors in MvsSln project. Please try with 1bd7675

I've tried it and came across this error.
Here is the project file along with the solution file.

@3F
Copy link
Owner Author

3F commented May 3, 2020

Thanks for the reply,

  1. Just to be sure, can you show result for the command DllExport -build-info
  2. About ProjectReference my bad, I meant the dependent project. More like WinTabby.DaemonWPF\WinTabby.Daemon.csproj. Can you attach this? or at least its ProjectReference record.

@NickAcPT
Copy link

NickAcPT commented May 3, 2020

  • Just to be sure, can you show result for the command DllExport -build-info
    There we go:
    S_NUM_REV:          1.7.0.1056
    S_REL:
    bSha1:              a2a0c69
    bName:              master
    MetaCor:            netstandard1.1
    MetaLib:            v2.0
    Wizard:             v4.0
    Configuration:      PublicRelease
    Platform:           Any CPU
    cfgname:            Release
    revDeltaBase:       2016/10/12
    revDeltaMin:        1000
    revDeltaMax:        65534

    :: generated by a vsSolutionBuildEvent 1.14.0.36854

2. About ProjectReference my bad, I meant the dependent project. More like WinTabby.DaemonWPF\WinTabby.Daemon.csproj. Can you attach this? or at least its ProjectReference record.

Oh, no problem 😄
Here are the files:
WinTabby.Events.csproj depends on WinTabby.Hooks.
And then WinTabby.Daemon.csproj depends on WinTabby.Events.

@3F
Copy link
Owner Author

3F commented May 3, 2020

@NickAcPT Thank you for the detailed information. Yep, I didn't think first about obsolete projectguid in ProjectReference, but yes, MS avoids an obsolete projectguid in any modern places. My bad. And my issue btw: 3F/MvsSln#18

So, this is it, MvsSln expects this format:

<ProjectReference Include="..\WinTabby.Hooks\WinTabby.Hooks.csproj" Project="18816D42-416D-4D2B-83F5-1DF7353184AD" />

But the new is:

<ProjectReference Include="..\WinTabby.Hooks\WinTabby.Hooks.csproj" />

The error is controlled by 3F/MvsSln#26

@3F
Copy link
Owner Author

3F commented May 4, 2020

Added ProjectReference support without obsolete projectguid. Please confirm solution e638674

@NickAcPT
Copy link

NickAcPT commented May 5, 2020

Can confirm it builds now! Thanks a lot for all the work!

@NickAcPT
Copy link

NickAcPT commented May 5, 2020

I've noticed a problem that doesn't go away after cleaning and rebuilding when the solution is structured as this:

  • ProjectA (WinTabby.Hooks; .NET Framework 4.8; uses DllExport)
  • ProjectB (WinTabby.Events; .NET Core 3.1; references ProjectA)
  • ProjectC (WinTabby.Daemon; .NET Core 3.1; references ProjectA and ProjectB)

My end-goal with ProjectC referencing ProjectA is to have it so it also would copy the x64/x86 folders to the output of the ProjectC.

Edit: forgot to attach the build error.
out.txt

@NickAcPT
Copy link

NickAcPT commented May 5, 2020

I'm not sure if it is related to the PR, but on the Pre-processing tab there is a task to run ILMerge, even on .NET Core/Standard projects, but it only works on .NET Framework projects.
Would it be possible to make something that can work everywhere (without the need of changing the code of ILMerge)?

I've read here that there's a way to make .NET Core executables/dlls into one single file with some changes, but it requires publishing the project (dotnet publish).

@3F
Copy link
Owner Author

3F commented May 5, 2020

Well, I forgot about recursion for multiple destination in msbuild task :)

That is, instead of @(DllExportDependentsTargetDir->'%(Identity)x86\') we need to use, for example, like Outputs splitting etc.

Since DllExportPostProc is a common case for Post-processing feature and Outputs should not interfere with it, for the case like yours and so on, we'll use additional target like:

<Target AfterTargets="DllExportPostProc" Name="DllExportPostProcForDependent"
      Outputs="%(DllExportDependentsTargetDir.Identity)">

<Copy SourceFiles="@(DllExportDirX64)" OverwriteReadOnlyFiles="true"
        DestinationFolder="%(DllExportDependentsTargetDir.Identity)\x64\" />
        
        ...
</Target>

As you can see, any other custom targets can easily extend our DllExportPostProc

So, I'll update logic ASAP. You can also try it yourself to make sure before I start.

@3F
Copy link
Owner Author

3F commented May 5, 2020

I'm not sure if it is related to the PR, but on the Pre-processing tab there is a task to run ILMerge, even on .NET Core/Standard projects, but it only works on .NET Framework projects.

As I remember, I tested both types including .NET Core based projects. Or please clarify the case. Better to open a new issue if you found error related to the other planned Pre-processing feature.

3F added a commit to 3F/MvsSln that referenced this pull request May 5, 2020
* NEW: Activating ProjectReferences for existing ProjectDependencies (shallow copy) through new flag.
       Issue #25.
        ```
        ProjectDependenciesXml = 0x0800 | ProjectDependencies | Env,
        ```
            Covers ProjectDependencies (SLN) logic using data from project files (XML).
            Helps eliminate miscellaneous units between VS and msbuild world:
            #25 (comment)
            Requires Env with loaded projects (LoadMinimalDefaultData or LoadDefaultData).

            A core feature in .NET DllExport Post-processing:
            3F/DllExport#148

* NEW: `ProjectReference` support without obsolete projectguid. Issue #26.

* NEW: IXProject methods:
       ```
       +IXProject.GetFullPath(string relative)
       ```

* FIXED: Fixed possible empty records in SlnParser.SetProjectItemsConfigs.

* FIXED: Fixed `The given key was not present...` when different case for keys in Item.Metadata.

* FIXED: Fixed protected XProject GetProjectGuid() + GetProjectName() when empty property.

* CHANGED: Compatible signature update for `ForEach<T>` extension method:
           ```
           IEnumerable<T> ForEach<T>(this IEnumerable<T> items, Action<T> act)
           ```

* CHANGED: Updated Microsoft.CSharp 4.7.0 (Only for: netstandard2.0 + netcoreapp2.1)

* CHANGED: Updated Microsoft.Build 16.5.0 (Only for: netcoreapp2.1)
@3F
Copy link
Owner Author

3F commented May 6, 2020

I updated logic for generating derivative targets as I mentioned above.

@NickAcPT Try this 0b09916

@NickAcPT
Copy link

NickAcPT commented May 6, 2020

I tried it.

  • ProjectA (WinTabby.Hooks; .NET Framework 4.8; uses DllExport)
  • ProjectB (WinTabby.Events; .NET Core 3.1; references ProjectA)
  • ProjectC (WinTabby.Daemon; .NET Core 3.1; references ProjectA and ProjectB)

Was I supposed to see any changes on ProjectC? Don't think it created a task there.

Even if not. Tried it and on ProjectA, I see these two targets:

<Target Name="DllExportPostProc" Label="8337224c9ad9e356" />
<Target Name="DllExportPostProcForTargetDir" AfterTargets="DllExportPostProc" Label="8337224c9ad9e356" Outputs="%(DllExportDependentsTargetDir.Identity)">
<Copy SourceFiles="@(DllExportDirX86)" DestinationFolder="%(DllExportDependentsTargetDir.Identity)x86\" OverwriteReadOnlyFiles="true" />
<Copy SourceFiles="@(DllExportDirX64)" DestinationFolder="%(DllExportDependentsTargetDir.Identity)x64\" OverwriteReadOnlyFiles="true" />
</Target>

@3F
Copy link
Owner Author

3F commented May 6, 2020

Tried it and on ProjectA, I see these two targets:

It looks correct.

Any data should be configured only in ProjectA!

While ProjectB + ProjectC will magically receive x86+x64 folders from ProjectA.
Please attach /v:d log. For me it works well.

@NickAcPT
Copy link

NickAcPT commented May 6, 2020

There you go.
out.txt

@3F
Copy link
Owner Author

3F commented May 6, 2020

I get it, we're talking about different cases when you're talking about ProjectC ... references ProjectA and ProjectB

Because:

ProjectC -> ProjectA + ProjectB
ProjectB -> ProjectA

Is not the same to:

ProjectC -> ProjectB -> ProjectA

That's more complex way because it also can be as:

... ProjectE -> ProjectD -> ProjectC -> ProjectB -> ProjectA

And I have a lot of questions if this type. For example,

  • Do we need to copy to all places? ProjectE + ProjectD + ProjectC + ProjectB
  • What if only ProjectE + ProjectB ? or more specific, if for ProjectD we don't need to copy this due to custom environment, but for ProjectB we expect this?

Well, don't know...

@3F
Copy link
Owner Author

3F commented May 7, 2020

Okay, I also added DllExportSeqDependents... items:

    @(DllExportSeqDependents + populated property name) 
       - each populated properties from DllExportProcEnv, 
          e.g. DllExportSeqDependentsTargetDir

This includes sequential referencing through other projects. For GUI find [ ] ++ button (0x10 bit in DllExportPostProcType)

See fc5344f

However, for other specific controls and options you need to open new PR. I will not personally consider this at least here.

@NickAcPT
Copy link

NickAcPT commented May 7, 2020

Can confirm that it works.
Thank you so much for all the work you've done. 👍

@3F
Copy link
Owner Author

3F commented May 7, 2020

Good! Thanks for the confirmation.

@3F 3F changed the title WIP. Adds Post-Processing feature. Related issue #144 Adds Post-Processing feature. Related issue #144 May 7, 2020
@3F 3F merged commit 7d54260 into master May 7, 2020
@3F 3F deleted the dev/postproc branch May 7, 2020 16:01
3F added a commit that referenced this pull request May 7, 2020
* NEW: Pre-Processing feature. PR #146.
       Related issue #40

       Official ILMerge support;
       https://github.com/dotnet/ILMerge

       Quick integration with Conari for most easy access to unmanaged features;
       https://github.com/3F/Conari

       +Other related tools and assembly manipulations.
       Manual configuring: #40 (comment)

* NEW: Post-Processing. PR #148.
       Continues direction of Pre-Processing feature. PR #146
       Related issue #144

       Explanation and details:
       https://ko-fi.com/Blog/Post/ILMerge---Conari---Debug-information---DllExport-=-O5O61MV8A

       1.7.1 Provides only basic GUI support for predefined options. Thus,

        *! Some Post-Proc features are not yet available in GUI.
        But you can already configure it with msbuild:
        ```
        <Target Name="DllExportPostProc">

            <!-- After activation, you can access the following properties and items:

            $(DllExport)     - version
            $(DllExportSln)  - full path to .sln which controls current project
            $(DllExportPrj)  - full path to current project where processed .NET DllExport

            @(DllExportDirX64)    - $(TargetDir)x64\*.*
            @(DllExportDirX86)    - $(TargetDir)x86\*.*
            @(DllExportDirBefore) - $(TargetDir)Before\*.*
            @(DllExportDirAfter)  - $(TargetDir)After\*.*

            @(DllExportDependents + populated property name)
               - each populated properties from DllExportProcEnv,
                  e.g. DllExportDependentsTargetDir

            @(DllExportDependencies + populated property name)
              - each populated properties from DllExportProcEnv,
                  e.g. DllExportDependenciesTargetDir

            @(DllExportSeqDependents + populated property name)
               - each populated properties from DllExportProcEnv,
                  e.g. DllExportSeqDependentsTargetDir

            -->

        </Target>
        ```
        #148 (comment)

* NEW: Optional copying of intermediate files + x86+x64 directories into output
           for projects that dependent on projects where used DllExport. Issue #144.

           Including sequential referencing through other projects:
           #148 (comment)

* FIXED: Fixed #140 ... failed to create safe SSL/TLS context.

* FIXED: Pack of fixes for .net.dllexport.targets. PR #147.

        * Fixed "Cannot modify an evaluated object originating in an imported file".
        * Fixed possible duplication in .net.dllexport.targets when configuring.
        * Adds removing TargetsFile if not used.
        * Fixed possible loss of settings in .targets when configuring.

* FIXED: Fixed #143 'Microsoft.NET.Sdk' specified could not be found.

* FIXED: A multiple empty `<PropertyGroup />` in project files during new configuration.

* CHANGED: Wizard. Dropped support for ssl3 + tls1.0 + tls1.1

* CHANGED: Wizard. Simplified notification for stable versions.

* CHANGED: Manager. Access to hMSBuild tool (packed) via `-hMSBuild` key.
                    https://github.com/3F/hMSBuild
                    Since it uses packed version (while GetNuTool is integrated inside),
                    you need use -dxp-version to control specific version.

* CHANGED: Updated Cecil 0.11.2
           https://github.com/jbevain/cecil/releases/tag/0.11.2

* CHANGED: Updated MvsSln 2.5.2
           https://github.com/3F/MvsSln/releases/tag/2.5.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Question regarding depending on the DllExported dlls on another project
2 participants