Skip to content
This repository has been archived by the owner on Dec 18, 2017. It is now read-only.

Adding dependency analyzer to kpm #1009

Closed
wants to merge 4 commits into from
Closed

Adding dependency analyzer to kpm #1009

wants to merge 4 commits into from

Conversation

troydai
Copy link
Contributor

@troydai troydai commented Dec 23, 2014

DPA: Dependency Analyzer - https://github.com/aspnet/KRuntime/tree/dev/src/DependencyAnalyzer is a tooled used during KRE build process to generate the minimal KRE assembly list and TPA list. It is ran in the second round of the build (d8a8fce) to ensure it relies on the latest build because it is run from source code.

This change adds dependency analyzer (dpa) into kpm. Makes it fully independent from the KRE under build. It allows us to build minimal KRE and TPA list in the first round of build. This is the first step towards a further reduced build script (#1010)

The kpm dpa will build a manifest file lists the files to be included in KRE from CoreCLR and other nuget packages. Additionally, the new dpa can also build manifest for CLR version of KRE. It is required because CLR version of KRE also packs a few assemblies from NuGet.

The dpa is also used to generate the tpa.cpp includes a minimal TPA list.

AssemblyInformation is moved from Microsoft.Framework.Project to Microsoft.Framework.Runtime because kpm dpa relies on it. Microsoft.Framework.PackageManager can't directly reference to Microsoft.Framework.Project

Related Issues: #779 #755
Follow up change: #1010

Here's an example of the KRE manifest (for CoreCLR x86)

-Runtime
C:\Users\trdai\.kpm\packages\Microsoft.CodeAnalysis.CSharp\1.0.0-beta2-20141221-02\lib\portable-net45+win8\Microsoft.CodeAnalysis.CSharp.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Text.Encoding.Extensions.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\mscorlib.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.InteropServices.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Core.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.Extensions.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Internal.Uri.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Internal.Runtime.Uri.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Dynamic.Runtime.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Reflection.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Text.Encoding.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Reflection.Primitives.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Resources.ResourceManager.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Tools.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Xml.XDocument.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Xml.Linq.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Xml.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Xml.ReaderWriter.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.IO.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.Tasks.Parallel.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Debug.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Tracing.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Collections.Concurrent.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Globalization.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Collections.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.Tasks.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\mscorlib.Extensions.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Linq.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Linq.Expressions.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Resources.ResourceWriter.dll
C:\Users\trdai\.kpm\packages\Microsoft.CodeAnalysis.Common\1.0.0-beta2-20141221-02\lib\portable-net45+win8\Microsoft.CodeAnalysis.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Reflection.Extensions.dll
C:\Users\trdai\.kpm\packages\Newtonsoft.Json\6.0.6\lib\portable-net45+wp80+win8+wpa81+aspnetcore50\Newtonsoft.Json.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Microsoft.CSharp.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.Serialization.Primitives.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.Serialization.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Text.RegularExpressions.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.ObjectModel.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.ComponentModel.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Console.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.IO.FileSystem.Primitives.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.IO.Compression.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.IO.FileSystem.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.ThreadPool.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.Handles.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.Overlapped.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.IO.FileSystem.Watcher.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Microsoft.Win32.Primitives.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Net.Sockets.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Networking.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.ComponentModel.EventBasedAsync.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Cryptography.Encryption.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Principal.Windows.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Principal.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Claims.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.SecureString.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Collections.NonGeneric.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Cryptography.X509Certificates.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Runtime.Loader.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.Timer.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Threading.Thread.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Xml.XmlSerializer.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Xml.Serialization.dll
C:\Users\trdai\.kpm\packages\System.Collections.Immutable\1.1.33-beta\lib\portable-net45+win8+wp8+wpa81\System.Collections.Immutable.dll
C:\Users\trdai\.kpm\packages\System.Reflection.Metadata\1.0.18-beta\lib\portable-net45+win8\System.Reflection.Metadata.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.AppContext.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Net.Primitives.dll
C:\Users\trdai\.kpm\packages\System.Reflection.Metadata\1.0.17-beta\lib\portable-net45+win8\System.Reflection.Metadata.dll
C:\Users\trdai\.kpm\packages\System.Collections.Immutable\1.1.32-beta\lib\portable-net45+win8+wp8+wpa81\System.Collections.Immutable.dll
-Microsoft.Framework.DesignTimeHost
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Process.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Microsoft.Win32.Registry.dll
-Microsoft.Framework.PackageManager
C:\Users\trdai\.kpm\packages\Microsoft.Net.Http.Client\1.0.0-rc1-10049\lib\aspnetcore50\Microsoft.Net.Http.Client.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Net.Security.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Net.Http.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Process.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Microsoft.Win32.Registry.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Reflection.TypeExtensions.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Cryptography.Hashing.Algorithms.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Contracts.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Cryptography.Hashing.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Security.Cryptography.Encoding.dll
-Microsoft.Framework.Project
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\System.Diagnostics.Process.dll
C:\myk\KRuntime\artifacts\build\CoreCLR\Runtime\x86\Microsoft.Win32.Registry.dll

{
if (string.IsNullOrEmpty(KreRoot) || !Directory.Exists(KreRoot))
{
Reports.Error.WriteLine("A valid path to the KRE folder is required");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Messages to end with a period.

@troydai
Copy link
Contributor Author

troydai commented Dec 23, 2014

@davidfowl here's the holiday gift

{
if (string.IsNullOrEmpty(CoreClrRoot) || !Directory.Exists(CoreClrRoot))
{
Reports.Error.WriteLine("A valid path to the CoreCLR folder is required when target framework is CoreCLR.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when target framework is CoreCLR => aspnetcore50?

@davidfowl
Copy link
Member

For this to be a proper kpm command it has to be general purpose. kpm dpa is a pretty cryptic name. Can I generally use this on any project? We need to figure out the layering here. There can be a dependency analyzer command and then maybe we use the output of that command to build the TPA list.

Putting it all into kpm doesn't make much sense as it's only used to build the KRE.

@Eilon
Copy link
Member

Eilon commented Dec 25, 2014

Yeah I think this should just share common code with kpm. I agree this has no business in kpm itself because no one except for us can use it.

@troydai
Copy link
Contributor Author

troydai commented Dec 25, 2014

The point here is we need to build some of the functionality, at least the dependency analyzing part, into the tooling contained in KRE. Otherwise the build process will always stay in the dilemma that the source codes of build tools are based on next version but the running framework is based on the previous version.

We can add a slim down version of dpa to the kpm for just print out the assembly dependency of a given project. It has its general purpose that customer can use it to analyze their own project.

@Eilon
Copy link
Member

Eilon commented Dec 26, 2014

I think everyone is agreement about the functionality. I think the key suggestions are:

  1. Still have it perform the same utility as it does today. Whatever it's doing, we still need.
  2. Move this logic out of KPM. If you can articulate a reasonable scenario where a customer would want this, we can consider that.
  3. Have this share common code with KPM.

@troydai
Copy link
Contributor Author

troydai commented Dec 30, 2014

Talked with David, will come up a new command name, slim down the function and implement with a more general purpose.

@troydai
Copy link
Contributor Author

troydai commented Jan 5, 2015

Undergo major design change.

@troydai troydai closed this Jan 5, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants