Allow neon to compile from the source (2x) #142

shargon · 2019-11-30T11:16:27Z

Close #78
Port of #131 to 2x

shargon · 2019-12-05T10:30:00Z

devhawk · 2019-12-05T16:50:32Z

I'll need to take a look at this. We have other changes to the compiler (#144 as well as the temporary neon-de fork that I'm not sure how we are going to replicate going C# -> AVM directly.

shargon · 2019-12-14T20:20:26Z

@devhawk Some ut was wrong and fixed, please take a look

nicolegys · 2019-12-17T08:32:27Z

Test:

Compile from .cs: Success

Compile from .csproj: Fail

Here failed, because the value of projDefinition wasn't the content in the .csproj file.

In addition, could you also add some help info to make neon more friendly?

shargon · 2019-12-17T08:41:14Z

Can you copy your csproj file?

nicolegys · 2019-12-17T09:53:44Z

Sorry, I've got it wrong. projDefinition has no problem, but I found.
1.If my .csproj file has xmlns, there will be some problem when reading this file.
2.If my .csproj file doesn't have some element like PackageReference, it will throw an exception.

<Project ToolsVersion="14.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    <ProductVersion>8.0.30703</ProductVersion>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{2712642F-46ED-47B6-95D9-FB2567BFA2F5}</ProjectGuid>
    <OutputType>Library</OutputType>
    <AppDesignerFolder>Properties</AppDesignerFolder>
    <RootNamespace>NEP5</RootNamespace>
    <AssemblyName>NEP5</AssemblyName>
    <TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
    <FileAlignment>512</FileAlignment>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>full</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\Debug\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugType>none</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release\</OutputPath>
    <DefineConstants>
    </DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
  </PropertyGroup>
  <ItemGroup>
    <Reference Include="Neo.SmartContract.Framework, Version=2.7.3.0, Culture=neutral, processorArchitecture=MSIL">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>.\Neo.SmartContract.Framework.dll</HintPath>
    </Reference>
    <Reference Include="System" />
    <Reference Include="System.Core" />
    <Reference Include="System.Xml.Linq" />
    <Reference Include="System.Data.DataSetExtensions" />
    <Reference Include="Microsoft.CSharp" />
    <Reference Include="System.Data" />
    <Reference Include="System.Numerics" />
    <Reference Include="System.Xml" />
  </ItemGroup>
  <ItemGroup>
    <Compile Include="NEP5.cs" />
    <Compile Include="Properties\AssemblyInfo.cs" />
  </ItemGroup>
  <ItemGroup>
    <None Include="packages.config" />
    <None Include="build.tasks" />
  </ItemGroup>
  <ItemGroup>
    <Content Include="Neo.ConvertTask.dll" />
    <Content Include="Neo.SmartContract.Framework.dll" />
  </ItemGroup>
  <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
  <Import Project="Build.tasks" />
  <Target Name="AfterBuild">
    <Message Text="Start NeoContract converter, Source File: $(TargetPath)" Importance="high">
    </Message>
    <ConvertTask DataSource="$(TargetPath)" />
  </Target>
</Project>

shargon · 2019-12-17T20:40:26Z

Thanks @nicolegys , could you test it again please? (If works we should port this change to 3x)

nicolegys · 2019-12-18T07:21:04Z

Compile from .csproj file:

Succeed!

shargon · 2019-12-18T15:32:03Z

Any update @lock9 @devhawk ?

devhawk · 2019-12-18T16:06:41Z

Has this been tested with https://github.com/neo-project/neo-debugger? The screen shot above shows the .avmdbgnfo file getting created, so that's a good sign

Neo.Compiler.MSIL/Program.cs

devhawk · 2019-12-18T16:16:11Z

Neo.Compiler.MSIL/Program.cs

            var log = new DefLogger();
            log.Log("Neo.Compiler.MSIL console app v" + Assembly.GetEntryAssembly().GetName().Version);

+            // Check argmuents
+            if (args.Length == 0)


Can we use a library for command line processing? neo-express uses https://github.com/natemcmaster/CommandLineUtils and I've been pretty happy with it

Now we don't have a lot of arguments, only a file name, was discarded in other PR
7364563

Neo.Compiler.MSIL/Program.cs

devhawk · 2019-12-18T16:20:42Z

Neo.Compiler.MSIL/Program.cs

+            if (args.Length == 0)
+            {
+                log.Log("You need a parameter to specify the DLL or the file name of the project.");
+                log.Log("Examples: ");


we also support single .cs files right?

We should have a --help option

Neo.Compiler.MSIL/Compiler.cs

devhawk · 2019-12-18T16:24:13Z

looks good, a few pieces of minor feedback. I want to test to make sure it also fixes #78 before I sign off

devhawk · 2019-12-18T17:30:34Z

I tried creating a contract from template and compiling it with neon built out of this PR branch and it didn't work:

C:\Users\harry\Source\neo\seattle\samples\hello2 > dotnet new neo-contract
The template "Neo Smart Contract" was created successfully.
C:\Users\harry\Source\neo\seattle\samples\hello2 > neon .\hello2.csproj
Neo.Compiler.MSIL console app v2.6.0.0
Compiling from csproj project
Unhandled exception. System.IO.FileNotFoundException: Could not find file 'C:\Users\harry\Source\neo\seattle\samples\hello2\Neo.SmartContract.Framework.dll'.
File name: 'C:\Users\harry\Source\neo\seattle\samples\hello2\Neo.SmartContract.Framework.dll'
   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)
   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at System.IO.File.OpenRead(String path)
   at Roslyn.Utilities.FileUtilities.OpenFileStream(String path)
   at Microsoft.CodeAnalysis.MetadataReference.CreateFromFile(String path, MetadataReferenceProperties properties, DocumentationProvider documentation)
   at Neo.Compiler.Compiler.<>c.<CreateReferences>b__6_0(String u) in C:\Users\harry\Source\neo\official\devpack-dotnet\Neo.Compiler.MSIL\Compiler.cs:line 195
   at System.Linq.Enumerable.SelectArrayIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at Neo.Compiler.Compiler.CreateReferences(String[] references) in C:\Users\harry\Source\neo\official\devpack-dotnet\Neo.Compiler.MSIL\Compiler.cs:line 195
   at Neo.Compiler.Compiler.CompileCSFile(String[] filenames, String[] references, Boolean releaseMode) in C:\Users\harry\Source\neo\official\devpack-dotnet\Neo.Compiler.MSIL\Compiler.cs:line 175
   at Neo.Compiler.Compiler.CompileCSProj(String filename, Boolean releaseMode) in C:\Users\harry\Source\neo\official\devpack-dotnet\Neo.Compiler.MSIL\Compiler.cs:line 77
   at Neo.Compiler.Program.Main(String[] args) in C:\Users\harry\Source\neo\official\devpack-dotnet\Neo.Compiler.MSIL\Program.cs:line 105

devhawk · 2019-12-18T17:34:04Z

Neo.Compiler.MSIL/Program.cs

+                default:
+                    {
+                        log.Log("File format not supported by neon: " + path);
+                        Environment.Exit(-1);


Again, should just return -1 instead of using environment exit

devhawk · 2019-12-18T17:58:41Z

Neo.Compiler.MSIL/Neo.Compiler.MSIL.csproj

    <PackageReference Include="Neo.VM" Version="2.4.1" />
  </ItemGroup>

  <ItemGroup>
+    <ProjectReference Include="..\Neo.SmartContract.Framework\Neo.SmartContract.Framework.csproj" />


I don't think this is a good idea. Today, the Neo.SmartContract.Framework versions independently from NEON. With this change, the version of Neo.SmartContract.Framework is hardcoded into the compiler.

Unfortunately, it appears this is the change that allows NEON to compile dlls on build instead of publish.

It was decided in 3x to include it to facilitate developers

devhawk

don't embed Neo.SmartContract.Framework in neon

devhawk · 2019-12-18T21:38:49Z

FYI, I have an idea on how to locate the Neo.SmartContract.Framework assembly at runtime.

The Microsoft.Extensions.DependencyModel nuget package can be used to parse the .deps.json file that is generated by dotnet build. I built a test app that shows how to load and resolve the dependencies to file paths.

Then with Cecil, we can build a custom assembly resolver that uses the DependencyModel information to resolve assemblies (deferring to cecil's DefaultAssemblyResolver for assemblies that aren't listed in the DependencyModel).

We'd still have to figure out how to resolve the Neo.SmartContract.Framework package for compile from source. Honestly, I'm OK with having an embedded version of the Neo.SmartContract.Framework assembly in NEON as long as it's only used for compile from source scenarios.

shargon · 2019-12-21T10:55:28Z

I tried creating a contract from template and compiling it with neon built out of this PR branch and it didn't work:

Could you try again, I think that was fixed here ee1aa41

shargon · 2019-12-21T10:57:05Z

don't embed Neo.SmartContract.Framework in neon

As you can see in the thread of #131 all was agree in include in neon

devhawk · 2019-12-23T17:59:15Z

don't embed Neo.SmartContract.Framework in neon

As you can see in the thread of #131 all was agree in include in neon

This is probably not as big a deal as I thought it was. We don't add new services in the blockchain very often and framework changes like FeaturesAttribute require a new version of the compiler anyway.

* Apply #142 changes * Some fixes * Plural

shargon added 2 commits November 30, 2019 12:11

Allow to compile code

3deb003

Fix UT

e3412bd

shargon requested review from devhawk and lock9 November 30, 2019 11:16

shargon added 3 commits December 14, 2019 20:40

Merge branch 'master-2.x' into compile-source-2x

ae3b032

Fix using

add6ffe

Fix UT

5043336

Fix csproj

829ea5e

Fix enumerable

bb9fd88

Merge branch 'master-2.x' into compile-source-2x

d281ba6

shargon added a commit that referenced this pull request Dec 18, 2019

Apply #142 changes

b02467a

shargon mentioned this pull request Dec 18, 2019

Apply #142 changes (3x) #175

Merged

Update Compiler.cs

a6c76d4