Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-4839: [C#] Add NuGet package metadata and instructions. #3891

Closed
wants to merge 7 commits into from

Conversation

eerhardt
Copy link
Contributor

This change adds the necessary changes for creating the Apache.Arrow nuget package.

  1. I've updated the .csproj with what I think is the correct package metadata. Please take a look and give any feedback on what I got wrong.
  2. I've added instructions to the README on how to build the NuGet package. By default, this will produce a -preview version of the package. You need to explicitly opt into creating a "stable" version by passing in -p:VersionSuffix=''.
  3. I've enabled strong naming on the library, which is a recommendation according to the .NET Open-source library guidance.
    • Note that "strong naming" isn't a security feature - it is just about creating a unique identity for the library.

/cc @stephentoub @ericstj @pgovind @chutchinson @wesm

Copy link
Contributor

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Thanks.

- Add csharp version update to the prepare script. To do this correctly, I decided to refactor the .csproj and .props files so the common msbuild properties are in a root Directory.Build.props file.

- Adjust the Description of the Apache.Arrow library.

- Add the new xml files to the rat_exclude list.
Copy link

@ericstj ericstj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple minor nits.

csharp/src/Apache.Arrow/Apache.Arrow.csproj Outdated Show resolved Hide resolved
csharp/src/Apache.Arrow/Apache.Arrow.csproj Outdated Show resolved Hide resolved
- Use latest serviced version for dependencies.
- Remove one unnecessary dependency.
@codecov-io
Copy link

Codecov Report

Merging #3891 into master will increase coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3891      +/-   ##
==========================================
+ Coverage   87.78%   87.85%   +0.06%     
==========================================
  Files         727      726       -1     
  Lines       87892    89340    +1448     
  Branches     1252     1252              
==========================================
+ Hits        77158    78490    +1332     
- Misses      10616    10736     +120     
+ Partials      118      114       -4
Impacted Files Coverage Δ
go/arrow/math/uint64_sse4_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/math/float64_sse4_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/memory/memory_sse4_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/math/int64_sse4_amd64.go 0% <0%> (-100%) ⬇️
cpp/src/arrow/util/decimal.h 85.71% <0%> (-14.29%) ⬇️
cpp/src/arrow/compute/logical_type.cc 52.68% <0%> (-9.65%) ⬇️
cpp/src/arrow/compute/expression.cc 40.1% <0%> (-7.23%) ⬇️
cpp/src/gandiva/tree_expr_builder.cc 51.3% <0%> (-6.43%) ⬇️
cpp/src/arrow/status.cc 40.74% <0%> (-6.04%) ⬇️
cpp/src/arrow/sparse_tensor.cc 76.04% <0%> (-5%) ⬇️
... and 263 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95d62ca...b3c6ab8. Read the comment docs.

rm -f Directory.Build.props.bak
git add Directory.Build.props
cd -

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

How about using empty VersionSuffix on release tag? If we use empty VersionSuffix on release tag, we don't need to pass -p:VersionSuffix='' to build release package.

diff --git a/csharp/Directory.Build.props b/csharp/Directory.Build.props
index fde2ea81..bc0eaa17 100644
--- a/csharp/Directory.Build.props
+++ b/csharp/Directory.Build.props
@@ -13,7 +13,7 @@
     <Copyright>2018 Apache Software Foundation</Copyright>
     <Company>Apache</Company>
     <VersionPrefix>0.13.0</VersionPrefix>
-    <VersionSuffix>preview</VersionSuffix>
+    <VersionSuffix>SNAPSHOT</VersionSuffix>
   </PropertyGroup>
 
   <PropertyGroup>
diff --git a/csharp/README.md b/csharp/README.md
index 1d8dbf83..1eeec818 100644
--- a/csharp/README.md
+++ b/csharp/README.md
@@ -147,7 +147,7 @@ To build the NuGet package run the following command to build a debug flavor, pr
 
 When building the officially released version run: (see Note below about current `git` repository)
 
-    dotnet pack -c Release -p:VersionSuffix=''
+    dotnet pack -c Release
 
 Which will build the final/stable package.
 
diff --git a/dev/release/00-prepare.sh b/dev/release/00-prepare.sh
index f8a5eddf..27101976 100755
--- a/dev/release/00-prepare.sh
+++ b/dev/release/00-prepare.sh
@@ -29,10 +29,14 @@ update_versions() {
   case ${type} in
     release)
       local version=${base_version}
+      local csharp_version_prefix=${base_version}
+      local csharp_version_suffix=
       local r_version=${base_version}
       ;;
     snapshot)
       local version=${next_version}-SNAPSHOT
+      local csharp_version_prefix=${next_version}
+      local csharp_version_suffix=SNAPSHOT
       local r_version=${base_version}.9000
       ;;
   esac
@@ -57,8 +61,9 @@ update_versions() {
   cd -
 
   cd "${SOURCE_DIR}/../../csharp"
-  sed -i.bak -E -e \
-    "s/^    <VersionPrefix>.+<\/VersionPrefix>/    <VersionPrefix>${version}<\/VersionPrefix>/" \
+  sed -i.bak -E \
+    -e "s/^    <VersionPrefix>.+<\/VersionPrefix>/    <VersionPrefix>${csharp_version_prefix}<\/VersionPrefix>/" \
+    -e "s/^    <VersionSuffix>.*<\/VersionSuffix>/    <VersionSuffix>${csharp_version_suffix}<\/VersionSuffix>/" \
     Directory.Build.props
   rm -f Directory.Build.props.bak
   git add Directory.Build.props

We don't use any suffix on release tag. For example, https://github.com/apache/arrow/blob/apache-arrow-0.12.1/cpp/CMakeLists.txt#L21 on apache-arrow-0.12.1 tag.

We use "SNAPSHOT" suffix on non release tag. For example,

set(ARROW_VERSION "0.13.0-SNAPSHOT")
on the current master.

How about using "SNAPSHOT" instead of "preview" in C# too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using "SNAPSHOT" instead of "preview" in C# too?

Good call. I am used to using preview on my team, but since we've already established SNAPSHOT in this project, I will make the name change.

How about using empty VersionSuffix on release tag?

When I started this work, I didn't know how the versioning was handled in this project. But what you describe above makes sense to me. My concerns when I started were:

  1. When a dev builds any given commit on their machine, they don't get a "stable" version NuGet package.
  2. You shouldn't need to change source code in order to build a "stable" package.

However, I now learned that we already have a process for switching out all the version numbers right before "tagging" for a release. This approach makes sense to me. That way if someone pulls the source code for a release and just "builds", they will get a "stable" package, because they built from the tagged commit for that release. I'll make this change, thanks for the feedback.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.
It's resonable.

@eerhardt
Copy link
Contributor Author

The appveyor error:

C:\projects\arrow\csharp>dotnet pack -c Release   || exit /B 
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
  Restore completed in 48.41 ms for C:\projects\arrow\csharp\test\Apache.Arrow.Benchmarks\Apache.Arrow.Benchmarks.csproj.
  Restore completed in 46.21 ms for C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj.
  Restore completed in 2.45 ms for C:\projects\arrow\csharp\test\Apache.Arrow.Tests\Apache.Arrow.Tests.csproj.
C:\Users\appveyor\.nuget\packages\microsoft.build.tasks.git\1.0.0-beta2-18618-05\build\Microsoft.Build.Tasks.Git.targets(36,5): warning : Submodule 'cpp/submodules/parquet-testing' doesn't have any commit, the source code won't be available via source link. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]
C:\Users\appveyor\.nuget\packages\microsoft.build.tasks.git\1.0.0-beta2-18618-05\build\Microsoft.Build.Tasks.Git.targets(36,5): warning : Submodule 'testing' doesn't have any commit, the source code won't be available via source link. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]
C:\Users\appveyor\.nuget\packages\microsoft.build.tasks.git\1.0.0-beta2-18618-05\build\Microsoft.Build.Tasks.Git.targets(36,5): warning : Submodule 'cpp/submodules/parquet-testing' doesn't have any commit, the source code won't be available via source link. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]
C:\Users\appveyor\.nuget\packages\microsoft.build.tasks.git\1.0.0-beta2-18618-05\build\Microsoft.Build.Tasks.Git.targets(36,5): warning : Submodule 'testing' doesn't have any commit, the source code won't be available via source link. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]
  Apache.Arrow -> C:\projects\arrow\csharp\artifacts\Release\netcoreapp2.1\Apache.Arrow.dll
  Apache.Arrow -> C:\projects\arrow\csharp\artifacts\Release\netstandard1.3\Apache.Arrow.dll
C:\Users\appveyor\.nuget\packages\microsoft.build.tasks.git\1.0.0-beta2-18618-05\buildCrossTargeting\Microsoft.Build.Tasks.Git.targets(36,5): warning : Submodule 'cpp/submodules/parquet-testing' doesn't have any commit, the source code won't be available via source link. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]
C:\Users\appveyor\.nuget\packages\microsoft.build.tasks.git\1.0.0-beta2-18618-05\buildCrossTargeting\Microsoft.Build.Tasks.Git.targets(36,5): warning : Submodule 'testing' doesn't have any commit, the source code won't be available via source link. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]
  Successfully created package 'C:\projects\arrow\csharp\/artifacts/Release\Apache.Arrow.0.13.0-preview.nupkg'.
C:\Program Files\dotnet\sdk\2.2.103\Sdks\NuGet.Build.Tasks.Pack\buildCrossTargeting\NuGet.Build.Tasks.Pack.targets(202,5): error NU5030: The license file 'LICENSE.txt' does not exist in the package. [C:\projects\arrow\csharp\src\Apache.Arrow\Apache.Arrow.csproj]

Is caused by NuGet/Home#7591. This has been fixed in the latest .NET Core SDK version 2.2.105. However, our appveyor is still using version 2.2.103.

We will either need appveyor to install the latest version - 2.2.105, or remove the pack command from CI until the new version is in place. I don't know how to upgrade the version on appveyor, so I will remove the pack command from CI for now.

Work around NuGet/Home#7591 until CI gets the updated SDK version.
- Handle csharp versioning like the rest of the project using SNAPSHOT suffix for non-release commits, and no suffix for release tags.
- Also, fixing a small error with the artifacts folder, AssemblyName isn't set at this time, so use MSBuildProjectName.
@wesm
Copy link
Member

wesm commented Mar 16, 2019

This looks OK to me at a quick glance, I'll leave to @kou to sign off and merge

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
I'll merge this.

rm -f Directory.Build.props.bak
git add Directory.Build.props
cd -

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.
It's resonable.

<PropertyGroup>
<Product>Apache Arrow library</Product>
<Copyright>2018 Apache Software Foundation</Copyright>
<Company>Apache</Company>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Apache Software Foundation" may be better.
If we need to change this, we can work on this as a follow-up task.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes the formal name of the organization is "The Apache Software Foundation"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.
@eerhardt Could you create a pull request for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do. Can I use the same JIRA?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be a simple PR without a JIRA since it's not notable for the changelog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants