Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Github actions build/test/publish/release pipeline #745

Merged
merged 93 commits into from
Jul 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
f3bde08
Remove appveyor
nozzlegear May 31, 2022
5adeeab
Updating build/test pipeline
nozzlegear May 31, 2022
a00ca9f
Use dotnet 6.0.x
nozzlegear May 31, 2022
60a2660
Remove quotations
nozzlegear May 31, 2022
24fe774
Use powershell to execute test script
nozzlegear Jun 2, 2022
8da181b
Debugging: temporarily remove --no-build
nozzlegear Jun 2, 2022
a97b8d7
Debugging: re-enable --no-build
nozzlegear Jun 2, 2022
4b1982d
Debugging: update package references
nozzlegear Jun 2, 2022
efb67dd
Only run tests with netcoreapp3.1 framework
nozzlegear Jun 2, 2022
bc02331
Use backticks for multi-line powershell commands
nozzlegear Jun 2, 2022
bb17a4a
Inject secrets into environment
nozzlegear Jun 2, 2022
632b059
Add a DotNetFramework test category for all infrastructure tests
nozzlegear Jun 2, 2022
ea08642
Add build script
nozzlegear Jun 4, 2022
610557e
Install fish during build
nozzlegear Jun 4, 2022
6f535da
Bugfix: use `run`
nozzlegear Jun 4, 2022
712d52f
Quiet apt update/install
nozzlegear Jun 4, 2022
bd4a3bf
Add github.run_number to environment variables during build
nozzlegear Jun 4, 2022
643e3a5
Remove unused warn function
nozzlegear Jun 4, 2022
974dd2c
Debug: echo fish version on script run
nozzlegear Jun 4, 2022
cd2d2e4
Add fish ppa on install
nozzlegear Jun 4, 2022
190cfbe
Clarify exits
nozzlegear Jun 4, 2022
906b8e3
Debug: explicitly exit with 0
nozzlegear Jun 4, 2022
c9937f3
Remove extra log message
nozzlegear Jun 4, 2022
7ff4a87
Remove old powershell build script
nozzlegear Jun 4, 2022
aef939c
Replace powershell test script with fish test script
nozzlegear Jun 11, 2022
451f38d
Fix test log filenames
nozzlegear Jun 11, 2022
b1ad1b9
Install fish on unit test runner
nozzlegear Jun 11, 2022
ab0ea30
Debug: use bash to run fish shell
nozzlegear Jun 11, 2022
0d4a6b8
Debug: put `or exit 1` on same line as command
nozzlegear Jun 11, 2022
56263f3
Debug: check status code explicitly after each test run
nozzlegear Jun 11, 2022
f00445f
Move exiting on test failure executeTests function
nozzlegear Jun 11, 2022
5fdaafc
Fix CustomerSavedSearch test fixture
nozzlegear Jun 11, 2022
e154232
Use a unique search name
nozzlegear Jun 11, 2022
ccc94f0
Trim saved search name length to 40 chars max
nozzlegear Jun 11, 2022
91a0ba8
Build test project for both net core and framework
nozzlegear Jun 12, 2022
0385259
Build test project for all targeted frameworks
nozzlegear Jun 12, 2022
662d9d9
Fix CustomerSavedSearch tests
nozzlegear Jun 12, 2022
c7cfc83
Fix property names
nozzlegear Jun 12, 2022
88d7574
Simplify CustomerSavedSearch test
nozzlegear Jun 12, 2022
56e6242
Test fix: set LocationId when creating fulfillment
nozzlegear Jun 13, 2022
e48f7e0
Don't run .NET Framework tests on arm64 machines
nozzlegear Jun 13, 2022
5f434ec
Temporarily disable .NET Framework tests
nozzlegear Jun 13, 2022
22dcebc
Use Github Actions format for surfacing warning/error messages
nozzlegear Jun 13, 2022
3fce439
Use GNU Parallel to execute tests in parallel
nozzlegear Jun 14, 2022
76622ad
Halt parallel on the first error
nozzlegear Jun 14, 2022
180579d
Check exit status of parallel
nozzlegear Jun 14, 2022
b9c70d3
Bugfix: use test to check status
nozzlegear Jun 14, 2022
8173080
Move pack/build/test commands into utility functions in utils.fish
nozzlegear Jun 14, 2022
1ef4334
Fix parallel test execution
nozzlegear Jun 14, 2022
a4f56fc
Always retry requests that fail due to rate limit
nozzlegear Jun 14, 2022
ecb80e0
Debug: echo parallel status if it's not 0
nozzlegear Jun 14, 2022
2a9bc27
Add explicit exit 0 command
nozzlegear Jun 14, 2022
41c9ba8
Exit with 1 when parallel fails
nozzlegear Jun 14, 2022
4f086b8
Explicitly return 0 if tests pass
nozzlegear Jun 14, 2022
6f63a2d
Use 'or exit 1' if parallel fails
nozzlegear Jun 14, 2022
8f34e16
Always use fixture services with execution policies when initializing…
nozzlegear Jun 15, 2022
c508dfc
Use base64 attachment to create collection image
nozzlegear Jun 15, 2022
25f02a5
Only try to delete collection if it was created
nozzlegear Jun 15, 2022
1b06f1a
Only exit if shell is not interactive
nozzlegear Jun 15, 2022
20e245c
Use base64 attachment to create collection image
nozzlegear Jun 15, 2022
20e2bb5
Test fix: implement IAsyncLifetime for all fixtures
nozzlegear Jun 16, 2022
2907ebf
Move OrderService instance to fixture
nozzlegear Jun 16, 2022
0f2f48d
Add script to count the number of missing execution policies in a tes…
nozzlegear Jun 17, 2022
04160c8
Add function to count number of misconfigured test fixtures in a file
nozzlegear Jun 17, 2022
0a264d2
Add missing execution policies
nozzlegear Jun 17, 2022
441457b
Revert changes in a4f56fcb1c
nozzlegear Jun 18, 2022
0ad6539
Run up to 5 test categories at once
nozzlegear Jun 18, 2022
b462a59
Refactor InventoryLevel tests, add shared execution policy
nozzlegear Jun 21, 2022
1a5868a
Catch expected errors during test cleanup
nozzlegear Jun 21, 2022
82c8ed5
Remove erroneous temporary file
nozzlegear Jun 21, 2022
3a23f5f
Upload artifacts during build phase
nozzlegear Jun 21, 2022
78d54e7
Eror if no artifact files are found
nozzlegear Jun 22, 2022
43a897a
Publish prerelease packages to Nuget after successful test runs
nozzlegear Jun 23, 2022
02ce7da
Rename pipeline to Build/Test Pipeline
nozzlegear Jun 23, 2022
a66f420
Skip duplicate package pushes, remove explicit symbol push
nozzlegear Jun 23, 2022
5507057
Create Github prerelease and publish artifacts to it
nozzlegear Jun 23, 2022
bda58c7
Add step to test .NET Framework tests
nozzlegear Jun 25, 2022
bfbd841
Run all DotNetFramework tests that are not ShopifyService and Shopify…
nozzlegear Jun 25, 2022
8966628
Add Exclusive category
nozzlegear Jun 28, 2022
156b1ec
Run all DotNetFramework tests that are not in the Exclusive category
nozzlegear Jun 28, 2022
71871ee
Replace Category=Exclusive with collection attributes
nozzlegear Jun 28, 2022
8c25285
Remove excluse category filter, rely on test collections
nozzlegear Jun 28, 2022
f511233
Add test fixture, clean up background/foreground test
nozzlegear Jun 29, 2022
7664681
Run shorter DotNetFramework tests first
nozzlegear Jun 29, 2022
de080ff
Fix yaml
nozzlegear Jun 29, 2022
d2f2cdb
Refactor background/foreground test
nozzlegear Jun 30, 2022
d0176ce
Debug: log foreground/background completion times
nozzlegear Jun 30, 2022
b0e503d
Temporarily disable test
nozzlegear Jun 30, 2022
f80badc
Add preliminary release workflow
nozzlegear Jul 2, 2022
b28d82d
Debugging: change inputs to github actions example
nozzlegear Jul 2, 2022
1c3253d
Fix yaml
nozzlegear Jul 2, 2022
9bca1e5
Remove obsolete deploy script
nozzlegear Jul 6, 2022
3263211
Do not throw if collection has already been deleted
nozzlegear Jul 6, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/scripts/build.fish
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#! /usr/bin/env fish

echo "Using $(fish --version)"

# Load utility functions
set utilsFilePath (dirname (status --current-filename))"/utils.fish"
source "$utilsFilePath"

if ! set -q "GITHUB_RUN_NUMBER"
error "Github run number (\$GITHUB_RUN_NUMBER) environment variable not set, cannot create prerelease build."
exit 1
end

set revision "b$GITHUB_RUN_NUMBER"
set outputDir "$ARTIFACT_DIR"

# Pack the projects
packProject "$revision" "$outputDir" "ShopifySharp/ShopifySharp.csproj"
packProject "$revision" "$outputDir" "ShopifySharp.Experimental/ShopifySharp.Experimental.fsproj"

exit 0
79 changes: 79 additions & 0 deletions .github/scripts/test.fish
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#! /usr/bin/env fish

echo "Using $(fish --version)"

set experimentalTestProject "ShopifySharp.Experimental.Tests/ShopifySharp.Experimental.Tests.fsproj"

# Load utility functions
set utilsFilePath (dirname (status --current-filename))"/utils.fish"
source "$utilsFilePath"

# Build and run the experimental project's tests all at once
echo "Testing experimental project."
buildProject "$experimentalTestProject"; or exit 1;
executeTests \
"ShopifySharp.Experimental" \
"$netCoreApp" \
"$experimentalTestProject"
success "Experimental tests succeeded."

# Build the test project once, then let all individual test runs skip build.
echo "Building test project."
buildProject "$testProjectFile"; or exit 1;
success "Test project build succeeded."

# Regex for parsing categories from Trait attributes in test files
set categoryRegex "Trait\(['\"]Category['\"], ['\"]([A-Za-z]+)['\"]"
set categories

# Read every C# file in the test directory and parse the categories
for file in (ls ./ShopifySharp.Tests/*.cs)
for category in (string match -agr "$categoryRegex" (cat "$file"))
switch (string lower "$category")
case "dotnetframework"
# This is a .NET Framework test, it will be run explicitly after the other categories.

case "shopifyexception"
# This is a test of Shopify rate limit exceptions, save it for last.

case "*"
set -a categories "$category"
end
end
end

# Run category tests
if command -q parallel
# Tests can be run in parallel. Run up to five test categories at once.
set threads 5
# This is a little weird, but parallel needs to run fish and source the utils file so it can call the executeTests function
set fishCommand "source $utilsFilePath; and executeTests {} $netCoreApp; and exit 0"

parallel \
-j "$threads" \
--halt-on-error 1 \
fish -c \"$fishCommand\" \
::: $categories
or exit 1;

else
warn "GNU Parallel is not installed or could not be found. Testing categories one by one."

for category in $categories
executeTests "$category" "$netCoreApp"
end
end

# Run .NET Framework tests using .NET Framework 4.7.2
if ! isArm64
# TODO: these tests need to be run on a Windows machine, they do not run on Linux/macOS
warn ".NET Framework tests are temporarily disabled."
#executeTests "DotNetFramework" "$netFramework"
else
warn "Machine is running on arm64 which does not support .NET Framework. Unable to run .NET Framework tests."
end

# Run Shopify exception tests which attempt to trip the rate limit
executeTests "ShopifyException" "$netCoreApp"

exit 0
234 changes: 234 additions & 0 deletions .github/scripts/utils.fish
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
#! /usr/bin/env fish

set config "Release"
set verbosity "quiet"
set netCoreApp "netcoreapp3.1"
set netFramework "net472"
set testDir "ShopifySharp.Tests"
set testProjectFile "$testDir/ShopifySharp.Tests.csproj"

function success
set_color green
echo "$argv"
set_color normal
end

function warn
# Surface warning messages in Github Actions:
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#example-setting-a-warning-message
set_color yellow
echo "::warning ::$argv"
set_color normal
end

function error
# Surface error messages in Github Actions:
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#example-setting-an-error-message
set_color red
echo "::error ::$argv"
set_color normal
end

function isArm64
# Note that functions in fish return exit codes -- success is 0, failure is anything > 0
if test (uname -m) = "arm64"
return 0
else
return 1
end
end

function printTestResults -a testOutput
# Parse the test output for the line indicating how many tests were run
# The `string match` function will automatically write to console if a match is found
string match -er "Total tests:|Passed!|Failed!|Skipped!" "$line"

echo "$testOutput" > testoutput.txt
end

function executeTests -a category -a framework -a projectFile
if test -z "$framework"
set framework "$netCoreApp"
end

if test -z "$projectFile"
set projectFile "$testProjectFile"
end

if test -z "$category"
set_color red
echo "[error] Empty category passed to test function."

if status --is-interactive
return 1
else
exit 1
end
end

echo ""
echo "Running $category tests..."

set testOutput "$(dotnet test \
-c "$config" \
-f "$framework" \
--verbosity "$verbosity" \
--logger "trx;LogFileName=$category.trx" \
--results-directory "TestResults" \
--filter "Category=$category" \
--no-build \
"$projectFile"
)"; or begin
echo "$testOutput"
error "$category tests failed!"

if status --is-interactive
return 1
else
exit 1
end
end

if string match "No test is available" "$testOutput"
# Bad category name, no test was run
error "$category test was not run! Category may not exist."

if status --is-interactive
return 1
else
exit 1
end
end

success "$category tests passed."
return 0
end

function buildProject -a projectFile
if ! test -e "$projectFile"
error "Project file at $projectFile does not exist"

if status --is-interactive
return 1
else
exit 1
end
end

dotnet build \
-c "$config" \
--verbosity "$verbosity" \
"$projectFile"

if test $status -ne 0
if status --is-interactive
return 1
else
exit 1
end
end
end

function packProject -a revision -a outputDir -a projectFile
if test -z "$revision"
error "revision value is empty."

if status --is-interactive
return 1
else
exit 1
end
end

if test -z "$outputDir"
error "outputDir value is empty."

if status --is-interactive
return 1
else
exit 1
end
end

if ! test -e "$projectFile"
error "Project file at $projectFile does not exist"

if status --is-interactive
return 1
else
exit 1
end
end

dotnet pack \
-c "$config" \
--version-suffix "$revision" \
-o "$outputDir" \
"$projectFile";
or begin
if status --is-interactive
return 1
else
exit 1
end
end

success "Packed $project for prerelease."

dotnet pack \
-c Release \
-o "$outputDir" \
"$projectFile";
or begin
if status --is-interactive
return 1
else
exit 1
end
end

success "Packed $projectFile for release."
end

function countMissingPolicies -a file -d "Counts the number of missing execution policies compared to the number of new ShopifySharp services instantiated in a file."
if ! test -e "$file"
error "File at $file does not exist."

if status --is-interactive
return 1
else
exit 1
end
end

set totalNewServices (count (rg "new \w+Service\(" "$file"))
set totalPoliciesSet (count (rg "\.SetExecutionPolicy\(" "$file"))

if test $totalPoliciesSet -ge $totalNewServices
echo 0
else
echo (math $totalNewServices - $totalPoliciesSet)
end
end

function countMisconfiguredFixtures -a file -d "Counts the number of missing or misconfigured test fixture classes in a file."
if ! test -e "$file"
error "File at $file does not exist."

if status --is-interactive
return 1
else
exit 1
end
end

set totalAsyncLifetimes (count (rg "class \w+ *: *IAsyncLifetime" "$file"))
set totalTestClasses (count (rg "class \w+ *: *IClassFixture<\w+>" "$file"))

if test $totalAsyncLifetimes -gt $totalTestClasses
# There are more IAsyncLifetime fixtures defined in this file than are used, which could indicate a misconfiguration
echo (math $totalAsyncLifetimes - $totalTestClasses)
else
echo (math $totalTestClasses - $totalAsyncLifetimes)
end
end
Loading