Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standalone workflow #3018

Open
wants to merge 114 commits into
base: master
Choose a base branch
from

Conversation

cschuchardt88
Copy link
Member

@cschuchardt88 cschuchardt88 commented Dec 7, 2023

NOTE: You need to run this workflow manually when it time for release.

How this works

  1. Run Build & Release (neo-cli) workflow job.
  2. Click Run workflow.
  3. Enter Version Number (Example: 3.7.0)

image

🍭 That's all!!!! 🍭

What Will the Output Look Like?

Workflow process

image

Caches Created

image

Release Title

image

Release Body

image

Releases Downloads

image

Folder Content

image

Working as Standalone App

image

Closes #3005
Closes #3062

@cschuchardt88
Copy link
Member Author

cschuchardt88 commented Dec 7, 2023

@superboyiii @shargon Where do you guys get libleveldb.dll from? I need a place to download it for github workflow. So I can bundle it up.

Edit:
Nevermind figured out how to compile it from source.

@cschuchardt88 cschuchardt88 marked this pull request as ready for review December 8, 2023 03:29
@cschuchardt88
Copy link
Member Author

cschuchardt88 commented Dec 8, 2023

@gsmachado Can you test on OSX? Don't forget to the install libleveldb-devel on MacOS.

@cschuchardt88
Copy link
Member Author

@shargon can you test/check out?

deps/libleveldb.arm64 Outdated Show resolved Hide resolved
@cschuchardt88 cschuchardt88 marked this pull request as draft December 8, 2023 10:39
@cschuchardt88 cschuchardt88 marked this pull request as ready for review December 8, 2023 11:44
@cschuchardt88
Copy link
Member Author

Would you like a portable version as well? That requires dotnet to be installed? The standalone version do NOT require dotnet to be installed!

@superboyiii
Copy link
Member

Look how big this already. I don't think we need to build leveldb for everything. That should the responsibility of the creator and/or distro that maintains the library. Developers SHOULD and MUST know that they have to setup an environment. That's the point of development, to be able to change or configure; how you want. Plus there is no easy clickonce server or blockchain that i know of. Every single one you need to install packages or configurations in some way. We may want to start thinking about a new way to storage. I recommended before to use FasterDb Its dotnet, easy to use, just as fast, if not faster. Since we are not switching to different storage. We should just remove that feature all together (to have plugins for storage).

Plus goal is to install neo-cli with brew anyways.

OK. Let's remove leveldb.

@cschuchardt88
Copy link
Member Author

Still needed for windows though.

@@ -0,0 +1,52 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use/copy the config files from the source?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are for the debian packages, They need a custom storage path.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it's possible to copy them in the script?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a script. build-dpkg.sh I think the name is. I need these templates. I can look into a way of changing the data for the config json files.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cschuchardt88, can you copy the files and change them with sed?
I think it will be better for maintenance.

Comment on lines +3 to +9
on:
workflow_dispatch:
inputs:
version:
type: string
description: 'Version Number (Example: 1.0.0)'
required: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a suggestion: don't we want to automatically run this job on GH release trigger?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We alway did it this way. This Creates a Release automatically after building it. With binaries add to release.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, but how about moving from manual trigger to automatical? Like this:

on:
  release:
    types:
      - published

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't that trigger after a release though? Workflow goes like this.

  1. Build Binaries & Cache binaries
  2. Get cache & Build Debian packages & Cache again
  3. Get cache & Publish Release with binaries
  4. Clean-up Caches

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, exactly for situations like this we have gh release upload. So the adjusted pipeline is the following:

  1. Create and publish release on GitHub, that will trigger release GH event
  2. Grab released ref. from release event, Build Binaries & Cache binaries
  3. Get cache & Build Debian packages
  4. Upload release artifacts from step 3 (IMO, caching isn't needed here).
  5. Get artifacts from step 4 & Attach binaries to the GH release
  6. Clean-up Caches

And with this pipeline you don't need a special person who manually pushes action activation button. Instead, workflow starts automatically immediately after release and in the end attaches resulting binaries to the release page. Take a look at this pipeline implemented in https://github.com/nspcc-dev/neo-go/blob/master/.github/workflows/build.yml, and especially pay attention to the following steps:

At the same time manual workflow dispatch can be kept "just in case", if something goes wrong with the automatically triggered job.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll read more into this when i have time. But i thought you should know, we can't use artifacts. We eat up space so fast and only have something like 500MB for that. Our space is very limited.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guys... we need the binaries/artifacts to be uploaded somewhere. This would enable people to just download and use it. Also, for brew, relying on binaries/artifacts with an SHA-256 file is the best fit.

If we're so miserable for not being able to pay more than 500Mb of storage in GitHub, my suggestion is: create an Amazon S3 bucket (or, in Digital Ocean) and add it there. Wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/nspcc-dev/gh-push-to-neofs/

Can even use some static nuget feed generator to have proper feeds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shargon need you here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're so miserable for not being able to pay more than 500Mb of storage in GitHub

Committee members should be able to aford it, core devs are the last link in neo economics,
I removed a lot of artifacts, and old workflows and still without free space, I don't know where is gone this space...

Copy link
Member

@vncoelho vncoelho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job, I will review it.

Copy link
Member

@vncoelho vncoelho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was sending some examples but they won't be needed.

Copy link
Member

@vncoelho vncoelho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cschuchardt88, it looks very good!

What do you need to finish it?

@cschuchardt88
Copy link
Member Author

@cschuchardt88, it looks very good!

What do you need to finish it?

fix up the xml files.

Comment on lines +133 to +149
- if: ${{ startsWith(matrix.runtime, 'win-x64') }}
name: Get Distribution Caches (win-x64)
uses: actions/cache@v3
with:
path: ./leveldb/build/Release/*
key: leveldb-windows-x64
enableCrossOsArchive: true
fail-on-cache-miss: true

- if: ${{ startsWith(matrix.runtime, 'win-arm64') }}
name: Get Distribution Caches (win-arm64)
uses: actions/cache@v3
with:
path: ./leveldb/build/Release/*
key: leveldb-windows-arm64
enableCrossOsArchive: true
fail-on-cache-miss: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move key to matrix parameter, exactly as in neo-project/neo-devpack-dotnet#855 (comment), and then two steps may be squashed. Cleaner code, no copy-pasting. The same optimisation can be applied to the instructions below.

@superboyiii
Copy link
Member

@Jim8y @vncoelho @cschuchardt88 @AnnaShaleva @roman-khimov @shargon Maybe we can try this on v3.7.0?

Copy link
Member

@shargon shargon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that need to be updated to net 8

.github/workflows/standalone.yml Outdated Show resolved Hide resolved
.github/workflows/standalone.yml Outdated Show resolved Hide resolved
.github/workflows/standalone.yml Outdated Show resolved Hide resolved
@cschuchardt88
Copy link
Member Author

cschuchardt88 commented Apr 17, 2024

This can be put on hold until

is completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants