Skip to content
This repository has been archived by the owner on Dec 26, 2023. It is now read-only.

Using the POST go lib in go-spacemesh (WIP) #48

Open
4 of 6 tasks
avive opened this issue Jun 9, 2021 · 3 comments
Open
4 of 6 tasks

Using the POST go lib in go-spacemesh (WIP) #48

avive opened this issue Jun 9, 2021 · 3 comments
Assignees

Comments

@avive
Copy link
Contributor

avive commented Jun 9, 2021

TL;DR - go-spacemesh dev env setup

Follow these steps to properly link go-spacemesh with the gpu-post platform libs:

  1. Copy all artifacts for your dev platform to the go-spacemesh exe path from a compatible gpu-post github release.
  2. Execute go-spacemesh only from its own directory, or set working dir for go-spacemesh exe to be its own directory, or export an VK_ICD_FILENAMES env var with the value set to the go-spacemesh directory.
  3. On windows - use MinGW-W64-builds-4.3.5 or newer version for build toolchain (make, gcc, etc...).
  4. On all platforms: change the permissions on all libs in the artifacts, so your user is allowed to execute them.
  5. On macOS: You MUST right-click-open via Finder on each of the 4 dylibs you downloaded from the internet to enable dynamic linking of apps with these libs because they are not notarized.

Quick test to verify linking works: use the SmesherService::GetProviders() method - it is implemented in the gpu-post c-lib.

macOS gpu-post artifacts

  • libgpu-setup.dylib
  • libMoltenVK.dylib
  • libvulkan.1.2.170.dylib
  • libvulkan.1.dylib
  • MoltenVK_icd.json

Q: Why are there so many files?
A: Apple doesn't include Vulkan drivers in macOS so they must be installed manually. The Vulkan runtime installer is much larger, so we provide these files to get Vulkan working. In Linux and Windows, full Vulkan support is included in the gpu driver if the gpu supports Vulkan.

Linux gpu-post artifacts

  • libgpu-setup.so

Windows gpu-post artifacts

  • gpu-setup.dll
  • gpu-setup.lib

Note that on unix based systems lib is prefixed to the lib name but not in Windows. This is why the Windows libs names are different than Linux and macOS libs.

Tasks

Verify dev env setup on the following:

  • macOS M1 cpu/gpu
  • macOS Intel cpu
  • Ubuntu 20 AMD gpu
  • Ubuntu 20 Nvidia gpu
  • Windows 10 home AMD gpu
  • Windows 10 home Nvidia gpu

Overview

go-spacemesh uses several dynamic libraries which are platform-specific. These libs needs to be linked by go-spacemesh in releases, testnets and ci.

Scenarios

  1. Developer - Running a single go-spacemesh node built locally from source code
  2. Developer - Running local-net via terminal
  3. Developer - Running go-spacemesh tests on local machine from cloned repo
  4. Automation - Running go-spacemesh tests in automation via GHA
  5. Automation - Building go-spacemesh releases
  6. Automation - Building Smapp releases
  7. Developer - Running go-spacemesh managed nodes in a testnet.
  8. User: Running go-spacemesh via terminal
  9. User: Running go-spacemesh via Smapp (or any other client which manages a node)

Platforms

  1. Windows 10
  2. macOS
  3. Linux (Ubuntu or other)

Scenario 1: Developer: running a single go-spacemesh built locally from source code

Solution: see working instructions at top of this doc.

Scenario 2: Developer - Running local-net via terminal

Localnet nodes run in a linux docker container without access to the host system gpu drivers apis.
The task here is to configure each node to link with the linux .so lib so they can generate post using cpu provider.

Scenario 3: Developer - Running go-spacemesh tests on local machine from cloned repo

Task: we need to find a setup that will enable running gpu-post integration tests in go-spacemesh.

Scenario 4: Developer - Running go-spacemesh tests in automation via GHA

Task: configure go-sm integration tests to link with libgpu-post.so so cpu provider can be used.

Scenario 5: Automation - Building go-spacemesh releases

  1. Modify the build process to add the artifacts from a gpu-post release (configured to be used in the go-spacemesh repo) so each release for one of the 3 platforms is a zip file which contains the go-sm binary for that platform as well the gpu-post artifacts for that platform.
  2. Use the modified make file for go-spacemesh which properly set the rpath to loader path

Scenario 6: 6. Automation - Building Smapp releases

Task: modify smapp release code to add all the gpu-post artifacts for each of the 3 platforms to the go-spacemesh directory on user's machine after install.

Scenario 7: Developer - Running go-spacemesh managed nodes in a testnet

Task: list here required changes to Spacecraft to support libs linking.

  • Go-spacemesh process working directory must be the executable file directory.
  • All artifacts must be in go-spacemesh root folder.

Scenario 8. User: Running go-spacemesh via terminal

  1. Download zip file from a go-spacemesh github release for one of the supported platforms and extract all to a directory.
  2. Run go-spacemesh via terminal from its directory (not from another directory).

The directory will have the artifacts for the user's platform so user doesn't need to do anything to get gpu-post to work.

Scenario 9. User: Running go-spacemesh via Smapp (or any other client which manages a node)

  1. Install Smapp as usual - no changes needed.

Smapp changes:

  • Smapp installer needs to copy the artifacts for the user's platform to the go-spacemesh directory so it will be able to link to them.

  • Smapp MUST set go-spacemesh process directory to the go-spacemesh executable directory if it launches it from a working directory different than the exe directory.

  • As an alternative, set VK_ICD_FILENAMES env VAR to the go-spacemesh exe directory and start it from any working dir.


Working Directory

  • Go-spacemesh process working folder MUST be the same as the exe folder and not any other.

  • As an alternative, VK_ICD_FILENAMES ENV VAR can be set to the go-spacemesh exe directory.

Go-spacemesh build changes

  1. Modify all go-spacemesh build script in MAKE files (in repo and in automation) to add the the executable directory to rpath embedded in the go-spacemesh executable. This enables go-spacemesh to dynamically link with libraries at execution time.

macOS

install_name_tool -add_rpath @loader_path/ go-spacemesh

Linux

Use patchelf

/patchelf --set-rpath . go-spacemesh

Windows

Q: is this needed? what's the cli to do this?


About MoltenVK_icd.json

  • It must be in the go-spacemesh directory for Vulkan providers to work properly.
  • The file specifies to the system the path of the Vulkan runtime lib.
  • The file will be in the artifacts of gpu-post for each platform.
@avive avive changed the title Packaging and linking platform specific libs with go-spacemesh (WIP) Lining platform-specific C libs to go-spacemesh (WIP) Jun 9, 2021
@avive avive changed the title Lining platform-specific C libs to go-spacemesh (WIP) Linking platform-specific C libs to go-spacemesh (WIP) Jun 10, 2021
@avive
Copy link
Contributor Author

avive commented Jun 10, 2021

Progress on dev setup using the instructions at top of this smip:

@avive avive changed the title Linking platform-specific C libs to go-spacemesh (WIP) Using the POST go lib in go-spacemesh (WIP) Jun 11, 2021
@sudachen
Copy link

For me it works on Linux/Nvidia, Linux-Docker/Nvida, Windows/Nvidia/UHD.

@avive
Copy link
Contributor Author

avive commented Jun 28, 2021

For me it works on Linux/Nvidia, Linux-Docker/Nvida, Windows/Nvidia/UHD.

if you share your tests flow i can try on all platforms for amd/vulkan - we have the hardware to test.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
No open projects
Development

No branches or pull requests

2 participants