silentarmy shared library with python CFFI #52
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I will consider merging something like this, yes. But I really like the fact that the solver instances are separate processes. Because if they crash (due to eg. flaky hardware) they do not crash the whole silentarmy process. But it seems with your work, we could still fork(), and have multiple python "solver" processes that hook into the library. So yes I would consider merging something like this. |
|
I actually run the library in separate processes for each solver instance for the exact reasons that you have stated. I created pyzcm - will publish it soon - that uses silentarmy in the library version. |
- move common declarations to silentarmy.h - add parameter to solve_equihash to skip solution verification when needed
- add declaration of ocl_code to silentarmy.h
- add SConstruct that is able to build both variants of the software - the CLI testing version along with shared library - use SHARED_LIB define to remove unnecessary code from main.c
…xtract_valid_solutions()
- move the common code into check_header_zero_pad function - use it in gpu_solver__find_sols
- add missing cast to prevent ffi warning since ffi.from_buffer() returns char[]. However, the block header is of uint8_t[] (unsigned char[]) type
- rely on system library search paths
…on encoding round - move all_sols into gpu_solver instance as all_found_sols - refactor all code as necessary
- refactor gpu_solver__{new,init}() accordingly
- update CFFI binding
- prevent calling the destructor when initialization must have previously failed
…the device to operate on - adjust python CFFI interface accordingly
…robust - optionally import an independent solver (used for validation) - consolidate test functionality into a function
janbraiins
force-pushed
the
library-2
branch
from
bb0e176
to
2687205
Compare
- move actual build targets into SConscript.{solver,lib} -> this has
simplified variant_dir handling (no need for get_sources function)
- move OpenCL kernel generator to each environment
- add OpenCL header and library detection (autoconf like). Paths to
each are command line configurable if necessary
- add command line option to enable windows mingw build
- enable use of mingw's printf instead of ms_printf - reasons
explained in SConscruct comment
- disable parts that are not directly supported by mingw. This affects only the sa-solver target that is not being compiled for windows anyway.
- this fixes problems when the python package is installed as an egg. Since egg's are zipped, we would NOT have proper access to the shared library for dlopen(). pkg_resources API abstracts this
- add more logging - detect system name a select the appropriate library name
…command - change empty array into an empty string as scons options are passed as a comma separated string. This way the split will work properly again.
- get_target_system_shared_library_name() that maps the --plat-name option into a target system specific name. Unfortunately, python package cross building support seems very limitted as otherwise it should be able to detect the libraries in case we use a custom build system. - fix incorrect specification of eager_resources - this is needed to support library access from an .egg - package_data adjusted with correct library name
- pass each environment in SConstruct via a dictionary. This creates an alias that can be imported in the SConscript - adjust solver and library SConscripts accordingly
- this allows using the pysa module from setup.py to map the correct shared library name without requiring other dependencies of the final python package
- remove LD variable as the correct variable name is 'LINK'. However, no override is needed since SMARTLINK handles the cross compiling build correctly.
…at triggers scons build
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi Marc,
I did some consolidation on the code to convert it into a standalone shared library, while maintaining the functionality in sa-solver in place. Let me know if you would consider to merge it.
I have started 1 week ago and I see you have made a big progress with the code. However, I believe the current version is broken in terms of finding proper solutions. I will open an issue for this.