Fix local build and install #620
Conversation
|
|
| # The mode determines how the stix-shifter is packaged | ||
| # 1 = Include everything in 1 whl package | ||
| # 3 - 3 whl packages respectively for stix-shifter, stix-shifter-utils and stix-shifter-modules | ||
| # N - stix-shifter, stix-shifter-utils, and each connector is packaged separately | ||
| # <module name> - package only the specified connector | ||
|
|
||
| mode = 'N' | ||
| mode = '3' |
There was a problem hiding this comment.
you should not change the default packaging mode, why are you using mode 3 at all? maybe mode 1 will fit you better?
There was a problem hiding this comment.
Agreed. Changing the default mode might be too much depending on the workflow. For the distribution purpose for developers (python setup.py bdist_wheel), it would be good to have separate packages.
But, for local build/install for users (python setup.py install), mode 3 or 1 is needed as the installed packages under site-packages will have the mix of egg zipfiles and directories for each stix_shifter_module, which will cause importing errors. For example, after python setup.py install (with mode N), importing qradar module (import stix_shifter_modules.qradar) from outside of the stix-shifter root directory will cause ModuleNotFoundError due to above issues.
There was a problem hiding this comment.
By the way, MODE 1/3 bdist/install fails due to incorrect path resolution for stix_shifter_modules. Proper fill_connectors is needed to support MODE 1/3.
There was a problem hiding this comment.
@mdazam1942 is looking into it.
|
Created another PR with the changes from jiyongj:local_build into branch opencybersecurityalliance:local-build. |
|
closing this PR since the the same commits are merged through a updated #779 |
Build environment: Python 3.6.9 on Ubuntu 18.04
Issue: Local build and install of
stix-shifterdidn't work properly as importingstix-shifter-modules.*failed after followingpython3 setup.py install.stix-shifter-modulesgenerated.eggzipfiles while others generated.eggdirectories. Missingzip-safeflag resulted in inconsistent module installation (https://setuptools.readthedocs.io/en/latest/deprecated/python_eggs.html#zip-safe-and-not-zip-safe), and failure of importing native namespace packages (https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package).zip-safeflag toFalse.eggdirectories caused incorrect path resolution at importing. For example, after installation, according to the order in the path (e.g.,easy-install.pth), allstix-shifter-modules.*imports look for the first one in the path (e.g.,./stix_shifter_modules_alertflex-1.0.0-py3.6.egg) regardless of their actual locations.3of installation is needed to have a proper installation structure. Change the default installation mode to3and fix the function to fill the connectors properly