Package version conflict encountered in requirements.txt.

[prasanthbalaji]

Information

• Onefuzz version: 4.0.0
• OS: Ubuntu 20.04
• Python version: 3.8.10
• pip version: 20.0.2

Provide detailed reproduction steps (if any)

Issues encountered while setting up requirements to build onefuzz cli locally.
(onefuzz-4.0.0/src/cli/requirements.txt)

1. pip3 install -r requirements.txt

Expected result

Successfully install all pre-requisites utils mentioned in requirements.txt

Actual result

Collecting msal~=1.14.0
Downloading msal-1.14.0-py2.py3-none-any.whl (75 kB)
|████████████████████████████████| 75 kB 1.9 MB/s
Collecting requests~=2.25.1
Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 3.3 MB/s
Collecting jmespath~=0.10.0
Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting semver~=2.13.0
Downloading semver-2.13.0-py2.py3-none-any.whl (12 kB)
Collecting signalrcore==0.9.2
Downloading signalrcore-0.9.2-py3-none-any.whl (34 kB)
Collecting asciimatics~=1.13.0
Downloading asciimatics-1.13.0-py2.py3-none-any.whl (158 kB)
|████████████████████████████████| 158 kB 2.8 MB/s
Collecting pydantic~=1.8.1
Downloading pydantic-1.8.2.tar.gz (263 kB)
|████████████████████████████████| 263 kB 2.7 MB/s
Collecting memoization~=0.4.0
Downloading memoization-0.4.0.tar.gz (41 kB)
|████████████████████████████████| 41 kB 473 kB/s
Collecting msrestazure==0.6.4
Downloading msrestazure-0.6.4-py2.py3-none-any.whl (40 kB)
|████████████████████████████████| 40 kB 3.0 MB/s
Collecting azure-storage-blob~=12.8
Downloading azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
|████████████████████████████████| 356 kB 2.6 MB/s
Collecting azure-applicationinsights==0.1.0
Downloading azure_applicationinsights-0.1.0-py2.py3-none-any.whl (104 kB)
|████████████████████████████████| 104 kB 4.6 MB/s
Collecting tenacity==8.0.1
Downloading tenacity-8.0.1-py3-none-any.whl (24 kB)
Collecting docstring_parser==0.8.1
Downloading docstring_parser-0.8.1.tar.gz (14 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting azure-cli-core==2.27.2
Downloading azure_cli_core-2.27.2-py3-none-any.whl (179 kB)
|████████████████████████████████| 179 kB 2.2 MB/s
Collecting packaging==20.9
Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
|████████████████████████████████| 40 kB 3.7 MB/s
Collecting urllib3[secure]>=1.26.5
Downloading urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
|████████████████████████████████| 138 kB 4.2 MB/s
Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 20)) (2.8)
Collecting cryptography<3.4,>=3.2
Downloading cryptography-3.3.2-cp36-abi3-manylinux2010_x86_64.whl (2.6 MB)
|████████████████████████████████| 2.6 MB 2.6 MB/s
Collecting PyJWT>=2.1.0
Downloading PyJWT-2.3.0-py3-none-any.whl (16 kB)
Collecting onefuzztypes==4.0.0
Downloading onefuzztypes-4.0.0-py3-none-any.whl (23 kB)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests~=2.25.1->-r requirements.txt (line 2)) (2019.11.28)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/lib/python3/dist-packages (from requests~=2.25.1->-r requirements.txt (line 2)) (3.0.4)
Collecting websocket-client==0.54.0
Downloading websocket_client-0.54.0-py2.py3-none-any.whl (200 kB)
|████████████████████████████████| 200 kB 3.9 MB/s
Collecting msgpack==1.0.2
Downloading msgpack-1.0.2-cp38-cp38-manylinux1_x86_64.whl (302 kB)
|████████████████████████████████| 302 kB 4.0 MB/s
Collecting future
Downloading future-0.18.2.tar.gz (829 kB)
|████████████████████████████████| 829 kB 3.0 MB/s
Collecting Pillow>=2.7.0
Downloading Pillow-9.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
|████████████████████████████████| 4.3 MB 2.8 MB/s
Collecting pyfiglet>=0.7.2
Downloading pyfiglet-0.8.post1-py2.py3-none-any.whl (865 kB)
|████████████████████████████████| 865 kB 3.7 MB/s
Collecting wcwidth
Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting typing-extensions>=3.7.4.3
Downloading typing_extensions-4.0.1-py3-none-any.whl (22 kB)
Collecting adal<2.0.0,>=0.6.0
Downloading adal-1.2.7-py2.py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 2.6 MB/s
Collecting msrest<2.0.0,>=0.6.0
Downloading msrest-0.6.21-py2.py3-none-any.whl (85 kB)
|████████████████████████████████| 85 kB 3.1 MB/s
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from msrestazure==0.6.4->-r requirements.txt (line 9)) (1.14.0)
Collecting azure-core<2.0.0,>=1.10.0
Downloading azure_core-1.22.0-py3-none-any.whl (178 kB)
|████████████████████████████████| 178 kB 4.1 MB/s
Collecting azure-common~=1.1
Downloading azure_common-1.1.28-py2.py3-none-any.whl (14 kB)
Collecting azure-nspkg>=2.0.0
Downloading azure_nspkg-3.0.2-py3-none-any.whl (1.5 kB)
Collecting argcomplete~=1.8
Downloading argcomplete-1.12.3-py2.py3-none-any.whl (38 kB)
Collecting azure-mgmt-core<1.3.0,>=1.2.0
Downloading azure_mgmt_core-1.2.2-py2.py3-none-any.whl (21 kB)
Collecting pkginfo>=1.5.0.1
Downloading pkginfo-1.8.2-py2.py3-none-any.whl (26 kB)
Collecting azure-cli-telemetry==1.0.6.*
Downloading azure_cli_telemetry-1.0.6-py3-none-any.whl (10 kB)
Collecting psutil~=5.8
Downloading psutil-5.9.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (283 kB)
|████████████████████████████████| 283 kB 3.6 MB/s
Collecting knack~=0.8.2
Downloading knack-0.8.2-py3-none-any.whl (59 kB)
|████████████████████████████████| 59 kB 2.9 MB/s
Collecting pyopenssl>=17.1.0
Downloading pyOpenSSL-22.0.0-py2.py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 1.6 MB/s
Collecting paramiko<3.0.0,>=2.0.8
Downloading paramiko-2.9.2-py2.py3-none-any.whl (210 kB)
|████████████████████████████████| 210 kB 3.3 MB/s
Collecting humanfriendly<10.0,>=4.7
Downloading humanfriendly-9.2-py2.py3-none-any.whl (86 kB)
|████████████████████████████████| 86 kB 3.5 MB/s
Collecting pyparsing>=2.0.2
Downloading pyparsing-3.0.7-py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 4.1 MB/s
Collecting cffi>=1.12
Downloading cffi-1.15.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB)
|████████████████████████████████| 446 kB 2.5 MB/s
Collecting python-dateutil<3,>=2.1.0
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
|████████████████████████████████| 247 kB 3.6 MB/s
Collecting isodate>=0.6.0
Downloading isodate-0.6.1-py2.py3-none-any.whl (41 kB)
|████████████████████████████████| 41 kB 1.0 MB/s
Collecting requests-oauthlib>=0.5.0
Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting portalocker~=1.2
Downloading portalocker-1.7.1-py2.py3-none-any.whl (10 kB)
Collecting applicationinsights<0.12,>=0.11.1
Downloading applicationinsights-0.11.10-py2.py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 1.5 MB/s
Collecting tabulate
Downloading tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting pygments
Downloading Pygments-2.11.2-py3-none-any.whl (1.1 MB)
|████████████████████████████████| 1.1 MB 3.3 MB/s
Collecting colorama
Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting pyyaml
Downloading PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
|████████████████████████████████| 701 kB 4.2 MB/s
Collecting pynacl>=1.0.1
Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
|████████████████████████████████| 856 kB 2.2 MB/s
Collecting bcrypt>=3.1.3
Downloading bcrypt-3.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (61 kB)
|████████████████████████████████| 61 kB 789 kB/s
Collecting pycparser
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
|████████████████████████████████| 118 kB 3.1 MB/s
Collecting oauthlib>=3.0.0
Downloading oauthlib-3.2.0-py3-none-any.whl (151 kB)
|████████████████████████████████| 151 kB 3.2 MB/s
Skipping wheel build for pydantic, due to binaries being disabled for it.
Building wheels for collected packages: memoization, docstring-parser, future
Building wheel for memoization (setup.py) ... done
Created wheel for memoization: filename=memoization-0.4.0-py3-none-any.whl size=50455 sha256=50028efe77514fd172d25d07438041cd50a19abfceb2e44bf802db79d5ad1bdb
Stored in directory: /root/.cache/pip/wheels/9f/bc/ae/e76d5ea088be013f58051186f0f397398a98c5ca8342d35a4f
Building wheel for docstring-parser (PEP 517) ... done
Created wheel for docstring-parser: filename=docstring_parser-0.8.1-py3-none-any.whl size=19679 sha256=183a4093d6da4af50a33d5fa659c0b8a2e62559ab384ea89a5573a27fe7c3f84
Stored in directory: /root/.cache/pip/wheels/7a/06/c4/9b2f0146899e8d1e7748897e42ad5412b6a025513f89cc4a0f
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=1b6bfddeb2ba52d859304c71b04ea46b905330cd50e773833c7998829d9c2d31
Stored in directory: /root/.cache/pip/wheels/8e/70/28/3d6ccd6e315f65f245da085482a2e1c7d14b90b30f239e2cf4
Successfully built memoization docstring-parser future
ERROR: pyopenssl 22.0.0 has requirement cryptography>=35.0, but you'll have cryptography 3.3.2 which is incompatible.
Installing collected packages: pycparser, cffi, cryptography, PyJWT, pyopenssl, urllib3, requests, msal, jmespath, semver, websocket-client, msgpack, signalrcore, future, Pillow, pyfiglet, wcwidth, asciimatics, typing-extensions, pydantic, memoization, python-dateutil, adal, isodate, oauthlib, requests-oauthlib, msrest, msrestazure, azure-core, azure-storage-blob, azure-common, azure-nspkg, azure-applicationinsights, tenacity, docstring-parser, argcomplete, azure-mgmt-core, pkginfo, portalocker, applicationinsights, azure-cli-telemetry, psutil, tabulate, pygments, colorama, pyyaml, knack, pynacl, bcrypt, paramiko, humanfriendly, azure-cli-core, pyparsing, packaging, onefuzztypes
Attempting uninstall: urllib3
Found existing installation: urllib3 1.25.8
Not uninstalling urllib3 at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'urllib3'. No files were found to uninstall.
Attempting uninstall: requests
Found existing installation: requests 2.22.0
Not uninstalling requests at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'requests'. No files were found to uninstall.
Running setup.py install for pydantic ... done
Successfully installed Pillow-9.0.1 PyJWT-2.3.0 adal-1.2.7 applicationinsights-0.11.10 argcomplete-1.12.3 asciimatics-1.13.0 azure-applicationinsights-0.1.0 azure-cli-core-2.27.2 azure-cli-telemetry-1.0.6 azure-common-1.1.28 azure-core-1.22.0 azure-mgmt-core-1.2.2 azure-nspkg-3.0.2 azure-storage-blob-12.9.0 bcrypt-3.2.0 cffi-1.15.0 colorama-0.4.4 cryptography-3.3.2 docstring-parser-0.8.1 future-0.18.2 humanfriendly-9.2 isodate-0.6.1 jmespath-0.10.0 knack-0.8.2 memoization-0.4.0 msal-1.14.0 msgpack-1.0.2 msrest-0.6.21 msrestazure-0.6.4 oauthlib-3.2.0 onefuzztypes-4.0.0 packaging-20.9 paramiko-2.9.2 pkginfo-1.8.2 portalocker-1.7.1 psutil-5.9.0 pycparser-2.21 pydantic-1.8.2 pyfiglet-0.8.post1 pygments-2.11.2 pynacl-1.5.0 pyopenssl-22.0.0 pyparsing-3.0.7 python-dateutil-2.8.2 pyyaml-6.0 requests-2.25.1 requests-oauthlib-1.3.1 semver-2.13.0 signalrcore-0.9.2 tabulate-0.8.9 tenacity-8.0.1 typing-extensions-4.0.1 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-0.54.0

[stishkin]

Hi @prasanthbalaji

Can you try version 5 of OneFuzz CLI.

I had to repro an installation issue:
#1642 (comment)

And I had no problems.

[prasanthbalaji]

Hi @stishkin, I'm getting this pkg conflict while building onefuzz in a docker container.
Are there any documentation on how to build onefuzz locally.
I assume ...

1. get all necessary packages from /src/cli/requirements.txt
2. install setup.py /src/cli/setup.py install

Also noticed onefuzztypes==0.0.0 in requirements.txt

[stishkin]

oh
do you want to develop onefuzz in docker container or you just want to use it to deploy ?

If you just want to deploy and use OneFuzz then this is all you need to run:
python3 -m pip install onefuzz

You do not need GitHub repo checked out.

We publish our releases to pypi, so pip install will work: https://pypi.org/project/onefuzz/

[prasanthbalaji]

whl files are working fine for me. But what are the instructions to build onefuzz after downloading the source files.

[stishkin]

Are you trying to use docker container to build and develop OneFuzz ?

Can you elaborate on what you trying to achieve ?

[prasanthbalaji]

Yes, I'm tying to download onefuzz source files and build it in docker. We will use this onefuzz to submit fuzzing jobs.
Our plan is to add additional tracing and updates to onefuzz source to suite our product fuzz testing and automation.

[stishkin]

1. for job submission you can do the following in your docker
   pip install onefuzz
python -m onefuzz --help

You can develop using Python any fuzzing submission flow you like. See this documentation https://github.com/microsoft/onefuzz/blob/main/docs/getting-started.md#launching-a-job-from-the-sdk

2. If you are interested in deplolying your own version of OneFuzz with modifications to backend and fuzzing agents - that requires a full build. And for full buid we use Github Actions:
   https://github.com/microsoft/onefuzz/blob/main/.github/workflows/ci.yml

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.