Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OWTF develop branch install error in Kali Linux 2.0 #516

Closed
dkorzhevin opened this issue Sep 1, 2015 · 56 comments
Closed

OWTF develop branch install error in Kali Linux 2.0 #516

dkorzhevin opened this issue Sep 1, 2015 · 56 comments

Comments

@dkorzhevin
Copy link

Hi,

I'm using Kali Linux 2.0 x86_64

I tried to install OWTF develop branch using official install script:

wget https://raw.githubusercontent.com/owtf/bootstrap-script/master/bootstrap.sh
bash bootstrap.sh

and meet next errors:

  Found existing installation: cryptography 0.9.3
    Not uninstalling cryptography at /usr/lib/python2.7/dist-packages, owned by OS
  Running setup.py install for cryptography

    Installed /tmp/pip-build-ARN7u7/cryptography/cffi-1.2.1-py2.7-linux-x86_64.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-ARN7u7/cryptography/setup.py", line 307, in <module>
        **keywords_with_side_effects(sys.argv)
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 266, in __init__
        _Distribution.__init__(self,attrs)
      File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 301, in finalize_options
        ep.load()(self, ep.name, value)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
        ['__name__'])
    ImportError: No module named setuptools_ext
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-ARN7u7/cryptography/setup.py';exec(compile(g
etattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8OnFdz-record/install-record.txt 
--single-version-externally-managed --compile:

Installed /tmp/pip-build-ARN7u7/cryptography/cffi-1.2.1-py2.7-linux-x86_64.egg

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip-build-ARN7u7/cryptography/setup.py", line 307, in <module>

    **keywords_with_side_effects(sys.argv)

  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup

    _setup_distribution = dist = klass(attrs)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 266, in __init__

    _Distribution.__init__(self,attrs)

  File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__

    self.finalize_options()

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 301, in finalize_options

    ep.load()(self, ep.name, value)

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load

    ['__name__'])

ImportError: No module named setuptools_ext

----------------------------------------

  Can't roll back cryptography; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-ARN7u7/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', 
open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8OnFdz-record/install-record.txt --single-version-external
ly-managed --compile failed with error code 1 in /tmp/pip-build-ARN7u7/cryptography
Traceback (most recent call last):
  File "/usr/bin/pip2", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip2')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 248, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 62: ordinal not in range(128)
[*] Finished
#? 1) OWTF 1.0.1 Lionheart       3) OWTF develop branch
2) OWTF master branch    4) Quit
#?
@dkorzhevin
Copy link
Author

Same, when I tried to install OWTF 1.0.1 Lionheart:

Downloading/unpacking pycparser from https://pypi.python.org/packages/source/p/pycparser/pycparser-2.14.tar.gz#md5=a2bc8d28c923b4fe2b2c3b4b51a4f935 (f
rom cffi>=1.1.0->cryptography>=0.2.1->pyOpenSSL==0.14->-r /root/tools/owtf/install/owtf.pip (line 16))
  Downloading pycparser-2.14.tar.gz (223kB): 223kB downloaded
  Running setup.py (path:/tmp/pip-build-FYnPy6/pycparser/setup.py) egg_info for package pycparser

    warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
    warning: no previously-included files matching 'lextab.*' found under directory 'tests'
    warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
    warning: no previously-included files matching 'lextab.*' found under directory 'examples'
Installing collected packages: cryptography, EasyProcess, isodate, SPARQLWrapper, html5lib, backports.ssl-match-hostname, pyasn1, setuptools, enum34, 
cffi, pycparser
  Found existing installation: cryptography 0.9.3
    Not uninstalling cryptography at /usr/lib/python2.7/dist-packages, owned by OS
  Running setup.py install for cryptography

    Installed /tmp/pip-build-FYnPy6/cryptography/cffi-1.2.1-py2.7-linux-x86_64.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-FYnPy6/cryptography/setup.py", line 307, in <module>
        **keywords_with_side_effects(sys.argv)
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 266, in __init__
        _Distribution.__init__(self,attrs)
      File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 301, in finalize_options
        ep.load()(self, ep.name, value)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
        ['__name__'])
    ImportError: No module named setuptools_ext
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-FYnPy6/cryptography/setup.py';exec(compile(g
etattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-VwF3qT-record/install-record.txt 
--single-version-externally-managed --compile:


Installed /tmp/pip-build-FYnPy6/cryptography/cffi-1.2.1-py2.7-linux-x86_64.egg

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip-build-FYnPy6/cryptography/setup.py", line 307, in <module>

    **keywords_with_side_effects(sys.argv)

  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup

    _setup_distribution = dist = klass(attrs)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 266, in __init__

    _Distribution.__init__(self,attrs)

  File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__

    self.finalize_options()

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 301, in finalize_options

    ep.load()(self, ep.name, value)

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load

    ['__name__'])

ImportError: No module named setuptools_ext

----------------------------------------
----------------------------------------
  Can't roll back cryptography; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-FYnPy6/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-VwF3qT-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-FYnPy6/cryptography
Storing debug log for failure in /root/.pip/pip.log
[*] Finished
#?

@dkorzhevin
Copy link
Author

When trying to start:

# ./owtf.py 
The following python libraries seem missing : 
   beautifulsoup4,lxml,Markdown,psycopg2,pycurl,six

Haven't you run the install script? 
   python2 install/install.py

If you are sure you ran the install script, install the missing libraries seperately
   pip install --upgrade -r install/owtf.pip

@viyatb
Copy link
Member

viyatb commented Sep 1, 2015

Hi @dkorzhevin, we are aware of installation issues in some Kali 2.0 images. The Kali people probably changed the permissions on usr/lib/python2.7/dist-packages/ dir so that the bundled tools do not break because of the dependencies. We are working on a solution; it should be available soon.

BTW which image did you used? I personally used the Kali 2.0 Virtualbox image x86_64 and faced no problems.

@dkorzhevin
Copy link
Author

Hi delta24

This is Kali Linux x86_64 installed on my notebook, not a virtualbox or vmware image.

If you need any info regarding installed packages, etc - please let me know - I will add more info.

@dkorzhevin
Copy link
Author

More info about system:

uname -a
Linux kali 4.0.0-kali1-amd64 #1 SMP Debian 4.0.4-1+kali2 (2015-06-03) x86_64 GNU/Linux

cat /etc/issue
Kali GNU/Linux 2.0 \n \l

List of installed packages:

https://dpaste.de/774A

@DoomTaper
Copy link
Contributor

I got this error and found out that it's because the command in install.py "sudo pip2 install --upgrade pip" will not upgrade pip and will display something like "Not uninstalling pip at /usr/lib/python2.7/dist-packages, owned by OS" in KALI 2.0 so instead we should use the command "sudo pip2 install --upgrade --force pip".

DoomTaper pushed a commit to DoomTaper/owtf that referenced this issue Sep 2, 2015
DoomTaper pushed a commit to DoomTaper/owtf that referenced this issue Sep 2, 2015
DoomTaper pushed a commit to DoomTaper/owtf that referenced this issue Sep 2, 2015
@7a
Copy link
Member

7a commented Sep 3, 2015

@dkorzhevin: Could you please try what Ankush is suggesting and let us know
how that works? Thank you!
On 2 Sep 2015 20:42, "Ayush Singh" notifications@github.com wrote:

I got this error and found out that it's because the command in install.py "sudo
pip2 install --upgrade pip" will not upgrade pip and will show an error
something like "Not uninstalling pip at
/usr/lib/python2.7/dist-packages, owned by OS"
in kali 2.0 so instead we
should use the command "sudo pip2 install --upgrade --force pip".


Reply to this email directly or view it on GitHub
#516 (comment).

@dkorzhevin
Copy link
Author

# grep -i -R "pip2 install" owtf/
owtf/install/install.py:        self.run_command("sudo -E pip2 install --upgrade -r %s"%(requirements_file))

I changed line:

sudo -E pip2 install --upgrade -r %s"%(requirements_file))

to:

sudo -E pip2 install --upgrade --force -r %s"%(requirements_file))

but this doesn't help me:

----------------------------------------
  Can't roll back cryptography; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-istbYR/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-ZxOfrq-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-istbYR/cryptography
Traceback (most recent call last):
  File "/usr/bin/pip2", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip2')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 248, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 62: ordinal not in range(128)
[*] Finished

Maby I should reinstall from scratch?

@dkorzhevin
Copy link
Author

I did:

# sudo pip2 install --upgrade --force pip
Downloading/unpacking pip
  Downloading pip-7.1.2-py2.py3-none-any.whl (1.1MB): 1.1MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Not uninstalling pip at /usr/lib/python2.7/dist-packages, owned by OS
Successfully installed pip
Cleaning up...

Full output of install after this command:

https://dpaste.de/VOHA

@dkorzhevin
Copy link
Author

root@kali:~/tools/owtf# ./owtf.py http://demo.testfire.net
Traceback (most recent call last):
  File "./owtf.py", line 41, in <module>
    verify_dependencies(os.path.dirname(os.path.abspath(sys.argv[0])) or '.')
  File "/root/tools/owtf/framework/dependency_check.py", line 57, in verify_dependencies
    owtf_libraries = [req.req.project_name for req in owtf_reqs]
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_file.py", line 72, in parse_requirements
    "parse_requirements() missing 1 required keyword argument: "
TypeError: parse_requirements() missing 1 required keyword argument: 'session'
root@kali:~/tools/owtf#

@dkorzhevin
Copy link
Author

Guys, can you please tell - are you have any update about this problem?

@flabbergastedbd
Copy link
Contributor

Hey @dkorzhevin Are you trying develop branch?

@DoomTaper
Copy link
Contributor

@dkorzhevin have you tried easy_install --upgrade pip ?

@dkorzhevin
Copy link
Author

Now I'm trying to install version 1.0.1

Upgraded pip:

# easy_install --upgrade pip
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 7.1.2
Downloading https://pypi.python.org/packages/source/p/pip/pip-7.1.2.tar.gz#md5=3823d2343d9f3aaab21cf9c917710196
Processing pip-7.1.2.tar.gz
Writing /tmp/easy_install-zQvi8G/pip-7.1.2/setup.cfg
Running pip-7.1.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-zQvi8G/pip-7.1.2/egg-dist-tmp-8E6APR
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching 'pip/_vendor/Makefile'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching 'dev-requirements.txt'
no previously-included directories found matching '.travis'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'contrib'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
Adding pip 7.1.2 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/pip-7.1.2-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

@dkorzhevin
Copy link
Author

After upgrading pip I ended with:

Building wheels for collected packages: cryptography, EasyProcess, isodate, SPARQLWrapper, html5lib, backports.ssl-match-hostname, enum34, cffi, pycparser
  Running setup.py bdist_wheel for cryptography
  Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-0EqWhn/cryptography/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_whe
el -d /tmp/tmpm9u64epip-wheel-:

  Installed /tmp/pip-build-0EqWhn/cryptography/cffi-1.2.1-py2.7-linux-x86_64.egg
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-0EqWhn/cryptography/setup.py", line 313, in <module>
      **keywords_with_side_effects(sys.argv)
    File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
      _setup_distribution = dist = klass(attrs)
    File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 266, in __init__
      _Distribution.__init__(self,attrs)
    File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
      self.finalize_options()
    File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 301, in finalize_options
      ep.load()(self, ep.name, value)
    File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
      ['__name__'])
  ImportError: No module named setuptools_ext

  ----------------------------------------
  Failed building wheel for cryptography
  Running setup.py bdist_wheel for EasyProcess
  Stored in directory: /root/.cache/pip/wheels/90/c5/e9/73acf9e60eafa1f24facd100c0113a646a23de27a010deda3b
  Running setup.py bdist_wheel for isodate
  Stored in directory: /root/.cache/pip/wheels/8a/b6/2c/9346712a1822e562d16c4d6a5ae242ebf45128a8cb2e7b4abe
  Running setup.py bdist_wheel for SPARQLWrapper
  Stored in directory: /root/.cache/pip/wheels/e4/87/7a/6ff49cbe7b5483dd8725c093e91abf31839f9b73b680ec275d
  Running setup.py bdist_wheel for html5lib
  Stored in directory: /root/.cache/pip/wheels/e1/62/7e/25731b7df4b5b564f5dc89ba14e220c238dd8b115ffea7b4b6
  Running setup.py bdist_wheel for backports.ssl-match-hostname
  Stored in directory: /root/.cache/pip/wheels/81/3d/36/f3c726a42caed585046d710e0e450bda1075c61c1c43443831
  Running setup.py bdist_wheel for enum34
  Stored in directory: /root/.cache/pip/wheels/d1/b2/2c/b51348698bd1921a226cf48d790b282d86fb4bcf9728afd6b3
  Running setup.py bdist_wheel for cffi
  Stored in directory: /root/.cache/pip/wheels/fa/ad/ab/28df6da3b6cf3c510c6dc64f336ef17ef37cd1b0d4d5eff3bc
  Running setup.py bdist_wheel for pycparser
  Stored in directory: /root/.cache/pip/wheels/c7/28/31/bac6d0b118c0bdcbf57f9219afdf2e624379c07efa6c769dbc
Successfully built EasyProcess isodate SPARQLWrapper html5lib backports.ssl-match-hostname enum34 cffi pycparser
Failed to build cryptography
Installing collected packages: pyasn1, setuptools, enum34, pycparser, cffi, cryptography, EasyProcess, isodate, SPARQLWrapper, html5lib, certifi, backports.ssl-match-hostname
  Found existing installation: pyasn1 0.1.7
    Uninstalling pyasn1-0.1.7:
      Successfully uninstalled pyasn1-0.1.7
  Found existing installation: setuptools 5.5.1
    Uninstalling setuptools-5.5.1:
      Successfully uninstalled setuptools-5.5.1
  Found existing installation: enum34 1.0.3
    DEPRECATION: Uninstalling a distutils installed project (enum34) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling enum34-1.0.3:
      Successfully uninstalled enum34-1.0.3
  Found existing installation: pycparser 2.10
    DEPRECATION: Uninstalling a distutils installed project (pycparser) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling pycparser-2.10:
      Successfully uninstalled pycparser-2.10
  Found existing installation: cffi 0.8.6
    Uninstalling cffi-0.8.6:
      Successfully uninstalled cffi-0.8.6
  Found existing installation: cryptography 0.9.3
    Can't uninstall 'cryptography'. No files were found to uninstall.
  Running setup.py install for cryptography
  Found existing installation: html5lib 0.999
    Uninstalling html5lib-0.999:
      Successfully uninstalled html5lib-0.999
  Found existing installation: certifi 2015.4.28
    Uninstalling certifi-2015.4.28:
      Successfully uninstalled certifi-2015.4.28
Successfully installed EasyProcess-0.1.9 SPARQLWrapper-1.6.4 backports.ssl-match-hostname-3.4.0.2 certifi-2015.9.6.2 cffi-1.2.1 cryptography-1.0.2 enum34-1.0.4 html5lib-0.9999999 isodate-0.5.4 pyasn1-0.1.9 pycparser-2.14 setuptools-18.3.2
[*] Finished
#?

@dkorzhevin
Copy link
Author

After this steps OWTF still doesn't works:

# ./owtf.py -l web
Traceback (most recent call last):
  File "./owtf.py", line 41, in <module>
    verify_dependencies(os.path.dirname(os.path.abspath(sys.argv[0])) or '.')
  File "/root/tools/owtf/framework/dependency_check.py", line 57, in verify_dependencies
    owtf_libraries = [req.req.project_name for req in owtf_reqs]
  File "/usr/local/lib/python2.7/dist-packages/pip-7.1.2-py2.7.egg/pip/req/req_file.py", line 72, in parse_requirements
    "parse_requirements() missing 1 required keyword argument: "
TypeError: parse_requirements() missing 1 required keyword argument: 'session'

@DoomTaper
Copy link
Contributor

@dkorzhevin have you done sudo apt-get update and sudo apt-get upgrade? If you are still getting ImportError: No module named setuptools_ext then try to upgrade cffi by using pip and rerun the install script

@viyatb
Copy link
Member

viyatb commented Oct 21, 2015

@dkorzhevin Until the complete install script is ready with the virtualenv stuff, you can follow these steps to install OWTF successfully.
You need to have pip installed.

# pip install virtualenv virtualenvwrapper
# source '/usr/local/bin/virtualenvwrapper.sh'
# echo 'source /usr/local/bin/virtualenvwrapper.sh' >> ~/.zshrc (or ~/.bashrc if use bash)
# mkvirtualenv owtf -p /usr/bin/python2.7

You will see something like this: (owtf)~/owtf/install %
This says a virtualenv named "owtf" is active. You can now run the install script.
You can deactivate the env by running: deactivate

But whenever you want to run OWTF, just cd into the directory and run: workon <env>
Like for the example above, workon owtf.

This should do as a temporary solution. :)

@dkorzhevin
Copy link
Author

Thank you

I will try and let you know how it works

@maximumgr
Copy link

To fix issue use: easy_install --upgrade cryptography

And after that install other missing python libraries, with the same command (easy_install --upgrade)

@viyatb
Copy link
Member

viyatb commented Dec 7, 2015

Hey @dkorzhevin, recent commits resolve all the installation errors (at least on lastest updated Kali 2.x)
fb72333
7ac6d90

Can you try the installing again by git clone https://github.com/owtf/owtf and running python install/install.py?

I'll reopen the issue if the installation still fails from your side. :)

@viyatb viyatb closed this as completed Dec 9, 2015
@dkorzhevin
Copy link
Author

Thank you, delta24 - I will try

@dkorzhevin
Copy link
Author

Hi

I can confirm, it works for me now:

Successfully installed EasyProcess-0.2.1 PTP-0.3.0 SPARQLWrapper-1.7.6 backports.ssl-match-hostname-3.5.0.1 certifi-2015.11.20.1 cryptography-1.1.2 enum34-1.1.2 ipaddress-1.0.15 keepalive-0.5 pyparsing-2.0.6 s
etuptools-18.7.1
[*] Running following command
sudo sh /root/owtf/scripts/db_setup.sh init

------------------------- OWTF Database Helper Script -------------------------
                    Helps in creation of user and database                     
-------------------------------------------------------------------------------

[+] Ensure that you have required values in /root/.owtf/db.cfg

Press Enter to continue

CREATE ROLE
CREATE DATABASE
[*] Running following command
sudo sh /root/owtf/scripts/db_run.sh
[*] Finished!
[*] Start OWTF by running ./owtf.py in the parent directory

@viyatb
Copy link
Member

viyatb commented Jun 18, 2016

@darelswag @h121h so I investigated and I found:

  • Errors like Not uninstalling pyparsing at /usr/lib/python2.7/dist-packages, outside environment /usr, is because in Kali rolling, the default path for pip install is /usr/lib/python2.7/dist-packages and not /usr/lib/python2.7/site-packages. As far as I understand, this is to separate the OS python packages (using apt) and user installed packages so that user packages do not break the system.

  • The fix is to use the pip's --user option which installs the packages specific to the user and not the system.

    When globally installed packages are on the python path, and they conflict with the installation requirements, they are ignored, and not uninstalled. --> hence the errors.

So the fix is:
After running python install/install.py, do this:
pip install --user --upgrade -r install/owtf.pip.

Some more info: http://matthew-brett.github.io/pydagogue/installing_on_debian.html, https://pip.readthedocs.io/en/latest/user_guide/#user-installs

Let me know if it helps. :-)

@h121h
Copy link

h121h commented Jun 18, 2016

@delta24, thanks for investigating this. I followed your suggestions, but I am still getting the same output as before (posted below.) It looks to me adding --user still doesn't use /usr/lib/python2.7/site-packages as mentioned.

Output:

pip install --user --upgrade -r install/owtf.pip
Requirement already up-to-date: beautifulsoup4==4.4.1 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 6))
Requirement already up-to-date: cookies==2.2.1 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 7))
Requirement already up-to-date: ipaddr==2.1.11 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 8))
Requirement already up-to-date: lxml==3.6.0 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 9))
Requirement already up-to-date: Markdown==2.6.6 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 10))
Requirement already up-to-date: pexpect==4.1.0 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 11))
Requirement already up-to-date: psycopg2==2.6.1 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 12))
Requirement already up-to-date: PTP==0.3.0 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 13))
Requirement already up-to-date: pycurl==7.43.0 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 14))
Requirement already up-to-date: six==1.10.0 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 16))
Requirement already up-to-date: pyOpenSSL==16 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 17))
Requirement already up-to-date: python-owasp-zap-v2==0.0.9 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 18))
Requirement already up-to-date: PyVirtualDisplay==0.2 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 19))
Requirement already up-to-date: rdflib==4.2.1 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 20))
Requirement already up-to-date: selenium==2.53.5 in /usr/local/lib/python2.7/dist-packages (from -r install/owtf.pip (line 21))
Requirement already up-to-date: SQLAlchemy==1.0.13 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 22))
Requirement already up-to-date: tornado==4.3.0 in /usr/lib/python2.7/dist-packages (from -r install/owtf.pip (line 23))
Requirement already up-to-date: ptyprocess>=0.5 in /usr/lib/python2.7/dist-packages (from pexpect==4.1.0->-r install/owtf.pip (line 11))
Requirement already up-to-date: EasyProcess in /usr/local/lib/python2.7/dist-packages (from PyVirtualDisplay==0.2->-r install/owtf.pip (line 19))
Requirement already up-to-date: isodate in /usr/local/lib/python2.7/dist-packages (from rdflib==4.2.1->-r install/owtf.pip (line 20))
Requirement already up-to-date: pyparsing in /usr/local/lib/python2.7/dist-packages (from rdflib==4.2.1->-r install/owtf.pip (line 20))
Requirement already up-to-date: html5lib in /usr/local/lib/python2.7/dist-packages (from rdflib==4.2.1->-r install/owtf.pip (line 20))
Requirement already up-to-date: SPARQLWrapper in /usr/local/lib/python2.7/dist-packages (from rdflib==4.2.1->-r install/owtf.pip (line 20))
Requirement already up-to-date: keepalive>=0.5 in /usr/local/lib/python2.7/dist-packages (from SPARQLWrapper->rdflib==4.2.1->-r install/owtf.pip (line 20))

./owtf.py
The following python libraries seem missing :
beautifulsoup4,Markdown,pyOpenSSL,SQLAlchemy

Haven't you run the install script?
python2 install/install.py

If you are sure you ran the install script, install the missing libraries seperately
pip install --upgrade -r install/owtf.pip

And this is very irrelevant but just FYI, in the last line of the output above, seperately should be spelled separately.

@xFreed0m
Copy link

same issue here as well. the --user doesn't fix it.

@bitsydarel
Copy link

tried with the --user , did not fix it ...

@viyatb
Copy link
Member

viyatb commented Jun 19, 2016

@FreedomShepherd @darelswag @h121h Until the complete install script is ready with the virtualenv stuff (so that I can fix this on Kali Linux), you can follow these steps to install and run OWTF successfully.
You need to have pip installed.

# pip install virtualenv virtualenvwrapper
# source '/usr/local/bin/virtualenvwrapper.sh'
# echo 'source /usr/local/bin/virtualenvwrapper.sh' >> ~/.zshrc (or ~/.bashrc if use bash)
# mkvirtualenv owtf -p /usr/bin/python2.7

You will see something like this: (owtf)~/owtf/install %
This says a virtualenv named "owtf" is active. You can now run the install script.
You can deactivate the env by running: deactivate

But whenever you want to run OWTF, just cd into the directory and run: workon <env>
Like for the example above, workon owtf.

This should do as a temporary solution. :)

@xFreed0m
Copy link

xFreed0m commented Jun 19, 2016

approved the comment above, it works. :)

@bitsydarel
Copy link

bitsydarel commented Jun 20, 2016

Here is a full log - https://pastee.org/vm2db.

@bitsydarel
Copy link

capture du 2016-06-20 13-51-56

@bitsydarel
Copy link

added the photo for you to check

@viyatb
Copy link
Member

viyatb commented Jul 11, 2016

@darelswag can you check again please? virtualenv creates an isolated env for the project. If you set up the env properly and activated it, there should not be any problem in installing/running OWTF.

@viyatb viyatb closed this as completed Jul 14, 2016
@DarKnight24
Copy link
Contributor

@delta24 If the virtualenv in active, even after running install script it didn't install python packages. I had to explicitly run pip install -r install/owtf.pip after the install script.

@hktalent
Copy link

hktalent commented Feb 9, 2017

./owtf.py

Traceback (most recent call last):
File "./owtf.py", line 17, in
from framework.core import Core
File "/usr/share/owtf/framework/core.py", line 21, in
from framework.http.proxy import proxy, transaction_logger
File "/usr/share/owtf/framework/http/proxy/proxy.py", line 31, in
from socket_wrapper import wrap_socket
File "/usr/share/owtf/framework/http/proxy/socket_wrapper.py", line 9, in
from gen_cert import gen_signed_cert
File "/usr/share/owtf/framework/http/proxy/gen_cert.py", line 6, in
from OpenSSL import crypto
File "/root/.virtualenvs/owtf/local/lib/python2.7/site-packages/OpenSSL/init.py", line 8, in
from OpenSSL import rand, crypto, SSL
File "/root/.virtualenvs/owtf/local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 105, in
SSL_ST_INIT = _lib.SSL_ST_INIT
AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

@viyatb
Copy link
Member

viyatb commented Feb 9, 2017

@hktalent Your Kali is not updated. This is not a problem with OWTF. Please refer to #812

@hktalent
Copy link

hktalent commented Feb 9, 2017

Thanks @delta24
do
pip install virtualenv virtualenvwrapper
source '/usr/local/bin/virtualenvwrapper.sh'
echo 'source /usr/local/bin/virtualenvwrapper.sh' >> ~/.zshrc (or ~/.bashrc if use bash)
mkvirtualenv owtf -p /usr/bin/python2.7
apt-get update;apt-get upgrade;apt autoremove;apt-get update;apt-get dist-upgrade;
pip install -U --trusted-host pypi.douban.com -i http://pypi.douban.com/simple beautifulsoup4 lxml Markdown pexpect psycopg2 pyOpenSSL PyVirtualDisplay rdflib selenium six SQLAlchemy tornado

run
./owtf.py

so is ok
thanks

@viyatb
Copy link
Member

viyatb commented Feb 9, 2017

@hktalent You do not need to do this manually. The installation script takes care of it. You just have to run source ~/.$SHELLrc; workon owtf; ./owtf.py

@hktalent
Copy link

hktalent commented Feb 10, 2017

(owtf) root@kali:/usr/local/share/owtf# git pull
Already up-to-date.
(owtf) root@kali:/usr/local/share/owtf# pwd
/usr/local/share/owtf

(owtf) root@kali:/usr/local/share/owtf# ./owtf.py
Current Path: ./owtf.py
Syntax: owtf.py [ options ] <target1 target2 target3 ..> where target can be: <target URL / hostname / IP>
NOTE: targets can also be provided via a text file

Examples:

Run all web plugins: owtf.py http://my.website.com
Run only passive + semi_passive plugins: owtf.py -t quiet http://my.website.com
Run only active plugins: owtf.py -t active http://my.website.com

Run all plugins except 'OWASP-CM-001: Testing_for_SSL-TLS': owtf.py -e 'OWASP-CM-001' http://my.website.com
Run all plugins except 'OWASP-CM-001: Testing_for_SSL-TLS': owtf.py -e 'Testing_for_SSL-TLS' http://my.website.com

Run only 'OWASP-CM-001: Testing_for_SSL-TLS': owtf.py -o 'OWASP-CM-001' http://my.website.com
Run only 'OWASP-CM-001: Testing_for_SSL-TLS': owtf.py -o 'Testing_for_SSL-TLS' http://my.website.com

Run only OWASP-IG-005 and OWASP-WU-VULN: owtf.py -o 'OWASP-IG-005,OWASP-WU-VULN' http://my.website.com
Run using my resources file and proxy: owtf.py -m r:/home/me/owtf_resources.cfg -x 127.0.0.1:8080 http://my.website.com

Run using TOR network: owtf.py -o OWTF-WVS-001 http://my.website.com --tor 127.0.0.1:9050:9051:password:1

Run Botnet-mode using miner: owtf.py -o OWTF-WVS-001 http://my.website.com -b miner

Run Botnet-mode using custom proxy list: owtf.py -o OWTF-WVS-001 http://my.website.com -b list:proxy_list_path.txt

ERROR: Start owtf in a different directory. 'cd path/to/pentest/directory; ./path/to/owtf.py'

@viyatb
Copy link
Member

viyatb commented Feb 10, 2017

@hktalent please refer to #766 and ERROR message carefully.

@hktalent
Copy link

@delta24
[] OWTF Version: 1.0.1, Release: LionHeart (Beta)
[
] /usr/share/owtf/scripts/run_nikto.sh: 69: /usr/share/owtf/scripts/run_nikto.sh: Syntax error: Missing '))'

@viyatb
Copy link
Member

viyatb commented Feb 10, 2017

@hktalent which version are you using? OWTF stable is at 2.0a not 1.0.1

@Tikam02
Copy link
Contributor

Tikam02 commented May 31, 2017

same problem

@Tikam02
Copy link
Contributor

Tikam02 commented May 31, 2017

i have fixed this issue

@Tikam02
Copy link
Contributor

Tikam02 commented May 31, 2017

You have to Make a folder like i made Pentest folder in Desktop.Then run it by changing it into OWTF folder.then run it.

  1. cd owtf
    2)source ~/.bashrc; workon owtf
    3)cd /root/Desktop/pentest; /root/owtf/owtf.py

It works 100%
owtf1

@dibrahim2050
Copy link

Hi
Could anybody help me fixing this:

oot@kali:# cd owtf/
root@kali:
/owtf# source /.bashrc; workon owtf
(owtf) root@kali:
/owtf# cd /root/Desktop/pentest; /root/owtf/owtf.py
The following python libraries seem missing :
cookies,ipaddr,lxml,Markdown,pexpect,psycopg2,PTP,pycurl,pyOpenSSL,python-owasp-zap-v2.4,PyVirtualDisplay,six,selenium,SQLAlchemy,tornado

Haven't you run the install script?
python2 install/install.py

If you are sure you ran the install script, install the missing libraries seperately
pip install --upgrade -r install/owtf.pip

(owtf) root@kali:~/Desktop/pentest#

@viyatb
Copy link
Member

viyatb commented Jan 11, 2018

which version of OWTF are you using?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests