setup.py: create a proper setup script

[blshkv]

Please create a proper script so it would be easier to install, upgrade and uninstall the tool and get rid of the current script which supports limited distros only: controllers/dependency_check.

Once it is done, it will be possible to install w3af on any platform using a standard command:
python setup.py install

[blshkv]

here is an example how other tools has fixed the same issue:
derv82/wifite2@8b786b7

[andresriancho]

controllers/dependency_check method is not perfect, I agree, but it does help (most?) users install w3af and all dependencies: both pypi and OS dependencies.

setup.py, AFAIK, only works with pypi dependencies.

How do we solve the OS dependencies with setup.py? The problem is that some pypi packages require OS dependencies to be installed. Some users will run python setup.py install and receive a "strange error" about a missing somelib.h. They won't know which package in their OS provide that.

[blshkv]

I have intentionally mentioned wifite2 because they call external tools too, so it is a good example.
A quick answer: all system dependencies must be documented and installed separately. By right, end users should not install the tool manually. Instead, it should be pushed to upstream distro and users should be able to install it with a standard <distro-package-manager> install w3af command. Distro package manager will take care of all deps. And when, non of distro maintainer will require controllers/dependency_check

[andresriancho]

Hmmm... let me answer line by line:

all system dependencies must be documented and installed separately

I see this as an extra step. Let me explain the two flows:

Proposed one

• User does git clone
• User runs python setup.py install
• Installation fails
• User doesn't know what to do, gets frustrated
• User finds commands to install OS dependencies in https://docs.w3af.org
• User installs dependencies
• User runs python setup.py install
• User runs w3af_console, it works!

Current

• User does git clone
• User runs ./w3af_console
• Run fails because of missing dependencies, script to install all dependencies is generated
• User runs dependency install script
• User runs ./w3af_console
• User runs w3af_console, it works!

Note that the user never left the console. Note that the user (if the distribution is supported by dependency check) never gets frustrated because w3af is telling him exactly what to do.

Support for distribution X or Y will always be a hard thing to achieve. Documentation and/or dependency check scripts will both get outdated.

By right, end users should not install the tool manually. Instead, it should be pushed to upstream distro and users should be able to install it with a standard install w3af command. Distro package manager will take care of all deps. And when, non of distro maintainer will require controllers/dependency_check

Agreed, but this never worked for me in the past. Maintaining packages for each distribution is hard, I personally dislike the task, and there is nobody helping me with it.

Getting a package to ubuntu / debian repos is a PITA. You need to get a package maintainer to help, there are too many rules to follow, many emails to get things going... and then... the same happens with the other 10 important distributions that there are out there... and each has a different set of rules and techniques to use to create "the package the right way".

With controllers/dependency_check the w3af community (mainly me) control whatever is run and all the installation process without caring about linux distribution rules.

These are the alternatives I see:

• Someone else takes care of maintaining all the w3af.deb/rpm/etc. packages
• We start using snap: a package manager that works in all distributions
• We keep using controllers/dependency_check and keep recommending people to install from git clone

[blshkv]

EDIT:

Proposed one

User runs the standard:
<distro-package-manager> install w3af
and everything works right way

I suggest the following:

• split the current dependency_check into a separate local_install.py for users (since you insist)
• create a proper setup.py, release a proper release and see how fast it goes into all distros.

I can guarantee you that Pentoo will be one of the first distro to pick up the latest version quickly (and no, I don't need your help if the tool is written properly).

Let me explain the current complexity from another point of view. As a package maintainer I need to do the following:

• review manually dependency_check source code and figure out which packages are required
• patch the code to remove dependency_check.
• install into system dir manually
• repeat it with each new version and make sure that custom patches work

You can have a look at the bash script here: https://github.com/pentoo/pentoo-overlay/blob/master/net-analyzer/w3af/w3af-1.6.54-r4.ebuild

As you can guess, we write such ebuilds with hope that it would be easier to install a next version by simply coping it with a different version number. It is not that simple today, and I had to waste a lot of time before our users can get a next version

[andresriancho]

create a proper setup.py, release a proper release and see how fast it goes into all distros.

It will be slow and painful, I promise. I've been through that path already and it is not nice. You're here and interested in having a better w3af, pentoo will be the first. There will be no second. The rest of the package maintainers don't care at all.

I've decided that implementing this is not the best use of my time, so I won't do it.

If you, or someone else form the community wants to send a PR, I'll be more than happy to review and merge if:

• The installation process for users outside pentoo is similar or easier than the current one
• The installation process helps users with different distributions figure out which OS packages they need to install to get the pip dependencies to work
• The installation process helps users install external tools such as retirejs
• The checks which assert that external tools are at specific versions are kept
• The installation pins dependencies to specific versions

[blshkv]

pentoo/pentoo-overlay#596