Installation requirements/dependencies issues

[AmosHason]

System

OS: Ubuntu 16.04
Python: 2.7.12
pip: 9.0.1
git: 2.7.4
PostgreSQL: 10.0
RabbitMQ: 3.6.12
virtualenv: 15.1.0

Expected Behavior

Following the steps in the repository README, you should not encounter any issues.

Actual Behavior

1. While executing "pip install -r requirements/dev.txt" (step 3)

Collecting psycopg2==2.6.2 (from -r requirements/common.txt (line 15))
Using cached psycopg2-2.6.2.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.
txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
Error: You need to install postgresql-server-dev-X.Y for building a server-s
ide extension or libpq-dev for building a client-side application.

Workaround: execute "apt-get install libpq-dev" first.

Suggested fix: add libpq-dev to the requirements.

2. While executing "pip install -r requirements/dev.txt" (step 3)

Collecting psycopg2==2.6.2 (from -r requirements/common.txt (line 15))
Using cached psycopg2-2.6.2.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.
txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
Error: could not determine PostgreSQL version from '10.0'

Workaround (suggested fix): set "psycopg2==2.7" in requirements/common.txt (see psycopg/psycopg2#594).

3. While trying to execute "bower install" (step 8)

bash: bower: command not found

Workaround: execute "npm install -g bower" first.

Suggested fix: add bower to the dependencies.

4. While trying to execute "gulp dev:runserver" (step 9)

bash: gulp: command not found

Workaround: execute "npm install -g gulp-cli" first.

Suggested fix: add gulp-cli to the dependencies.

5. While executing "gulp dev:runserver" (step 9)

events.js:160
throw er; // Unhandled 'error' event
^
Error: Gem sass is not installed.

Workaround: execute "gem install sass" first.

Suggested fix: add a preceding step in the repository README to execute "gem install sass".

[deshraj]

@AmosHason closing this since you mentioned on the gitter channel that you were able to solve this by installing sass on your machine.

[AmosHason]

@deshraj it only solves the fifth issue. The README should include a step for installing sass.

However, all of the other issues are due to requirements/dependencies gaps, and can be solved by editing those files.

[deshraj]

I see. @AmosHason please feel free to go ahead and open a PR adding instructions to install sass in the README.

[mlandry22]

Not just sass. I can confirm that the Postgres10.1 has a similar requirement issues as listed in (2).

Recent postgres versions appear to be a known conflict with psycopg2.6, which is used in common.txt
I updated to 2.7.1 and it worked for me, based on this issue (where they say they will not backport the updates to 2.6):
psycopg/psycopg2#594

For me, Mac Sierra 10.12.6 with brew, fresh postgres 10.1, current clone of EvalAI.

Here was the output originally from pip install -r requirements/dev.txt :

Collecting psycopg2==2.6.2 (from -r requirements/common.txt (line 15))
  Downloading psycopg2-2.6.2.tar.gz (376kB)
    100% |████████████████████████████████| 378kB 1.6MB/s 
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/psycopg2.egg-info
    writing pip-egg-info/psycopg2.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
    Error: could not determine PostgreSQL version from '10.1'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/q1/6gsv_7p15b5bj7w4_nynntbm0000gp/T/pip-build-k_61ZR/psycopg2/

And after the update, the pip install -r requirements/dev.txt finished with:

Successfully built commonmark django-import-export django-rest-auth drfdocs PyYaml rstr django-autofixture django-spaghetti-and-meatballs msgpack-python twisted tablib diff-match-patch django-allauth docopt configparser tornado port-for watchdog pathtools zope.interface odfpy openpyxl unicodecsv python-openid scandir MarkupSafe jdcal et-xmlfile oauthlib
Installing collected packages: six, msgpack-python, asgiref, redis, asgi-redis, txaio, autobahn, zope.interface, constantly, incremental, attrs, Automat, hyperlink, twisted, daphne, django, channels, commonmark, odfpy, jdcal, et-xmlfile, openpyxl, unicodecsv, xlrd, xlwt, PyYaml, tablib, diff-match-patch, django-import-export, djangorestframework, djangorestframework-expiring-authtoken, django-cors-headers, python-openid, oauthlib, idna, urllib3, certifi, chardet, requests, requests-oauthlib, django-allauth, django-rest-auth, drfdocs, pika, scandir, pathlib2, pickleshare, Pillow, psycopg2, verboselogs, monotonic, humanfriendly, property-manager, fasteners, coloredlogs, executor, proc, rstr, pep8, autopep8, coverage, docopt, coveralls, django-autofixture, sqlparse, django-debug-toolbar, django-extensions, pytz, MarkupSafe, Jinja2, Pygments, python-dateutil, django-silk, django-spaghetti-and-meatballs, enum34, configparser, mccabe, pyflakes, pycodestyle, flake8, py, pytest, pytest-cov, babel, docutils, alabaster, snowballstemmer, imagesize, sphinx, singledispatch, backports-abc, tornado, livereload, argh, port-for, pathtools, watchdog, sphinx-autobuild, sphinx-rtd-theme, pytest-django, ipaddress, Faker
Successfully installed Automat-0.6.0 Faker-0.7.7 Jinja2-2.10 MarkupSafe-1.0 Pillow-3.4.2 PyYaml-3.12 Pygments-2.2.0 alabaster-0.7.10 argh-0.26.2 asgi-redis-0.14.1 asgiref-2.0.1 attrs-17.3.0 autobahn-17.10.1 autopep8-1.2.4 babel-2.5.1 backports-abc-0.5 certifi-2017.11.5 channels-0.17.2 chardet-3.0.4 coloredlogs-7.3.1 commonmark-0.5.4 configparser-3.5.0 constantly-15.1.0 coverage-4.2 coveralls-1.1 daphne-0.15.0 diff-match-patch-20121119 django-1.10.2 django-allauth-0.34.0 django-autofixture-0.12.1 django-cors-headers-1.3.1 django-debug-toolbar-1.8 django-extensions-1.7.8 django-import-export-0.5.1 django-rest-auth-0.9.2 django-silk-1.0.0 django-spaghetti-and-meatballs-0.2.2 djangorestframework-3.5.2 djangorestframework-expiring-authtoken-0.1.4 docopt-0.6.2 docutils-0.14 drfdocs-0.0.11 enum34-1.1.6 et-xmlfile-1.0.1 executor-18.0 fasteners-0.14.1 flake8-3.0.4 humanfriendly-4.4.1 hyperlink-17.3.1 idna-2.6 imagesize-0.7.1 incremental-17.5.0 ipaddress-1.0.18 jdcal-1.3 livereload-2.5.1 mccabe-0.5.3 monotonic-1.4 msgpack-python-0.4.8 oauthlib-2.0.6 odfpy-1.3.5 openpyxl-2.4.9 pathlib2-2.3.0 pathtools-0.1.2 pep8-1.7.1 pickleshare-0.7.4 pika-0.10.0 port-for-0.3.1 proc-0.10.1 property-manager-2.2 psycopg2-2.7.1 py-1.5.2 pycodestyle-2.0.0 pyflakes-1.2.3 pytest-3.0.3 pytest-cov-2.4.0 pytest-django-3.1.2 python-dateutil-2.6.1 python-openid-2.2.5 pytz-2017.3 redis-2.10.6 requests-2.18.4 requests-oauthlib-0.8.0 rstr-2.2.5 scandir-1.6 singledispatch-3.4.0.3 six-1.11.0 snowballstemmer-1.2.1 sphinx-1.4.8 sphinx-autobuild-0.6.0 sphinx-rtd-theme-0.1.9 sqlparse-0.2.4 tablib-0.12.1 tornado-4.5.2 twisted-17.9.0 txaio-2.8.2 unicodecsv-0.14.1 urllib3-1.22 verboselogs-1.7 watchdog-0.8.3 xlrd-1.1.0 xlwt-1.3.0 zope.interface-4.4.3

[AmosHason]

Regarding the first issue: it's currently impossible to install libpq-dev using pip (see ncbi/python-libpq-dev#1).

[rupesh1798]

While installing dependencies for the project got the following error.

Collecting psycopg2==2.6.2 (from -r requirements/common.txt (line 15))
 Downloading psycopg2-2.6.2.tar.gz (376kB)
   100% |████████████████████████████████| 378kB 334kB/s
   Complete output from command python setup.py egg_info:
   running egg_info
   creating pip-egg-info/psycopg2.egg-info
   writing pip-egg-info/psycopg2.egg-info/PKG-INFO
   writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
   writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
   writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
   Error: could not determine PostgreSQL version from '10.1'

  ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/42/54d9118s4xx7qnjygmsbx7th0000gn/T/pip-build-1e_DjJ/psycopg2/

After changing "psycopg2==2.6.2" to "psycopg2==2.7.1" in requirements/common.txt the dependencies installed correctly.

should I make a PR for this issue?

[deshraj]

Thanks, @mlandry22 for pointing this out. Please feel free to create a pull request for the Postgres dependency updates.

[deshraj]

@Obtin: yeah that is a great idea. Please feel free to comment on the thread or create a pull request. :)

[gautamjajoo]

@deshraj Added the common error in my PR :- #1455

[deshraj]

Did anyone check if PostgreSQL 9.6 is compatible with psycopg2==2.7.3.1? If it is, then:
a) we can just update the requirement to psycopg2==2.7.3.1

Otherwise,
a) we can update the requirement to psycopg2==2.7.3.1
b) Mention in the README to use PostgreSQL 10.1 and above

Does that sound good @Obtin ?

[deshraj]

Already fixed. Closing this.