make update-admin-pip-requirements target fails

[kushaldas]

Description

Trying to update the requirements*.txt files for the onion v3 branch.

Steps to Reproduce

make update-admin-pip-requirements

Expected Behavior

The command should not error out and update the dependencies as required.

Actual Behavior

 $ make update-admin-pip-requirements
███ Updating admin pip requirements...
Run with DOCKER_BUILD_VERBOSE=true for more information
Docker image build in progress ..................................................................................... done !
Traceback (most recent call last):
  File "/opt/.venv/bin/pip-compile", line 10, in <module>
    sys.exit(cli())
  File "/opt/.venv/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/.venv/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/.venv/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/.venv/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/.venv/local/lib/python2.7/site-packages/piptools/scripts/compile.py", line 129, in cli
    repository = PyPIRepository(pip_options, session, build_isolation)
  File "/opt/.venv/local/lib/python2.7/site-packages/piptools/repositories/pypi.py", line 73, in __init__
    self.finder = PackageFinder(**finder_kwargs)
TypeError: __init__() got an unexpected keyword argument 'index_urls'
make[1]: *** [Makefile:10: update-pip-requirements] Error 1
make: *** [Makefile:32: update-admin-pip-requirements] Error 2

[redshiftzero]

was this solved? I just tried to repro locally and wasn't able to but from the traceback here it looks like it might be related to the versions of pip and pip-tools being used, were there any other steps to reproduce?

[redshiftzero]

It looks like this is due to a refactor on pip's side: pypa/pip@cad71a7

[redshiftzero]

indeed pip-tools also has some refactoring to do in response: jazzband/pip-tools#853

does this diff resolve for you? I also hit this locally on your v3 onion branch and it resolved for me

diff --git a/admin/Dockerfile b/admin/Dockerfile
index 934a77cf1..58c3213cc 100644
--- a/admin/Dockerfile
+++ b/admin/Dockerfile
@@ -17,4 +17,7 @@ ENV VIRTUAL_ENV /opt/.venv
 ENV PATH="$VIRTUAL_ENV/bin:$PATH"
 COPY requirements-dev.txt .
 RUN pip install --require-hashes -r requirements-dev.txt
+# Now also pin pip due to https://github.com/jazzband/pip-tools/issues/853
+RUN pip install pip==19.1
+
 RUN chown -R $USER_NAME /opt