Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

django.db.utils.OperationalError: no such table: main.core_analysistype__old #7

Open
moldach opened this issue Apr 28, 2020 · 19 comments

Comments

@moldach
Copy link

moldach commented Apr 28, 2020

I'm getting an error trying to install/run genesysv. After following the installation examples the following code is partially working. I'm able to open but something in the error isn't loading the example data in properly.

(genesysv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python manage.py runserver 0.0.0.0:8000
Performing system checks...

System check identified no issues (0 silenced).
April 27, 2020 - 21:03:38
Django version 2.0.5, using settings 'genesysv.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.

I'm able to open local host and login:

Screenshot from 2020-04-27 19-12-42

but cannot click on anything (pretty sure the example is the next part).

Trying the example

(genesysv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python utils/load_vcf.py \
        --vcf test_data/test_4families_annovar.vcf.gz \
        --tmp_dir /tmp --annot annovar \
        --hostname 127.0.0.1 --port 9200 \
        --index test_4families_annovar --study_name test_4families_annovar \
        --dataset_name test_4families_annovar --num_cores 4 \
        --assembly hg19 --ped test_data/test_4families.ped --cleanup
Pid 25941: processed 5000 variants
Pid 25942: processed 5000 variants
Pid 25944: processed 5000 variants
Pid 25943: processed 5000 variants
Pid 25941: processed 10000 variants
Pid 25944: processed 10000 variants
Pid 25942: processed 10000 variants
Pid 25943: processed 10000 variants
Pid 25941: processed 12501 variants
Process 25941 finished ...
Pid 25942: processed 12501 variants
Pid 25944: processed 12498 variants
Process 25942 finished ...
Pid 25943: processed 12501 variants
Process 25943 finished ...
Process 25944 finished ...
Finished parsing vcf file in 11.873375177383423 seconds, now creating ElasticSearch index ...
deleting 'test_4families_annovar' index...
response: '{'acknowledged': True}'
creating 'test_4families_annovar' index...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   590  100    98  100   492   1689   8482 --:--:-- --:--:-- --:--:-- 10000
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 25391  100    28  100 25363   1750  1548k --:--:-- --:--:-- --:--:-- 1549k
Response: '{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "test_4families_annovar"
}
{
  "acknowledged" : true
}
'
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_0.json
Took: 3.564603328704834 seconds
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_1.json
Took: 3.5286946296691895 seconds
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_2.json
Took: 3.4319169521331787 seconds
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_3.json
Took: 3.6023032665252686 seconds
Finished creating ES index, parsing time: 11.873375177383423 seconds, indexing time: 14.294233560562134 seconds, vcf: test_data/test_4families_annovar.vcf.gz

Creating Web user interface, please wait ...
Unmapped GUI CLNVAR_nested
Unmapped GUI COSMIC_nested
Unmapped GUI GTEx_nested
Unmapped GUI ICGC_nested
Unmapped GUI RGQ
Unmapped GUI mendelian_diseases
Unmapped GUI RGQ
Unmapped GUI mendelian_diseases
********************************************************************************

Study Name: test_4families_annovar
Dataset Name: test_4families_annovar_hg19
Dataset ES Index Name: test_4families_annovar
Dataset ES Type Name: test_4families_annovar_
Dataset ES Host: 127.0.0.1
Dataset ES Port: 9200
Traceback (most recent call last):
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 303, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: main.core_analysistype__old

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "utils/load_vcf.py", line 1715, in <module>
    make_gui(es, hostname, port, index_name, study, dataset_name, type_name, gui_mapping)
  File "/home/mtg/bin/GenESysV/utils/make_gui.py", line 961, in make_gui
    dataset_obj.analysis_type.add(*a)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 898, in add
    self._add_items(self.source_field_name, self.target_field_name, *objs)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 1065, in _add_items
    for obj_id in new_ids
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/models/query.py", line 466, in bulk_create
    ids = self._batched_insert(objs_without_pk, fields, batch_size)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/models/query.py", line 1148, in _batched_insert
    self._insert(item, fields=fields, using=self.db)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/models/query.py", line 1125, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1283, in execute_sql
    cursor.execute(sql, params)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/mtg/bin/genesysv/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 303, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: main.core_analysistype__old

Not sure what I'm missing.

@jianxinwang
Copy link
Contributor

jianxinwang commented Apr 28, 2020 via email

@moldach
Copy link
Author

moldach commented Apr 28, 2020

Could the error be happening here:

(base) mtg@mtg-ThinkPad-P53:~$ sudo add-apt-repository -y ppa:webupd8team/java
Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:2 https://dl.yarnpkg.com/debian stable InRelease                           
Hit:3 http://dl.google.com/linux/chrome/deb stable Release                     
Hit:4 http://ca.archive.ubuntu.com/ubuntu eoan InRelease                       
Hit:5 http://ca.archive.ubuntu.com/ubuntu eoan-updates InRelease               
Ign:6 http://ppa.launchpad.net/webupd8team/java/ubuntu eoan InRelease          
Hit:7 http://ca.archive.ubuntu.com/ubuntu eoan-backports InRelease
Hit:9 http://security.ubuntu.com/ubuntu eoan-security InRelease                
Err:10 http://ppa.launchpad.net/webupd8team/java/ubuntu eoan Release           
  404  Not Found [IP: 2001:67c:1560:8008::15 80]
Reading package lists... Done
E: The repository 'http://ppa.launchpad.net/webupd8team/java/ubuntu eoan Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

I've gone into Software and updates -> other software -> and then removed webupd8team PPA

Then I run

sudo apt-get --allow-releaseinfo-change update
sudo apt-get update

Trying this again: sudo add-apt-repository ppa:webupd8team/java:

(base) mtg@mtg-ThinkPad-P53:~$ sudo add-apt-repository ppa:webupd8team/java
 The Oracle JDK License has changed for releases starting April 16, 2019.

The new Oracle Technology Network License Agreement for Oracle Java SE is substantially different from prior Oracle JDK licenses. The new license permits certain uses, such as personal use and development use, at no cost -- but other uses authorized under prior Oracle JDK licenses may no longer be available. Please review the terms carefully before downloading and using this product. An FAQ is available here: https://www.oracle.com/technetwork/java/javase/overview/oracle-jdk-faqs.html

Oracle Java downloads now require logging in to an Oracle account to download Java updates, like the latest Oracle Java 8u211 / Java SE 8u212. Because of this I cannot update the PPA with the latest Java (and the old links were broken by Oracle).

For this reason, THIS PPA IS DISCONTINUED.

For Oracle Java 11, see a different PPA -> https://www.linuxuprising.com/2019/06/new-oracle-java-11-installer-for-ubuntu.html

That same PPA also has Oracle Java 13 -> https://www.linuxuprising.com/2019/09/install-oracle-java-13-on-ubuntu-linux.html

and Oracle Java 14 -> 
https://www.linuxuprising.com/2020/03/how-to-install-oracle-java-14-jdk14-on.html

Old description:

Oracle Java (JDK) Installer (automatically downloads and installs Oracle JDK8). There are no actual Java files in this PPA.

Important -> Why Oracle Java 7 And 6 Installers No Longer Work: http://www.webupd8.org/2017/06/why-oracle-java-7-and-6-installers-no.html

Update: Oracle Java 9 has reached end of life: http://www.oracle.com/technetwork/java/javase/downloads/jdk9-downloads-3848520.html

The PPA supports Ubuntu 18.10, 18.04, 16.04, 14.04 and 12.04.

More info (and Ubuntu installation instructions):
- http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html

Debian installation instructions:
- Oracle Java 8: http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html
 More info: https://launchpad.net/~webupd8team/+archive/ubuntu/java
Press [ENTER] to continue or Ctrl-c to cancel adding it.

Ign:1 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:2 https://dl.yarnpkg.com/debian stable InRelease                           
Hit:3 http://dl.google.com/linux/chrome/deb stable Release                     
Hit:4 http://ca.archive.ubuntu.com/ubuntu eoan InRelease                       
Hit:5 http://security.ubuntu.com/ubuntu eoan-security InRelease                
Ign:6 http://ppa.launchpad.net/webupd8team/java/ubuntu eoan InRelease
Hit:7 http://ca.archive.ubuntu.com/ubuntu eoan-updates InRelease
Hit:8 http://ca.archive.ubuntu.com/ubuntu eoan-backports InRelease
Err:10 http://ppa.launchpad.net/webupd8team/java/ubuntu eoan Release
  404  Not Found [IP: 2001:67c:1560:8008::15 80]
Reading package lists... Done
E: The repository 'http://ppa.launchpad.net/webupd8team/java/ubuntu eoan Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

It say's:

The PPA supports Ubuntu 18.10, 18.04, 16.04, 14.04 and 12.04.

I'm running Ubuntu 19.04. Which is the proper PPA I should be using?

@jianxinwang
Copy link
Contributor

jianxinwang commented Apr 30, 2020 via email

@moldach
Copy link
Author

moldach commented May 6, 2020

Okay I've updated to Java 8

(venv) mtg@mtg-ThinkPad-P53:~/GenESysV$ java -version
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

Now I get errors running the requirements.txt script:

(base) mtg@mtg-ThinkPad-P53:~/GenESysV$ source venv/bin/activate
(venv) (base) mtg@mtg-ThinkPad-P53:~/GenESysV$ pip install wheel
Requirement already satisfied: wheel in /home/mtg/anaconda3/lib/python3.7/site-packages (0.34.2)
(venv) (base) mtg@mtg-ThinkPad-P53:~/GenESysV$ pip install -r requirements.txt
Requirement already satisfied: alabaster==0.7.10 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (0.7.10)
Requirement already satisfied: amqp==2.2.2 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (2.2.2)
Requirement already satisfied: Babel==2.5.3 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (2.5.3)
Requirement already satisfied: backcall==0.1.0 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (0.1.0)
Requirement already satisfied: beautifulsoup4==4.6.0 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (4.6.0)
Requirement already satisfied: billiard==3.5.0.3 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (3.5.0.3)
Processing /home/mtg/.cache/pip/wheels/a7/2f/90/ec1e6518209e71889d65e3173b3ced45553f7058c38ed9ce91/bokeh-0.12.16-py3-none-any.whl
Collecting celery==4.1.1
  Using cached celery-4.1.1-py2.py3-none-any.whl (394 kB)
Collecting certifi==2018.4.16
  Using cached certifi-2018.4.16-py2.py3-none-any.whl (150 kB)
Requirement already satisfied: chardet==3.0.4 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 10)) (3.0.4)
Collecting decorator==4.3.0
  Using cached decorator-4.3.0-py2.py3-none-any.whl (9.2 kB)
Collecting Django==2.0.5
  Using cached Django-2.0.5-py3-none-any.whl (7.1 MB)
Collecting django-crispy-forms==1.7.2
  Using cached django_crispy_forms-1.7.2-py2.py3-none-any.whl (105 kB)
Collecting django-formtools==2.1
  Using cached django_formtools-2.1-py2.py3-none-any.whl (132 kB)
Collecting django-htmlmin==0.10.0
  Using cached django_htmlmin-0.10.0-py2.py3-none-any.whl (18 kB)
Processing /home/mtg/.cache/pip/wheels/77/ed/3e/589cbed6c95214c28f79f73470f68f6ddd80f4666d0e0ff4ab/django_sortedm2m-1.5.0-py3-none-any.whl
Collecting django-tinymce==2.7.0
  Using cached django_tinymce-2.7.0-py2.py3-none-any.whl (4.1 MB)
Collecting docutils==0.14
  Using cached docutils-0.14-py3-none-any.whl (543 kB)
Collecting elasticsearch==6.2.0
  Using cached elasticsearch-6.2.0-py2.py3-none-any.whl (69 kB)
Requirement already satisfied: html5lib==1.0.1 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 20)) (1.0.1)
Collecting idna==2.6
  Using cached idna-2.6-py2.py3-none-any.whl (56 kB)
Collecting imagesize==1.0.0
  Using cached imagesize-1.0.0-py2.py3-none-any.whl (5.0 kB)
Processing /home/mtg/.cache/pip/wheels/07/98/cf/7750e16d38de8b75222963249bfebaba55bbcc011248ef23d8/ipdb-0.11-py3-none-any.whl
Collecting ipython==6.4.0
  Using cached ipython-6.4.0-py3-none-any.whl (750 kB)
Requirement already satisfied: ipython-genutils==0.2.0 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 25)) (0.2.0)
Collecting isort==4.3.4
  Using cached isort-4.3.4-py3-none-any.whl (45 kB)
Collecting jedi==0.12.0
  Using cached jedi-0.12.0-py2.py3-none-any.whl (172 kB)
Collecting Jinja2==2.10
  Using cached Jinja2-2.10-py2.py3-none-any.whl (126 kB)
Collecting kombu==4.2.0
  Using cached kombu-4.2.0-py2.py3-none-any.whl (853 kB)
Processing /home/mtg/.cache/pip/wheels/8f/73/61/ad4c803fb36f6f2225882a6557fa2fbf5cf24fbe4be3f20b24/MarkupSafe-1.0-cp37-cp37m-linux_x86_64.whl
Collecting natsort==5.3.2
  Using cached natsort-5.3.2-py2.py3-none-any.whl (32 kB)
Processing /home/mtg/.cache/pip/wheels/cb/38/86/4ac7ecc2bf189cfe58fb0148b61ccc4b17d00dfcec88d5b973/numpy-1.14.3-cp37-cp37m-linux_x86_64.whl
Collecting packaging==17.1
  Using cached packaging-17.1-py2.py3-none-any.whl (24 kB)
Processing /home/mtg/.cache/pip/wheels/24/ec/f9/ff38cd2b56f71ceb81dc2a01d5b3417fa37a01ca381cb01c80/pandas-0.23.0-cp37-cp37m-linux_x86_64.whl
Collecting parso==0.2.1
  Using cached parso-0.2.1-py2.py3-none-any.whl (91 kB)
Collecting pexpect==4.5.0
  Using cached pexpect-4.5.0-py2.py3-none-any.whl (57 kB)
Collecting pickleshare==0.7.4
  Using cached pickleshare-0.7.4-py2.py3-none-any.whl (6.9 kB)
Collecting prompt-toolkit==1.0.15
  Using cached prompt_toolkit-1.0.15-py3-none-any.whl (247 kB)
Collecting ptyprocess==0.5.2
  Using cached ptyprocess-0.5.2-py2.py3-none-any.whl (39 kB)
Collecting pycodestyle==2.4.0
  Using cached pycodestyle-2.4.0-py2.py3-none-any.whl (62 kB)
Collecting Pygments==2.2.0
  Using cached Pygments-2.2.0-py2.py3-none-any.whl (841 kB)
Collecting pyparsing==2.2.0
  Using cached pyparsing-2.2.0-py2.py3-none-any.whl (56 kB)
Collecting python-dateutil==2.7.3
  Using cached python_dateutil-2.7.3-py2.py3-none-any.whl (211 kB)
Collecting python-memcached==1.59
  Using cached python_memcached-1.59-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: pytz==2018.4 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 46)) (2018.4)
Processing /home/mtg/.cache/pip/wheels/73/1c/5c/852e1974c0417219b83bc4eafda096b1952364b6f7c5fa5a7b/PyYAML-3.12-cp37-cp37m-linux_x86_64.whl
Collecting requests==2.18.4
  Using cached requests-2.18.4-py2.py3-none-any.whl (88 kB)
Requirement already satisfied: simplegeneric==0.8.1 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 49)) (0.8.1)
Requirement already satisfied: six==1.11.0 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 50)) (1.11.0)
Collecting snowballstemmer==1.2.1
  Using cached snowballstemmer-1.2.1-py2.py3-none-any.whl (64 kB)
Collecting Sphinx==1.7.4
  Using cached Sphinx-1.7.4-py2.py3-none-any.whl (1.9 MB)
Collecting sphinxcontrib-websupport==1.0.1
  Using cached sphinxcontrib_websupport-1.0.1-py2.py3-none-any.whl (26 kB)
Processing /home/mtg/.cache/pip/wheels/db/ab/30/d20afc027250c8a73c258e77493d9b5e9498e59fb5742ae619/tornado-5.0.2-cp37-cp37m-linux_x86_64.whl
Collecting tqdm==4.23.3
  Using cached tqdm-4.23.3-py2.py3-none-any.whl (42 kB)
Collecting traitlets==4.3.2
  Using cached traitlets-4.3.2-py2.py3-none-any.whl (74 kB)
Collecting urllib3==1.22
  Using cached urllib3-1.22-py2.py3-none-any.whl (132 kB)
Requirement already satisfied: vine==1.1.4 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 58)) (1.1.4)
Collecting wcwidth==0.1.7
  Using cached wcwidth-0.1.7-py2.py3-none-any.whl (21 kB)
Requirement already satisfied: webencodings==0.5.1 in /home/mtg/anaconda3/lib/python3.7/site-packages (from -r requirements.txt (line 60)) (0.5.1)
Requirement already satisfied: setuptools in /home/mtg/anaconda3/lib/python3.7/site-packages (from ipdb==0.11->-r requirements.txt (line 23)) (45.2.0.post20200210)
ERROR: spyder 4.0.1 requires pyqt5<5.13; python_version >= "3", which is not installed.
ERROR: spyder 4.0.1 requires pyqtwebengine<5.13; python_version >= "3", which is not installed.
ERROR: pytest-astropy 0.8.0 requires pytest-cov>=2.0, which is not installed.
ERROR: pytest-astropy 0.8.0 requires pytest-filter-subpackage>=0.1, which is not installed.
ERROR: spyder 4.0.1 has requirement jedi==0.14.1, but you'll have jedi 0.12.0 which is incompatible.
ERROR: python-language-server 0.31.7 has requirement jedi<0.16,>=0.14.1, but you'll have jedi 0.12.0 which is incompatible.
ERROR: numba 0.48.0 has requirement numpy>=1.15, but you'll have numpy 1.14.3 which is incompatible.
ERROR: jupyter-console 6.1.0 has requirement prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.15 which is incompatible.
ERROR: flask 1.1.1 has requirement Jinja2>=2.10.1, but you'll have jinja2 2.10 which is incompatible.
ERROR: flake8 3.7.9 has requirement pycodestyle<2.6.0,>=2.5.0, but you'll have pycodestyle 2.4.0 which is incompatible.
ERROR: astropy 4.0 has requirement numpy>=1.16, but you'll have numpy 1.14.3 which is incompatible.
Installing collected packages: pyparsing, packaging, numpy, tornado, MarkupSafe, Jinja2, PyYAML, python-dateutil, bokeh, kombu, celery, certifi, decorator, Django, django-crispy-forms, django-formtools, django-htmlmin, django-sortedm2m, django-tinymce, docutils, urllib3, elasticsearch, idna, imagesize, pickleshare, parso, jedi, ptyprocess, pexpect, wcwidth, prompt-toolkit, Pygments, traitlets, ipython, ipdb, isort, natsort, pandas, pycodestyle, python-memcached, requests, snowballstemmer, sphinxcontrib-websupport, Sphinx, tqdm
  Attempting uninstall: pyparsing
    Found existing installation: pyparsing 2.4.6
    Uninstalling pyparsing-2.4.6:
      Successfully uninstalled pyparsing-2.4.6
  Attempting uninstall: packaging
    Found existing installation: packaging 20.1
    Uninstalling packaging-20.1:
      Successfully uninstalled packaging-20.1
  Attempting uninstall: numpy
    Found existing installation: numpy 1.18.1
    Uninstalling numpy-1.18.1:
      Successfully uninstalled numpy-1.18.1
  Attempting uninstall: tornado
    Found existing installation: tornado 6.0.3
    Uninstalling tornado-6.0.3:
      Successfully uninstalled tornado-6.0.3
  Attempting uninstall: MarkupSafe
    Found existing installation: MarkupSafe 1.1.1
    Uninstalling MarkupSafe-1.1.1:
      Successfully uninstalled MarkupSafe-1.1.1
  Attempting uninstall: Jinja2
    Found existing installation: Jinja2 2.11.1
    Uninstalling Jinja2-2.11.1:
      Successfully uninstalled Jinja2-2.11.1
  Attempting uninstall: PyYAML
    Found existing installation: PyYAML 5.3
ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

Despite having run conda remove django and then pip install django from within the Python/3.5 virtualenv I get later errors about it not being available? Also the error above is claiming Python > 3 when it clearly:

(venv) (base) mtg@mtg-ThinkPad-P53:~/GenESysV$ sudo apt-get install memcached
[sudo] password for mtg: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
memcached is already the newest version (1.5.22-2).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
(venv) (base) mtg@mtg-ThinkPad-P53:~/GenESysV$ python manage.py makemigrations core
Traceback (most recent call last):
  File "manage.py", line 8, in <module>
    from django.core.management import execute_from_command_line
ImportError: No module named 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 14, in <module>
    import django
ImportError: No module named 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 17, in <module>
    "Couldn't import Django. Are you sure it's installed and "
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
(venv) (base) mtg@mtg-ThinkPad-P53:~/GenESysV$ pip install django
Collecting django
  Using cached Django-3.0.6-py3-none-any.whl (7.5 MB)
Collecting asgiref~=3.2
  Using cached asgiref-3.2.7-py2.py3-none-any.whl (19 kB)
Requirement already satisfied: pytz in /home/mtg/anaconda3/lib/python3.7/site-packages (from django) (2018.4)
Collecting sqlparse>=0.2.2
  Using cached sqlparse-0.3.1-py2.py3-none-any.whl (40 kB)
Installing collected packages: asgiref, sqlparse, django
Successfully installed asgiref-3.2.7 django-3.0.6 sqlparse-0.3.1
(venv) (base) mtg@mtg-ThinkPad-P53:~/GenESysV$ python manage.py makemigrations core
Traceback (most recent call last):
  File "manage.py", line 8, in <module>
    from django.core.management import execute_from_command_line
ImportError: No module named 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 14, in <module>
    import django
ImportError: No module named 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 17, in <module>
    "Couldn't import Django. Are you sure it's installed and "
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?

@jianxinwang
Copy link
Contributor

Looks like you have anaconda3 installed and its on your path:
"/home/mtg/anaconda3/lib/python3.7/site-packages"

This could be the source of trouble. Can you try to remove it from your env?

@moldach
Copy link
Author

moldach commented May 20, 2020

Hi @jianxinwang

I've deactivated conda and tried again.

(base) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ conda deactivate
mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ source venv/bin/activate

This time it looks like the issue is it cannot find elasticsearch - although I installed it and tested:

(venv) mtg@mtg-ThinkPad-P53:~$ wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
OK
(venv) mtg@mtg-ThinkPad-P53:~$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.3.deb
--2020-05-20 09:45:35--  https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.3.deb
Resolving artifacts.elastic.co (artifacts.elastic.co)... 2a04:4e42:d::734, 151.101.54.222
Connecting to artifacts.elastic.co (artifacts.elastic.co)|2a04:4e42:d::734|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28379628 (27M) [binary/octet-stream]
Saving to: ‘elasticsearch-6.1.3.deb’

elasticsearch-6.1.3.deb 100%[=============================>]  27.06M  6.54MB/s    in 4.6s    

2020-05-20 09:45:40 (5.94 MB/s) - ‘elasticsearch-6.1.3.deb’ saved [28379628/28379628]

(venv) mtg@mtg-ThinkPad-P53:~$ sudo dpkg -i elasticsearch-6.1.3.deb
(Reading database ... 249736 files and directories currently installed.)
Preparing to unpack elasticsearch-6.1.3.deb ...
Unpacking elasticsearch (6.1.3) over (6.1.3) ...
Setting up elasticsearch (6.1.3) ...
Processing triggers for systemd (245.4-4ubuntu3) ...
(venv) mtg@mtg-ThinkPad-P53:~$ sudo su -
root@mtg-ThinkPad-P53:~# nano /etc/elasticsearch/jvm.options 
root@mtg-ThinkPad-P53:~# nano /etc/elasticsearch/elasticsearch.yml 
root@mtg-ThinkPad-P53:~# systemctl daemon-reload
root@mtg-ThinkPad-P53:~# systemctl enable elasticsearch.service
Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable elasticsearch
root@mtg-ThinkPad-P53:~# systemctl start elasticsearch.service
root@mtg-ThinkPad-P53:~# curl 127.0.0.1:9200
{
  "name" : "fC5Em3D",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "paQ90Q54TNyBYqaI-A_0GQ",
  "version" : {
    "number" : "6.1.3",
    "build_hash" : "af51318",
    "build_date" : "2018-01-26T18:22:55.523Z",
    "build_snapshot" : false,
    "lucene_version" : "7.1.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

So next I exit root (I'm now within the Python3.5 venv and not Conda) and then try the example:

root@mtg-ThinkPad-P53:~# exit
logout

(venv) mtg@mtg-ThinkPad-P53:~$ cd ~/bin/GenESysV/
(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python utils/load_vcf.py --vcf test_data/test_4families_annovar.vcf.gz --tmp_dir /tmp --annot annovar --hostname 127.0.0.1 --port 9200 --index test_4families_annovar --study_name test_4families_annovar --dataset_name test_4families_annovar --num_cores 4 --assembly hg19 --ped test_data/test_4families.ped --cleanup
Traceback (most recent call last):
  File "utils/load_vcf.py", line 21, in <module>
    import elasticsearch
ImportError: No module named 'elasticsearch'

If I open the python and try to import elasticsearch it fails.
I cannot download with pip? but seemingly can with pip3 although the Python versions don't seem to line-up and I still cannot find elasticsearch from the venv running Python3.5:

(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ pip3 install elasticsearch
Requirement already satisfied: elasticsearch in /home/mtg/.local/lib/python3.8/site-packages (7.7.0)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from elasticsearch) (2019.11.28)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3/dist-packages (from elasticsearch) (1.25.8)
(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python --version
Python 3.5.0
(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ pip install elasticsearch

Command 'pip' not found, but there are 18 similar ones.

Trying to install pip this way then forces elasticsearch to be installed under Python2.7 instead of Python3.5:

(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ sudo python /usr/lib/python3/dist-packages/easy_install.py pip
/usr/lib/python3/dist-packages/pkg_resources/py2_warn.py:22: UserWarning: Setuptools will stop working on Python 2
************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release (no sooner than 2020-04-20).
Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
WARNING: The easy_install command is deprecated and will be removed in a future version.
Searching for pip
Best match: pip 20.0.2
Adding pip 20.0.2 to easy-install.pth file
Installing pip script to /usr/local/bin
Installing pip3.8 script to /usr/local/bin
Installing pip3 script to /usr/local/bin

Using /usr/lib/python3/dist-packages
Processing dependencies for pip
Finished processing dependencies for pip
(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ pip install elasticsearch
/usr/lib/python3/dist-packages/pkg_resources/py2_warn.py:22: UserWarning: Setuptools will stop working on Python 2
************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release (no sooner than 2020-04-20).
Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please comment in
https://github.com/pypa/setuptools/issues/1458
about the steps that led to this unsupported combination.
************************************************************
  sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting elasticsearch
  Using cached elasticsearch-7.7.0-py2.py3-none-any.whl (99 kB)
Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3/dist-packages (from elasticsearch) (1.25.8)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from elasticsearch) (2019.11.28)
Installing collected packages: elasticsearch
Successfully installed elasticsearch-7.7.0
(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python utils/load_vcf.py --vcf test_data/test_4families_annovar.vcf.gz --tmp_dir /tmp --annot annovar --hostname 127.0.0.1 --port 9200 --index test_4families_annovar --study_name test_4families_annovar --dataset_name test_4families_annovar --num_cores 4 --assembly hg19 --ped test_data/test_4families.ped --cleanup
Traceback (most recent call last):
  File "utils/load_vcf.py", line 21, in <module>
    import elasticsearch
ImportError: No module named 'elasticsearch'

How do I fix this issue?

@jianxinwang
Copy link
Contributor

jianxinwang commented May 20, 2020 via email

@jianxinwang
Copy link
Contributor

Hi Matt,

I have updated our code to make it work on recent versions of Elasticsearch. The installation process had been simplified a bit. Please pull the new code and give it a try. Hopefully this will work out-of-the-box.

@moldach
Copy link
Author

moldach commented May 28, 2020

Hi Matt, I recommend that you start an installation from a clean ubuntu environment or do a test install on a virtualbox running Ubuntu as guest OS.

I opened a new issue for a clean Ubuntu 20.04 OS on a VM (above.)

I have updated our code to make it work on recent versions of Elasticsearch. The installation process had been simplified a bit. Please pull the new code and give it a try. Hopefully this will work out-of-the-box.

I will continue to troubleshoot this for my computer here.

@moldach
Copy link
Author

moldach commented May 28, 2020

Pulling the new code I get the following error:

^C(venv) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python utils/load_vcf.py --vcf test_data/test_4families_annovar.vcf.gz --tmp_dir /tmp --annot annovar --hostname 127.0.0.1 --port 9200 --index test_4families_annovar --study_name test_4families_annovar --dataset_name test_4families_annovar --num_cores 4 --assembly hg19 --ped test_data/test_4families.ped --cleanup
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8c88>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8d68>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fc0b8>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8e10>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8c88>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8400>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e82b0>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff261156c88>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fc160>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fcba8>: Failed to establish a new connection: [Errno 111] Connection refused
GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s]
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "utils/load_vcf.py", line 1587, in <module>
    es.cluster.health(wait_for_status='yellow')
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/client/cluster.py", line 33, in health
    'health', index), params=params)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/transport.py", line 314, in perform_request
    status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 175, in perform_request
    raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused)

@jianxinwang
Copy link
Contributor

jianxinwang commented May 28, 2020 via email

@moldach
Copy link
Author

moldach commented May 28, 2020

Looks like your elasticserch is not running yet. You need to start it first.

On Thu, May 28, 2020 at 2:13 PM Matthew J. Oldach @.***> wrote: Pulling the new code I get the following error: ^C(venv) @.***ThinkPad-P53:~/bin/GenESysV$ python utils/load_vcf.py --vcf test_data/test_4families_annovar.vcf.gz --tmp_dir /tmp --annot annovar --hostname 127.0.0.1 --port 9200 --index test_4families_annovar --study_name test_4families_annovar --dataset_name test_4families_annovar --num_cores 4 --assembly hg19 --ped test_data/test_4families.ped --cleanup GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8c88>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8d68>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fc0b8>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8e10>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8c88>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e8400>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610e82b0>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff261156c88>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fc160>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fcba8>: Failed to establish a new connection: [Errno 111] Connection refused GET http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow [status:N/A request:0.000s] Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, **httplib_request_kw) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/home/mtg/.pyenv/versions/3.6.0/lib/python3.6/http/client.py", line 964, in send self.connect() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "utils/load_vcf.py", line 1587, in es.cluster.health(wait_for_status='yellow') File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped return func(*args, params=params, **kwargs) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/client/cluster.py", line 33, in health 'health', index), params=params) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/transport.py", line 314, in perform_request status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout) File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 175, in perform_request raise ConnectionError('N/A', str(e), e) elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff2610fcc88>: Failed to establish a new connection: [Errno 111] Connection refused) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX6JWZXO7THMIKCUJ73NN3RT2SUPANCNFSM4MSMV7JQ .

Ahhh okay, right. So I run ~/elasticsearch-7.7.0/bin/elasticsearch from one terminal window:

(venv) ubuntu@ubunu2004:~/GenESysV$  ~/elasticsearch-7.7.0/bin/elasticsearch
[2020-05-29T01:08:45,169][WARN ][o.e.b.JNANatives         ] [ubunu2004] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
[2020-05-29T01:08:45,184][WARN ][o.e.b.JNANatives         ] [ubunu2004] This can result in part of the JVM being swapped out.
[2020-05-29T01:08:45,186][WARN ][o.e.b.JNANatives         ] [ubunu2004] Increase RLIMIT_MEMLOCK, soft limit: 67108864, hard limit: 67108864
[2020-05-29T01:08:45,194][WARN ][o.e.b.JNANatives         ] [ubunu2004] These can be adjusted by modifying /etc/security/limits.conf, for example: 
	# allow user 'ubuntu' mlockall
	ubuntu soft memlock unlimited
	ubuntu hard memlock unlimited
[2020-05-29T01:08:45,219][WARN ][o.e.b.JNANatives         ] [ubunu2004] If you are logged in interactively, you will have to re-login for the new limits to take effect.
[2020-05-29T01:08:45,632][INFO ][o.e.e.NodeEnvironment    ] [ubunu2004] using [1] data paths, mounts [[/home (/dev/sda6)]], net usable_space [7.2gb], net total_space [9.3gb], types [ext4]
[2020-05-29T01:08:45,638][INFO ][o.e.e.NodeEnvironment    ] [ubunu2004] heap size [4gb], compressed ordinary object pointers [true]
[2020-05-29T01:08:45,835][INFO ][o.e.n.Node               ] [ubunu2004] node name [ubunu2004], node ID [4DE9bxYmSQyAAkAADSBl0g], cluster name [elasticsearch]
[2020-05-29T01:08:45,841][INFO ][o.e.n.Node               ] [ubunu2004] version[7.7.0], pid[2299], build[default/tar/81a1e9eda8e6183f5237786246f6dced26a10eaf/2020-05-12T02:01:37.602180Z], OS[Linux/5.4.0-33-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14/14+36]
[2020-05-29T01:08:45,842][INFO ][o.e.n.Node               ] [ubunu2004] JVM home [/home/ubuntu/elasticsearch-7.7.0/jdk]
[2020-05-29T01:08:45,842][INFO ][o.e.n.Node               ] [ubunu2004] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms4g, -Xmx4g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-15431798403291466034, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=2147483648, -Des.path.home=/home/ubuntu/elasticsearch-7.7.0, -Des.path.conf=/home/ubuntu/elasticsearch-7.7.0/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2020-05-29T01:08:49,776][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [aggs-matrix-stats]
[2020-05-29T01:08:49,785][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [analysis-common]
[2020-05-29T01:08:49,788][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [constant-keyword]
[2020-05-29T01:08:49,788][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [flattened]
[2020-05-29T01:08:49,789][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [frozen-indices]
[2020-05-29T01:08:49,789][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [ingest-common]
[2020-05-29T01:08:49,789][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [ingest-geoip]
[2020-05-29T01:08:49,789][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [ingest-user-agent]
[2020-05-29T01:08:49,790][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [lang-expression]
[2020-05-29T01:08:49,790][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [lang-mustache]
[2020-05-29T01:08:49,790][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [lang-painless]
[2020-05-29T01:08:49,791][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [mapper-extras]
[2020-05-29T01:08:49,791][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [parent-join]
[2020-05-29T01:08:49,792][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [percolator]
[2020-05-29T01:08:49,792][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [rank-eval]
[2020-05-29T01:08:49,792][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [reindex]
[2020-05-29T01:08:49,792][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [repository-url]
[2020-05-29T01:08:49,793][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [search-business-rules]
[2020-05-29T01:08:49,793][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [spatial]
[2020-05-29T01:08:49,804][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [tasks]
[2020-05-29T01:08:49,806][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [transform]
[2020-05-29T01:08:49,806][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [transport-netty4]
[2020-05-29T01:08:49,806][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [vectors]
[2020-05-29T01:08:49,807][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-analytics]
[2020-05-29T01:08:49,807][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-async-search]
[2020-05-29T01:08:49,807][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-autoscaling]
[2020-05-29T01:08:49,808][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-ccr]
[2020-05-29T01:08:49,809][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-core]
[2020-05-29T01:08:49,809][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-deprecation]
[2020-05-29T01:08:49,809][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-enrich]
[2020-05-29T01:08:49,809][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-eql]
[2020-05-29T01:08:49,810][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-graph]
[2020-05-29T01:08:49,810][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-identity-provider]
[2020-05-29T01:08:49,810][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-ilm]
[2020-05-29T01:08:49,811][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-logstash]
[2020-05-29T01:08:49,811][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-ml]
[2020-05-29T01:08:49,811][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-monitoring]
[2020-05-29T01:08:49,811][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-ql]
[2020-05-29T01:08:49,812][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-rollup]
[2020-05-29T01:08:49,812][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-security]
[2020-05-29T01:08:49,812][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-sql]
[2020-05-29T01:08:49,812][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-voting-only-node]
[2020-05-29T01:08:49,813][INFO ][o.e.p.PluginsService     ] [ubunu2004] loaded module [x-pack-watcher]
[2020-05-29T01:08:49,814][INFO ][o.e.p.PluginsService     ] [ubunu2004] no plugins loaded
[2020-05-29T01:08:56,508][INFO ][o.e.x.s.a.s.FileRolesStore] [ubunu2004] parsed [0] roles from file [/home/ubuntu/elasticsearch-7.7.0/config/roles.yml]
[2020-05-29T01:08:57,902][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [ubunu2004] [controller/2461] [Main.cc@110] controller (64 bit): Version 7.7.0 (Build a8939d3da43f33) Copyright (c) 2020 Elasticsearch BV
[2020-05-29T01:08:58,677][INFO ][o.e.d.DiscoveryModule    ] [ubunu2004] using discovery type [zen] and seed hosts providers [settings]
[2020-05-29T01:09:00,191][INFO ][o.e.n.Node               ] [ubunu2004] initialized
[2020-05-29T01:09:00,193][INFO ][o.e.n.Node               ] [ubunu2004] starting ...
[2020-05-29T01:09:00,463][INFO ][o.e.t.TransportService   ] [ubunu2004] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2020-05-29T01:09:00,997][WARN ][o.e.b.BootstrapChecks    ] [ubunu2004] memory locking requested for elasticsearch process but memory is not locked
[2020-05-29T01:09:00,998][WARN ][o.e.b.BootstrapChecks    ] [ubunu2004] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2020-05-29T01:09:01,002][INFO ][o.e.c.c.Coordinator      ] [ubunu2004] cluster UUID [98cEay2-Sm6dVD4ER3IQsw]
[2020-05-29T01:09:01,049][INFO ][o.e.c.c.ClusterBootstrapService] [ubunu2004] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2020-05-29T01:09:01,264][INFO ][o.e.c.s.MasterService    ] [ubunu2004] elected-as-master ([1] nodes joined)[{ubunu2004}{4DE9bxYmSQyAAkAADSBl0g}{Cg60PWenSoeaFxR6cxCzhw}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=2084360192, xpack.installed=true, transform.node=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 2, version: 28, delta: master node changed {previous [], current [{ubunu2004}{4DE9bxYmSQyAAkAADSBl0g}{Cg60PWenSoeaFxR6cxCzhw}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=2084360192, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}
[2020-05-29T01:09:01,346][INFO ][o.e.c.s.ClusterApplierService] [ubunu2004] master node changed {previous [], current [{ubunu2004}{4DE9bxYmSQyAAkAADSBl0g}{Cg60PWenSoeaFxR6cxCzhw}{127.0.0.1}{127.0.0.1:9300}{dilmrt}{ml.machine_memory=2084360192, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}, term: 2, version: 28, reason: Publication{term=2, version=28}
[2020-05-29T01:09:01,440][INFO ][o.e.h.AbstractHttpServerTransport] [ubunu2004] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2020-05-29T01:09:01,441][INFO ][o.e.n.Node               ] [ubunu2004] started
[2020-05-29T01:09:01,793][INFO ][o.e.l.LicenseService     ] [ubunu2004] license [ccb73465-ee3a-4b3b-b1d9-1f5f8ddff57c] mode [basic] - valid
[2020-05-29T01:09:01,794][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [ubunu2004] Active license is now [BASIC]; Security is disabled
[2020-05-29T01:09:01,823][INFO ][o.e.g.GatewayService     ] [ubunu2004] recovered [0] indices into cluster_state

Then open another window and run the following:

(venv) ubuntu@ubunu2004:~/GenESysV$ python utils/load_vcf.py --vcf test_data/test_4families_annovar.vcf.gz --tmp_dir /tmp --annot annovar --hostname 127.0.0.1 --port 9200 --index test_4families_annovar --study_name test_4families_annovar --dataset_name test_4families_annovar --num_cores 4 --assembly hg19 --ped test_data/test_4families.ped --cleanup
Sqlite error: no such table: core_dataset



Pid 2800: processed 5000 variants
Pid 2801: processed 5000 variants
Pid 2802: processed 5000 variants
Pid 2803: processed 5000 variants
Pid 2800: processed 10000 variants
Pid 2801: processed 10000 variants
Pid 2802: processed 10000 variants
Pid 2803: processed 10000 variants
Pid 2800: processed 12501 variants
Process 2800 finished ...
Pid 2801: processed 12501 variants
Process 2801 finished ...
Pid 2802: processed 12501 variants
Process 2802 finished ...
Pid 2803: processed 12498 variants
Process 2803 finished ...
Finished parsing vcf file in 69.25930881500244 seconds, now creating ElasticSearch index ...
HEAD http://127.0.0.1:9200/test_4families_annovar [status:N/A request:0.001s]
Traceback (most recent call last):
  File "/home/ubuntu/GenESysV/venv/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/ubuntu/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/home/ubuntu/GenESysV/venv/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

@jianxinwang
Copy link
Contributor

Did you run these two lines?

python manage.py makemigrations core
python manage.py migrate

These commands create and populate the sqlite tables.

@moldach
Copy link
Author

moldach commented May 28, 2020

I thought I had but maybe I didn't?

Trying again:

(venv) (base) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python manage.py makemigrations core
No changes detected in app 'core'
(venv) (base) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, core, sessions
Running migrations:
  No migrations to apply.

Able to run the following:

~/elasticsearch-7.7.0/bin/elasticsearch # then open a new terminal

now run

python manage.py runserver 0.0.0.0:8000 

Close it with Ctrl + C and then tried

(venv) (base) mtg@mtg-ThinkPad-P53:~/bin/GenESysV$ python utils/load_vcf.py --vcf test_data/test_4families_annovar.vcf.gz --tmp_dir /tmp --annot annovar --hostname 127.0.0.1 --port 9200 --index test_4families_annovar --study_name test_4families_annovar --dataset_name test_4families_annovar --num_cores 4 --assembly hg19 --ped test_data/test_4families.ped --cleanup
Pid 161606: processed 5000 variants
Pid 161607: processed 5000 variants
Pid 161605: processed 5000 variants
Pid 161604: processed 5000 variants
Pid 161606: processed 10000 variants
Pid 161604: processed 10000 variants
Pid 161607: processed 10000 variants
Pid 161605: processed 10000 variants
Pid 161606: processed 12501 variants
Pid 161604: processed 12501 variants
Process 161604 finished ...
Pid 161607: processed 12498 variants
Pid 161605: processed 12501 variants
Process 161605 finished ...
Process 161606 finished ...
Process 161607 finished ...
Finished parsing vcf file in 21.185588836669922 seconds, now creating ElasticSearch index ...
deleting 'test_4families_annovar' index...
response: '{'acknowledged': True}'
creating 'test_4families_annovar' index...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   590  100    98  100   492    453   2277 --:--:-- --:--:-- --:--:--  2731
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 23038  100    28  100 23010    528   423k --:--:-- --:--:-- --:--:--  424k
Response: '{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "test_4families_annovar"
}
{
  "acknowledged" : true
}
'
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_0.json
Took: 5.210810899734497 seconds
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_1.json
Took: 4.1324498653411865 seconds
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_2.json
Took: 4.524986743927002 seconds
Indexing file /tmp/test_4families_annovar.vcf.gz.chunk_3.json
Took: 4.066014051437378 seconds
Finished creating ES index, parsing time: 21.185588836669922 seconds, indexing time: 18.4201762676239 seconds, vcf: test_data/test_4families_annovar.vcf.gz

Creating Web user interface, please wait ...
Unmapped GUI CLNVAR_nested
Unmapped GUI COSMIC_nested
Unmapped GUI GTEx_nested
Unmapped GUI ICGC_nested
Unmapped GUI RGQ
Unmapped GUI mendelian_diseases
Unmapped GUI RGQ
Unmapped GUI mendelian_diseases
********************************************************************************

Study Name: test_4families_annovar
Dataset Name: test_4families_annovar_hg19
Dataset ES Index Name: test_4families_annovar
Dataset ES Type Name: test_4families_annovar_
Dataset ES Host: 127.0.0.1
Dataset ES Port: 9200
Traceback (most recent call last):
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 303, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: main.core_analysistype__old

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "utils/load_vcf.py", line 1715, in <module>
    make_gui(es, hostname, port, index_name, study, dataset_name, type_name, gui_mapping)
  File "/home/mtg/bin/GenESysV/utils/make_gui.py", line 959, in make_gui
    dataset_obj.analysis_type.add(*a)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/models/fields/related_descriptors.py", line 898, in add
    self._add_items(self.source_field_name, self.target_field_name, *objs)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/models/fields/related_descriptors.py", line 1065, in _add_items
    for obj_id in new_ids
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/models/query.py", line 466, in bulk_create
    ids = self._batched_insert(objs_without_pk, fields, batch_size)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/models/query.py", line 1148, in _batched_insert
    self._insert(item, fields=fields, using=self.db)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/models/query.py", line 1125, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1283, in execute_sql
    cursor.execute(sql, params)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/mtg/bin/GenESysV/venv/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 303, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: main.core_analysistype__old

@jianxinwang
Copy link
Contributor

jianxinwang commented May 29, 2020 via email

@moldach
Copy link
Author

moldach commented May 29, 2020

I'm using Ubuntu 20.04 not 18.04.

When trying to run pip install -r requirements.txt I get an error from pkg-resources==0.0.0 which caused the script to stop at that line. I needed to comment it out with #:

ERROR: Could not find a version that satisfies the requirement pkg-resources==0.0.0 (from -r requirements.txt (line 38)) (from versions: none)
ERROR: No matching distribution found for pkg-resources==0.0.0 (from -r requirements.txt (line 38))

@jianxinwang
Copy link
Contributor

jianxinwang commented May 29, 2020 via email

@linnil1
Copy link

linnil1 commented Jun 15, 2021

I reproduce this error in docker container, too.

I solve it by upgrade Django from 2.0.5 to 2.1.15

Maybe it's related to this thread https://stackoverflow.com/questions/53637182/django-no-such-table-main-auth-user-old

@jianxinwang
Copy link
Contributor

jianxinwang commented Jun 15, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants