Getting HTTP connection error

[InosRahul]

p2o.py --enrich --index git_raw --index-enrich git -e http://localhost:9200 --no_inc --debug git https://github.com/chaoss/grimoirelab-perceval.git
Output:
2019-03-08 17:14:10,910 Debug mode activated
2019-03-08 17:14:10,910 Feeding Ocean from git (git_raw)
2019-03-08 17:14:10,917 Retrying (Retry(total=20, connect=20, read=8, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6700565ac8>: Failed to establish a new connection: [Errno 111] Connection refused',)': /
2019-03-08 17:14:11,318 Retrying (Retry(total=19, connect=19, read=8, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f67005655c0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /
2019-03-08 17:14:12,122 Retrying (Retry(total=18, connect=18, read=8, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f67005653c8>: Failed to establish a new connection: [Errno 111] Connection refused',)': /
2019-03-08 17:14:13,725 Retrying (Retry(total=17, connect=17, read=8, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f67005650f0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /
2019-03-08 17:14:16,930 Retrying (Retry(total=16, connect=16, read=8, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f670055eef0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /
2019-03-08 17:14:23,338 Retrying (Retry(total=15, connect=15, read=8, redirect=5, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f670055eba8>: Failed to establish a new connection: [Errno 111] Connection refused',)': /

[valeriocos]

Is your elasticsearch (http://localhost:9200) up and running?

[valeriocos]

@InosRahul I close this issue, feel free to reopen it if needed

[InosRahul]

Actually I solved the error, Elasticsearch was not running, configured everything in docker using the grimoirelab/full image from docker hub

[valeriocos]

thank you for reporting the solution @InosRahul

[InosRahul]

I have actually ran into another issue, when using this command
p2o.py --enrich --index git_raw --index-enrich git -e http://localhost:9200 --no_inc --debug git https://github.com/grimoirelab/perceval.git

I am able to create the git_raw index, but the git index is not being created, the error I am getting is something like this,

es'
Traceback (most recent call last):
File "/root/chaoss/lib/python3.5/site-packages/grimoire_elk/elk.py", line 550, in enrich_backend
db_user, db_password, db_host)
File "/root/chaoss/lib/python3.5/site-packages/grimoire_elk/enriched/git.py", line 94, in init
db_user, db_password, db_host)
File "/root/chaoss/lib/python3.5/site-packages/grimoire_elk/enriched/enrich.py", line 124, in init
Enrich.sh_db = Database(db_user, db_password, db_sortinghat, db_host)
File "/root/chaoss/lib/python3.5/site-packages/sortinghat/db/database.py", line 58, in init
self.__create_schema(self._engine)
File "/root/chaoss/lib/python3.5/site-packages/sortinghat/db/database.py", line 167, in __create_schema
ModelBase.metadata.create_all(engine)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/sql/schema.py", line 4200, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 2010, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/lib/python3.5/contextlib.py", line 59, in enter
return next(self.gen)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 2002, in _optional_conn_ctx_manager
with self.contextual_connect() as conn:
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 2192, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 2228, in _wrap_pool_connect
return fn()
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 425, in connect
return _ConnectionFairy._checkout(self)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 822, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 554, in checkout
rec = pool._do_get()
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 1250, in _do_get
self._dec_overflow()
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 67, in exit
compat.reraise(exc_type, exc_value, exc_tb)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 277, in reraise
raise value
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 1247, in _do_get
return self._create_connection()
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 370, in _create_connection
return _ConnectionRecord(self)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 499, in init
self.__connect(first_connect_check=True)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/pool.py", line 701, in __connect
connection = pool._invoke_creator(self)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/root/chaoss/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 437, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/root/chaoss/lib/python3.5/site-packages/pymysql/init.py", line 94, in Connect
return Connection(*args, **kwargs)
File "/root/chaoss/lib/python3.5/site-packages/pymysql/connections.py", line 325, in init
self.connect()
File "/root/chaoss/lib/python3.5/site-packages/pymysql/connections.py", line 599, in connect
self._request_authentication()
File "/root/chaoss/lib/python3.5/site-packages/pymysql/connections.py", line 846, in _request_authentication
data += self.db + b'\0'
TypeError: unsupported operand type(s) for +: 'bool' and 'bytes'

@valeriocos

[valeriocos]

the problem could be related to which version of mariadb you are using. it should be 10.2.

[InosRahul]

I am running mariadb version 10.2 through a docker container but the problem still persists, also it occurs when using the grimoirelab/(full) image in docker.

[valeriocos]

Sorry for the late reply @InosRahul , it seems that the problem was related to a recent change in ELK, which was reflected in p2o. The following PR should fix the issue you reported: #576 (I guess it will be merged to the master branch in the following days).

Could you check if this solves your issue? Thank you!

[harshalmittal4]

Hey @valeriocos ,
The problem still persists when I use the command
p2o.py --enrich --index perceval_git_raw --index-enrich perceval_git -e http://localhost:9200 --no_inc --debug --db-host 127.0.0.1 --db-sortinghat sortinghatDB --db-user root git https://github.com/chaoss/grimoirelab-perceval
to generate index for git date source. I was doing this for generating the report.
It is using grimoire-elk==0.36.0.

[valeriocos]

Thank you @harshalmittal4 for reporting this issue.
I have just checked it with the latest version of ELK 0.41.0 and I don't see any error.
Could you please try it with the latest version ? Thank you

[valeriocos]

any news on this issue @harshalmittal4 , can we close it ?

[valeriocos]

any news on this issue @harshalmittal4 , can we close it ?

[harshalmittal4]

Yes sure @valeriocos 😅 I already closed the related issue by me here since the error was resolved some time back!

[valeriocos]

perfect! thank you for the feedback @harshalmittal4

[harshalmittal4]

Sure @valeriocos, getting off of this one ☺️