Error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters' #62

andy812 · 2020-09-14T12:21:34Z

I use query-exporter with oracle database. My config:

databases:
  devdb:
    dsn: oracle://user:passvord@example.com:1521/devdb
    keep-connected: true

metrics:
  new_test_metric_oracle:
    type: gauge
    description: test metrics

queries:
  query1:
    interval: 1m
    databases: [primedb]
    metrics: [new_test_metric_oracle]
    sql: |
      select count(1) as new_test_metric_oracle from TCTDBS.ACQUIRERBILATERALS;

I added instantclient 19.8 in adonato/query-exporter:2.4.0 image according fb08e31
Logs:

2020-09-14 12:04:45,647 - INFO - aiohttp.web - Listening on http://0.0.0.0:9560
2020-09-14 12:04:45,761 - ERROR - query-exporter - error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters'.
2020-09-14 12:04:45,762 - DEBUG - query-exporter - updating metric "database_errors" inc 1 {database="devdb"}
2020-09-14 12:04:45,841 - ERROR - query-exporter - error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters'.
2020-09-14 12:04:45,842 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="devdb",status="error"}

Any ideas how to fix it?

The text was updated successfully, but these errors were encountered:

xuanyuanaosheng · 2020-09-21T08:11:51Z

@andy812 I had the same problem. The new oracledb 19.6 has this problem. The oracledb version 11.2.0.4 works fine.

Teriand · 2020-09-23T13:48:02Z

same problem today with default user right.
after add all right - work fine

I don't know exactly which rights from the list helped

-- Create the user 
create user CLEAR_LOG
  default tablespace USERS
  temporary tablespace TEMP
  profile DEFAULT
  quota unlimited on users;
-- Grant/Revoke role privileges 
grant connect to CLEAR_LOG;
-- Grant/Revoke system privileges 
grant alter any table to CLEAR_LOG;
grant create any index to CLEAR_LOG;
grant create database link to CLEAR_LOG;
grant create job to CLEAR_LOG;
grant create procedure to CLEAR_LOG;
grant create sequence to CLEAR_LOG;
grant create table to CLEAR_LOG;
grant debug connect session to CLEAR_LOG;
grant delete any table to CLEAR_LOG;
grant drop any index to CLEAR_LOG;
grant drop any table to CLEAR_LOG;
grant select any dictionary to CLEAR_LOG;
grant select any table to CLEAR_LOG;

manartezekov · 2020-11-10T03:35:43Z

connect to Oracle 19.* doesn't work? I have same problem with connect?

searchgithub · 2020-11-17T08:46:22Z

I have the same issue on "Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production"
ERROR - query-exporter - error from database "my_db": 'AsyncioEngine' object has no attribute 'hide_parameters' with "query-exporter 2.2.0"

But "1.4.0" is working fine.

Just give this access is enough. But our "security team" still need the reason why need this.
grant select any dictionary to CLEAR_LOG;

albertodonato · 2020-11-17T22:14:31Z

Hi. I unfortunately don't have access to an oracle db to test.

Is there any way to setup a test server locally with docker?

aorfanos · 2020-11-24T18:11:58Z

Issue also exists in 'query-exporter:latest/2.5.0', connecting to MySQL 5.7.

I receive the following message:

2020-11-24 17:52:00,927 - ERROR - query-exporter - error from database "devtestdb": 'AsyncioEngine' object has no attribute 'hide_parameters'.

My config.yaml looks like:

databases:
    dsn: mysql://user:pass@host:port/testdb

metrics:
  sample_metric:
    type: gauge
    description: How many seconds have passed since update

queries:
  sample_metric:
    interval: 15
    databases: [testdb]
    metrics: [sample_metric]
    sql: SELECT UNIX_TIMESTAMP(now()) - UNIX_TIMESTAMP(lastsent) AS sample_metric FROM randomtable ORDER BY lastsent DESC LIMIT 1

albertodonato · 2020-11-24T18:24:04Z

Thanks for the info, I'll try to reproduce locally

albertodonato · 2020-11-24T20:12:57Z

I just tested with the same config and 2.5.0, it works fine for me.

albertodonato · 2020-11-24T20:13:59Z

Does running with -LDEBUG give any more details in the error?

aorfanos · 2020-11-25T12:18:00Z

thanks for looking at it so quickly @albertodonato . Running with -LDEBUG showed the following:

socket.gaierror: [Errno -2] Name or service not known

UPDATE:

It was my mistake after all, I was connecting to a proxysql host. I changed the DSN to the proper hostname and this fixed the error, also added custom DNS resolvers. False alarm. Thanks again @albertodonato for your help.

albertodonato · 2020-11-25T12:50:41Z

This seems to point at a DNS issue. Maybe the hostname for the db doesn't get resolved. Can you try with the IP?

…

On Wed, Nov 25, 2020 at 1:18 PM aorfanos ***@***.***> wrote: thanks for looking at it so quickly @albertodonato <https://github.com/albertodonato> . Running with -LDEBUG showed the following: socket.gaierror: [Errno -2] Name or service not known — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#62 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2PWD75B4FW2TCCLMS2UQ3SRTYYNANCNFSM4RLRBUOA> .

albertodonato · 2020-11-25T23:45:20Z

FTR I just released 2.5.1, which adds tracebacks for errors when -L DEBUG is passed.
If someone can reproduce this issue, please attach log for the error, which should hopefully include useful info.

igna-v · 2020-12-28T19:58:42Z

Im having the same issue using with an informix database

databases:
db1:
dsn: mysql://exporter_user:pass@xxx.xxx.xxx.xxx:3306/test?auth_plugin='mysql_native_password'
db2:
dsn: ibm_db_sa://informix:in4mix@xxx.xxx.xxx.xxx:9089/test
#dsn: db2+ibm_db://informix:in4mix@xxx.xxx.xxx.xxx:9089/test

Im running in debug mode and this is what I get

2020-12-28 19:49:55,690 - INFO - aiohttp.web - Listening on http://0.0.0.0:9560
2020-12-28 19:49:55,729 - DEBUG - query-exporter - connected to database "db1"
2020-12-28 19:49:55,866 - ERROR - query-exporter - error from database "db2": 'AsyncioEngine' object has no attribute 'hide_parameters'.
2020-12-28 19:49:55,866 - DEBUG - query-exporter - updating metric "database_errors" inc 1 {database="db2"}
2020-12-28 19:49:55,867 - DEBUG - query-exporter - running query "query1" on database "db1"
2020-12-28 19:49:55,896 - ERROR - query-exporter - error from database "db2": 'AsyncioEngine' object has no attribute 'hide_parameters'.
2020-12-28 19:49:55,896 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="db2",query="query2",status="error"}

albertodonato · 2020-12-28T20:52:54Z

@igna-v what version are you using, and is it via snap, docker or directly installed from pip?

igna-v · 2020-12-28T21:03:21Z

@albertodonato Im using via docker and the latest tag
For informix Im using https://hub.docker.com/r/ibmcom/informix-developer-database/ also the latest one

I tried running the same query from the container and it worked (this comands in the python console):

import sqlalchemy
from sqlalchemy import *
import ibm_db_sa
db2 = sqlalchemy.create_engine("ibm_db_sa://informix:in4mix@xxx.xxx.xxx.xxx:9089/test")
with db2.connect() as connection:
result = connection.execute("select * from tablaPrueba;")
for row in result:
print(row)

Thanks for the quick responce

albertodonato · 2020-12-29T09:55:53Z

@albertodonato Im using via docker and the latest tag
For informix Im using https://hub.docker.com/r/ibmcom/informix-developer-database/ also the latest one

I tried running the same query from the container and it worked (this comands in the python console):

import sqlalchemy
from sqlalchemy import *
import ibm_db_sa
db2 = sqlalchemy.create_engine("ibm_db_sa://informix:in4mix@xxx.xxx.xxx.xxx:9089/test")
with db2.connect() as connection:
result = connection.execute("select * from tablaPrueba;")
for row in result:
print(row)

Thanks for the quick responce

@igna-v I've never used informix. could you paste some instructions on how to set up the test db (database, user, ...) with that docker so I can run query-exporter against it?

igna-v · 2020-12-29T12:09:53Z

It also is my first experience with it, so sorry I cant provide much information, I have to test the tool so the DBA can then create the queries to monitor the productive instances.

I used the instructions here, and use the default informix user (informix:in4mix).
https://github.com/informix/informix-dockerhub-readme/blob/master/12.10.FC12/informix-developer-database.md

Here I checked and the ibm-db-sa dialect is mentioned in sql alchemy as usefull for both db2 and informix.
https://docs.sqlalchemy.org/en/14/dialects/index.html

And I tried the connection from the query exporter container using this ibm guide (which is aimed at db2, but worked perfectly with informix)
https://www.ibm.com/support/knowledgecenter/es/SSEPGG_9.7.0/com.ibm.swg.im.dbclient.python.doc/doc/t0060891.html

igna-v · 2020-12-29T14:51:56Z

@albertodonato
As an update, I asked in
https://gitter.im/sqlalchemy/community
And sqlalchemy documentation is outdated as ibm_db_sa stopped supporting informix.
It kind of works when executed manually, with some issued though, which I think might trigger the 'AsyncioEngine' object has no attribute 'hide_parameters'.

Nevertheless if you can figure out how to use this query exporter with informix that would be really appreciated. Trying manually with ibm_db and ibm_db_dbi works perfectly.

albertodonato · 2020-12-30T08:58:09Z

I suspect the issues is actually RazerM/sqlalchemy_aio#36.

I pushed a test image to adonato/query-exporter:test-sqlalchemy_aio, which uses a modified sqlalchemy_aio package to address that issue.

@igna-v (or anyone else hitting this issue) mind testing with that?

igna-v · 2020-12-30T12:16:59Z

@albertodonato

Hi it solves the 'AsyncioEngine' object has no attribute 'hide_parameters'.
Still it doesnt work for Informix (I suspect because sqlalchemy stopped supporting it), but thats a different issue.

Here is the error I get in case it is useful for someone. If I find a solution or work around for informix I´ll post it.

2020-12-30 12:08:58,157 - ERROR - query-exporter - query "query2" on database "db2" failed: (ibm_db_dbi.Error) ibm_db_dbi::Error: SystemError(' returned a result with an error set')
(Background on this error at: http://sqlalche.me/e/13/dbapi)
2020-12-30 12:08:58,158 - DEBUG - query-exporter - File "/virtualenv/lib/python3.8/site-packages/query_exporter/db.py", line 306, in execute
result = await self._execute_query(query)
File "/virtualenv/lib/python3.8/site-packages/query_exporter/db.py", line 339, in _execute_query
return await self.execute_sql(
File "/virtualenv/lib/python3.8/site-packages/query_exporter/db.py", line 332, in execute_sql
return await asyncio.wait_for(
File "/usr/local/lib/python3.8/asyncio/tasks.py", line 455, in wait_for
return await fut
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy_aio/base.py", line 291, in execute
rp = await self._run_in_thread(
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy_aio/base.py", line 234, in _run_in_thread
return await _self._worker.run(_func, args, kwargs)
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy_aio/asyncio.py", line 73, in run
return request.response.unwrap()
File "/virtualenv/lib/python3.8/site-packages/outcome/_impl.py", line 138, in unwrap
raise captured_error
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self.handle_dbapi_exception(
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1514, in handle_dbapi_exception
util.raise(exc_info[1], with_traceback=exc_info[2])
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise
raise exception
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1301, in _execute_context
self._root._commit_impl(autocommit=True)
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 773, in _commit_impl
self.handle_dbapi_exception(e, None, None, None, None)
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in handle_dbapi_exception
util.raise(
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise
raise exception
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 771, in _commit_impl
self.engine.dialect.do_commit(self.connection)
File "/virtualenv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 562, in do_commit
dbapi_connection.commit()
File "/virtualenv/lib/python3.8/site-packages/ibm_db_dbi.py", line 723, in commit
raise _get_exception(inst)

Thanks!

igna-v · 2020-12-30T13:08:02Z

I know its a bit off topic but if someone is having the same issues with informix I fixed it by adding sqlalchemy-informix python package.

ibm_db_sa can stay for db2 support if needed.

I made a simple Dockerfile to add the package:

FROM adonato/query-exporter:test-sqlalchemy_aio

RUN pip install sqlalchemy-informix

ENV PATH="/virtualenv/bin:$PATH"
ENV VIRTUAL_ENV="/virtualenv"
ENV LD_LIBRARY_PATH="/opt/oracle/instantclient"

EXPOSE 9560/tcp
# IPv6 support is not enabled by default, only bind IPv4
ENTRYPOINT ["query-exporter", "/config.yaml", "-H", "0.0.0.0"]

And the dsn for informix should look like this

informix://user:pass@host[:port]/database

Thank you very much @albertodonato for all the help

albertodonato · 2020-12-30T15:52:04Z

@igna-v thatnks for testing, I'll leave this open until the issue is fixed in sqlalchemy_aio (RazerM/sqlalchemy_aio#37)

albertodonato · 2021-01-04T23:40:35Z

added a workaround for now

kullmanp · 2021-01-22T16:43:49Z

I still have this issue with 2.6.1 for an oracle database. Running with -LDEBUG it only says

2021-01-22 16:37:14,493 - INFO - aiohttp.web - Listening on http://0.0.0.0:9560
2021-01-22 16:37:15,276 - ERROR - query-exporter - error from database "ados": 'AsyncioEngine' object has no attribute 'hide_parameters'.
2021-01-22 16:37:15,277 - DEBUG - query-exporter - updating metric "database_errors" inc 1 {database="ados"}
2021-01-22 16:37:16,084 - ERROR - query-exporter - error from database "ados": 'AsyncioEngine' object has no attribute 'hide_parameters'.
2021-01-22 16:37:16,085 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="ados",query="query1",status="error"}

Running the same query with test-sqlalchemy_aio it works and I get the following output:

2021-01-22 16:41:39,428 - INFO - aiohttp.web - Listening on http://0.0.0.0:9560
/virtualenv/lib/python3.8/site-packages/sqlalchemy/dialects/oracle/base.py:1400: SAWarning: Oracle version (12, 2, 0, 1, 0) is known to have a maximum identifier length of 128, rather than the historical default of 30. SQLAlchemy 1.4 will use 128 for this database; please set max_identifier_length=128 in create_engine() in order to test the application with this new length, or set to 30 in order to assure that 30 continues to be used.  In particular, pay close attention to the behavior of database migrations as dynamically generated names may change. See the section 'Max Identifier Lengths' in the SQLAlchemy Oracle dialect documentation for background.
  util.warn(
2021-01-22 16:41:40,274 - DEBUG - query-exporter - connected to database "ados"
2021-01-22 16:41:40,275 - DEBUG - query-exporter - running query "query1" on database "ados"
2021-01-22 16:41:40,318 - DEBUG - query-exporter - updating metric "loeschung_auftrag_count" set 20354 {database="ados"}
2021-01-22 16:41:40,318 - DEBUG - query-exporter - updating metric "query_latency" observe 0.021305000002030283 {database="ados",query="query1"}
2021-01-22 16:41:40,318 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="ados",query="query1",status="success"}
2021-01-22 16:41:50,286 - DEBUG - query-exporter - running query "query1" on database "ados"
2021-01-22 16:41:50,332 - DEBUG - query-exporter - updating metric "loeschung_auftrag_count" set 20354 {database="ados"}
2021-01-22 16:41:50,333 - DEBUG - query-exporter - updating metric "query_latency" observe 0.026991199993062764 {database="ados",query="query1"}
2021-01-22 16:41:50,333 - DEBUG - query-exporter - updating metric "queries" inc 1 {database="ados",query="query1",status="success"}

mrcrch · 2021-02-25T21:36:07Z

I suspect the issues is actually RazerM/sqlalchemy_aio#36.

I pushed a test image to adonato/query-exporter:test-sqlalchemy_aio, which uses a modified sqlalchemy_aio package to address that issue.

@igna-v (or anyone else hitting this issue) mind testing with that?

I did some tests using this image and everything worked perfectly
Oracle Version: Oracle Database 18c Standard Edition 2 Release 18.0.0.0.0 - Production

Can we have a new release with this fix?

andy812 added the bug label Sep 14, 2020

albertodonato mentioned this issue Dec 29, 2020

AsyncEngine missing hide_parameters attribute RazerM/sqlalchemy_aio#36

Closed

albertodonato closed this as completed in fe7c4eb Jan 4, 2021

HEP85 mentioned this issue Jan 22, 2021

Error from database "devdb": 'AsyncioEngine' object has no attribute 'url' #76

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters' #62

Error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters' #62

andy812 commented Sep 14, 2020 •

edited

Loading

xuanyuanaosheng commented Sep 21, 2020

Teriand commented Sep 23, 2020

manartezekov commented Nov 10, 2020 •

edited

Loading

searchgithub commented Nov 17, 2020 •

edited

Loading

albertodonato commented Nov 17, 2020

aorfanos commented Nov 24, 2020

albertodonato commented Nov 24, 2020

albertodonato commented Nov 24, 2020

albertodonato commented Nov 24, 2020

aorfanos commented Nov 25, 2020 •

edited

Loading

albertodonato commented Nov 25, 2020 via email

albertodonato commented Nov 25, 2020

igna-v commented Dec 28, 2020

albertodonato commented Dec 28, 2020

igna-v commented Dec 28, 2020 •

edited

Loading

albertodonato commented Dec 29, 2020

igna-v commented Dec 29, 2020

igna-v commented Dec 29, 2020

albertodonato commented Dec 30, 2020

igna-v commented Dec 30, 2020

igna-v commented Dec 30, 2020 •

edited

Loading

albertodonato commented Dec 30, 2020 •

edited

Loading

albertodonato commented Jan 4, 2021

kullmanp commented Jan 22, 2021

mrcrch commented Feb 25, 2021

Error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters' #62

Error from database "devdb": 'AsyncioEngine' object has no attribute 'hide_parameters' #62

Comments

andy812 commented Sep 14, 2020 • edited Loading

xuanyuanaosheng commented Sep 21, 2020

Teriand commented Sep 23, 2020

manartezekov commented Nov 10, 2020 • edited Loading

searchgithub commented Nov 17, 2020 • edited Loading

albertodonato commented Nov 17, 2020

aorfanos commented Nov 24, 2020

albertodonato commented Nov 24, 2020

albertodonato commented Nov 24, 2020

albertodonato commented Nov 24, 2020

aorfanos commented Nov 25, 2020 • edited Loading

albertodonato commented Nov 25, 2020 via email

albertodonato commented Nov 25, 2020

igna-v commented Dec 28, 2020

albertodonato commented Dec 28, 2020

igna-v commented Dec 28, 2020 • edited Loading

albertodonato commented Dec 29, 2020

igna-v commented Dec 29, 2020

igna-v commented Dec 29, 2020

albertodonato commented Dec 30, 2020

igna-v commented Dec 30, 2020

igna-v commented Dec 30, 2020 • edited Loading

albertodonato commented Dec 30, 2020 • edited Loading

albertodonato commented Jan 4, 2021

kullmanp commented Jan 22, 2021

mrcrch commented Feb 25, 2021

andy812 commented Sep 14, 2020 •

edited

Loading

manartezekov commented Nov 10, 2020 •

edited

Loading

searchgithub commented Nov 17, 2020 •

edited

Loading

aorfanos commented Nov 25, 2020 •

edited

Loading

igna-v commented Dec 28, 2020 •

edited

Loading

igna-v commented Dec 30, 2020 •

edited

Loading

albertodonato commented Dec 30, 2020 •

edited

Loading