New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

worker is not supported dbfilter with %d or %h parameter. #58

Closed
emiprotech opened this Issue Apr 22, 2015 · 12 comments

Comments

Projects
None yet
7 participants
@emiprotech

emiprotech commented Apr 22, 2015

This connector worker is not supported with the dbfilter having either %d or %h.
i.e. dbfilter = '^%d.*' then it is not working with the following line of code.
dbfilter = config['dbfilter']
if dbfilter and db_names:
db_names = [d for d in db_names if re.match(dbfilter, d)]

here dbfilter variable having '^%d.*' thats why re.match will not found any matches with this and it is returning [ ](blank list).

same this openerp-connector-worker script is not working with dbfilter.

@guewen

This comment has been minimized.

Show comment
Hide comment
@guewen

guewen Jul 2, 2015

Member

The --db-filter option is an option filtering what databases are shown on the web interface.
I'm not sure that it should restrict the job workers. The --database option seems more appropriate for this to me.

Member

guewen commented Jul 2, 2015

The --db-filter option is an option filtering what databases are shown on the web interface.
I'm not sure that it should restrict the job workers. The --database option seems more appropriate for this to me.

@sbidoul

This comment has been minimized.

Show comment
Hide comment
@sbidoul

sbidoul Jul 2, 2015

Member

I think the runner should honor --db-filter. For example, you may want a configuration with two odoo instances on the same machine, each serving a different set of databases controlled by --db-filter. The runner should listen to the exact same set of databases than the Odoo instance it's running in.

I did not know dbfilter could include %d. I thought I had copied the same db filtering algorithm as the Odoo server. To be verified.

Member

sbidoul commented Jul 2, 2015

I think the runner should honor --db-filter. For example, you may want a configuration with two odoo instances on the same machine, each serving a different set of databases controlled by --db-filter. The runner should listen to the exact same set of databases than the Odoo instance it's running in.

I did not know dbfilter could include %d. I thought I had copied the same db filtering algorithm as the Odoo server. To be verified.

@guewen

This comment has been minimized.

Show comment
Hide comment
@guewen

guewen Jul 2, 2015

Member

@sbidoul here is my interpretation:
--db-filter only restricts HTTP to serve databases which are not in the filter. It does not prevent for instance the crons to run on those databases. The option that really makes Odoo (including crons) run on a selection of databases is --database. So in your case, with 2 odoo on the same machine but different set of databases, you should probably configure a different --database option otherwise your cron will be mixed up between all the databases. I think the job runner should have the same behavior than the cron.

Member

guewen commented Jul 2, 2015

@sbidoul here is my interpretation:
--db-filter only restricts HTTP to serve databases which are not in the filter. It does not prevent for instance the crons to run on those databases. The option that really makes Odoo (including crons) run on a selection of databases is --database. So in your case, with 2 odoo on the same machine but different set of databases, you should probably configure a different --database option otherwise your cron will be mixed up between all the databases. I think the job runner should have the same behavior than the cron.

@sbidoul

This comment has been minimized.

Show comment
Hide comment
@sbidoul

sbidoul Jul 2, 2015

Member

@guewen It seems you are right indeed. I remember now where I got this part of code from: I copied it from the connector worker 😉

Still it's a bit strange to me. If you use %h and %d in the dbfilter, this means that you probably don't use --database and the Odoo instance sees databases served by other hosts. So the crons will run on all of them and compete to catch the jobs. That's probably a minor issues for crons which will lock the jobs anyway and prevent the same job to be run by two instance at the same time.

But for the runner, if it ignores dbfilter, it will listen (and create connections) to all visible databases.

So I would rather keep using dbfilter and use http.db_filter() in https://github.com/OCA/connector/blob/8.0/connector/jobrunner/runner.py#L221

Member

sbidoul commented Jul 2, 2015

@guewen It seems you are right indeed. I remember now where I got this part of code from: I copied it from the connector worker 😉

Still it's a bit strange to me. If you use %h and %d in the dbfilter, this means that you probably don't use --database and the Odoo instance sees databases served by other hosts. So the crons will run on all of them and compete to catch the jobs. That's probably a minor issues for crons which will lock the jobs anyway and prevent the same job to be run by two instance at the same time.

But for the runner, if it ignores dbfilter, it will listen (and create connections) to all visible databases.

So I would rather keep using dbfilter and use http.db_filter() in https://github.com/OCA/connector/blob/8.0/connector/jobrunner/runner.py#L221

@flotho

This comment has been minimized.

Show comment
Hide comment
@flotho

flotho Aug 27, 2015

Member

Hi folks,

I think I have this kind of issue. I run successfully the connector-worker yet no worker is added in the db that has the connector installed.
I'm not sure to have understand if there is a workaround.

Thanks in advance.

Edit : adding the db filter on the connector command works very well of course!

Member

flotho commented Aug 27, 2015

Hi folks,

I think I have this kind of issue. I run successfully the connector-worker yet no worker is added in the db that has the connector installed.
I'm not sure to have understand if there is a workaround.

Thanks in advance.

Edit : adding the db filter on the connector command works very well of course!

@lepri

This comment has been minimized.

Show comment
Hide comment
@lepri

lepri Sep 10, 2015

I have this issue using dbfilter. After remover the filter ^%d$ the connector works fine.

lepri commented Sep 10, 2015

I have this issue using dbfilter. After remover the filter ^%d$ the connector works fine.

@guewen

This comment has been minimized.

Show comment
Hide comment
@guewen

guewen Sep 11, 2015

Member

So I would rather keep using dbfilter and use http.db_filter() in https://github.com/OCA/connector/blob/8.0/connector/jobrunner/runner.py#L221

Ok, I trust you on that, I'm not very familiar with --db-filter because usually we create postgres users so that we have 1 database available at a time.

Member

guewen commented Sep 11, 2015

So I would rather keep using dbfilter and use http.db_filter() in https://github.com/OCA/connector/blob/8.0/connector/jobrunner/runner.py#L221

Ok, I trust you on that, I'm not very familiar with --db-filter because usually we create postgres users so that we have 1 database available at a time.

@YannickB

This comment has been minimized.

Show comment
Hide comment
@YannickB

YannickB Oct 13, 2015

Contributor

Same for me, I confirm that when you use a db_filter no workers appear on the connector interface, whether you use monoworker, multiworker with openerp-connector-worker or jobrunner.

In fact, you never see in the log anything related to the database, so for me the connector is just unable to see the databases, probably because of db_filter. Even if you specify the database directly with -d option

My configuration :
db_filter = ^odoo_%h$
database name : odoo_customer_clouder_at
my url : customer.clouder.at

Contributor

YannickB commented Oct 13, 2015

Same for me, I confirm that when you use a db_filter no workers appear on the connector interface, whether you use monoworker, multiworker with openerp-connector-worker or jobrunner.

In fact, you never see in the log anything related to the database, so for me the connector is just unable to see the databases, probably because of db_filter. Even if you specify the database directly with -d option

My configuration :
db_filter = ^odoo_%h$
database name : odoo_customer_clouder_at
my url : customer.clouder.at

@guewen

This comment has been minimized.

Show comment
Hide comment
@guewen

guewen Oct 13, 2015

Member

@YannickB the correct option is dbfilter, is it a typo in your comment?

Member

guewen commented Oct 13, 2015

@YannickB the correct option is dbfilter, is it a typo in your comment?

@YannickB

This comment has been minimized.

Show comment
Hide comment
@YannickB

YannickB Oct 13, 2015

Contributor

Yep it is a typo

Contributor

YannickB commented Oct 13, 2015

Yep it is a typo

@LeartS

This comment has been minimized.

Show comment
Hide comment
@LeartS

LeartS Oct 19, 2015

Contributor

I have the same issue, spent a long time trying to find out why connector wasn't working!

Contributor

LeartS commented Oct 19, 2015

I have the same issue, spent a long time trying to find out why connector wasn't working!

2cadz pushed a commit to 2cadz/connector that referenced this issue Dec 8, 2015

sbidoul added a commit to acsone/connector that referenced this issue Dec 27, 2015

[FIX] ignore dbfilter, fixes #58
dbfilter can contains %d or %h, referring to the client hostname.
Since connector does run independently of client, it can not rely
on the dbfilter concept.

sbidoul added a commit to acsone/connector that referenced this issue Dec 27, 2015

[FIX] ignore dbfilter containing %d or %h, fixes #58
This is a temporary fix. In version 4.0,
dbfilter will be completely ignored by connector.

@sbidoul sbidoul added the bug label Dec 27, 2015

sbidoul added a commit to acsone/connector that referenced this issue Dec 27, 2015

[FIX] ignore dbfilter containing %d or %h, fixes #58
This is a temporary fix. In version 4.0,
dbfilter will be completely ignored by connector.

sbidoul added a commit to acsone/connector that referenced this issue Dec 27, 2015

[FIX] ignore dbfilter, fixes #58
dbfilter can contains %d or %h, referring to the client hostname.
Since connector does run independently of client, it can not rely
on the dbfilter concept.

guewen added a commit that referenced this issue Jan 4, 2016

Merge pull request #166 from acsone/8.0-fix-dbfilter-sbi
[FIX] ignore dbfilter containing %d or %h, fixes #58

sbidoul added a commit to acsone/connector that referenced this issue Jan 4, 2016

lmignon added a commit to acsone/connector that referenced this issue Feb 15, 2016

[FIX] ignore dbfilter containing %d or %h, fixes #58
This is a temporary fix. In version 4.0,
dbfilter will be completely ignored by connector.

lmignon added a commit to acsone/connector that referenced this issue Feb 29, 2016

@sbidoul

This comment has been minimized.

Show comment
Hide comment
@sbidoul

sbidoul Feb 29, 2016

Member

Fixed in

Member

sbidoul commented Feb 29, 2016

Fixed in

@sbidoul sbidoul closed this Feb 29, 2016

agb80 added a commit to OpenPymeMx/connector that referenced this issue Dec 22, 2016

agb80 added a commit to OpenPymeMx/connector that referenced this issue Feb 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment