[8.0] can't import dateparser from an Odoo module

[alexis-via]

Here are the steps to reproduce the bug:

1. install the dateparser python lib:

sudo pip install dateparser

2. Unzip the module bug_dateparser available from https://people.via.ecp.fr/~alexis/bug_dateparser.zip in a directory of your addons path. This is a very simple module that only does the following:

import dateparser

3. Update module list, then install the module bug_dateparser: you get a crash:

2016-01-28 21:13:23,200 3940 INFO o8_test1 openerp.modules.loading: loading 92 modules...
2016-01-28 21:13:23,265 3940 CRITICAL o8_test1 openerp.modules.module: Couldn't load module bug_dateparser
2016-01-28 21:13:23,266 3940 CRITICAL o8_test1 openerp.modules.module: find_module() takes exactly 3 arguments (2 given)
> /usr/lib/python2.7/pkgutil.py(475)find_loader()
-> loader = importer.find_module(fullname)
(Pdb) bt
  /home/alexis/new_boite/dev/odoo8/openerp/http.py(574)dispatch()
-> result = self._call_function(**self.params)
  /home/alexis/new_boite/dev/odoo8/openerp/http.py(310)_call_function()
-> return checked_call(self.db, *args, **kwargs)
  /home/alexis/new_boite/dev/odoo8/openerp/service/model.py(118)wrapper()
-> return f(dbname, *args, **kwargs)
  /home/alexis/new_boite/dev/odoo8/openerp/http.py(307)checked_call()
-> return self.endpoint(*a, **kw)
  /home/alexis/new_boite/dev/odoo8/openerp/http.py(803)__call__()
-> return self.method(*args, **kw)
  /home/alexis/new_boite/dev/odoo8/openerp/http.py(403)response_wrap()
-> response = f(*args, **kw)
  /home/alexis/new_boite/dev/odoo8/addons/web/controllers/main.py(948)call_button()
-> action = self._call_kw(model, method, args, {})
  /home/alexis/new_boite/dev/odoo8/addons/web/controllers/main.py(936)_call_kw()
-> return getattr(request.registry.get(model), method)(request.cr, request.uid, *args, **kwargs)
  /home/alexis/new_boite/dev/odoo8/openerp/api.py(268)wrapper()
-> return old_api(self, *args, **kwargs)
  /home/alexis/new_boite/dev/odoo8/openerp/addons/base/module/module.py(448)button_immediate_install()
-> return self._button_immediate_function(cr, uid, ids, self.button_install, context=context)
  /home/alexis/new_boite/dev/odoo8/openerp/api.py(268)wrapper()
-> return old_api(self, *args, **kwargs)
  /home/alexis/new_boite/dev/odoo8/openerp/addons/base/module/module.py(496)_button_immediate_function()
-> registry = openerp.modules.registry.RegistryManager.new(cr.dbname, update_module=True)
  /home/alexis/new_boite/dev/odoo8/openerp/modules/registry.py(370)new()
-> openerp.modules.load_modules(registry._db, force_demo, status, update_module)
  /home/alexis/new_boite/dev/odoo8/openerp/modules/loading.py(355)load_modules()
-> loaded_modules, update_module)
  /home/alexis/new_boite/dev/odoo8/openerp/modules/loading.py(255)load_marked_modules()
-> loaded, processed = load_module_graph(cr, graph, progressdict, report=report, skip_modules=loaded_modules, perform_checks=perform_checks)
  /home/alexis/new_boite/dev/odoo8/openerp/modules/loading.py(143)load_module_graph()
-> load_openerp_module(package.name)
  /home/alexis/new_boite/dev/odoo8/openerp/modules/module.py(315)load_openerp_module()
-> __import__('openerp.addons.' + module_name)
  /home/alexis/new_boite/dev/odoo8/openerp/modules/module.py(80)load_module()
-> mod = imp.load_module('openerp.addons.' + module_part, f, path, descr)
  /home/alexis/new_boite/dev/test8/bug_dateparser/__init__.py(3)<module>()
-> from . import bug
  /home/alexis/new_boite/dev/test8/bug_dateparser/bug.py(21)<module>()
-> import dateparser
  /usr/local/lib/python2.7/dist-packages/dateparser-0.3.2-py2.7.egg/dateparser/__init__.py(4)<module>()
-> from .date import DateDataParser
  /usr/local/lib/python2.7/dist-packages/dateparser-0.3.2-py2.7.egg/dateparser/date.py(11)<module>()
-> from dateparser.date_parser import date_parser
  /usr/local/lib/python2.7/dist-packages/dateparser-0.3.2-py2.7.egg/dateparser/date_parser.py(16)<module>()
-> from .conf import apply_settings
  /usr/local/lib/python2.7/dist-packages/dateparser-0.3.2-py2.7.egg/dateparser/conf.py(61)<module>()
-> settings = Settings()
  /usr/local/lib/python2.7/dist-packages/dateparser-0.3.2-py2.7.egg/dateparser/conf.py(31)__init__()
-> self._updateall(self._get_settings_from_yaml().items())
  /usr/local/lib/python2.7/dist-packages/dateparser-0.3.2-py2.7.egg/dateparser/conf.py(44)_get_settings_from_yaml()
-> data = get_data('data', 'settings.yaml')
  /usr/lib/python2.7/pkgutil.py(578)get_data()
-> loader = get_loader(package)
  /usr/lib/python2.7/pkgutil.py(464)get_loader()
-> return find_loader(fullname)
> /usr/lib/python2.7/pkgutil.py(475)find_loader()
-> loader = importer.find_module(fullname)

This is a real problem for me, because the lib invoice2data that is used in my module account_invoice_import (cf this PR OCA/account-invoicing#122) now uses dateparser, so, if you have the latest version of invoice2data, you won't be able to install the module account_invoice_import :-(

I don't know if this bug is Odoo's fault or dateparser's fault. I have some difficulty to understand the exact cause of the problem. One thing I noticed:

• in /usr/lib/python2.7/pkgutil.py, we have a find_module method with 3 args and the third has a default value, so it can be called with only 2 args:

def find_module(self, fullname, path=None)

• in odoo/openerp/module/module.py (cf https://github.com/odoo/odoo/blob/8.0/openerp/modules/module.py#L68) we have the same method with 3 args and the 3rd one doesn't have a default value so it has to be called with 3 args:

def find_module(self, module_name, package_path):

So I was thinking it may be mixing the two... but I really don't understand what's happening, so I may be completely wrong.

[rvalyi]

@alexis-via may be that's could help you understand what is going on: http://dangerontheranger.blogspot.com.br/2012/07/how-to-use-sysmetapath-with-python.html

But yeah, at first sight I would say Odoo should have package_path=None instead in the signature, so it can cope with non Odoo calls and not match anything but that's probably fine. At least I would try that...

[xmo-odoo]

@alexis-via should be fixed in the v8 branch (revision edeb5a8). It was our (Odoo's) fault: we setup a PEP302 import hook so that e.g. account can be imported as both account and openerp.addons.account (without importing it twice and executing the code twice ideally), but did fail to correctly follow the PEP: it defines one of the methods of the protocol as find_module(fullname, path=None) and we made the second parameter mandatory.

It works in most cases because callers generally provide a path, but some clients like pkgutil.find_loader (which iterates over all package loaders to try and find the loader for a specific module name) take advantage of the spec and pass in a single parameter which caused the crash you experienced.

Thanks for the report and sorry about that.

[alexis-via]

@rvalyi @xmo-odoo @beledouxdenis Thank you very very much for your help and for the patch ! I updated my odoo v8 to the latest version (which includes your patch) and it works well now ! I can continue the development of my module account_invoice_import - a great module by the way, you should try it :)