New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pymongo and rpyc #165

Closed
capaulson opened this Issue Apr 13, 2015 · 1 comment

Comments

Projects
None yet
2 participants
@capaulson

capaulson commented Apr 13, 2015

I'm trying to use pymongo out of IronPython, and I'm having issues. For a simple example, I just want to extract the database names:

The code:

import rpyc
c = rpyc.classic.connect('localhost', port=12345)
pymongo = c.modules['pymongo']
conn = pymongo.MongoClient('localhost')
print conn.database_names()

This returns the following from IronPython:

Message: 'Database' object is not callable. If you meant to call the 'exposed_database_names' method on a 'MongoClient' object it is failing because no such method exists.

This works perfectly under 'regular' python, which is running a classic server. Any ideas on why this doesn't work or what the error message means?

@coldfix

This comment has been minimized.

Collaborator

coldfix commented Dec 27, 2017

Hi, quick note:

This issue occurs on CPython as well and is not in any way specific to IronPython or windows.

The reason is the __getattr__ implementation of MongoClient that returns a Database object if asked for a non-existing attribute. Since we try for exposed_database_names before we try for database_names itself, we always run into this case.

Possible server-side fixes:

  • set config['allow_exposed_attrs'] = False in the server application, globally. However, this may break other parts of your program
  • override Connection._check_attr to not use exposed_prefix when accessing a MongoClient instance.

Possible fixes in rpyc:

  • in _check_attr: move allow_all_attrs (etc) checks above allow_exposed_attrs check and return there only if hasattr(obj, name) succeeds. The implication would be that exposed_XXX attributes will no longer override plain XXX attributes
  • disable allow_exposed_attrs for classic server (you will be able to do anything anyway..)
  • let everything as is but document how to fix it in user code

EDIT: I finally came to the conclusion that these changes are a good idea on their own to keep the principle of least surprise and have therefore applied these fixes to rpyc. It should be included in the next release (4.0).

@capaulson capaulson closed this Dec 27, 2017

coldfix added a commit that referenced this issue Dec 27, 2017

Fix SlaveService with allow_exposed_attrs=False
Users may need allow_exposed_attrs=False to avoid problems as in #165.

coldfix added a commit that referenced this issue Dec 27, 2017

Ignore exposed_prefix for SlaveService
Fixes #165 for classic connections.

coldfix added a commit that referenced this issue Dec 27, 2017

Serve plain attributes before exposed ones
Fixes #165 for services with allow_exposed_attrs=True.

One may argue whether it was a security feature that an exposed
attribute would always override a plain attribute even if one would have
the permissions to access that one otherwise…

However, this case was not documented and I believe such a policy calls
for a more careful implementation anyway (you probably don't want to
serve objects where a private attribute is only protected by shadowing).

coldfix added a commit that referenced this issue Jun 11, 2018

Release rpyc 4.0.0
This release brings a few minor backward incompatibilities, so be sure to read
on before upgrading. However, fear not: the ones that are most likely relevant
to you have a relatively simple migration path.

Backward Incompatibilities
^^^^^^^^^^^^^^^^^^^^^^^^^^

* ``classic.teleport_function`` now executes the function in the connection's
  namespace by default. To get the old behaviour, use
  ``teleport_function(conn, func, conn.modules[func.__module__].__dict__)``
  instead.

* Changed signature of ``Service.on_connect`` and ``on_disconnect``, adding
  the connection as argument.

* Changed signature of ``Service.__init__``, removing the connection argument

* no longer store connection as ``self._conn``. (allows services that serve
  multiple clients using the same service object, see `#198`_).

* ``SlaveService`` is now split into two asymetric classes: ``SlaveService``
  and ``MasterService``. The slave exposes functionality to the master but can
  not anymore access remote objects on the master (`#232`_, `#248`_).
  If you were previously using ``SlaveService``, you may experience problems
  when feeding the slave with netrefs to objects on the master. In this case, do
  any of the following:

  * use ``ClassicService`` (acts exactly like the old ``SlaveService``)
  * use ``SlaveService`` with a ``config`` that allows attribute access etc
  * use ``rpyc.utils.deliver`` to feed copies rather than netrefs to
    the slave

* ``RegistryServer.on_service_removed`` is once again called whenever a service
  instance is removed, making it symmetric to ``on_service_added`` (`#238`_)
  This reverts PR `#173`_ on issue `#172`_.

* Removed module ``rpyc.experimental.splitbrain``. It's too confusing and
  undocumented for me and I won't be developing it, so better remove it
  altogether. (It's still available in the ``splitbrain`` branch)

* Removed module ``rpyc.experimental.retunnel``. Seemingly unused anywhere, no
  documentation, no clue what this is about.

* ``bin/rpyc_classic.py`` will bind to ``127.0.0.1`` instead of ``0.0.0.0`` by
  default

* ``SlaveService`` no longer serves exposed attributes (i.e., it now uses
  ``allow_exposed_attrs=False``)

* Exposed attributes no longer hide plain attributes if one otherwise has the
  required permissions to access the plain attribute. (`#165`_)

.. _#165: #165
.. _#172: #172
.. _#173: #173
.. _#198: #198
.. _#232: #232
.. _#238: #238
.. _#248: #248

What else is new
^^^^^^^^^^^^^^^^

* teleported functions will now be defined by default in the globals dict

* Can now explicitly specify globals for teleported functions

* Can now use streams as context manager

* keep a hard reference to connection in netrefs, may fix some ``EOFError``
  issues, in particular on Jython related (`#237`_)

* handle synchronous and asynchronous requests uniformly

* fix deadlock with connections talking to each other multithreadedly (`#270`_)

* handle timeouts cumulatively

* fix possible performance bug in ``Win32PipeStream.poll`` (oversleeping)

* use readthedocs theme for documentation (`#269`_)

* actually time out sync requests (`#264`_)

* clarify documentation concerning exceptions in ``Connection.ping`` (`#265`_)

* fix ``__hash__`` for netrefs (`#267`_, `#268`_)

* rename ``async`` module to ``async_`` for py37 compatibility (`#253`_)

* fix ``deliver()`` from IronPython to CPython2 (`#251`_)

* fix brine string handling in py2 IronPython (`#251`_)

* add gevent_ Server. For now, this requires using ``gevent.monkey.patch_all()``
  before importing for rpyc. Client connections can already be made without
  further changes to rpyc, just using gevent's monkey patching. (`#146`_)

* add function ``rpyc.lib.spawn`` to spawn daemon threads

* fix several bugs in ``bin/rpycd.py`` that crashed this script on startup
  (`#231`_)

* fix problem with MongoDB, or more generally any remote objects that have a
  *catch-all* ``__getattr__`` (`#165`_)

* fix bug when copying remote numpy arrays (`#236`_)

* added ``rpyc.utils.helpers.classpartial`` to bind arguments to services (`#244`_)

* can now pass services optionally as instance or class (could only pass as
  class, `#244`_)

* The service is now charged with setting up the connection, doing so in
  ``Service._connect``. This allows using custom protocols by e.g. subclassing
  ``Connection``.  More discussions and related features in `#239`_-`#247`_.

* service can now easily override protocol handlers, by updating
  ``conn._HANDLERS`` in ``_connect`` or ``on_connect``. For example:
  ``conn._HANDLERS[HANDLE_GETATTR] = self._handle_getattr``.

* most protocol handlers (``Connection._handle_XXX``) now directly get the
  object rather than its ID as first argument. This makes overriding
  individual handlers feel much more high-level. And by the way it turns out
  that this fixes two long-standing issues (`#137`_, `#153`_)

* fix bug with proxying context managers (`#228`_)

* expose server classes from ``rpyc`` top level module

* fix logger issue on jython

.. _#137: #137
.. _#146: #146
.. _#153: #153
.. _#165: #165
.. _#228: #228
.. _#231: #231
.. _#236: #236
.. _#237: #237
.. _#239: #239
.. _#244: #244
.. _#247: #247
.. _#251: #251
.. _#253: #253
.. _#264: #264
.. _#265: #265
.. _#267: #267
.. _#268: #268
.. _#269: #269
.. _#270: #270

.. _gevent: http://www.gevent.org/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment