Improve standalone RPC more reliable and fix nameko shell loading #711
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tly (#4) * Changes 1: Improve standalone RPC to be more reliable + Add support for heartbeat in standalone RPC + Auto recover connection and retry if encounter network failure (only issue which know to be recoverable) + Change the behavior of PollingQueueConsumer to ack message on received instead of when result() method is called (this is to match the behavior of non-standalone rpc QueueConsumer) * Changes 2: Fix nameko shell to load correctly + Fix nameko shell to properly import and init ClusterRpcProxy after PYTHONSTARTUP script loaded and ShellRunner started + Add current working directory to System path when start nameko shell (This is to solved auto import feature of other framework such as: Django ORM) * Changes 3: Fix compatible issue with system was running older version of nameko (prior to 2.12) + Add back auto_delete=True option when init Exchange for events to fix compatible with system was running older version of nameko. Also this feature is not deprecated from AMQP 0-9-1 anymore (see: celery/py-amqp#286)
|
Its seem travis-ci build is failed with Docker pull error: |
…`timeout` kwarg in connection.ensure(). Fix a bug when nameko shell doesn't load all the variable defined in PYTHONSTARTUP script. Fix a couple of test-case to use @Retry decorator instead of sleep and gen queue_name with random uuid
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow me to introduce my self, i am leader developer of a start up company which recently got acquired in Singapore. The core micro services internal RPC system of the product we developed was built using this nameko framework. In the past 3 years i've put a few changes to the framework to fix a couple of reliability issue (most are network issue). The main changes are the Changes 1 mentioned below, which is to fix many weird network issues we were encountering on Production environment (RabbitMQ 3.6.x). Unfortunately there is no easy way to write a test-case for all these network issue so this PR may have to relied on the people review, comment and trust. Hope the owner of
namekoframework can take a look in this PR and give some comment for all the changes implemented here.p/s: I'm also the active maintainer/owner of these libraries:
+django-nameko : RPC connection pool implementation to use in conjunction with Nameko standalone RPC Proxy in any multi-threaded based non-nameko services such as: Django View, Django-Restframework API, Flask, ..)
+nameko-django : A custom kombu serializer which does auto convert Django ORM/Queryset/DateTime/Date/Decimal using msgpack which can be configure to used in nameko services
Changes 1: Improve standalone RPC to be more reliable
Changes 2: Fix nameko shell to load correctly
Changes 3: Fix compatible issue with system was running older version of nameko (prior to 2.12)
auto_delete=Trueoption when init Exchange for events to fix compatible with system was running older version of nameko. Also this feature is not deprecated from AMQP 0-9-1 anymore (see: auto_delete flag for exchanges is deprecated since 2014 celery/py-amqp#286)