New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Message Provider Implementation #200
Closed
tuan-pham
wants to merge
25
commits into
pact-foundation:master
from
tuan-pham:feature/message-provider
Closed
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
887909c
wip: message provider implementation
tuan-pham ee29b77
wip: add basic verify flow
tuan-pham 65e0e02
wip: add missing flask config
tuan-pham 5964311
feat: initial flask start setup
williaminfante 5e7b99e
wip: and sample provider test, rename MessageProvider constructor
tuan-pham cbbc099
feat: move handler to provider
williaminfante 7fd2dd6
feat: pass handler as python argument
williaminfante 2299c93
feat: pass handler as python argument
williaminfante a67ce37
feat: create setup endpoint for message handlers, add setup_state fn,…
tuan-pham 05e499a
feat: enable context manager in message provider, allow provider to p…
tuan-pham ddb108e
fix: flake8
tuan-pham 0ecfda8
fix: revert bad merge to http_proxy; add pydocstyle
tuan-pham 6baf3b3
feat: parse content, update readme and test
williaminfante f009afb
test: add missing tests for message provider
tuan-pham 1b1175c
fix: check the pact files exists before running the vefivication
tuan-pham 18fbe9f
fix: flake8
tuan-pham af52bfd
fix: revert changes to __exit__, refactor the example/tests to sync u…
tuan-pham 532d4da
fix: remove dead code
tuan-pham b1827ad
feat: add http_proxy test, replace print with log, use flask localsta…
tuan-pham efca9f1
fix: change PROXY_PORT to 1234 to fix broken build
tuan-pham 95dcde0
fix: flake8
tuan-pham e0a6974
chore: skip provider test to make the build pass (troubleshooting)
tuan-pham 715b0f9
chore: skip 2 tests that causes BrokenPipeError for investigation
tuan-pham d88a821
chore: comment out the broken tests
tuan-pham 6e2ac1a
fix: change default proxy port to 1234
tuan-pham File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
import pytest | ||
from pact import MessageProvider | ||
|
||
def document_created_handler(): | ||
return { | ||
"event": "ObjectCreated:Put", | ||
"bucket": "bucket_name", | ||
"key": "path_to_file_in_s3.pdf", | ||
"documentType": "application/pdf" | ||
} | ||
|
||
def document_deleted_handler(): | ||
return { | ||
"event": "ObjectCreated:Delete", | ||
"bucket": "bucket_name", | ||
"key": "existing_file_in_s3.pdf", | ||
"documentType": "application/pdf" | ||
} | ||
|
||
def test_verify_success(): | ||
provider = MessageProvider( | ||
message_providers={ | ||
'A document created successfully': document_created_handler, | ||
'A document deleted successfully': document_deleted_handler | ||
}, | ||
provider='DocumentService', | ||
consumer='DetectContentLambda' | ||
) | ||
with provider: | ||
provider.verify() | ||
|
||
def test_verify_failure_when_a_provider_missing(): | ||
provider = MessageProvider( | ||
message_providers={ | ||
'A document created successfully': document_created_handler, | ||
}, | ||
provider='DocumentService', | ||
consumer='DetectContentLambda' | ||
) | ||
|
||
with pytest.raises(AssertionError): | ||
with provider: | ||
provider.verify() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
"""Http Proxy to be used as provider url in verifier.""" | ||
from werkzeug.local import LocalStack | ||
from flask import Flask, jsonify, request | ||
from werkzeug.exceptions import HTTPException | ||
import json | ||
import logging | ||
log = logging.getLogger(__name__) | ||
logging.basicConfig(level=logging.DEBUG) | ||
|
||
app = Flask(__name__) | ||
localstack = LocalStack() | ||
PROXY_PORT = 1234 | ||
|
||
def shutdown_server(): | ||
"""Shutdown Http Proxy server.""" | ||
shutdown = request.environ.get('werkzeug.server.shutdown') | ||
if shutdown is None: | ||
raise RuntimeError('Not running with the Werkzeug Server') | ||
shutdown() | ||
|
||
def _match_states(payload): | ||
"""Match states in payload against stored message handlers.""" | ||
log.debug(f'Find handler from payload: {payload}') | ||
handlers = localstack.top | ||
states = handlers['messageHandlers'] | ||
log.debug(f'Setup states: {handlers}') | ||
provider_states = payload['providerStates'] | ||
|
||
for state in provider_states: | ||
matching_state = state['name'] | ||
if matching_state in states: | ||
return states[matching_state] | ||
raise RuntimeError('No matched handler.') | ||
|
||
@app.route('/', methods=['POST']) | ||
def home(): | ||
"""Match states with provided message handlers.""" | ||
payload = request.json | ||
message = _match_states(payload) | ||
res = jsonify({ | ||
'contents': message | ||
}) | ||
res.status_code = 200 | ||
return res | ||
|
||
@app.route('/ping', methods=['GET']) | ||
def ping(): | ||
"""Check whether the server is available before setting up states.""" | ||
res = jsonify({ | ||
'ping': 'pong' | ||
}) | ||
res.status_code = 200 | ||
return res | ||
|
||
@app.route("/setup", methods=['POST']) | ||
def setup(): | ||
"""Endpoint to setup states. | ||
|
||
Use localstack to store payload. | ||
""" | ||
payload = request.json | ||
# Store payload in localstack | ||
localstack.push(payload) | ||
res = jsonify(payload) | ||
res.status_code = 201 | ||
return res | ||
|
||
@app.route('/shutdown', methods=['POST']) | ||
def shutdown(): | ||
"""Shutdown Http Proxy server.""" | ||
shutdown_server() | ||
return 'Server shutting down...' | ||
|
||
@app.errorhandler(HTTPException) | ||
def handle_exception(e): | ||
"""Return JSON instead of HTML for HTTP errors.""" | ||
res = e.get_response() | ||
res.data = json.dumps({ | ||
"code": e.code, | ||
"name": e.name, | ||
"description": e.description, | ||
}) | ||
res.content_type = "application/json" | ||
return res | ||
|
||
@app.errorhandler(RuntimeError) | ||
def handle_runtime_error(e): | ||
"""Handle the RuntimeError. | ||
|
||
Handle HTML stacktrace when RuntimeError occurs due to no matched handler. | ||
when the verifier fails. | ||
""" | ||
res = jsonify({ | ||
"name": "RuntimeError", | ||
"description": str(e), | ||
}) | ||
res.status_code = 500 | ||
res.content_type = "application/json" | ||
return res | ||
|
||
|
||
if __name__ == '__main__': | ||
app.run(debug=True, port=PROXY_PORT) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,7 +7,7 @@ | |
|
||
from .broker import Broker | ||
from .constants import MESSAGE_PATH | ||
|
||
from .matchers import from_term | ||
|
||
class MessagePact(Broker): | ||
""" | ||
|
@@ -136,7 +136,7 @@ def with_content(self, contents): | |
:rtype: Pact | ||
""" | ||
self._insert_message_if_complete() | ||
self._messages[0]['contents'] = contents | ||
self._messages[0]['contents'] = from_term(contents) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Well, we need test for this! |
||
return self | ||
|
||
def expects_to_receive(self, description): | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the intention that Flask will run in the same process as out tests but in the background? Just beware you might have problems with later versions (Python and Flask) of this being a subprocess to do with tickle? Not a joke!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In case the context isn't clear. Basically, the way the provider verification for messages works still uses the same underlying verification binary, which expects to talk to an HTTP server. Instead of making the end user do the mapping, this proxy server is responsible for mapping the "message" to be verified, to a "function" that will produce the message - i.e. the actual provider.
See the sequence diagrams here for more: https://github.com/pact-foundation/pact-message-demo
If you knew all that already... sorry!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@elliottmurray yes, the idea is to run it as a subprocess. There maybe a problem with later version with subprocess here but I'm not 100% sure what can we replace a subproces with.
@mefellows Thanks for reminding about sequence diagram. It helps understand the verify flow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you will need to use FastAPI over Flask @tuan-pham . I have an example in the examples. It is why I actually can't use the provider python in the e2e (which is Flask) and have to use a shell script instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mefellows I still don't understand this! I mean I get the provider side but the consumer side just makes my head hurt!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi I download @tuan-pham code. I'm using python 3.7 and lastest greatest flask. Which version of flask works? The issue I'm seeing is in the setup we set values on localstack. Then in the request we are getting None from the localstack. Not sure how to fix this. I figure different version of flask.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @elliottmurray,
I'm working on the porting to FastAPI of the @tuan-pham work, any suggestions on how to replace werkzeug.local.LocalStack?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've not looked at this recently due to personal stuff. I am going to start looking at moving to the underlying Rust impl and v3. However, I did do some work on this around messaging generally. I can't remember if I pulled Flask out on this branch or was just local:
https://github.com/pact-foundation/pact-python/commits/docs/kafka_example
The main issue IIRC was getting it to start up. But didn't get very far.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pulphix Yes, it's the plan to move to FastAPI over Flask. Unfortunately, it's not on my radar due to more higher priority assignment at my end. I may have a look into it at some point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I setup maemcahce to solve the problem but fast api will solve this issue.