Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Messaging polyglot support #1472

merged 21 commits into from Aug 21, 2020

Messaging polyglot support #1472

merged 21 commits into from Aug 21, 2020


Copy link

@marcingrzejszczak marcingrzejszczak commented Aug 18, 2020

Passing part of the documentation as the description of what we're trying to achieve for adding polyglot messaging support.

How can you help

Please read the extract from the documentation below and answer the following questions

  • did you get anything from the documentation?
    • tell us how to fix the docs so they are more understandable
  • does the flow make any sense?
    • if not what would you change?

If you know python:

  • would any python person write such an endpoint to trigger a production message
  • does it make any sense from the python perspective?

If you know any other language

  • does this flow make sense for your language?
  • would you be willing to write such an HTTP endpoint to benefit from contract testing?
    • if that's the case would you be willing to test the feature out?
    • if that's the case would you be willing to provide a sample that we could embed in the docs? (simple controller that delegates a call to messaging code)

In addition to this please provide any feedback that you have with regards to this feature.

Additional links

Documentation of the feature

Example of Usage via Messaging

If you want to use Spring Cloud Contract with messaging via the Docker images (e.g.
in case of polyglot applications) then you'll have to have the following prerequisites met:

  • Middleware (e.g. RabbitMQ or Kafka) must be running before generating tests
  • Your contract needs to call a method triggerMessage(...) with a String parameter that is equal to the contract's label.
  • Your application needs to have a HTTP endpoint via which we can trigger a message
    • That endpoint should not be available on production (could be enabled via an environment variable)

Example of a Messaging Contract

The contract needs to call a triggerMessage(...) method. That method is already provided in the base class for all tests in the docker image and will send out a request to the HTTP endpoint on the producer side. Below you can find examples of such contracts.

Contract.make {
    description 'Send a pong message in response to a ping message'
    label 'ping_pong'
    input {
        // You have to provide the `triggerMessage` method with the `label`
        // as a String parameter of the method
    outputMessage {
            message: 'pong'
           outputMessage: [
               connectToBroker: [
                   declareQueueWithName: "queue"
                messageProperties: [
                    receivedRoutingKey: '#'
description: 'Send a pong message in response to a ping message'
label: 'ping_pong'
    # You have to provide the `triggerMessage` method with the `label`
    # as a String parameter of the method
    triggeredBy: 'triggerMessage("ping_pong")'
    sentTo: 'output'
        message: 'pong'
                declareQueueWithName: "queue"
                receivedRoutingKey: '#'

HTTP Endpoint to Trigger a Message

Why is there need to develop such an endpoint? Spring Cloud Contract
would have to generate code in various languages (as it does in Java) to make it possible to trigger production
code that sends a message to a broker. If such code is not generated then we need to be able to trigger the message anyways, and the way to do it is to provide an HTTP endpoint that the user will prepare in the language of their choosing.

The endpoint must have the following configuration:

  • URL: /springcloudcontract/{label} where label can be any text
  • Method: POST
  • Basing on the label will generate a message that will be sent to a given destination according to the contract definition

Below you have an example of such an endpoint. If you're interested in
providing an example in your language don't hesitate to file an issue in
the Spring Cloud Contract repository at Github -

#!/usr/bin/env python
from flask import Flask
from flask import jsonify
import pika
import os
app = Flask(__name__)
# Production code that sends a message to RabbitMQ
def message(cmd):
    connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
    channel =
            delivery_mode=2,  # make message persistent
    return " [x] Sent via Rabbit: %s" % cmd
# This should be ran in tests (shouldn't be publicly available)
if 'CONTRACT_TEST' in os.environ:
    @app.route('/springcloudcontract/<label>', methods=['POST'])
    def springcloudcontract(label):
        if label == "ping_pong":
            return message('{"message":"pong"}')
            raise ValueError('No such label expected.')

Running Message Tests on the Producer Side

Now, let's generate tests from contracts to test the producer side.
We will run bash code to start the Docker image
with attached contracts, however we will also add variables for the messaging
code to work. In this case let's assume that the contracts are being stored in
a Git repository.

set -x
CURRENT_DIR="$( pwd )"
export APP_IP="$( ./ )"
export APP_PORT="${APP_PORT:-8000}"
export PROJECT_NAME="${PROJECT_NAME:-application}"
# In our Python app we want to enable the HTTP endpoint
export CONTRACT_TEST="true"
# In the Verifier docker container we want to add support for RabbitMQ
export MESSAGING_TYPE="rabbit"
# Let's start the infrastructure (e.g. via Docker Compose)
yes | docker-compose kill || echo "Nothing running"
docker-compose up -d
echo "SC Contract Version [${SC_CONTRACT_DOCKER_VERSION}]"
echo "Application URL [${APPLICATION_BASE_URL}]"
echo "Project Version [${PROJECT_VERSION}]"
# Let's run python app
gunicorn -w 4 --bind main:app &
# Generate and run tests
docker run  --rm \
                --name verifier \
                # For the image to find the RabbitMQ running in another container
                -e "SPRING_RABBITMQ_ADDRESSES=${APP_IP}:5672" \
                # We need to tell the container what messaging middleware we will use
                -e "MESSAGING_TYPE=${MESSAGING_TYPE}" \
                -e "PUBLISH_STUBS_TO_SCM=false" \
                -e "PUBLISH_ARTIFACTS=false" \
                -e "PROJECT_NAME=${PROJECT_NAME}" \
                -e "PROJECT_GROUP=${PROJECT_GROUP}" \
                -e "PROJECT_VERSION=${PROJECT_VERSION}" \
                -v "${CURRENT_DIR}/build/spring-cloud-contract/output:/spring-cloud-contract-output/" \
kill $APP_PID
yes | docker-compose kill

What will happen is:

  • Tests will be generated from contracts taken from Git
  • In the contract we've provided an entry in metadata called declareQueueWithName that will lead to creation of a queue in RabbitMQ with the given name before the request to trigger the message is sent
  • Via the triggerMessage("ping_pong") method call a POST request to the Python application to the /springcloudcontract/ping_pong endpoint will be made
  • The Python application will generate and send a '{"message":"pong"}' JSON via RabbitMQ to an exchange called output
  • The generated test will poll for a message sent to the output exchange
  • Once the message was received will assert its contents

After the tests have passed we know that the message was properly sent from the Python app to RabbitMQ.

Example of Usage of Stub Runner with Messaging

In order to make messaging work it's enough to pass the MESSAGING_TYPE environment variable with kafka or rabbit values. This will lead to setting up
the Stub Runner Boot Docker image with dependencies required to connect to the broker.

In order to set the connection properties you can check out Spring Cloud Stream properties page to set proper environment variables.

The most common property you would set is the location of the running middleware.
If a property to set it is called spring.rabbitmq.addresses or spring.kafka.bootstrap-servers then you should name the environment variable SPRING_RABBITMQ_ADDRESSES and SPRING_KAFKA_BOOTSTRAP_SERVERS` respectively.

Documentation of the feature with standalone mode (aka with running middleware)

Running Contract Tests against Existing Middleware

There is legitimate reason to run your contract tests against existing middleware. Some
testing frameworks might give you false positive results - the test within your build
passes whereas on production the communication fails.

In Spring Cloud Contract docker images we give an option to connect to existing middleware.
As presented in previous subsections we do support Kafka and RabbitMQ out of the box. However,
via[Apache Camel Components] we can support
other middleware too. Let's take a look at the following examples of usage.

Spring Cloud Contract Docker and running Middleware

In order to connect to arbitrary middleware, we'll leverage the standalone metadata entry
in the contract section.

description: 'Send a pong message in response to a ping message'
label: 'standalone_ping_pong' <1>
  triggeredBy: 'triggerMessage("ping_pong")' <2>
  sentTo: 'rabbitmq:output' <3>
  body: <4>
    message: 'pong'
  standalone: <5>
    setup: <6>
      options: rabbitmq:output?queue=output&routingKey=# <7>
    outputMessage: <8>
      additionalOptions: routingKey=#&queue=output <9>
  • <1> Label by which we'll be able to trigger the message via Stub Runner
  • <2> As in the previous messaging examples we'll need to trigger the HTTP endpoint in the running application to make it send a message according to the provided protocol
  • <3> protocol:destination as requested by Apache Camel
  • <4> Output message body
  • <5> Standalone metadata entry
  • <6> Setup part will contain information about how to prepare for running contract tests before the actual call to HTTP endpoint of the running application is made
  • <7> Apache Camel URI to be called in the setup phase. In this case we will try to poll for a message at the output exchange and due to to having the queue=output and routingKey=# a queue with name output will be set and bound to the output exchange with routing key #
  • <8> Additional options (more technical ones) to be appended to the protocol:destination from point (3) - together will be combined in the following format rabbitmq:output?routingKey=#&queue=output.

For the contract tests to pass we will need as usual in case of messaging in polyglot environment
a running application and running middleware. This time we will have different environment variables set for the Spring Cloud Contract Docker image.

set -x

# Setup
# Run the middleware
docker-compose up -d rabbitmq <1>

# Run the python application
gunicorn -w 4 --bind main:app & <2>

docker run  --rm \
                --name verifier \
                -e "STANDALONE_PROTOCOL=rabbitmq" \ <3>
                -e "CAMEL_COMPONENT_RABBITMQ_ADDRESSES=" \ <4>
                -e "PUBLISH_STUBS_TO_SCM=false" \
                -e "PUBLISH_ARTIFACTS=false" \
                -e "APPLICATION_BASE_URL=" \
                -e "PROJECT_NAME=application" \
                -e "PROJECT_GROUP=group" \
                -e "EXTERNAL_CONTRACTS_ARTIFACT_ID=application" \
                -e "EXTERNAL_CONTRACTS_GROUP_ID=group" \
                -e "EXTERNAL_CONTRACTS_VERSION=0.0.1-SNAPSHOT" \
                -v "${CURRENT_DIR}/build/spring-cloud-contract/output:/spring-cloud-contract-output/" \

# Teardown
kill $APP_PID
yes | docker-compose kill

Stub Runner Docker and running Middleware

In order to trigger a stub message against running middleware, we can run Stub Runner Docker image in the following manner.

Example of usage

$ docker run \
    -e "STUBRUNNER_IDS=group:application:0.0.1-SNAPSHOT" \ <2>
    -e "STUBRUNNER_REPOSITORY_ROOT=git://" \ <3>
    -e ADDITIONAL_OPTS="" \ <4>
    -v "${HOME}/.m2/:/root/.m2:ro" \ <6>
    -p 8750:8750 \ <7>
    springcloud/spring-cloud-contract-stub-runner:3.0.0-SNAPSHOT <8>
  • <1> We're injecting the address of RabbitMQ via[Apache Camel's Spring Boot Auto-Configuration]
  • <2> We're telling Stub Runner which stubs to download
  • <3> We're providing an external location for our stubs (Git repository)
  • <4> Via the property we're telling Stub Runner which additional dependency to fetch at runtime. In this case we want to fetch camel-rabbitmq-starter so XXX is a random string and we want to fetch org.apache.camel.springboot:camel-rabbitmq-starter artifact in version 3.4.0.
  • <5> Since we're using Git, the remote option of fetching stubs needs to be set
  • <6> So that we speed up launching of Stub Runner, we're attaching our local Maven repository .m2 as a volume. If you don't have it populated you can consider setting the write permissions via :rw instead read only :ro.
  • <7> We expose the port 8750 at which Stub Runner is running.
  • <8> Coordinates of the Stub Runner Docker image.

After a while you'll notice the following text in your console, which means that Stub Runner is ready to accept requests.

o.a.c.impl.engine.AbstractCamelContext   : Apache Camel 3.4.3 (camel-1) started in 0.007 seconds
o.s.c.c.s.server.StubRunnerBoot          : Started StubRunnerBoot in 14.483 seconds (JVM running for 18.666)
o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
o.s.web.servlet.DispatcherServlet        : Completed initialization in 2 ms

To get the list of triggers you can send an HTTP GET request to localhost:8750/triggers endpoint. To trigger a stub message, you can send a HTTP POST request to localhost:8750/triggers/standalone_ping_pong. In the console you'll see:

o.s.c.c.v.m.camel.CamelStubMessages      : Will send a message to URI [rabbitmq:output?routingKey=#&queue=output]

If you check the RabbitMQ management console, you'll see that there's 1 message available in the output queue.

cc @Buzzardo there are some documentation changes here

@marcingrzejszczak marcingrzejszczak added this to the 3.0.0-M4 milestone Aug 18, 2020
@marcingrzejszczak marcingrzejszczak added this to In progress in 2020.0.0-M4 via automation Aug 18, 2020
Copy link

I get that I can set up an application that has a messaging endpoint to use with contracts.
I get that there's a triggerMessage() method to let me communicate with such an endpoint.

I don't know Python, so I can't comment on that code.

I did have a couple questions as I went along:

What setup steps do I need to do in Docker?
Are we going to have a Java sample? (Since Spring is mostly Java, that seems like something people would want.)

Copy link
Contributor Author

What setup steps do I need to do in Docker?

Set the env vars really. Also mind you that in the documentation we already have a whole chapter on how to configure the docker stuff. So this is just a flow with messaging basing on previously presented docker example.

Are we going to have a Java sample? (Since Spring is mostly Java, that seems like something people would want.)

With Java you don't really need to use Docker cause you can generate tests directly in your codebase

Copy link

I would definitely add those bits to the docs. I'll read it again after a meeting, to see if I have any further ideas.

@marcingrzejszczak marcingrzejszczak merged commit a915cf1 into master Aug 21, 2020
3 of 4 checks passed
2020.0.0-M4 automation moved this from In progress to Done Aug 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
No open projects

Successfully merging this pull request may close these issues.

Possibility to set MessageProperties in the AMQP contract DSL
2 participants