Consumes data from a queue, stores the raw elements and exports the data to i2b2
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


C3-PRO-Consumer is a system that consumes from an AWS SQS queue, stores the raw elements in an oracle DB and exports the data into an i2b2 instance ( The elements in the queue are FHIR resources pushed trough C3-PRO-Server (

The system serves the following REST methods to start and stop the consumption of elements in the queue:

    HTTP/1.1 POST /c3pro-consumer/rest/actions/start
    HTTP/1.1 POST /c3pro-consumer/rest/actions/stop

Configuration and Deployment


The system uses the following external resources:

  • SQS queue: A queue deployed in AWS to consume from. This queue must be configured and populated as described in
  • Oracle DB: An oracle schema is needed to store the raw data from the SQS. Ideally, this schema should be located in the intranet of an organization.
  • FHIR DSTU2-1.0.2 or DSTU2-0.9.0 compliant system: To store the consumed resourced. In the current release we store the data in i2b2 through the newly created i2b2 fhir cell.

Installing Maven, Java && JBoss AS7

The system uses java 7 and we recommend to use JBoss AS7, although other java-based web servers can be used, like tomcat7. To install the basic tools in a Debian-based Linux distribution:

sudo apt-get clean
sudo apt-get update
sudo apt-get install openjdk-7-jdk
sudo apt-get install unzip
sudo apt-get install maven
sudo unzip -d /usr/share/
sudo chown -fR {{you_chosen_user}}:{{you_chosen_user}} /usr/share/jboss-as-7.1.1.Final/

Oracle DB configuration

The system uses an oracle DB to store the raw information extracted form the queue. Here are the steps to configure the DB properly:

  • Run the table creation script: {{src/main/scripts/create_tables.sql}}

  • Deploy the provided oracle jdbc driver in jBoss or anywhere accessible through the project:

    $HOME_C3PRO_CONSUMER/cp ojdbc14.jar $JBOSS_HOME/standalone/deployments
  • Configure the data source by editing the file $JBOSS_HOME/standalone/configuration/standalone.xml. In the data source section place the following:
<datasource jndi-name="java:jboss/datasources/c3proDS" pool-name="c3proDS" enabled="true" use-java-context="true">
  • Note for production deployments: It's not recommended to display raw DB credentials in the configuration files, even when the servers are protected. One possible way is to use security domains to wrap encrypted credentials. For instance:
<datasource jndi-name="java:jboss/datasources/c3proDS" pool-name="c3proDS" enabled="true" use-java-context="true">

and in the security domain section:

<security-domain name="secure-c3pro-credentials" cache-type="default">
      <login-module code="" flag="required">
          <module-option name="username" value="{{db_username}}"/>
          <module-option name="password" value="{{ENCRYPTED PASSWORD}}"/>

The encrypted password can be generated running picketbox security module as follows:

java {{db_password}}

The output will be the encrypted password to place in the security domain element. Make sure that your CLASS_PATH includes the appropriate jar file. PICKET BOX is included by default in JBOSS AS7 distribution as a module.

Building and deploying in DEV

Once the project is cloned or download, in the root of the project:

mvn clean package
mvn jboss-as:deploy

The previous instructions take the resource files located in src/main/resources/dev and place them as the resource files of the deployment. This requires JBoss on:


To stop JBoss:

$JBOSS_HOME/bin/ --connect command=:shutdown

Building in QA and PROD environment

In QA:

mvn clean package -Pqa
mvn jboss-as:deploy


mvn clean package -Pprod
mvn jboss-as:deploy

These commands take the resource files located in src/main/resources/qa or src/main/resources/prod respectively, and place them as the resource files of the deployment.

Deploying on web server containers different than JBOSS##

Generate the war files for the desired environment

mvn clean package
mvn clean package -Pqa
mvn clean package -Pprod

and copy the generated war located in target/c3pro-consumer.war to the corresponding deployment directory. In tomcat7 the default directory is:


AWS SDK credentials

The system uses the Java AWS SDK provided by Amazon. The SDK will be installed automatically since it is a maven dependency. However, it grabs the credentials to access the S3 bucket and SQS from a file that should be located here:


The content of the file should be something like:


To obtain access keys and secrets from AWS, visit We suggest to create a user in AWS-IAM with only permissions to access SQS, and generate the access key and secret for this user.

Generating and installing public-private keys

The information retrieved from the SQS is encrypted using a symmetric key. Such symmetric key is sent via metadata of the elements inserted in the queue, encrypted using a public key. Also, the ID of the public key is sent as metadata. The C3-PRO-Consumer uses the corresponding private key to decrypt the symmetric key and finally decrypt the fhir resource.

To generate and install a new key pair follow the steps:

(1) Generate a new UUID This will be the new ID of the key

(2) Inform the server about the new ID In the corresponding S3 Bucket, upload a new text file containing the new UUID. See (

(3) Generate the public-private keys

Execute the following command:


These files will be generated:


public-c3pro.der contains the public key and must be uploaded to the S3 bucket used by C3-PRO-Server. See for details.

private-c3pro.der contains the private key and must not be shared nor distributed under any circumstance. It should be backed up in a secure device and installed in the following directory:

~/.c3pro/{{new UUID}}/private-c3pro.der

If this private key is lost, you won't be able to recuperate the messages in the queue.

Configuration Parameters

There is one configuration parameters file for each environment (dev, qa and prod). They are located here:


SQS configuration access

Url connection to Amazon SQS queue

name of the SQS

Amazon profile for the SQS connection

Amazon region where the SQS is deployed

Property names of the Queue message (should not be changed! or changed in tune with the Research Kit App)

They are the property names of the messages in the queue

The property name that holds the private symmetric AES key

The property name that holds the public key id used to encrypt the private symmetric AES key

The property name that holds the fhir version


The default version of fhir in case the version is not informed in the queue message


Encryption parameters (should not be changed! or changed in tune with the Research Kit App and the c3pro-server)

*The asymmetric full algorithm used to encrypt and decrypt the symetric random key

The asymmetric BASE algorithm used to encrypt and decrypt the symmetric random key

The symmetric full algorithm used to encrypt and decrypt resources

The symmetric BASE algorithm used to encrypt and decrypt resources

The key size in bytes of the random symmetric key

The private key file name

The private key base path name. The complete path where will be ''/pkey_id/'' where pkey_id is the key id

End point and connection information of the running fhir compliant instance to store the resources. In this case, the i2b2 fhir cell

The host name

The end point pattern. The %s will be replaced by the corresponding fhir version. For instance, if the version received in the queue message is 1.0.2, under the above settings the end point will be: /fhir-i2b2/1.0.2/fhir


The connection port


The transport protocol

Integration test variables (optional)###