AcraConnector and AcraWriter

karenswry edited this page Aug 22, 2018 · 12 revisions
:warning: Attention! This documentation is for Acra v0.82.0. For the latest documentation, examples, tutorials, and demos, please visit Cossack Labs Documentation Server. :warning:

AcraConnector

AcraConnector is a (separate) service running alongside your application — it pretends to be a database listener, relays all the requests to AcraServer, receives the responses, and returns them to the app, just like a normal database listener would do.

To talk to AcraServer, you'll need to run AcraConnector on the same host as your application, in a separate container or as a separate user. You'll also need to route database requests to its address.

To talk to AcraTranslator you'll need to do the same: run AcraConnector on the same host as your application, in a separate container or as a separate user, and use its URL as destination URL from your application.

Why use a special piece of software to talk to your other piece of software?

Acra needs a trusted agent on the application's side to protect the sensitive decrypted responses, to provide basic channel authentication, and to enforce certain behaviours.

AcraConnector acts as a local proxy that receives requests from the app and returns decrypted answers. AcraConnector provides an encrypted and authenticated connection to AcraServer, which, in turn, fetches the response to the request from the database and decrypts all the data. AcraServer then returns the data to AcraConnector via a secure connection.

AcraConnector works in a similar fashion with AcraTranslator, redirecting AcraStructs from application to AcraTranslator and delivering the decrypted response back.

This enforces maximum secrecy, employing authentication that is easy to manage (pre-shared private keys), and requires minimal intervention into your code for successful implementation!

Getting started with AcraConnector

1a. Launch AcraConnector using Docker (the fastest way to try AcraConnector)

Note: Using Docker is recommended for testing purposes only. Please don't rely on Docker in real-life production settings.
Note: The following examples focus on using AcraConnector and AcraWriter with PostgreSQL, but Acra also supports MySQL.

Clone the Acra repository, build images, and start Docker compose with PostgreSQL, AcraServer, AcraConnector, and Secure Session between them:

git clone https://github.com/cossacklabs/acra.git
make docker
docker-compose -f docker-compose.pgsql-nossl-server-ssession-connector.yml up

Okay, now the basic infrastructure is all set up, all components are connected, and keys are distributed into the appropriate folder.

Now, move to step 2 – launch AcraConnector.

1b. Manual launch

Note: Skip this if you used the Docker method described above.

  • Install dependencies: Themis cryptographic library:
git clone https://github.com/cossacklabs/themis.git
cd themis
make
sudo make install
  • Install AcraConnector:
go get github.com/cossacklabs/acra/cmd/acra-connector

Note: All the commands starting with 'go' are meant to be executed from the 'acra' folder (the folder with the repository code).

  • Install the key generation utility:
go get github.com/cossacklabs/acra/cmd/acra-keymaker
  • Use acra-keymaker to generate a master key into master.key file and assign it into the environment variable like this:
$GOPATH/bin/acra-keymaker --generate_master_key=master.key
export ACRA_MASTER_KEY=`cat master.key | base64`

Read more about different types of keys used in Acra on the Key Management page.

  • Generate the "client" proxy keypair:
$GOPATH/bin/acra-keymaker --client_id=client_name --generate_acraconnector_keys

The name of the key should be longer than 5 characters. It is also used as an identifier for the Secure Session connection between AcraConnector and AcraServer.

The generated keypair client_name and client_name.pub will appear in .acrakeys (or anywhere you ask with --keys_output_dir=/path/to/dir argument).

  • Exchange public keys:
    • You should put public key client_name.pub into the corresponding folder (.acrakeys) on AcraServer/AcraTranslator.
    • You should put AcraServer's/AcraTranslator's public key (called client_name_server.pub) to AcraConnector's key folder (.acrakeys or anything you chose in --keys_output_dir).

2. Launching AcraConnector

A one-line command.

By default, AcraConnector is ready to talk to AcraServer:

$GOPATH/bin/acra-connector --client_id=client_name --acraserver_connection_host=acra.server.host

To point AcraConnector to AcraTranslator, configure an appropriate connection host/port/string and mode:

$GOPATH/bin/acra-connector --client_id=client_name --acratranslator_connection_host=acra.translator.host --mode=acratranslator

AcraConnector CLI reference

--acraserver_api_connection_port
  	Port of Acra HTTP api (default 9090)
--acraserver_api_connection_string
  	Connection string to Acra's API like tcp://x.x.x.x:yyyy or unix:///path/to/socket
--acraserver_connection_host
  	IP or domain to AcraServer daemon
--acraserver_connection_port
  	Port of AcraServer daemon (default 9393)
--acraserver_connection_string
  	Connection string to AcraServer like tcp://x.x.x.x:yyyy or unix:///path/to/socket
--acraserver_securesession_id
  	Expected id from AcraServer for Secure Session (default "acra_server")
--acraserver_tls_transport_enable
  	Use tls to encrypt transport between AcraServer and AcraConnector/client
--acraserver_transport_encryption_disable
  	Use raw transport (tcp/unix socket) between acraserver and acraproxy/client (don't use this flag if you not connect to database with ssl/tls
--acratranslator_connection_host
  	IP or domain to AcraTranslator daemon (default "0.0.0.0")
--acratranslator_connection_port
  	Port of AcraTranslator daemon (default 9696)
--acratranslator_connection_string
  	Connection string to AcraTranslator like grpc://0.0.0.0:9696 or http://0.0.0.0:9595
--acratranslator_securesession_id
  	Expected id from AcraTranslator for Secure Session (default "acra_translator")
--client_id
  	Client ID
--config_file
  	path to config
--dump_config
  	dump config
--http_api_enable
  	Enable AcraServer HTTP API
--incoming_connection_api_port
  	Port for AcraConnector HTTP api (default 9191)
--incoming_connection_api_string
  	Connection string like tcp://x.x.x.x:yyyy or unix:///path/to/socket (default "tcp://127.0.0.1:9191/")
--incoming_connection_port
  	Port to AcraConnector (default 9494)
--incoming_connection_string
  	Connection string like tcp://x.x.x.x:yyyy or unix:///path/to/socket (default "tcp://127.0.0.1:9494/")
--keys_dir
  	Folder from which will be loaded keys (default ".acrakeys")
--logging_format
  	Logging format: plaintext, json or CEF (default "plaintext")
--mode
  	Expected mode of connection. Possible values are: AcraServer or AcraTranslator. Corresponded connection host/port/string/session_id will be used. (default "AcraServer")
--tls_acraserver_sni
  	Expected Server Name (SNI) from AcraServer
--tls_auth
  	Set authentication mode that will be used in TLS connection with AcraServer/AcraTranslator. Values in range 0-4 that set auth type (https://golang.org/pkg/crypto/tls/#ClientAuthType). Default is tls.RequireAndVerifyClientCert (default 4)
--tls_ca
  	Path to root certificate which will be used with system root certificates to validate AcraServer's certificate
--tls_cert
  	Path to certificate
--tls_key
  	Path to private key that will be used in TLS handshake with AcraServer
--user_check_disable
  	Disable checking that connections from app running from another user
-v	Log to stderr

Changing configuration options

You can run with the options from config or you can use the CLI parameters described above.

Copy the example config ($REPO_DIR/configs/acra-connector.yaml or from GOPATH $GOPATH/src/github.com/cossacklabs/acra/configs/acra-connector.yaml) or generate the config yourself:

$GOPATH/bin/acra-connector --dump_config --config_file=<path_to_config_file/acra-connector.yaml>

and run:

$GOPATH/bin/acra-connector --config_file=<path_to_config_file/acra-connector.yaml>

AcraConnector will start listening on port 9494, and will attempt to connect to AcraServer on port 9393.

Configuring incoming connections
--incoming_connection_host
--incoming_connection_port
--incoming_connection_string

You can change the host and port on which AcraConnector will be listening to receive the data from AcraServer/AcraTranslator:

For example, setting the listening port to 5432:

$GOPATH/bin/acra-connector --client_id=client_name --acraserver_connection_host=acra.server.host --incoming_connection_port=5432
--http_api_enable

AcraServer supports two modes: binary data and HTTP API commands.

Start AcraConnector with HTTP API port open to accept API requests for AcraWebConfig and new Zones' requests by sending a request to http://127.0.0.1:9191/getNewZone. With the former, you will receive a json file {"id": "zoneid", "public_key": "base 64 encoded zone public key"} in the response.

$GOPATH/bin/acra-connector --client_id=client_name --acraserver_connection_host=acra.server.host --http_api_enable
--incoming_connection_api_port
--incoming_connection_api_string

You can change the port and host for listening to AcraServer responses made through HTTP API:

$GOPATH/bin/acra-connector --client_id=client_name --acraserver_connection_host=acra.server.host --http_api_enable --incoming_connection_api_port=12345

If you're running AcraConnector from the same machine that contains your app's code (and remember - you must run AcraConnector from a different user!), you can connect to locahost:5432 from your app to have a normal database experience.

Configuring connections to AcraServer
--acraserver_connection_string
--acraserver_connection_port
--acraserver_api_connection_string
--acraserver_api_connection_port

You might specify custom binary data and API connection strings (or a host/port pair) to connect to AcraServer:

$GOPATH/bin/acra-connector --client_id=client_name --acraserver_connection_host=127.0.0.1 --acraserver_connection_port=10000
--acraserver_securesession_id
--acraserver_tls_transport_enable

Set SecureSessionId to use Themis Secure Session between AcraConnector and AcraServer.

Alternatively, you can use TLS: set acraserver_tls_transport_enable to true to enable using TLS between AcraConnector and AcraServer. AcraConnector requires the path to keys and certificates: tls_ca is the path to root certificate; tls_cert path to the TLS server certificate and tls_key is the path to the TLS server key.

Configuring connections to the database
--tls_acraserver_sni

If the client application is connecting to the database with SSL enabled (i.e. sslmode values either allowed or required), you should define tls_acraserver_sni as the database hostname that AcraConnector checks during TLS handshake on the connection attempt.

Configuring connections to AcraTranslator
--mode=acratranslator
--acratranslator_connection_host
--acratranslator_connection_port
--acratranslator_connection_string

To connect to AcraTranslator, you should specify the exact mode and provide the connection parameters:

$GOPATH/bin/acra-connector --client_id=client_name --mode=acratranslator --acratranslator_connection_host=127.0.0.1 --acratranslator_connection_port=10000
Other important configuration options:
--keys_dir

Sets a custom key directory location. By default, AcraConnector is looking for keys in .acrakeys, but you can specify another location.

--disable_user_check

Turns off the test that checks if AcraConnector has started from under a user different from the one running the app. This option was only added to simplify your tests!

$GOPATH/bin/acra-connector --client_id=client_name --acraserver_connection_host=127.0.0.1 --disable_user_check
--logging_format

Changes logging format to plaintext, CEF, or json. Read more about analysing logs here.

AcraWriter

After you have configured AcraConnector, your application can keep using your database handling code as before — all the extra work will now be taken care of by Acra's components.

AcraWriter is a library for your code that can be used anywhere within your app whenever you need to encrypt sensitive records.

Under the hood, it is basically the Themis library generating AcraStructs with the keys you've provided to AcraWriter.

To start protecting your data, pick a place in your code where you would like to integrate AcraWriter (or any self-made AcraStruct encrypter).

You can use our libraries for Go, Python, Ruby, PHP, Node.js.

Or you can take a look at our examples: Go, Python, Ruby, PHP, Node.js.

If you plan to use Acra with:

  • PostgreSQL — use BYTEA binary type;
  • MySQL — use mysql binary type.

Usage scenario

You can encrypt the sensitive data by generating AcraStructs with AcraWriter anywhere across your app. Send INSERTS/UPDATES to the database either through AcraConnector or directly via a separate connection.

You can decrypt AcraStructs by sending database requests to AcraConnector and receive responses that went through AcraServer.

Acra does not dictate a way of storing database requests in the database. If your code is a monolith running on one server as one service, it might make sense to point the writes into the same connections as reads, i.e. AcraConnector.

However, if your code is a huge family of microservices where some of them just write data and some just read it, it is essential to be able to do it the way you want it. You can write AcraStructs generated by AcraWriter directly into the database so that you don't need to carry all the AcraConnector keys with every piece of code on every machine.

For example, we're aware of a setup where AcraStructs are sent down the Kafka stream and emerge in the database at some further point in the future.

Client-side without zones for PostgreSQL

Writes

When you need to store some sensitive data:

  • before doing INSERT/UPDATE on data, you should use AcraWriter libraries specific to your language, encrypting sensitive data into AcraStruct with AcraServer's public key with suffix _storage (some_client_id_storage.pub) that is different from the keypair used by AcraConnector for communicating with AcraServer;
  • then you can proceed with doing INSERT/UPDATE, either through AcraConnector or directly.

Reads

Read requests are the same as the regular reads, but their remote address is AcraConnector instead of your database listener. A typical PostgreSQL connection address would look like this: postgresql://user:password@127.0.0.1:9494/db_name.

A note for PostgreSQL: In the current layout, you aren't required to use SSL when trying to connect to the database. Transport protection for the sensitive (decrypted) data is carried out between AcraServer and AcraConnector via Secure Session. However, you can use SSL connection, too.

Client-side with zones for PostgreSQL

Zone-based encryption is the best way to cryptographically compartmentalise the data that comes from different sources, following the user's choice. Cryptographically, the Zones are a mechanism for modifying the AcraServer's key according to the Zone ID and for mapping Zone's private keys during the decryption. In theory, you could use a new key for every record. However, it would introduce an unwanted overhead on AcraServer's performance.

Writing

To write AcraStruct with Zone keys, you need to:

  • generate the Zone keys on AcraServer,
  • acquire Zone key and Zone ID from AcraServer,
  • encrypt data using this key as AcraServer key in a standard AcraStruct process.

There are two ways you can generate the keys:

  1. First, make sure that you've already generated ACRA_MASTER_KEY and put it into an environmental variable. If you haven't, you can generate it like this:
export ACRA_MASTER_KEY=$(echo -n "My_Very_Long_Key_Phrase_ge_32_chars" | base64)

Read more about the different types of keys used in Acra on Key Management page.

  1. Then build the AcraAddZone utility in AcraServer from Acra repository:
go build acra-addzone
./acra-addzone

When you run it, you should get JSON object that looks something like this:

{"id":"DDDDDDDDQHpbUSOgYTzqCktp","public_key":"VUVDMgAAAC3yMBGsAmK/wBXZkL8iBv/C+7hqoQtSZpYoi4fZYMafkJbWe2dL"}

By decoding base64-wrapped public key, you get the binary Zone key, which can be used for generating AcraStructs.

  1. AcraConnector locally provides HTTP API to add Zones and to return public keys on 127.0.0.1:9191/getNewZone. You can change the port via --incoming_connection_api_port argument.

Both approaches bring identical results.

Reading

Reading with Zones works just like it does without Zones. However, to point AcraServer to the exact Zone key it has to use, you need to structure your query in such a way that AcraStruct is preceded by the corresponding Zone ID in its answer. For example, you may want to store the Zone ID in the preceding column and execute the query in the following way:

SELECT id, lastname, zone, data, datetime FROM table;

where zone contains ZoneID and data stands for AcraStruct.

If you keep the Zone ID in some other place (i.e. configuration file, other table, or other database storage), you need to explicitly state it in the select, for example:

SELECT id, lastname, 'DDDDDDDDQHpbUSOgYTzqCktp'::bytea, data, datetime FROM table;

Note: It's worth mentioning that for PostgreSQL to work properly, ::bytea conversion is necessary.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.