description |
---|
A detailed guide for installing and configuring a hybrid APIM deployment |
This page focuses on the installation of the Self-Hosted Data-Plane, which is part of the API Management platform in a hybrid architecture (SaaS Control-Plane + Self-Hosted Data-Plane).
Component | Description |
---|---|
APIM Console (for API producers) | This web UI gives easy access to some key APIM Management API services. API publishers can use it to publish APIs. Administrators can also configure global platform settings and specific portal settings. |
APIM Management API | This RESTful API exposes services to manage and configure the APIM Console and APIM Developer Portal web UIs. All exposed services are restricted by authentication and authorization rules. For more information, see the Management API Reference section. |
APIM Developer Portal (for API consumers) | This web UI gives easy access to some key APIM API services. It allows API Consumers to manage their applications and search for, view, try out, and subscribe to a published API. |
APIM SaaS API Gateways | APIM Gateway is the core component of the APIM platform. You can think of it like a smart reverse proxy. Unlike a traditional HTTP proxy, APIM Gateway has the capability to apply policies (i.e., rules or logic) to both the request and response phases of an API transaction. With these policies, you can transform, secure, monitor, etc., your APIs. |
Bridge Gateways | A bridge API Gateway exposes extra HTTP services for bridging HTTP calls to the underlying repository (which can be any of our supported repositories: MongoDB, JDBC, etc.) |
Config Database | All the API Management platform management data, such as API definitions, users, applications, and plans. |
S3 Bucket + Analytics Database | Analytics and logs data. |
Gravitee Cockpit | Gravitee Cockpit is a centralized, multi-environments / organizations tool for managing all your Gravitee API Management and Access Management installations in a single place. |
[Optional] API Designer | Drag-and-Drop graphical (MindMap) API designer to quickly and intuitively design your APIs (Swagger / OAS) and deploy mocked APIs for quick testing. |
[Optional] Alert Engine | Alert Engine (AE) provides APIM and AM users with efficient and flexible API platform monitoring, including advanced alerting configuration and notifications sent through their preferred channels, such as email, Slack and using Webhooks. AE does not require any external components or a database as it does not store anything. It receives events and sends notifications under the conditions which have been pre-configured upstream with triggers. |
Self-Hosted Data-Plane components¶
Component | Description |
---|---|
APIM Gateway | APIM Gateway is the core component of the APIM platform. You can think of it like a smart reverse proxy. Unlike a traditional HTTP proxy, APIM Gateway has the capability to apply policies (i.e., rules or logic) to both the request and response phases of an API transaction. With these policies, you can transform, secure, monitor, etc., your APIs. |
Logstash | Collect and send local Gateway logs and metrics to the Gravitee APIM SaaS Control Plane. |
Redis | The database used locally for rate limit synchronized counters (RateLimit, Quota, Spike Arrest) and, optionally, as an external cache for the Cache policy. |
{% hint style="danger" %} Make sure the version you are installing aligns with the Control-Plane SaaS version. {% endhint %}
{% hint style="info" %} Don't forget to add the Gravitee.io License file. {% endhint %}
{% tabs %} {% tab title="Kubernetes (Helm)" %}
- Follow the APIM installation instructions in the Install on Kubernetes guide.
- Install only the Gateway and disable the other components in your
values.yaml
configuration file.
{% hint style="info" %} Additional assets
- Hybrid Deployment on Kubernetes
- Gravitee Helm charts {% endhint %} {% endtab %}
{% tab title="Docker" %} Follow the APIM installation instructions in the Install on Docker guide.
{% hint style="info" %} Download and mount the required plugins for the Gravitee.io Gateway:
- Redis Repository: The database used locally for rate limits synchronized counters
- TCP Reporter: To push events to Logstash {% endhint %} {% endtab %}
{% tab title="VM" %}
Follow the APIM installation instructions in the Install with .ZIP
guide.
{% hint style="info" %} Download and mount the required plugins for the Gravitee.io Gateway:
- Redis Repository: The database used locally for rate limits synchronized counters
- TCP Reporter: To push events to Logstash {% endhint %} {% endtab %} {% endtabs %}
There are at least 3 connections to configure:
- The connection to the SaaS Control-Plane via the Bridge Gateway.
- The connection to push Analytics and Logs using the file or TCP reporter to push data to Logstash and send to the SaaS storage.
- The connection to the local rate limits database (Redis).
- (Optional) The connection to the SaaS Alert Engine.
Management (SaaS Control-Plane Bridge Gateway)¶
{% tabs %} {% tab title="Kubernetes (Helm)" %} {% code title="values.yaml" lineNumbers="true" %}
management:
type: http
gateway:
management:
http:
url: https://bridge-gateway-url:bridge-gateway-port
username: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
password: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
# ssl:
# trustall: true
# verifyHostname: true
# keystore:
# type: jks # Supports jks, pem, pkcs12
# path: ${gravitee.home}/security/keystore.jks
# password: secret
# truststore:
# type: jks # Supports jks, pem, pkcs12
# path: ${gravitee.home}/security/truststore.jks
# password: secret
# proxy:
# host:
# port:
# type: http
# username:
# password:
{% endcode %}
{% hint style="info" %} Additional assets
- Hybrid deployment on Kubernetes
- Gravitee Helm charts {% endhint %} {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" lineNumbers="true" %}
version: '3'
services:
gateway:
image: graviteeio/apim-gateway:<VERSION-ALIGNED-WITH-CONTROL-PLANE>
container_name: gio_apim_gateway
restart: always
ports:
- "8082:8082"
environment:
# --- BRIDGE GATEWAYS ---
- gravitee_management_type=http
- gravitee_management_http_url=https://bridge-gateway-url:bridge-gateway-port
- gravitee_management_http_authentication_basic_username=bridge-gateway-username
- gravitee_management_http_authentication_basic_password=bridge-gateway-password
{% endcode %} {% endtab %}
{% tab title="VM" %} {% code title="gravitee.yaml" lineNumbers="true" %}
management:
type: http
http:
url: https://bridge-gateway-url:bridge-gateway-port
keepAlive: true
idleTimeout: 30000
connectTimeout: 10000
authentication:
basic:
username: bridge-gateway-username
password: bridge-gateway-password
# ssl:
# trustAll: true
# verifyHostname: true
# keystore:
# type: # can be jks / pem / pkcs12
# path:
# password:
# trustore:
# type: # can be jks / pem / pkcs12
# path:
# password:
{% endcode %} {% endtab %} {% endtabs %}
{% tabs %} {% tab title="Kubernetes (Helm)" %} {% code title="values.yaml" lineNumbers="true" %}
gateway:
reporters:
elasticsearch:
enabled: false
tcp:
enabled: true
host: logstash-host
port: 8379
output: elasticsearch
{% endcode %} {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" overflow="wrap" %}
version: '3'
services:
gateway:
image: graviteeio/apim-gateway:<VERSION-ALIGNED-WITH-CONTROL-PLANE>
container_name: gio_apim_gateway
restart: always
ports:
- "8082:8082"
environment:
# --- LOGSTASH ---
- gravitee_reporters_elasticsearch_enabled=false
- gravitee_reporters_tcp_enabled=true
- gravitee_reporters_tcp_host=logstash-host
- gravitee_reporters_tcp_port=8379
- gravitee_reporters_tcp_output=elasticsearch
{% endcode %} {% endtab %}
{% tab title="VM" %} {% code title="gravitee.yaml" lineNumbers="true" %}
reporters:
elasticsearch:
enabled: false
tcp:
enabled: true
host: logstash-host
port: 8379
output: elasticsearch
{% endcode %} {% endtab %} {% endtabs %}
{% tabs %} {% tab title="Kubernetes (Helm)" %} {% code title="values.yaml" lineNumbers="true" %}
ratelimit:
type: redis
redis:
host: redis-host
port: 6379
password: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
download: true
{% endcode %}
{% hint style="info" %} Additional assets
- Full
values.yaml
example {% endhint %} {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" overflow="wrap" %}
version: '3'
services:
gateway:
image: graviteeio/apim-gateway:<VERSION-ALIGNED-WITH-CONTROL-PLANE>
container_name: gio_apim_gateway
restart: always
ports:
- "8082:8082"
environment:
# --- RATE LIMIT REPO ---
- gravitee_ratelimit_type=redis
- gravitee_ratelimit_redis_host=redis-host
- gravitee_ratelimit_redis_port=6379
- gravitee_ratelimit_redis_password=${REDIS_PASS:-redis-password}
{% endcode %} {% endtab %}
{% tab title="VM" %} {% code title="gravitee.yaml" lineNumbers="true" %}
ratelimit:
type: redis
redis:
host: redis-host
port: 6379
password: redis-password
{% endcode %} {% endtab %} {% endtabs %}
{% tabs %} {% tab title="Kubernetes (Helm)" %} {% code title="values.yaml" lineNumbers="true" %}
alerts:
enabled: true
endpoints:
- https://alert-engine-url:alert-engine-port
security:
enabled: true
username: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
password: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
{% endcode %} {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" lineNumbers="true" %}
version: '3'
services:
gateway:
image: graviteeio/apim-gateway:<VERSION-ALIGNED-WITH-CONTROL-PLANE>
container_name: gio_apim_gateway
restart: always
ports:
- "8082:8082"
environment:
# --- ALERT ENGINE ---
- gravitee_alerts_alertengine_enabled=true
- gravitee_alerts_alertengine_ws_discovery=true
- gravitee_alerts_alertengine_ws_endpoints_0=https://alert-engine-url:alert-engine-port
- gravitee_alerts_alertengine_ws_security_username=alert-engine-username
- gravitee_alerts_alertengine_ws_security_password=alert-engine-password
{% endcode %} {% endtab %}
{% tab title="VM" %} {% code title="gravitee.yaml" lineNumbers="true" %}
alerts:
alert-engine:
enabled: true
ws:
discovery: true
endpoints:
- https://alert-engine-url:alert-engine-port
security:
username: alert-engine-username
password: alert-engine-password
{% endcode %} {% endtab %} {% endtabs %}
{% tabs %} {% tab title="Kubernetes (Helm)" %} {% code title="values.yaml" lineNumbers="true" %}
management:
type: http
gateway:
management:
http:
url: https://bridge-gateway-url:bridge-gateway-port
username: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
password: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
reporters:
elasticsearch:
enabled: false
tcp:
enabled: true
host: logstash
port: 8379
output: elasticsearch
alerts:
enabled: true
endpoints:
- https://alert-engine-url:alert-engine-port
security:
enabled: true
username: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
password: kubernetes://<namespace>/secrets/<my-secret-name>/<my-secret-key>
{% endcode %} {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" lineNumbers="true" %}
version: '3'
volumes:
data-redis:
services:
gateway:
image: graviteeio/apim-gateway:<VERSION-ALIGNED-WITH-CONTROL-PLANE>
container_name: gio_apim_hybrid_gateway
restart: always
ports:
- "8082:8082"
depends_on:
redis:
condition: service_healthy
logstash:
condition: service_healthy
volumes:
# --- LOCAL LOG FILES ---
- ./logs/apim-gateway-dev:/opt/graviteeio-gateway/logs
# --- EE LICENSE FILE ---
- ${GIO_LICENSE}:/opt/graviteeio-gateway/license/license.key
# --- ADDITIONAL PLUGINS ---
- ./plugins:/opt/graviteeio-gateway/plugins-ext
environment:
# --- PLUGINS LOCATIONS ---
- gravitee_plugins_path_0=/opt/graviteeio-gateway/plugins
- gravitee_plugins_path_1=/opt/graviteeio-gateway/plugins-ext
# --- COCKPIT ORGS & ENVS ---
- gravitee_organizations=<YOUR-COCKPIT-ORG-HRID>,<YOUR-COCKPIT-ORG-HRID>
- gravitee_environments=<YOUR-COCKPIT-ENV-HRID>,<YOUR-COCKPIT-ENV-HRID>
# --- SHARDING TAGS & TENANTS ---
# - gravitee_tags=internal
# - gravitee_tenant=xxx
# --- BRIDGE GATEWAYS ---
- gravitee_management_type=http
- gravitee_management_http_url=https://bridge-gateway-url:bridge-gateway-port
- gravitee_management_http_authentication_basic_username=bridge-gateway-username
- gravitee_management_http_authentication_basic_password=bridge-gateway-password
# --- RATE LIMIT REPO ---
- gravitee_ratelimit_type=redis
- gravitee_ratelimit_redis_host=redis
- gravitee_ratelimit_redis_port=6379
- gravitee_ratelimit_redis_password=${REDIS_PASS:-redis-password}
# - gravitee_ratelimit_type=hazelcast
# --- LOGSTASH ---
- gravitee_reporters_elasticsearch_enabled=false
- gravitee_reporters_tcp_enabled=true
- gravitee_reporters_tcp_host=logstash
- gravitee_reporters_tcp_port=8379
- gravitee_reporters_tcp_output=elasticsearch
# --- ALERT ENGINE ---
- gravitee_alerts_alertengine_enabled=true
- gravitee_alerts_alertengine_ws_discovery=true
- gravitee_alerts_alertengine_ws_endpoints_0=https://alert-engine-url:alert-engine-port
- gravitee_alerts_alertengine_ws_security_username=alert-engine-username
- gravitee_alerts_alertengine_ws_security_password=alert-engine-password
# --- SECRETS ---
- gravitee_api_properties_encryption_secret=your-own-api-32-caracters-secret
redis:
# https://hub.docker.com/_/redis?tab=tags
image: redis:${REDIS_VERSION:-7.2.1-alpine}
container_name: gio_apim_hybrid_redis
hostname: redis
restart: always
ports:
- '6379:6379'
command: redis-server --requirepass ${REDIS_PASS:-28kjzEGquZYrztGyPMofR8eWuNbn4YqR}
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 1s
timeout: 3s
retries: 30
volumes:
- data-redis:/data
logstash:
# https://www.docker.elastic.co/r/logstash/logstash-oss
image: docker.elastic.co/logstash/logstash-oss:${LOGSTASH_VERSION:-8.10.2}
container_name: gio_apim_hybrid_logstash
hostname: logstash
ports:
- "8379:8379"
healthcheck:
test: curl -f -I http://localhost:9600/_node/pipelines/main || exit 1
start_period: 20s
interval: 3s
timeout: 5s
retries: 30
volumes:
- ./config/logstash:/usr/share/logstash/pipeline:ro
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
{% endcode %} {% endtab %}
{% tab title="VM" %} {% code title="gravitee.yaml" lineNumbers="true" %}
############################################################################################################
#################################### Gravitee.IO Gateway - Configuration ###################################
############################################################################################################
############################################################################################################
# This file is the general configuration of Gravitee.IO Gateway:
# - Properties (and respective default values) in comment are provided for information.
# - You can reference other property by using ${property.name} syntax
# - gravitee.home property is automatically set-up by launcher and refers to the installation path. Do not override it !
#
# Please have a look to http://docs.gravitee.io/ for more options and fine-grained granularity
############################################################################################################
organizations: cockpit-org-hrid
environments: cockpit-env-hrid
tags: your, sharding, tags #example: internal
plugins:
path:
- /opt/graviteeio-gateway/plugins
- /opt/graviteeio-gateway/plugins-ext
management:
type: http
http:
url: https://bridge-gateway-url:bridge-gateway-port
authentication:
basic:
username: bridge-gateway-username
password: bridge-gateway-password
ratelimit:
# type: hazelcast
type: redis
redis:
host: redis-host
port: 6379
password: redis-password
cache:
type: ehcache
reporters:
elasticsearch:
enabled: false # Is the reporter enabled or not (default to true)
tcp:
enabled: true
host: logstash-host
port: logstash-port
output: elasticsearch
services:
core:
http:
enabled: true
port: 18082
host: localhost
authentication:
type: basic
users:
admin: internal-api-password
sync:
delay: 5000
unit: MILLISECONDS
distributed: false # By enabling this mode, data synchronization process is distributed over clustered API Gateways.
bulk_items: 100 # Defines the number of items to retrieve during synchronization (events, plans, api keys, ...).
local:
enabled: false
path: ${gravitee.home}/apis # The path to API descriptors
monitoring:
delay: 5000
unit: MILLISECONDS
distributed: false # By enabling this mode, data monitoring gathering process is distributed over clustered API Gateways.
metrics:
enabled: false
prometheus:
enabled: true
tracing:
enabled: false
api:
properties:
encryption:
secret: your-own-api-32-caracters-secret
alerts:
alert-engine:
enabled: true
ws:
discovery: true
endpoints:
- https://alert-engine-url:alert-engine-port
security:
username: alert-engine-username
password: alert-engine-password
classloader:
legacy:
enabled: false
{% endcode %} {% endtab %} {% endtabs %}
{% tabs %} {% tab title="Kubernetes (Helm)" %} Bitnami Helm charts {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" lineNumbers="true" %}
version: '3'
volumes:
data-redis:
services:
redis:
# https://hub.docker.com/_/redis?tab=tags
image: redis:${REDIS_VERSION:-7.2.1-alpine}
container_name: gio_apim_hybrid_redis
hostname: redis
restart: always
ports:
- '6379:6379'
command: redis-server --requirepass ${REDIS_PASS:-28kjzEGquZYrztGyPMofR8eWuNbn4YqR}
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 1s
timeout: 3s
retries: 30
volumes:
- data-redis:/data
{% endcode %} {% endtab %}
{% tab title="VM" %} Installing Redis from redis.io {% endtab %} {% endtabs %}
{% tabs %} {% tab title="Kubernetes (Helm)" %}
- Official Helm charts
- Bitnami Helm charts {% endtab %}
{% tab title="Docker" %} {% code title="docker-compose.yaml" lineNumbers="true" %}
version: '3'
services:
logstash:
# https://www.docker.elastic.co/r/logstash/logstash-oss
image: docker.elastic.co/logstash/logstash-oss:${LOGSTASH_VERSION:-8.10.2}
container_name: gio_apim_hybrid_logstash
hostname: logstash
ports:
- "8379:8379"
healthcheck:
test: curl -f -I http://localhost:9600/_node/pipelines/main || exit 1
start_period: 20s
interval: 3s
timeout: 5s
retries: 30
volumes:
- ./config/logstash:/usr/share/logstash/pipeline:ro
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
{% endcode %} {% endtab %}
{% tab title="VM" %} Download Logstash OSS {% endtab %} {% endtabs %}
{% code title="logstash.conf" lineNumbers="true" %}
input {
tcp {
port => 8379
codec => "json_lines"
}
}
filter {
if [type] != "request" or [type] != "v4-metrics" {
mutate { remove_field => ["path", "host"] }
}
}
output {
s3 {
access_key_id => "${S3_ACEESS_KEY_ID}"
secret_access_key => "${S3_SECRET_ACCESS_KEY}"
region => "${S3_REGION}"
bucket => "${S3_BUCKET_NAME}"
rotation_strategy => time
time_file => 1
codec => "json_lines"
}
}
{% endcode %}
{% hint style="info" %} Additional assets
- Configuring Logstash {% endhint %}
{% tabs %} {% tab title="Kubernetes (Helm)" %}
- Official Helm charts
- Bitnami Helm charts {% endtab %}
{% tab title="Docker" %} You have to build your own docker image:
{% code title="Dockerfile" lineNumbers="true" %}
FROM fluent/fluentd:v1.16.2-1.0
USER root
RUN ["gem", "install", "fluent-plugin-s3"]
USER fluent
{% endcode %}
{% code title="docker-compose.yaml" lineNumbers="true" %}
version: '3'
services:
fluentd:
image: fluentd:s3
container_name: gio_apim_fluentd
hostname: fluentd
restart: always
ports:
- "9000:9000"
volumes:
- ./fluentd_conf:/fluentd/etc
{% endcode %} {% endtab %}
{% tab title="VM" %} Download Fluentd {% endtab %} {% endtabs %}
{% code title="fluentd.conf" lineNumbers="true" %}
<source>
@type tcp
tag tcp
<parse>
@type json
</parse>
port 9000
</source>
<match *.**>
@type s3
aws_key_id "xxxxxxxxxxxxxxx"
aws_sec_key "xxxxxxxxxxxxxxx"
s3_bucket "my-s3-bucket"
s3_region "my-s3-region"
path /
time_slice_format %Y%m%d%H
time_slice_wait 10m
time_format %Y%m%d%H%M
buffer_type file
buffer_path /fluentd/log
buffer_chunk_limit 256m
buffer_queue_limit 512
flush_interval 10s
flush_at_shutdown true
<format>
@type json
</format>
</match>
{% endcode %}
{% hint style="info" %} Additional assets
- Configuring Fluentd {% endhint %}