Configuring Jetstream Application
In jetstream, there are two types of configuration data.
- Static config
- Dynamic config
Jetstream application gets wired at runtime by the spring container. Spring container requires the wiring specification to be specified as XML document. This is made available through static configuration.
Static config is either sourced from the class path or file path.
Configuration in file path supersedes configuration in class path for the same bean in given application.
All Jetstream applications need to set JETSTREAM_HOME environment variable to specify the configuration directory for the application.
Set JETSTREAM_HOME to the parent of "JetstreamConf" directory
For example :
JETSTREAM_HOME = C:\source directory\Jetstream\realtimedefender\buildsrc
Jetstream will look for "JetstreamConf" directory and then load all the xml files from that directory.
Jetstream allows dynamic configuration for beans, config that can be changed at runtime.
Dynamic config simplifies and facilitates the following
Bean config can be deployed in multiple applications.
Bean can have application specific configuration
Bean can also have data center specific configuration per application
Bean can also have node specific configuration per application
In this model if configuration for a particular version of a bean for a particular app needs to be applied, the application will look for bean in this order: local/node, data center and global scope.
If local/node scope is provisioned, it is applied. If node is not specified, then it will lookup data center scope. If this is provisioned then the configuration at the data center scope will be applied. If no configuration is found at the data center scope, then global scope will be looked up for the specified version of config for this bean for the specified application.
If no configuration is found for the specified version of the bean in dynamic configuration repository, it means application does not have any dynamic config.
Dynamic configuration requires that your application is assigned to run with a unique name and config version. This is supplied to the application through 2 command line parameters viz. "-n" for app name and "-cv" for config version. When you submit new config for the app using the Configuration Management Tool described below you have to specify the name and config version provided to the app. Without this dynamic configuration will not work properly.
Repository for storing bean config is MongoDB database. It is very simple to hook MongoDB as a resource to the application and it can be done by setting MONGO_HOME environment variable.
URL format for MONGO_HOME is mongo://databaseserver:port/database
For example on dev machines
Set MONGO_HOME=mongo://mongohost1:27017/jetstream;mongo://mongohost2:27017/jetstream;mongo://mongohost3:27017/jetstream
Above sample format is a standard eBay model and it represents all the nodes in a Mongo replica set that gets provisioned.
Minimum number of nodes for MongoDB in eBay production environment is 3 node replica set to make it highly available.
At application startup, it loads bean definitions from MongoDB.
It running on two http ports: 8080 and 9999, itself is also a jetstream app. The 8080 port is for configuration UI, 9999 is jetstream monitoring port.
The configuration management depends on MongoDB and zookeeper, and it have been dockerized. Itself is also a jetstream app, and you can use same way to start the configuration app.
The config use spring security to manage authentication, and by default it use admin/admin as the user name and password. It can be customized by using the buildsrc/webapp/WEB-INF/spring-security.xml
-
mvn clean install
-
Should make sure zkserver and mongoserver running first
sudo docker run -d --name zkserver -t "pulsar/zookeeper"
sudo docker run -d --name mongoserver -t "mongo"
-
sudo docker build -t yourusername/config .
-
sudo docker run -p 0.0.0.0:9999:9999 -p 0.0.0.0:8080:8080 --link zkserver:zkserver --link mongoserver:mongoserver -t yourusername/config
Then access the http://host:8080/configuration to access the UI.
- Access the http://host:port/configuration, enter admin/admin.
- First need register the app by click the link 'can't find your app name? ', the appname should align with the name with app '-n' start parameter.
- Then use the Create/Update/Delete/Import to manipulate configuration.
- It allows an app have multiple version configuration with same bean name, only the config version matched with eth '-cv' parameter with the app will be take effect.
- It also allow global/dc-aware/local(IP specific) configuration
- When click the 'Push to Mongo' after Update the configuration, it will trigger a message via zookeeper to all apps, and the mathched will reload the configuration.
You can apply dynamic config to a specific server(s) using local scope. In the UI, select local and also input the hostname of the server.
You can apply dynamic config to specific datacenter. All the nodes running in the datacenter will pickup the dynamic config update. In the UI, select dc, and then input dc name. For exmaple - hostname.xyz.com, it will use xyz and com for DC matching.
It means the configuration will be push to all running apps.
- Check the console log when the tool runs. It outputs the bean definition, version, application details etc
- Check the console log of Jetstream application. You will see bean change event notification along with logs about spring container being updated with the new bean definition.
- Monitoring page of every application provides details about config updates received from mongo. - http://machinename:port/config/mongo/MongoDbResource?$format=help
Only the bean with version above the version in app will be take effected.
e.g
Application have bean1 with version 10 in it's Spring container. When it gets notified that a new version is pushed and the version is lower than the local one. App won't pick it up.