This section describes the various properties that can be set to control the behavior of your Genie node and cluster. For more information on Spring properties you should see the Spring Boot reference documentation and the Spring Cloud documentation. The Spring properties described here are ones that we have overridden from Spring defaults.
Properties marked 'dynamic' reflect change of property value in the environment happening at runtime. Whereas static properties values are bound during application startup and do not change after the application is up and running.
| Property | Description | Default Value | Dynamic |
|---|---|---|---|
genie.aws.s3.buckets.[bucketName].roleARN |
For the bucket with name |
no |
|
genie.aws.s3.buckets.[bucketName].region |
The AWS region the bucket with |
no |
|
genie.agent.filter.enabled |
If set to |
no |
|
genie.agent.filter.version.minimum |
The minimum version an agent needs to be (e.g., |
yes |
|
genie.agent.filter.version.blacklist |
A regex matched against agent version (e.g., |
yes |
|
genie.agent.filter.version.whitelist |
A regex matched against agent version (e.g., |
yes |
|
genie.aws.credentials.role |
The AWS role ARN to assume when connecting to S3. If this is set Genie will create a credentials provider that will attempt to assume this role on the host Genie is running on |
no |
|
genie.file.cache.location |
Where to store cached files on local disk |
file:///tmp/genie/cache |
no |
genie.grpc.server.enabled |
Whether to start the gRPC server and services during server startup |
true |
no |
genie.grpc.server.services.job-file-sync.ackIntervalMilliseconds |
How many milliseconds to wait between checks whether some acknowledgement should be sent to the agent regardless of
whether the |
30,000 |
no |
genie.grpc.server.services.job-file-sync.maxSyncMessages |
How many messages to receive from the agent before an acknowledgement message is sent back from the server |
10 |
no |
genie.health.maxCpuLoadConsecutiveOccurrences |
Defines the threshold of consecutive occurrences of CPU load crossing the <maxCpuLoadPercent>. Health of the system is marked unhealthy if the CPU load of a system goes beyond the threshold 'maxCpuLoadPercent' for 'maxCpuLoadConsecutiveOccurrences' consecutive times. |
3 |
no |
genie.health.maxCpuLoadPercent |
Defines the threshold for the maximum CPU load percentage to consider for an instance to be unhealthy. Health of the system is marked unhealthy if the CPU load of a system goes beyond this threshold for 'maxCpuLoadConsecutiveOccurrences' consecutive times. |
80 |
no |
genie.http.connect.timeout |
The number of milliseconds before HTTP calls between Genie nodes should time out on connection |
2000 |
no |
genie.http.read.timeout |
The number of milliseconds before HTTP calls between Genie nodes should time out on attempting to read data |
10000 |
no |
genie.jobs.cleanup.deleteDependencies |
Whether or not to delete the dependencies directories for applications, cluster, command to save disk space after job completion |
true |
no |
genie.jobs.clusters.load-balancers.script.destination |
The location on disk where the script source file should be stored after it is downloaded from
|
file:///tmp/genie/loadbalancers/script/destination/ |
no |
genie.jobs.clusters.load-balancers.script.enabled |
Whether the script based load balancer should be enabled for the system or not.
See also: |
false |
no |
genie.jobs.clusters.load-balancers.script.refreshRate |
How frequently to refresh the load balancer script (in milliseconds) |
300000 |
no |
genie.jobs.clusters.load-balancers.script.source |
The location of the script the load balancer should load to evaluate which cluster to use for a job request |
file:///tmp/genie/loadBalancers/script/source/loadBalance.js |
no |
genie.jobs.clusters.load-balancers.script.timeout |
The amount of time (in milliseconds) that the system will attempt to run the cluster load balancer script before it forces a timeout |
5000 |
no |
genie.jobs.forwarding.enabled |
Whether or not to attempt to forward kill and get output requests for jobs |
true |
no |
genie.jobs.forwarding.port |
The port to forward requests to as it could be different than ELB port |
8080 |
no |
genie.jobs.forwarding.scheme |
The connection protocol to use (http or https) |
http |
no |
genie.jobs.locations.archives |
The default root location where job archives should be stored. Scheme should be included. Created if doesn’t exist. |
file:///tmp/genie/archives/ |
no |
genie.jobs.locations.attachments |
The default root location where job attachments will be temporarily stored. Scheme should be included. Created if doesn’t exist. |
file:///tmp/genie/attachments/ |
no |
genie.jobs.locations.jobs |
The default root location where job working directories will be placed. Created by system if doesn’t exist. |
file:///tmp/genie/jobs/ |
no |
genie.jobs.max.stdOutSize |
The maximum number of bytes the job standard output file can grow to before Genie will kill the job |
8589934592 |
no |
genie.jobs.max.stdErrSize |
The maximum number of bytes the job standard error file can grow to before Genie will kill the job |
8589934592 |
no |
genie.jobs.memory.maxSystemMemory |
The total number of MB out of the system memory that Genie can use for running jobs |
30720 |
no |
genie.jobs.memory.defaultJobMemory |
The total number of megabytes Genie will assume a job is allocated if not overridden by a command or user at runtime |
1024 |
no |
genie.jobs.memory.maxJobMemory |
The maximum amount of memory, in megabytes, that a job client can be allocated |
10240 |
no |
genie.jobs.users.creationEnabled |
Whether Genie should attempt to create a system user in order to run the job as or not. Genie user must have sudo rights for this to work. |
false |
no |
genie.jobs.users.runAsUserEnabled |
Whether Genie should run the jobs as the user who submitted the job or not. Genie user must have sudo rights for this to work. |
false |
no |
genie.jobs.active-limit.enabled |
Enables the per-user active job limit. The number of jobs is controlled by the |
false |
no |
genie.jobs.active-limit.count |
The maximum number of active jobs a user is allowed to have. Once a user hits this limit, jobs submitted are rejected. This is property is ignored unless |
100 |
no |
genie.jobs.active-limit.overrides.<user-name> |
The maximum number of active jobs that user 'user-name' is allowed to have. This is property is ignored unless |
- |
yes |
genie.jobs.completion-check-back-off.min-interval |
The minimum time between checks for job completion in milliseconds. Must be greater than zero. |
100 |
no |
genie.jobs.completion-check-back-off.max-interval |
The maximum time between checks for job completion in milliseconds. This is a fallback value, the value used in most cases is specified as part of the |
10000 |
no |
genie.jobs.completion-check-back-off.factor |
Multiplication factor that grows the delay between checks for job completions. Must be greater than 1. |
1.2 |
no |
genie.leader.enabled |
Whether this node should be the leader of the cluster or not. Should only be used if leadership is not being determined by Zookeeper or other mechanism via Spring |
false |
no |
genie.mail.fromAddress |
The e-mail address that should be used as the from address when alert emails are sent |
no |
|
genie.mail.user |
The user to log into the e-mail server with |
no |
|
genie.mail.password |
The password for the e-mail server |
no |
|
genie.redis.enabled |
Whether to enable storage of HTTP sessions inside Redis via Spring Session |
false |
no |
genie.retry.initialInterval |
The amount of time to wait after initial failure before retrying the first time in milliseconds |
10000 |
no |
genie.retry.maxInterval |
The maximum amount of time to wait between retries for the final retry in the back-off policy |
60000 |
no |
genie.retry.noOfRetries |
The number of times to retry requests to before failure |
5 |
no |
genie.retry.s3.noOfRetries |
The number of times to retry requests to S3 before failure |
5 |
no |
genie.security.oauth2.enabled |
Whether to enable oauth2 based security or not for REST APIs |
false |
no |
genie.security.oauth2.pingfederate.enabled |
Whether Ping Federate is being used as the OAuth2 server and Genie should assume default configuration for its tokens |
false |
no |
genie.security.oauth2.pingfederate.jwt.enabled |
Whether to assume that the bearer tokens coming with API requests are JWT tokens or not |
false |
no |
genie.security.oauth2.pingfederate.jwt.keyValue |
The public key used to verify the JWT signature |
no |
|
genie.security.saml.enabled |
Whether SAML security should be turned on to protect access to the user interface |
false |
no |
genie.security.saml.attributes.user |
The key in the SAML assertion to get the user name from |
no |
|
genie.security.saml.attributes.groups.name |
The key in the SAML assertion to get group information for the user from |
no |
|
genie.security.saml.attributes.groups.admin |
The group a user needs to be a member of in order to be granted an admin role |
no |
|
genie.security.saml.idp.serviceProviderMetadataUrl |
The URL where metadata for Genie service SAML configuration can be pulled from |
no |
|
genie.security.saml.keystore.name |
The name of the keystore file on the classpath for SAML assertions |
no |
|
genie.security.saml.keystore.password |
The password for opening the keystore |
no |
|
genie.security.saml.keystore.defaultKey.name |
The name of the default key to use for signing the SAML request |
no |
|
genie.security.saml.keystore.defaultKey.password |
The password to open the default key |
no |
|
genie.security.saml.loadBalancer.contextPath |
The context path for Genie |
/ |
no |
genie.security.saml.loadBalancer.includeServerPortInRequestURL |
Whether or not to include the port of the load balancer in the redirect request |
false |
no |
genie.security.saml.loadBalancer.scheme |
The scheme the load balancer Genie cluster is run behind uses (http or https). Used for SAML post back |
no |
|
genie.security.saml.loadBalancer.serverName |
Root context for the Genie load balancer e.g. genie.prod.com |
no |
|
genie.security.saml.loadBalancer.serverPort |
The port the load balancer is listening on. Used for SAML post back |
no |
|
genie.security.saml.sp.entityId |
The id that Genie is identified by in the identity provider |
no |
|
genie.security.saml.sp.entityBaseURL |
Where the SAML assertion should be posted back to. e.g. https://genie.prod.com |
no |
|
genie.security.x509.enabled |
Whether to enable x509 certificate security on the REST APIs |
false |
no |
genie.swagger.enabled |
Whether to enable Swagger to be bootstrapped into the Genie service so that the endpoint /swagger-ui.html shows API documentation generated by the swagger specification |
false |
no |
genie.tasks.agent-cleanup.enabled |
Whether to enable the task that detects jobs whose agent has gone AWOL, and marks them failed |
true |
no |
genie.tasks.agent-cleanup.refreshInterval |
How often the AWOL agent tasks executed, in milliseconds |
10000 |
no |
genie.tasks.agent-cleanup.timeLimit |
How long of a leeway to give a job after it agent disconnected and before the job is marked failed, in milliseconds |
120000 |
no |
genie.tasks.cluster-checker.healthIndicatorsToIgnore |
The health indicator groups from the actuator /health endpoint to ignore when determining if a node is lost or not as a comma separated list |
genieMemory,mail,genieAgent |
no |
genie.tasks.cluster-checker.lostThreshold |
The number of times a Genie nodes need to fail health check in order for jobs running on that node to be marked as lost and failed by the Genie leader |
3 |
no |
genie.tasks.cluster-checker.port |
The port to connect to other Genie nodes on |
8080 |
no |
genie.tasks.cluster-checker.rate |
The number of milliseconds to wait between health checks to other Genie nodes |
300000 |
no |
genie.tasks.cluster-checker.scheme |
The scheme (http or https) for connecting to other Genie nodes |
http |
no |
genie.tasks.database-cleanup.enabled |
Whether or not to delete old and unused records from the database at a scheduled interval.
See: |
true |
no |
genie.tasks.database-cleanup.maxDeletedPerTransaction |
The number of job records (across multiple tables) to delete from the database in a single transaction. Genie will loop and perform multiple transactions until all jobs older than the retention time are deleted. This is a soft limit, it could be rounded up to the next multiple of page size. |
1000 |
no |
genie.tasks.database-cleanup.pageSize |
The page size used within each cleanup transaction to iterate through the job records |
1000 |
no |
genie.tasks.database-cleanup.expression |
The cron expression for how often to run the database cleanup task |
0 0 0 * * * |
no |
genie.tasks.database-cleanup.retention |
The number of days to retain jobs in the database |
90 |
no |
genie.tasks.database-cleanup.skipJobsCleanup |
Skip the Jobs table when performing database cleanup |
false |
no |
genie.tasks.database-cleanup.skipClustersCleanup |
Skip the Clusters table when performing database cleanup |
false |
no |
genie.tasks.database-cleanup.skipFilesCleanup |
Skip the Files table when performing database cleanup |
false |
no |
genie.tasks.database-cleanup.skipTagsCleanup |
Skip the Tags table when performing database cleanup |
false |
no |
genie.tasks.disk-cleanup.enabled |
Whether or not to remove old job directories on the Genie node or not |
true |
no |
genie.tasks.disk-cleanup.expression |
How often to run the disk cleanup task as a cron expression |
0 0 0 * * * |
no |
genie.tasks.disk-cleanup.retention |
The number of days to leave old job directories on disk |
3 |
no |
genie.tasks.executor.pool.size |
The number of executor threads available for tasks to be run on within the node in an adhoc manner. Best to set to the number of CPU cores x 2 + 1 |
1 |
no |
genie.tasks.scheduler.pool.size |
The number of available threads for the scheduler to use to run tasks on the node at scheduled intervals. Best to set to the number of CPU cores x 2 + 1 |
1 |
no |
genie.tasks.user-metrics.enabled |
Whether or not to publish user-tagged metrics |
true |
no |
genie.tasks.user-metrics.refresh-interval |
Publish/refresh interval in milliseconds |
30000 |
no |
genie.zookeeper.leader.path |
The namespace to use for Genie leadership election of a given cluster |
/genie/leader/ |
no |
genie.s3filetransfer.strictUrlCheckEnabled |
Whether to strictly check an S3 URL for illegal characters before attempting to use it |
false |
no |
| Property | Description | Default Value |
|---|---|---|
info.genie.version |
The Genie version to be displayed by the UI and returned by the actuator /info endpoint. Set by the build. |
Current build version |
management.endpoints.web.base-path |
The default base path for the Spring Actuator[https://docs.spring.io/spring-boot/docs/current/actuator-api/html/]
management endpoints. Switched from default |
/admin |
security.oauth2.client.client-id |
The id of the OAuth2 client |
|
security.oauth2.client.client-secret |
The secret for the oauth2 client |
|
security.oauth2.resource.filter-order |
The order the OAuth2 resource filter is places within the spring security chain |
3 |
security.oauth2.resource.id |
Id of the resource server |
|
security.oauth2.resource.token-info-uri |
URI where to get token information from |
|
spring.application.name |
The name of the application in the Spring context |
genie |
spring.banner.location |
Banner file location |
genie-banner.txt |
spring.data.redis.repositories.enabled |
Whether Spring data repositories should attempt to be created for Redis |
false |
spring.datasource.url |
JDBC URL of the database |
jdbc:h2:mem:genie |
spring.datasource.username |
Username for the datasource |
root |
spring.datasource.password |
Database password |
|
spring.datasource.hikari.auto-commit |
Whether to auto commit transactions |
false |
spring.datasource.hikari.connection-test-query |
Query to use to check if connections are ok |
Select 0; |
spring.datasource.hikari.leak-detection-threshold |
How long to wait (in milliseconds) before a connection should be considered leaked out of the pool if it hasn’t been returned |
30000 |
spring.datasource.hikari.maximum-pool-size |
Maximum connection pool size |
40 |
spring.datasource.hikari.minimum-idle |
Fewest allowed idle connections |
5 |
spring.datasource.hikari.pool-name |
The name of the connection pool. Will show up in logs under this name. |
genie-hikari-db-pool |
spring.flyway.baselineDescription |
Description for the initial baseline of a database instance |
Base Version |
spring.flyway.baselineOnMigrate |
Whether or not to baseline when Flyway is present and the datasource targets a DB that isn’t managed by Flyway |
true |
spring.flyway.baselineVersion |
Initial DB version (When Genie migrated to Flyway is current setting. Shouldn’t touch) |
3.2.0 |
spring.flyway.locations |
Where flyway should look for database migration files |
classpath:db/migration/{vendor} |
spring.jackson.serialization.write-dates-as-timestamps |
Whether to serialize instants as timestamps or ISO8601 strings |
false |
spring.jackson.time-zone |
Time zone used when formatting dates. For instance |
UTC |
spring.jpa.hibernate.ddl-auto |
DDL mode. This is actually a shortcut for the "hibernate.hbm2ddl.auto" property. |
validate |
spring.jpa.hibernate.properties.hibernate.jdbc.time_zone |
The timezone to use when writing dates to the database see article |
UTC |
spring.profiles.active |
The default active profiles when Genie is run |
dev |
spring.mail.host |
The hostname of the mail server |
|
spring.mail.testConnection |
Whether to check the connection to the mail server on startup |
false |
spring.redis.host |
Endpoint for the Redis cluster used to store HTTP session information |
|
spring.servlet.multipart.max-file-size |
Max attachment file size. Values can use the suffixed "MB" or "KB" to indicate a Megabyte or Kilobyte size. |
100MB |
spring.servlet.multipart.max-request-size |
Max job request size. Values can use the suffixed "MB" or "KB" to indicate a Megabyte or Kilobyte size. |
200MB |
spring.session.store-type |
The back end storage system for Spring to store HTTP session information. See Spring Boot Session for more information. Currently on classpath only none, redis and jdbc will work. |
none |
Properties set by default to manipulate various Spring Cloud libraries.
| Property | Description | Default Value |
|---|---|---|
cloud.aws.credentials.useDefaultAwsCredentialsChain |
Whether to attempt creation of a standard AWS credentials chain. See Spring Cloud AWS for more information. |
true |
cloud.aws.region.auto |
Whether the AWS region will be attempted to be auto recognized via the AWS metadata services on EC2. See Spring Cloud AWS for more information. |
false |
cloud.aws.region.static |
The default AWS region. See Spring Cloud AWS for more information. |
us-east-1 |
cloud.aws.stack.auto |
Whether auto stack detection is enabled. See Spring Cloud AWS for more information. |
false |
spring.cloud.zookeeper.enabled |
Whether to enable zookeeper functionality or not |
false |
spring.cloud.zookeeper.connectString |
The connection string for the zookeeper cluster |
localhost:2181 |
| Property | Description | Default Value |
|---|---|---|
spring.datasource.url |
JDBC URL of the database |
jdbc:mysql://127.0.0.1/genie?useUnicode=yes&characterEncoding=UTF-8&useLegacyDatetimeCode=false |
spring.datasource.username |
Username for the datasource |
root |
spring.datasource.password |
Database password |
|
spring.datasource.hikari.data-source-properties.cachePrepStmts |
true |
|
spring.datasource.hikari.data-source-properties.prepStmtCacheSize |
250 |
|
spring.datasource.hikari.data-source-properties.prepStmtCacheSqlLimit |
2048 |
|
spring.datasource.hikari.data-source-properties.useServerPrepStmts |
true |
|
spring.datasource.hikari.data-source-properties.useLocalSessionState |
true |
|
spring.datasource.hikari.data-source-properties.useLocalTransactionState |
true |
|
spring.datasource.hikari.data-source-properties.rewriteBatchedStatements |
true |
|
spring.datasource.hikari.data-source-properties.cacheResultSetMetadata |
true |
|
spring.datasource.hikari.data-source-properties.elideSetAutoCommits |
true |
|
spring.datasource.hikari.data-source-properties.maintainTimeStats |
false |