Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unblock elastic/security for public serverless #505

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

gbanasiak
Copy link
Contributor

@gbanasiak gbanasiak commented Oct 31, 2023

Similar to #465.

Remarks:

  • Assumption: elastic/security will be run against serverless Security projects.
  • With the above assumption logs-endpoint composable templates and all referenced objects are already included, so can by skipped entirely by Rally.
  • The track-empty-index-template is a workaround for empty composable-templates array which is not accepted by Rally track schema. This can happen if the only integration type is logs-enpoint.
  • ILM is not supported in serverless, so I'm adopting a solution from elastic/logs - lifecycle parameter and track-data-stream-lifecycle component template which either configures security ILM policy or sets a default/empty DLM policy.

Testing:

# logs-endpoint-params.json
{
  "query_warmup_time_period": "1",
  "query_time_period": "1",
  "workflow_time_interval": "1",
  "think_time_interval": "1",
  "integration_ratios": {
    "logs-endpoint": {
      "corpora": {
        "endpoint-events-file": 0.2,
        "endpoint-events-library": 0.1,
        "endpoint-events-network": 0.2,
        "endpoint-events-process": 0.3,
        "endpoint-events-registry": 0.1,
        "endpoint-events-security": 0.1
      }
    }
  }
}

TRACK=elastic/security

# logs-endpoint integration
esrally race --track-path=../rally-tracks/$TRACK --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort --challenge="security-indexing" --test-mode --track-params=../<path>/logs-endpoint-params.json
esrally race --track-path=../rally-tracks/$TRACK --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort --challenge="security-querying" --test-mode --track-params=../<path>/logs-endpoint-params.json --exclude-tasks="tag:setup"
esrally race --track-path=../rally-tracks/$TRACK --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort --challenge="security-indexing-querying" --test-mode --track-params=../<path>/logs-endpoint-params.json

# integrations other than logs-endpoint
esrally race --track-path=../rally-tracks/$TRACK --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort --challenge="security-indexing" --test-mode 
esrally race --track-path=../rally-tracks/$TRACK --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort --challenge="security-querying" --test-mode --exclude-tasks="tag:setup"
esrally race --track-path=../rally-tracks/$TRACK --target-hosts=${ES_HOST}:443 --pipeline=benchmark-only --client-options="use_ssl:true,api_key:${ES_API_KEY}" --on-error=abort --challenge="security-indexing-querying" --test-mode --track-params="query_warmup_time_period:1,query_time_period:1,workflow_time_interval:1,think_time_interval:1"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants