Feat/cypress buildkite upgrade #119833
Feat/cypress buildkite upgrade #119833
Conversation
patrykkopycinski
force-pushed
the
feat/cypress-buildkite-upgrade
branch
from
c6325aa
to
4f1333d
Compare
There was a problem hiding this comment.
This is looking awesome! Just left some suggestions.
I have some questions about the whole thing:
-
Could this be done in TS? I think we could create and import some external services and as kibana is a TS project I think it should be a better way to go (understood if this is the initial approach in JS)
-
About share data between steps (to avoid having a single file) I just read this documentation and seems we can use the Meta-data to share info between the steps. Again, understood that this is the initial approach but this could be a next step.
I can imagine the pipeline definition as:
-
Create instance in X version (where X is defined un the pipeline and the script creates the instance with the desired X if config exists)
-
A set of scripts to load data, for example, a step to load data Y, another step to load data Z, etc. Each step using the desired method (by cypress, ESArchive, using API, etc.). We could have different kind of scripts (one for each method) that gets a param and runs whatever we want: ./cypress load_trusted_apps, ./ESArchive load_agent_events, ./api load_policy_integration, etc. Where each script is a wrapper that runs the code in, for example, load_trusted_apps.
-
Run an upgrade to version W.
-
Run desired tests (in the same way we loaded the data) or even load more data to the upgraded version.
-
Run another upgrade if needed
-
Run desired tests or load more data.
-
Repeat steps above how many times you want...
Doing this we will be able to create an infinite set of stages and test cases just using buildkite pipeline.
Note: This is just a suggestion but I would love to discuss this and find a common solution that works for most of the team mates.
Also, at some point we should move the trusted apps test files outside the osquery path :)
Other than that, this is an amazing work, you have rocked it! 🎉
|
|
||
| console.log('credentials', credentials); | ||
|
|
||
| const clusterInfo = await pRetry(() => isInstanceReady({ deploymentId }), { |
There was a problem hiding this comment.
Could you move this pRetry inside the isInstanceReady function? I think it's done more than one time and could be incapsulated inside the function.
|
|
||
| const response = await request.put(`deployments/${deploymentId}?validate_only=false`, newConfig); | ||
|
|
||
| await new Promise((r) => setTimeout(r, 30000)); |
There was a problem hiding this comment.
Why this timoeut? If there is a retry just below this I think we could get rid of this setTimeout, thoughts?
| }); | ||
|
|
||
| it('Add legacy trusted app', () => { | ||
| addAndVerifyIntegration(); |
There was a problem hiding this comment.
Could this be moved to the before clause just right after the addAndVerifyAgentPolicy ?
| @@ -0,0 +1,291 @@ | |||
| /* | |||
There was a problem hiding this comment.
Could this be reused form the one in security solution?
…ildkite-upgrade # Conflicts: # x-pack/plugins/osquery/cypress/screens/integrations.ts # x-pack/plugins/osquery/cypress/tasks/integrations.ts # x-pack/plugins/osquery/cypress/tasks/login.ts
…ildkite-upgrade # Conflicts: # x-pack/plugins/osquery/cypress/cypress.json # x-pack/plugins/osquery/cypress/tasks/live_query.ts # x-pack/plugins/osquery/cypress/tasks/navigation.ts # x-pack/plugins/osquery/server/routes/fleet_wrapper/get_agent_details.ts # yarn.lock
💔 Build FailedFailed CI StepsMetrics [docs]
History
To update your PR or re-run it, just comment with: |
Summary
Add Cypress-based buildkite upgrade testing pipeline