feat: generate plan yamls

[tedim52]

Description:

This change implements the generation of a yaml that represents the effect of a sequence of instructions on an enclave. The major changes are as follows:

• Adds gRPC endpoints GetStarlarkPackge/ScriptPlanYaml to APIC and Enclave Manager for returning this yaml
• Implements PlanYaml object and yaml generation logic in startosis_engine
• Adds UpdatePlan(plan *PlanYaml) method to KurtosisInstruction interface so each instruction implements logic for updating the plan yaml
  • Most of the knowledge needed to generate the yaml comes from the interpretation phase and is simply passed into yaml generation logic

Tests are in startosis_interpreter_plan_yaml_tests.go and demonstrate how the InstructionsPlan generates the yaml via the PlanYaml object.

eg. starlark script turned plan yaml:

def run(plan, hi_files_artifact):
	service = plan.add_service(
		name="db",
		config=ServiceConfig(
			image="postgres:latest",
			env_vars={
				"POSTGRES_DB": "kurtosis",
				"POSTGRES_USER": "kurtosis",
				"POSTGRES_PASSWORD": "kurtosis",
			},
			files = {
				"/root": hi_files_artifact,
			}
		)
	)
	execResult = plan.exec(
		service_name="db",
		recipe=ExecRecipe(
			command=["echo", service.ip_address + " " + service.hostname]
		),
		acceptable_codes=[0],
	)	
	runShResult = plan.run_sh(
		run="echo " + execResult["code"] + " " + execResult["output"],
	)
	plan.run_sh(
		run="echo " + runShResult.code + " " + runShResult.output,
	)

plan yaml:

packageId: DEFAULT_PACKAGE_ID_FOR_SCRIPT
services:
- uuid: "1"
  name: db
  image:
    name: postgres:latest
  envVars:
  - key: POSTGRES_DB
    value: kurtosis
  - key: POSTGRES_PASSWORD
    value: kurtosis
  - key: POSTGRES_USER
    value: kurtosis
  files:
  - mountPath: /root
    filesArtifacts:
    - uuid: "2"
      name: hi-file
filesArtifacts:
- uuid: "2"
  name: hi-file
tasks:
- uuid: "3"
  taskType: exec
  command:
  - echo
  - '{{ kurtosis.1.ip_address }} {{ kurtosis.1.hostname }}'
  serviceName: db
  acceptableCodes:
  - 0
- uuid: "4"
  taskType: sh
  command:
  - echo {{ kurtosis.3.code }} {{ kurtosis.3.output }}
  image: badouralix/curl-jq
- uuid: "5"
  taskType: sh
  command:
  - echo {{ kurtosis.4.code }} {{ kurtosis.4.output }}
  image: badouralix/curl-jq

Is this change user facing?

NO

References:

The Enclave Manager uses this plan yaml to render packages in the Enclave Builder:
#2250

[gitguardian]

⚠️ GitGuardian has uncovered 39 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
9966505	Triggered	Generic Password	ec3cf48	core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go	View secret
9966505	Triggered	Generic Password	ec3cf48	core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go	View secret
9966505	Triggered	Generic Password	f69701f	core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go	View secret
9966505	Triggered	Generic Password	b254a49	core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go	View secret
9966505	Triggered	Generic Password	b254a49	core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go	View secret
9583343	Triggered	Generic Password	3998b57	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	8c7a419	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	323ac97	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	80ace33	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	908ad7d	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	ccc256b	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	458fd75	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	ccc256b	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	21e4ead	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	13a5cba	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	0544d1b	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	3998b57	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	9109cac	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	21e4ead	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	8c7a419	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	13a5cba	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	679ed73	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	d190067	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	a4e7031	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	13a5cba	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	aef291e	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	13a5cba	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	aef291e	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	ae9f7ce	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	80ace33	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	21e4ead	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	a4e7031	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	9109cac	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	21e4ead	core/server/api_container/server/startosis_engine/plan_yaml_generator_test.go	View secret
9583343	Triggered	Generic Password	13a5cba	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	323ac97	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	d190067	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	03a7535	core/server/api_container/server/startosis_engine/plan.yml	View secret
9583343	Triggered	Generic Password	ec3cf48	core/server/api_container/server/startosis_engine/startosis_interpreter_plan_yaml_test.go	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

<sup>🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!</sup>

[tedim52]

The GitGuardian secrets are only picking up the PASSWORD env vars that I set when writing tests, no secrets were actually used in this PR.

[tedim52]

I considered renaming PlanYaml to EnclaveState for a while as the yaml represents state of an enclave post an instructions plan containing services and files artifacts, but the fact that tasks are in it throws me off a bit as tasks aren't really state, but maybe we consider them so. Open to suggestions on naming.

[h4ck3rk3y]

spent some time reviewing this - overall happy as long as it works!

some of the plan_yaml_generator stuff can be cleaned up but we can get to it in our own time.