Skip to content

Commit

Permalink
feat: generate plan yamls (#2177)
Browse files Browse the repository at this point in the history
## Description:

This change implements the generation of a yaml that represents the
effect of a sequence of instructions on an enclave. The major changes
are as follows:

- Adds gRPC endpoints `GetStarlarkPackge/ScriptPlanYaml` to APIC and
Enclave Manager for returning this yaml
- Implements `PlanYaml` object and yaml generation logic in
`startosis_engine`
- Adds `UpdatePlan(plan *PlanYaml)` method to `KurtosisInstruction`
interface so each instruction implements logic for updating the plan
yaml
- Most of the knowledge needed to generate the yaml comes from the
interpretation phase and is simply passed into yaml generation logic

Tests are in `startosis_interpreter_plan_yaml_tests.go` and demonstrate
how the `InstructionsPlan` generates the yaml via the `PlanYaml` object.

eg. starlark script turned plan yaml:

```
def run(plan, hi_files_artifact):
	service = plan.add_service(
		name="db",
		config=ServiceConfig(
			image="postgres:latest",
			env_vars={
				"POSTGRES_DB": "kurtosis",
				"POSTGRES_USER": "kurtosis",
				"POSTGRES_PASSWORD": "kurtosis",
			},
			files = {
				"/root": hi_files_artifact,
			}
		)
	)
	execResult = plan.exec(
		service_name="db",
		recipe=ExecRecipe(
			command=["echo", service.ip_address + " " + service.hostname]
		),
		acceptable_codes=[0],
	)	
	runShResult = plan.run_sh(
		run="echo " + execResult["code"] + " " + execResult["output"],
	)
	plan.run_sh(
		run="echo " + runShResult.code + " " + runShResult.output,
	)
```
plan yaml:
```
packageId: DEFAULT_PACKAGE_ID_FOR_SCRIPT
services:
- uuid: "1"
  name: db
  image:
    name: postgres:latest
  envVars:
  - key: POSTGRES_DB
    value: kurtosis
  - key: POSTGRES_PASSWORD
    value: kurtosis
  - key: POSTGRES_USER
    value: kurtosis
  files:
  - mountPath: /root
    filesArtifacts:
    - uuid: "2"
      name: hi-file
filesArtifacts:
- uuid: "2"
  name: hi-file
tasks:
- uuid: "3"
  taskType: exec
  command:
  - echo
  - '{{ kurtosis.1.ip_address }} {{ kurtosis.1.hostname }}'
  serviceName: db
  acceptableCodes:
  - 0
- uuid: "4"
  taskType: sh
  command:
  - echo {{ kurtosis.3.code }} {{ kurtosis.3.output }}
  image: badouralix/curl-jq
- uuid: "5"
  taskType: sh
  command:
  - echo {{ kurtosis.4.code }} {{ kurtosis.4.output }}
  image: badouralix/curl-jq
  ```


## Is this change user facing?
NO

## References:

The Enclave Manager uses this plan yaml to render packages in the Enclave Builder:
#2250

---------

Co-authored-by: Ben Gazzard <ben@dartoxia.com>
  • Loading branch information
tedim52 and Dartoxian committed Mar 13, 2024
1 parent be6d7d3 commit 806a13e
Show file tree
Hide file tree
Showing 50 changed files with 4,533 additions and 350 deletions.

Large diffs are not rendered by default.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions api/protobuf/core/api_container_service.proto
Expand Up @@ -53,6 +53,12 @@ service ApiContainerService {

// Get last Starlark run
rpc GetStarlarkRun(google.protobuf.Empty) returns (GetStarlarkRunResponse) {};

// Gets yaml representing the plan the script will execute in an enclave
rpc GetStarlarkScriptPlanYaml(StarlarkScriptPlanYamlArgs) returns (PlanYaml) {};

// Gets yaml representing the plan the package will execute in an enclave
rpc GetStarlarkPackagePlanYaml(StarlarkPackagePlanYamlArgs) returns (PlanYaml) {};
}

// ==============================================================================================
Expand Down Expand Up @@ -564,3 +570,34 @@ message GetStarlarkRunResponse {

RestartPolicy restart_policy = 8;
}

// ==============================================================================================
// Get Starlark Plan Yaml
// ==============================================================================================

message PlanYaml {
string plan_yaml = 1;
}

message StarlarkScriptPlanYamlArgs {
string serialized_script = 1;

optional string serialized_params = 2;

// The name of the main function, the default value is "run"
optional string main_function_name = 5;
}

message StarlarkPackagePlanYamlArgs {
string package_id = 1;

// Serialized parameters data for the Starlark package main function
// This should be a valid JSON string
optional string serialized_params = 2;

// The relative main file filepath, the default value is the "main.star" file in the root of a package
optional string relative_path_to_main_file = 3;

// The name of the main function, the default value is "run"
optional string main_function_name = 4;
}

0 comments on commit 806a13e

Please sign in to comment.