Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add env to activation table #1156

Merged
merged 9 commits into from
May 24, 2024
21 changes: 17 additions & 4 deletions docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -507,22 +507,35 @@ Typical examples of build dependencies are:

## The `activation` table

If you want to run an activation script inside the environment when either doing a `pixi run` or `pixi shell` these can be defined here.
The scripts defined in this table will be sourced when the environment is activated using `pixi run` or `pixi shell`
The activation table is used for specialized activation operations that need to be run when the environment is activated.

There are two types of activation operations a user can modify in the manifest:

- `scripts`: A list of scripts that are run when the environment is activated.
- `env`: A mapping of environment variables that are set when the environment is activated.

These activation operations will be run before the `pixi run` and `pixi shell` commands.

!!! note
The activation scripts are run by the system shell interpreter as they run before an environment is available.
The activation operations are run by the system shell interpreter as they run before an environment is available.
This means that it runs as `cmd.exe` on windows and `bash` on linux and osx (Unix).
Only `.sh`, `.bash` and `.bat` files are supported.

If you have scripts per platform use the [target](#the-target-table) table.
And the environment variables are set in the shell that is running the activation script, thus take note when using e.g. `$` or `%`.

If you have scripts or env variable per platform use the [target](#the-target-table) table.

```toml
[activation]
scripts = ["env_setup.sh"]
env = { ENV_VAR = "value" }

# To support windows platforms as well add the following
[target.win-64.activation]
scripts = ["env_setup.bat"]

[target.linux-64.activation.env]
ENV_VAR = "linux-value"
```

## The `target` table
Expand Down
5 changes: 4 additions & 1 deletion examples/ros2-nav2/pixi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,11 @@ authors = ["Ruben Arts <ruben@prefix.dev>"]
channels = ["conda-forge", "robostack-staging"]
platforms = ["linux-64", "osx-arm64", "osx-64"]

[activation.env]
TURTLEBOT3_MODEL = "waffle"

[tasks.start]
env = { GAZEBO_MODEL_PATH = "$GAZEBO_MODEL_PATH:$PIXI_PROJECT_ROOT/.pixi/env/share/turtlebot3_gazebo/models", TURTLEBOT3_MODEL = "waffle" }
env = {GAZEBO_MODEL_PATH = "$GAZEBO_MODEL_PATH:$PIXI_PROJECT_ROOT/.pixi/env/share/turtlebot3_gazebo/models"}
cmd = "ros2 launch nav2_bringup tb3_simulation_launch.py headless:=False"

[dependencies]
Expand Down
689 changes: 683 additions & 6 deletions pixi.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions pixi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -69,5 +69,6 @@ pyyaml = ">=6.0.1,<6.1"
taplo = ">=0.9.1,<0.10"

[environments]
default = { features = ["docs", "schema"] }
docs = { features = ["docs"], no-default-feature = true }
schema = { features = ["schema"], no-default-feature = true }
11 changes: 11 additions & 0 deletions schema/examples/invalid/bad_env_variable.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#:schema ./../../schema.json
[project]
name = "project"
platforms = ["linux-64"]
channels = ["conda-forge"]

[activation.env]
# It can not be a list
test = ["pytest"]
# It can not be a number
test2 = 1
4 changes: 4 additions & 0 deletions schema/examples/valid/full.toml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ prod = {features = ["test2"], solve-group = "test"}

[activation]
scripts = ["activate.sh", "deactivate.sh"]
env = { TEST = "bla"}

[target.unix.activation.env]
TEST2 = "bla2"

[target.win-64.activation]
scripts = ["env_setup.bat"]
Expand Down
5 changes: 5 additions & 0 deletions schema/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,11 @@ class Activation(StrictBaseModel):
description="The scripts to run when the environment is activated",
examples=["activate.sh", "activate.bat"],
)
env: dict[NonEmptyStr, NonEmptyStr] | None = Field(
None,
description="A map of environment variables to values, used in the activation of the environment. These will be set in the shell. Thus these variables are shell specific. Using '$' might not expand to a value in different shells.",
examples=[{"key": "value"}, {"ARGUMENT": "value"}],
)


##################
Expand Down
17 changes: 17 additions & 0 deletions schema/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,23 @@
"type": "object",
"additionalProperties": false,
"properties": {
"env": {
"title": "Env",
"description": "A map of environment variables to values, used in the activation of the environment. These will be set in the shell. Thus these variables are shell specific. Using '$' might not expand to a value in different shells.",
"type": "object",
"additionalProperties": {
"type": "string",
"minLength": 1
},
"examples": [
{
"key": "value"
},
{
"ARGUMENT": "value"
}
]
},
"scripts": {
"title": "Scripts",
"description": "The scripts to run when the environment is activated",
Expand Down
1 change: 1 addition & 0 deletions schema/test_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def valid_manifest(request) -> str:
params=[
"empty",
"no_channel",
"bad_env_variable",
],
)
def invalid_manifest(request) -> str:
Expand Down
10 changes: 8 additions & 2 deletions src/activation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,16 @@ impl Environment<'_> {
}
EnvironmentName::Default => self.project().name().to_string(),
};
HashMap::from_iter([
let mut map = HashMap::from_iter([
(format!("{ENV_PREFIX}NAME"), self.name().to_string()),
(
format!("{ENV_PREFIX}PLATFORMS"),
self.platforms().iter().map(|plat| plat.as_str()).join(","),
),
("PIXI_PROMPT".to_string(), format!("({}) ", prompt)),
])
]);
map.extend(self.activation_env(Some(Platform::current())));
map
}
}

Expand Down Expand Up @@ -239,6 +241,9 @@ mod tests {
channels = ["conda-forge"]
platforms = ["linux-64", "osx-64", "win-64"]

[activation.env]
TEST = "123test123"

[feature.test.dependencies]
pytest = "*"
[environments]
Expand All @@ -260,6 +265,7 @@ mod tests {
assert_eq!(env.get("PIXI_ENVIRONMENT_NAME").unwrap(), "test");
assert!(env.get("PIXI_PROMPT").unwrap().contains("pixi"));
assert!(env.get("PIXI_PROMPT").unwrap().contains("test"));
assert!(env.get("TEST").unwrap().contains("123test123"));
}

#[test]
Expand Down
12 changes: 12 additions & 0 deletions src/project/environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,18 @@ impl<'p> Environment<'p> {
.collect()
}

/// Returns the environment variables that should be set when activating this environment.
///
/// The environment variables of all features are combined in the order they are defined for the environment.
pub fn activation_env(&self, platform: Option<Platform>) -> HashMap<String, String> {
self.features()
.filter_map(|f| f.activation_env(platform))
.fold(HashMap::new(), |mut acc, env| {
acc.extend(env.iter().map(|(k, v)| (k.clone(), v.clone())));
acc
})
}

/// Validates that the given platform is supported by this environment.
fn validate_platform_support(
&self,
Expand Down
3 changes: 3 additions & 0 deletions src/project/manifest/activation.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
use indexmap::IndexMap;
use serde::Deserialize;

#[derive(Default, Clone, Deserialize, Debug)]
#[serde(deny_unknown_fields)]
pub struct Activation {
pub scripts: Option<Vec<String>>,
/// Environment variables to set before running the scripts.
pub env: Option<IndexMap<String, String>>,
}
12 changes: 12 additions & 0 deletions src/project/manifest/feature.rs
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,18 @@ impl Feature {
.next()
}

/// Returns the activation environment for the most specific target that matches the given
/// `platform`.
///
/// Returns `None` if this feature does not define any target with an activation.
pub fn activation_env(&self, platform: Option<Platform>) -> Option<&IndexMap<String, String>> {
self.targets
.resolve(platform)
.filter_map(|t| t.activation.as_ref())
.filter_map(|a| a.env.as_ref())
.next()
}

/// Returns true if the feature contains any reference to a pypi dependencies.
pub fn has_pypi_dependencies(&self) -> bool {
self.targets
Expand Down
115 changes: 115 additions & 0 deletions src/project/manifest/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1315,6 +1315,121 @@ mod tests {
);
}

#[test]
fn test_activation_env() {
let contents = r#"
[project]
name = "foo"
channels = []
platforms = ["win-64", "linux-64"]

[activation.env]
FOO = "main"

[target.win-64.activation]
env = { FOO = "win-64" }

[target.linux-64.activation.env]
FOO = "linux-64"

[feature.bar.activation]
env = { FOO = "bar" }

[feature.bar.target.win-64.activation]
env = { FOO = "bar-win-64" }

[feature.bar.target.linux-64.activation]
env = { FOO = "bar-linux-64" }
"#;

let manifest = Manifest::from_str(Path::new("pixi.toml"), contents).unwrap();
let default_targets = &manifest.default_feature().targets;
let default_activation_env = default_targets
.default()
.activation
.as_ref()
.and_then(|a| a.env.as_ref());
let win64_activation_env = default_targets
.for_target(&TargetSelector::Platform(Platform::Win64))
.unwrap()
.activation
.as_ref()
.and_then(|a| a.env.as_ref());
let linux64_activation_env = default_targets
.for_target(&TargetSelector::Platform(Platform::Linux64))
.unwrap()
.activation
.as_ref()
.and_then(|a| a.env.as_ref());

assert_eq!(
default_activation_env,
Some(&IndexMap::from([(
String::from("FOO"),
String::from("main")
)]))
);
assert_eq!(
win64_activation_env,
Some(&IndexMap::from([(
String::from("FOO"),
String::from("win-64")
)]))
);
assert_eq!(
linux64_activation_env,
Some(&IndexMap::from([(
String::from("FOO"),
String::from("linux-64")
)]))
);

// Check that the feature activation env is set correctly
let feature_targets = &manifest
.feature(&FeatureName::Named(String::from("bar")))
.unwrap()
.targets;
let feature_activation_env = feature_targets
.default()
.activation
.as_ref()
.and_then(|a| a.env.as_ref());
let feature_win64_activation_env = feature_targets
.for_target(&TargetSelector::Platform(Platform::Win64))
.unwrap()
.activation
.as_ref()
.and_then(|a| a.env.as_ref());
let feature_linux64_activation_env = feature_targets
.for_target(&TargetSelector::Platform(Platform::Linux64))
.unwrap()
.activation
.as_ref()
.and_then(|a| a.env.as_ref());

assert_eq!(
feature_activation_env,
Some(&IndexMap::from([(
String::from("FOO"),
String::from("bar")
)]))
);
assert_eq!(
feature_win64_activation_env,
Some(&IndexMap::from([(
String::from("FOO"),
String::from("bar-win-64")
)]))
);
assert_eq!(
feature_linux64_activation_env,
Some(&IndexMap::from([(
String::from("FOO"),
String::from("bar-linux-64")
)]))
);
}

#[test]
fn test_target_specific_tasks() {
let contents = format!(
Expand Down