Make environment variables available inside local server

[GeeWee]

In our test suite I have a few places I'd like to e.g. toggle some authentication logic based on environment variables.

This currently seems impossible, as Viceroy only exposes the environment variables it has predefined such as FASTLY_HOSTNAME, but it is not possible to set your own env variables and have them be accessible inside fastly compute serve

[cratelyn]

hello! so, i am a bit hesitant to add this because the environment variables in a Compute@Edge program are predefined in the production environment. these variables are specified here, in case that's helpful: https://developer.fastly.com/reference/compute/ecp-env/

for situations like this i might suggest using an Edge Dictionary, which can contain this sort of runtime configuration. this isn't especially well documented at the moment, but the fastly.toml reference here outlines the dictionary section. #11 introduced that support, which may also be a helpful guide.

let me know if this answers your question @GeeWee!

[GeeWee]

Yeah I absolutely understand why you're a bit hesitant as you wouldn't be able to read the variables inside the production environment.

We actually do use dictionaries reasonably heavily for something similar. Let me perhaps try to outline the situation we're in a little bit.

• We would like to be able to run integration tests using Viceroy, where each test has it's own database key. In a perfect world we'd like these tests to run simultaneously.
• In production scenarios we use an edge dictionary to look up the database key.
• In integration test scenarios we have a local dictionary with a predefined key.

However, I'd like to be able to swap out the contents of my database key in a, preferably easy way, that means I can run parallel tests.
The way I see it there's a few options

• Environment variables. I'd then use those, and if they don't exist, I'd look in the dictionary for the key. That's what I'm proposing here.
• A way to configure or override dictionaries with env variables. This would solve our problem as well, and might be more appetitizing to you as well? As you're not introducing env vars that won't exist in production.
• Using different fastly.toml environments such as fastly.staging.toml, but if I want to run e.g. 10 or 20 tests simultaneously, that's quite a lot of files I'd have to write in my filesystem

What are your thoughts?

[cratelyn]

after a bit more thought, going with environment variables feels like it won't be the workable answer to this. keeping the environment in which a Wasm program runs in parity with production is an important goal for Viceroy, so i'd like for us to look elsewhere.

the fastly.toml environments feel like the best fit for this situation, but i hear you that this would involve a lot of filesystem interaction to run tests. using viceroy_lib directly in your tests may be a better route that i might propose, if possible.

to that effect, @GeeWee, i opened up #148 to hopefully accommodate this. this should let you pass a string directly into a FastlyConfig constructor, without needing to write files to disk. does that possibly help with your problems?

and to help you on your journeys, this should look vaguely like this (note that today this still involves writing a config to disk, until the branch above lands.)

use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::Arc;
use viceroy_lib::config::Backend;

fn example_invocation() {
    // Load the wasm module into an execution context
    let mut ctx = viceroy_lib::ExecuteCtx::new("./program.wasm")
        .unwrap()
        .with_log_stderr(true)
        .with_log_stdout(true);

    let backend = Backend {
        uri: "http://127.0.0.1:8080".parse().unwrap(),
        override_host: None,
    };
    let mut backends = HashMap::new();
    backends.insert("backend1".to_owned(), Arc::new(backend));

    ctx = ctx
        .with_backends(backends.clone())
        .with_config_path(PathBuf::from("./fastly.toml"));

    tokio::spawn(async move {
        viceroy_lib::ViceroyService::new(ctx)
            .serve("127.0.0.1:8081".parse().unwrap())
            .await
            .unwrap()
    });

    let response = reqwest::get("http://127.0.0.1:8081").await.unwrap();
}

edit: clippy lints have helpfully reminded me that the config structure implements std::str::FromStr. we can do this without touching the filesystem!

[GeeWee]

Totally understand your reasoning.
FromStr, might actually work! I'm unsure on one point though - will something like this also allow me to create the edge dictionaries from a string, or will I still have to write those to disk?
That would be the way that I would end up changing the env variables to allow for multiple tests.

[cratelyn]

will something like this also allow me to create the edge dictionaries from a string, or will I still have to write those to disk? That would be the way that I would end up changing the env variables to allow for multiple tests.

thank you for pointing that out, i'd forgotten about the need to place .json files on disk for dictionaries. so! 🥁 (drum roll) i have put together #150, which allows us to provide dictionaries directly inside of fastly.toml.

an example of the configuration is included in the PR description over there, but i wanted to ask if this would solve your problems @GeeWee? my hope is that this should give you a more convenient avenue to define distinct values in each of your test iterations 🙂

[GeeWee]

Yes! #150 and the already existing FromStr should definitely solve these issues and allow for a much nicer testing experience as far as I can tell! Thank you!