Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add local dictionary support #61

Merged
merged 34 commits into from
Aug 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
e765522
Add local dictionary support
JakeChampion Aug 8, 2021
005c9d2
memoize reading a local dictionaries json file to make calls to the s…
JakeChampion Aug 9, 2021
5662469
remove no required `use serde_json`
JakeChampion Aug 9, 2021
8c722b6
remove redundant clone of `dictionaries` within `with_dictionaries_by…
JakeChampion Aug 9, 2021
576c25a
Check dictionary definitions are within the allowed limits of Fastly'…
JakeChampion Aug 9, 2021
c83dd0d
Move dictionary validaton logic into dictionaries module
JakeChampion Aug 10, 2021
0ec8eb8
use a PrimaryMap to avoid having to manually track the IDs for dictio…
JakeChampion Aug 10, 2021
107e38e
Read the dictionary files on every Dictionary.get call
JakeChampion Aug 10, 2021
d8f552c
use a pathbuf instead of a string for Dictionary.file field as the fi…
JakeChampion Aug 10, 2021
e9a054e
format the imports
JakeChampion Aug 10, 2021
e70788a
Use the name of the dictionary table as the name for the dictonary
JakeChampion Aug 10, 2021
2b047a5
remove dead code
JakeChampion Aug 10, 2021
d273601
correct the comment for the FastlyConfig.dictionaries function
JakeChampion Aug 11, 2021
e378898
correct the comment for the DictionariesConfig struct
JakeChampion Aug 11, 2021
92faacd
avoid copying `name` by using deref coercion to convert GuestStr into…
JakeChampion Aug 11, 2021
4be64b0
avoid always copying the key in FastlyDictionary.get - instead only c…
JakeChampion Aug 11, 2021
235d9a4
assign a new DictionaryHandle for every given DictionaryName to align…
JakeChampion Aug 11, 2021
901e706
Stop double escaping strings retrieved from the dictionary files
JakeChampion Aug 11, 2021
d28f41a
improve the error message for an invalid dictionary item value by men…
JakeChampion Aug 11, 2021
fef0155
Add error enum for when the dictionary file contents is not a json ob…
JakeChampion Aug 11, 2021
edd7237
Add custom errors for misformatted dictionary file and not being able…
JakeChampion Aug 11, 2021
9350f1e
Extract the dictionary limits into a separate limits file
JakeChampion Aug 15, 2021
276590e
Do not write zero into the dictionary item buffer when the buffer len…
JakeChampion Aug 15, 2021
0fe9aed
refactor setting default values for local backends and dictionaries i…
JakeChampion Aug 16, 2021
915338b
use std::fmt instead of core::fmt
JakeChampion Aug 16, 2021
2ad330c
Remove unused field `name` from Dictionary struct
JakeChampion Aug 16, 2021
51a32ea
refactor how we read the dictionary item key to not use `unwrap` and …
JakeChampion Aug 16, 2021
c4f7549
Move tempfile into dev-dependencies
JakeChampion Aug 16, 2021
ed92569
Correct comment to mention it is a dictionary definition and not a ba…
JakeChampion Aug 16, 2021
9782342
Add format field to local dictionary to figure out what format the di…
JakeChampion Aug 16, 2021
0d641bd
Fix formatting issues via `cargo fmt --all`
JakeChampion Aug 17, 2021
e2373aa
Change from DictionaryFormat::JSON to DictionaryFormat::Json
JakeChampion Aug 17, 2021
ecda15a
update lockfile with new dependencies
JakeChampion Aug 17, 2021
728d924
use single quotes for paths in toml config
JakeChampion Aug 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 5 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ tracing-futures = "0.2.5"
hyper = {version = "0.14.0", features = ["full"]}
hyper-tls = "0.5.0"
wat = "1.0.38"
serde_json = "1.0.66"

[dev-dependencies]
anyhow = "1.0.31"
Expand Down
2 changes: 2 additions & 0 deletions cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,12 @@ pub async fn serve(opts: Opts) -> Result<(), Error> {
if let Some(config_path) = opts.config_path() {
let config = FastlyConfig::from_file(config_path)?;
let backends = config.backends();
let dictionaries = config.dictionaries();
let backend_names = itertools::join(backends.keys(), ", ");

ctx = ctx
.with_backends(backends.clone())
.with_dictionaries(dictionaries.clone())
.with_config_path(config_path.into());

if backend_names.is_empty() {
Expand Down
18 changes: 18 additions & 0 deletions cli/tests/trap-test/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions lib/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ regex = "1.3.9"
semver = "0.10.0"
serde = "1.0.114"
serde_derive = "1.0.114"
serde_json = "1.0.59"
thiserror = "1.0.20"
tokio = {version = "1.2", features = ["full"]}
toml = "0.5.6"
Expand All @@ -41,6 +42,9 @@ wasmtime = "0.29.0"
wasmtime-wasi = {version = "0.29.0", features = ["tokio"]}
wiggle = {version = "0.29.0", features = ["wasmtime_async"]}

[dev-dependencies]
tempfile = "3.2.0"

[features]
default = []
test-fatalerror-config = []
43 changes: 37 additions & 6 deletions lib/src/config.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! Fastly-specific configuration utilities.

use {
self::backends::BackendsConfig,
self::{backends::BackendsConfig, dictionaries::DictionariesConfig},
crate::error::FastlyConfigError,
serde_derive::Deserialize,
std::{collections::HashMap, convert::TryInto, fs, path::Path, str::FromStr, sync::Arc},
Expand All @@ -12,6 +12,15 @@ use {
#[cfg(test)]
mod unit_tests;

/// Fastly limits
mod limits;

/// Types and deserializers for dictionaries configuration settings.
mod dictionaries;
pub use self::dictionaries::Dictionary;
pub use self::dictionaries::DictionaryName;
pub type Dictionaries = HashMap<DictionaryName, Dictionary>;

/// Types and deserializers for backend configuration settings.
mod backends;
pub use self::backends::Backend;
Expand Down Expand Up @@ -55,6 +64,11 @@ impl FastlyConfig {
&self.local_server.backends.0
}

/// Get the dictionaries configuration.
pub fn dictionaries(&self) -> &Dictionaries {
&self.local_server.dictionaries.0
}

/// Parse a `fastly.toml` file into a `FastlyConfig`.
pub fn from_file(path: impl AsRef<Path>) -> Result<Self, FastlyConfigError> {
fs::read_to_string(path.as_ref())
Expand Down Expand Up @@ -130,6 +144,7 @@ impl TryInto<FastlyConfig> for TomlFastlyConfig {
#[derive(Clone, Debug, Default)]
pub struct LocalServerConfig {
backends: BackendsConfig,
dictionaries: DictionariesConfig,
}

/// Internal deserializer used to read the `[testing]` section of a `fastly.toml` file.
Expand All @@ -138,15 +153,31 @@ pub struct LocalServerConfig {
/// a [`LocalServerConfig`] with [`TryInto::try_into`].
#[derive(Deserialize)]
struct RawLocalServerConfig {
backends: Table,
backends: Option<Table>,
dictionaries: Option<Table>,
}

impl TryInto<LocalServerConfig> for RawLocalServerConfig {
type Error = FastlyConfigError;
fn try_into(self) -> Result<LocalServerConfig, Self::Error> {
let Self { backends } = self;
backends
.try_into()
.map(|backends| LocalServerConfig { backends })
let Self {
backends,
dictionaries,
} = self;
let backends = if let Some(backends) = backends {
backends.try_into()?
} else {
BackendsConfig::default()
};
let dictionaries = if let Some(dictionaries) = dictionaries {
dictionaries.try_into()?
} else {
DictionariesConfig::default()
};

Ok(LocalServerConfig {
backends,
dictionaries,
})
}
}