Conversation
For most components this will simply generate an empty table with boilerplate fields (`type`, `inputs`, etc). Eventually, however, components will have curated examples written for them which are baked into the Vector binary and can be used to generate more useful files. Signed-off-by: Ashley Jeffs <ash@jeffail.uk>
|
Closes #1058 |
|
Docs look great 👍 |
lukesteensen
left a comment
lukesteensen
left a comment
There was a problem hiding this comment.
I left a few comments on ways the implementation could maybe be more idiomatic, but I don't think that needs to block merging. Looks good!
| pub struct SinkOuter { | ||
| pub healthcheck: bool, | ||
| pub inputs: Vec<String>, | ||
| #[serde(flatten)] | ||
| pub inner: Value, | ||
| pub buffer: crate::buffers::BufferConfig, | ||
| } | ||
|
|
||
| #[derive(Serialize)] | ||
| pub struct TransformOuter { | ||
| pub inputs: Vec<String>, | ||
| #[serde(flatten)] | ||
| pub inner: Value, | ||
| } | ||
|
|
||
| #[derive(Serialize, Default)] | ||
| pub struct Config { | ||
| #[serde(flatten)] | ||
| pub global: GlobalOptions, | ||
| pub sources: IndexMap<String, Value>, | ||
| pub transforms: IndexMap<String, TransformOuter>, | ||
| pub sinks: IndexMap<String, SinkOuter>, | ||
| } |
There was a problem hiding this comment.
It's a little funky to have "duplicates" of these structs just for this purpose, but it's probably simpler than trying to unify.
There was a problem hiding this comment.
Yeah it felt a bit off, but it does give us tighter control over what gets generated. I've just updated the sections to be optionals to avoid generating empty sections.
src/generate.rs
Outdated
| let components: Vec<Vec<String>> = opts | ||
| .expression | ||
| .split('|') | ||
| .map(|s| { | ||
| s.to_owned() | ||
| .split(',') | ||
| .map(|s| s.trim().to_owned()) | ||
| .filter(|s| s.len() > 0) | ||
| .collect() | ||
| }) | ||
| .collect(); |
There was a problem hiding this comment.
It doesn't seem like we actually need owned strings here, so you can probably drop all of the to_owned calls. The type annotation can also just be let components: Vec<Vec<_>>.
src/generate.rs
Outdated
| components | ||
| .get(0) | ||
| .unwrap_or(&Vec::new()) | ||
| .iter() | ||
| .for_each(|c| { | ||
| i += 1; |
There was a problem hiding this comment.
This is a little tricky to follow. It might be a little more idiomatic to do something like
if let Some(source_types) = components.get(0) {
for (i, source_type) in source_types.iter().enumerate() {
let name = ...There was a problem hiding this comment.
Thanks! Looks much nicer now.
src/generate.rs
Outdated
| config.sources.insert(name, { | ||
| let mut d = SourceDescription::example(c) | ||
| .map_err(|e| { | ||
| match e { | ||
| ExampleError::MissingExample => {} | ||
| _ => errs.push(e.clone()), | ||
| } | ||
| e | ||
| }) | ||
| .unwrap_or(Value::Table(BTreeMap::new())); | ||
| d.as_table_mut().map(|s| { | ||
| s.insert("type".to_owned(), c.to_owned().into()); | ||
| s | ||
| }); | ||
| d | ||
| }); |
There was a problem hiding this comment.
Lots of nesting going on here too. My shot at a more idiomatic version:
let example = match SourceDescription::example(c) {
Ok(example) => example,
Err(err) => {
if err != ExampleError::MissingExample {
errs.push(e);
}
Value::Table(BTreeMap::new())
}
}
example.as_table_mut().expect("examples are always tables").insert("type".into(), c.into());
config.sources.insert(name, example);There was a problem hiding this comment.
Same as above, thanks!
Signed-off-by: Ashley Jeffs <ash@jeffail.uk>
3 participants
For most components this will simply generate an empty table with
boilerplate fields (
type,inputs, etc). Eventually, however,components will have curated examples written for them which are baked
into the Vector binary and can be used to generate more useful files.
Signed-off-by: Ashley Jeffs ash@jeffail.uk