Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Common: api_request_enrichment_config schema version 1-0-1 support #670

Merged
merged 1 commit into from
Aug 10, 2022

Conversation

lmath
Copy link
Contributor

@lmath lmath commented Aug 5, 2022

See: #669

check 1 - api request enrichment with schema version 1-0-0 works on a pipeline

this change is in 3.2.4-rc2

  1. I brought up a GCP pipeline and ensured enrichment version was set to 3.2.4-rc2
  2. I set up a minimal api_request_enrichment_config.json which configures an api, one input and one output which is valid for schema version 1-0-0 (has a non-zero value for cache size)
  3. I added a schema to the dev sandbox iglu server to configure the output in api_request_enrichment_config.json
  4. I added a subscription to the good event topic so that I'd be able to pull and look at good events
  5. I sent a test event to the collector of my GCP pipeline
  6. I was able to see the good event coming through, enriched with the output field I had defined in api_request_enrichment_config.json and matching the schema I added to the iglu server.

check 2 - api request enrichment with schema version 1-0-1 works on a pipeline

same as above, except for 2:
I set up a minimal api_request_enrichment_config.json which configures an api, one input and one output which is valid for schema version 1-0-1 (where cache size is 0) and then re-ran terraform apply

@@ -40,7 +40,6 @@ object ApiRequestEnrichment extends ParseableEnrichment {
"api_request_enrichment_config",
"jsonschema",
1,
0,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered initially if the criteria should be valid == 1-0-0 or 1-0-1 rather than valid == 1-0-x

but it seems like ApiRequestEnrichment was the outlier here. See: WeatherEnrichement, YauaaEnrichement, HttpHeaderExtractorEnrichment

@lmath lmath force-pushed the api-enrichment-config-support-schema-1-0-1 branch from 01e9c04 to c54fa80 Compare August 8, 2022 16:26
@lmath lmath marked this pull request as ready for review August 8, 2022 17:14
@peel peel added this to the 3.2.4 milestone Aug 9, 2022
@dilyand dilyand self-requested a review August 10, 2022 10:35
Copy link
Contributor

@dilyand dilyand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loved the added history and reasoning.👍🏻

@lmath lmath merged commit fb0daa9 into develop Aug 10, 2022
@lmath lmath deleted the api-enrichment-config-support-schema-1-0-1 branch August 10, 2022 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants