Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scala Common Enrich: port JSON Schema validation from Scala Hadoop Shred #1637

Closed
alexanderdean opened this issue Apr 24, 2015 · 4 comments
Closed

Scala Common Enrich: port JSON Schema validation from Scala Hadoop Shred #1637

alexanderdean opened this issue Apr 24, 2015 · 4 comments
Assignees
@fblundun
Labels
3. Enrich

Comments

@alexanderdean
Copy link
Member

This will validate:

  1. unstructured events
  2. custom contexts

Note that Scala Hadoop Shred will retain its existing validation code too (i.e., we won't assume an enriched event is valid).
@alexanderdean alexanderdean self-assigned this Apr 24, 2015
@alexanderdean alexanderdean added this to the Avro support #1 milestone Apr 24, 2015
@alexanderdean alexanderdean modified the milestones: Release 69 XXX, Avro support #1 Jun 23, 2015
@alexanderdean
Copy link
Member Author

@danisola will work on this.

@alexanderdean alexanderdean mentioned this issue Jun 23, 2015
Closed
@pvdb
Copy link

pvdb commented Jun 24, 2015

:g/will/s/will/might/g

😄

@alexanderdean
Copy link
Member Author

From an architecture perspective, my suggestion is to:

  1. Extract the current JSON Validation logic from Scala Hadoop Shred into Common Enrich
  2. Use Common Enrich's JSON Validation logic in Scala Hadoop Shred, Scala Hadoop Enrich and Scala Kinesis Enrich

@danisola danisola mentioned this issue Aug 6, 2015
Merged
@alexanderdean
Copy link
Member Author

Duplicate of #1947

@alexanderdean alexanderdean removed this from the Release 71 [HAD] Bird TBC milestone Aug 6, 2015
fblundun pushed a commit that referenced this issue Aug 19, 2015
@fblundun
Port JSON Schema validation to Common Enrich from Shred (closes #1637)
ff2f687
fblundun pushed a commit that referenced this issue Aug 26, 2015
@fblundun
Port JSON Schema validation to Common Enrich from Shred (closes #1637)
124a00e
fblundun pushed a commit that referenced this issue Sep 25, 2015
@fblundun
Port JSON Schema validation to Common Enrich from Shred (closes #1637)
15b25d4
fblundun pushed a commit that referenced this issue Sep 25, 2015
@fblundun
Port JSON Schema validation to Common Enrich from Shred (closes #1637)
551e5ff
fblundun pushed a commit that referenced this issue Sep 25, 2015
@fblundun
Port JSON Schema validation to Common Enrich from Shred (closes #1637)
f400e03
alexanderdean pushed a commit that referenced this issue Jan 14, 2016
@alexanderdean
Port JSON Schema validation to Common Enrich from Shred (closes #1637)
457b101
chuwy pushed a commit to snowplow/snowplow-rdb-loader that referenced this issue Sep 5, 2017
@chuwy
Port JSON Schema validation to Common Enrich from Shred (snowplow/sno…
68129bd 
…wplow#1637)
oguzhanunlu pushed a commit to snowplow/common-enrich that referenced this issue May 29, 2020
@fblundun
Port JSON Schema validation to Common Enrich from Shred (closes snowp…
ee1c36c 
…low/snowplow#1637)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fblundun fblundun

Labels
3. Enrich
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pvdb @alexanderdean @fblundun