Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Tabular and Errors to Structure #47

Closed
3 of 7 tasks
b5 opened this issue Jan 19, 2018 · 1 comment · Fixed by #58
Closed
3 of 7 tasks

Add a Tabular and Errors to Structure #47

b5 opened this issue Jan 19, 2018 · 1 comment · Fixed by #58
Assignees
Milestone

Comments

@b5
Copy link
Member

b5 commented Jan 19, 2018

Two new fields on Structure before we ship 0.1.0:

type Structure struct {
  // ... current fields omitted

  // Tabular indicates if this data is intended to be interpreted in a row/column format
 Tabular bool
  // The number of errors determined by validating this data against Schema
  Errors int
}

Note that Errors should always be encoded to json (as in, no omitempty), even if the number of errors is 0 (because 0 errors is good!).

Tabular is to indicate "this data is intended to be consumed as a table", which might apply to high-dimensional data, but almost always applies to certain formats like csv.

Steps to complete

@b5 b5 added this to the 0.1.0 milestone Jan 19, 2018
@b5
Copy link
Member Author

b5 commented Jan 24, 2018

Ok quick updates on this, after some thought, I think we can drop the Tabular field. I initially thought we'd need to rely on the tabular flag to manage the transition from tabular schemas to jsonschema, but we've found a way around that. If anything, Tabular may end up being a field of VizConfig. So two updates to this issue:

  1. No more tabular field
  2. Let's change the name of the new Errors field to ErrCount, which seems more accurate

b5 added a commit that referenced this issue Jan 24, 2018
dataset.Structure now has a field to store the number of validation
errors in a given dataset. dsfs.CreateDataset now fills this out
at time of dataset creation automatically.

closes #47
@ghost ghost assigned b5 Jan 24, 2018
@ghost ghost added the in progress label Jan 24, 2018
@b5 b5 closed this as completed in #58 Jan 24, 2018
@ghost ghost removed the in progress label Jan 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant