Skip to content
This repository has been archived by the owner on Apr 1, 2019. It is now read-only.
/ schemaker Public archive

Web service to perform JSON and Parquet schema format conversion and validation

License

Notifications You must be signed in to change notification settings

mozilla/schemaker

Repository files navigation

Schemaker

The "schema service" is a collection of tools, including a web service and some glue code that is used to validate incoming telemetry and other data against the specified schema that the data is supposed to honor, and support the efforts of converting data from the ingested format (initially JSON) to the desired storage format (initially Parquet).

CircleCI codecov

Development Setup

This application uses Docker for local development. Please make sure to install Docker and Docker Compose.

To start the application, run:

make up

Run the tests

Run the tests using the following command on your computer:

make test

This will spin up a Docker container to run the tests, so please set up the development setup first.

API Endpoints

POST /convert

Converts the provided schema to the requested output schema.

Query parameters:

  • output (required): The requested output format. Currently the only format supported is 'parquet-mr'.

Provide the input schema as the body of the POST request.

Example output:

{
  "parquet-mr": "<full output of schema>"
}

About

Web service to perform JSON and Parquet schema format conversion and validation

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors 4

  •  
  •  
  •  
  •