Skip to content
Branch: master
Find file History
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
alpha
BUILD.bazel
README.md
generate_doc.sh Use JSON schema to define metadata. (#10) Apr 19, 2019
schema_test.go Init type spec for model, metrics and workspace (#26) May 14, 2019

README.md

Metadata Schema

This directory contains all predefined metadata types in the form of JSON schema. We expect every piece of metadata to have the following fields:

  • idof type string. Unique identifier assigned by this metadata service.
  • name of type string. Name of the metadata assigned by external users.
  • type information. We need type information to make the metadata self-explanatory and require explicit version of the type, following the Kubernetes convention. The following three fields together uniquely identify a version of a type.
    • kind of type string. Name of the type.
    • namespace of type string. The namespace of the type to avoid naming collision.
    • apiversion of type string. The version of the type.
  • category of type string. We categorize metadata based on its role in Kubeflow systems:
    • "artifact" represents input data and derived data in a workflow.E.g. data set, model.
    • "execution" represents a run of an excutable, which can have artifacts as input and/or output.
    • "container" represents a group of artifacts, executions, and other containers. E.g. workspace for solving a ML problem and Katib experiment for creating multiple models.

It is not necessary, but the easiest way to comply with these requirements is to extend the alpha/entity.json schema.

Predefined Metadata

This directory contains versions of predefined metadata schemas, which are loaded by the metadata service before it starts. Therefore metadata of these types can be directly logged to the metadata store.

Folder Structure

  • Different versions of metadata schema should be organized as <version>/<relative path>.
  • Markdown documentaion of schemas are at <version>/docs.
  • schema_test.go validates all schemas in sub-directories and examples in /examples.

Customized Metadata

Customized metadata is defined in the same schema format as predefined metadata. The only difference between them is that customized metadata schemas are loaded by sending requests to the schema registration endpoint. (TODO: add link)

You can’t perform that action at this time.