Skip to content

Latest commit

 

History

History
1037 lines (858 loc) · 51.3 KB

README.md

File metadata and controls

1037 lines (858 loc) · 51.3 KB
Raw

BIDS schema description

Portions of the BIDS specification are defined using YAML files in order to make the specification machine-readable.

Currently the portions of the specification that rely on this schema are:

  • the entity tables,
  • entity definitions,
  • filename templates,
  • metadata tables.

Any changes to the specification should be mirrored in the schema.

Organization and syntax

At the time of this writing, the schema has the following file layout:

├── meta
│   ├── ...
│   └── versions.yaml
├── objects
│   ├── ...
│   └── suffixes.yaml
├── rules
│   ├── checks
│   │   ├── ...
│   │   └── references.yaml
│   ├── files
│   │   ├── common
│   │   │   ├── core.yaml
│   │   │   └── tables.yaml
│   │   ├── deriv
│   │   │   ├── imaging.yaml
│   │   │   └── preprocessed_data.yaml
│   │   └── raw
│   │       ├── ...
│   │       └── task.yaml
│   ├── sidecars
│   │   ├── derivatives
│   │   │   └── common_derivatives.yaml
│   │   ├── ...
│   │   └── pet.yaml
│   ├── tabular_data
│   │   ├── derivatives
│   │   │   └── common_derivatives.yaml
│   │   ├── ...
│   │   └── task.yaml
│   ├── ...
│   └── modalities.yaml
├── BIDS_VERSION
└── SCHEMA_VERSION

The top-level organization includes objects, where terms are defined; rules, where constraints (such as valid filenames or required metadata fields) are defined; and meta, where definitions useful for interpreting the schema are defined.

Each file is made up of YAML data, most often an object. For example, the file rules/checks/mri.yaml contains the contents:

PhasePartUnits:
  issue:
    code: PHASE_UNITS
    message: |
      Phase images (with the `part-phase` entity) must have units
      "rad" or "arbitrary".
    level: error
  selectors:
    - modality == "mri"
    - entities.part == "phase"
    - '"Units" in sidecar'
  checks:
    - intersects([sidecar.Units], ["rad", "arbitrary"])

When we wish to refer to a file we might write rules/checks/mri.yaml. Alternately, we can use rules.checks.mri to refer to the object contained by the file. Using this notation, the qualified name, the contents of an entire directory or a portion of a file can be referred to unambiguously. For example, the entire rules/checks/ directory is referred to as rules.checks, and rules.checks.mri.PhasePartUnits.issue refers to the object:

{
  "code": "PHASE_UNITS",
  "message": "Phase images (with the `part-phase` [...]\n\"rad\" or \"arbitrary\".\n",
  "level": "error"
}

These qualified names may be used in this README, as well as in references and expressions.

Description formatting

Many objects throughout the schema have a description field, which will typically be rendered somewhere in the specification. Because the specification is written in Markdown, these description fields may also contain Markdown, including links to other locations in the specification.

Because the same description may be used in multiple locations, a mechanism is needed to ensure that the correct path is discovered to render the description in each location. To do this, the path should follow the form SPEC_ROOT/path/within/source.md#anchor. For example, to link to the Definitions section of Common principles, use the path SPEC_ROOT/common-principles.md#definitions:

[Common principles - Definitions](SPEC_ROOT/common-principles.md#definitions)

Note that the Markdown extension .md MUST be used for this to render correctly.

For more information please see the following pull request and linked discussions: #1096

References

Some schema entries take the form:

ObjectName:
  $ref: objects.metadata.OtherObjectName

This object may be dereferenced by replacing the $ref entry with the object being referenced. The following two prototypical examples are presented to clarify the semantics of references (the cases in which they are used will be presented later):

  1. In objects.enums:

    _GeneticLevelEnum:
  type: string
  enum:
    - $ref: objects.enums.Genetic.value
    - $ref: objects.enums.Genomic.value
    - $ref: objects.enums.Epigenomic.value
    - $ref: objects.enums.Transcriptomic.value
    - $ref: objects.enums.Metabolomic.value
    - $ref: objects.enums.Proteomic.value

    and in objects.metadata:

    GeneticLevel:
  name: GeneticLevel
  display_name: Genetic Level
  description: |
    Describes the level of analysis.
    Values MUST be one of `"Genetic"`, `"Genomic"`, `"Epigenomic"`,
    `"Transcriptomic"`, `"Metabolomic"`, or `"Proteomic"`.
  anyOf:
    - $ref: objects.enums._GeneticLevelEnum
    - type: array
      items:
        $ref: objects.enums._GeneticLevelEnum

    Here _GeneticLevelEnum is used to describe the valid values of GeneticLevel, (which are in turn references to individual values), and the references inside GeneticLevel.anyOf indicate that there may be a single such value or a list of values.

  2. In rules.files.deriv.preprocessed_data:

    anat_nonparametric_common:
  $ref: rules.files.raw.anat.nonparametric
  entities:
    $ref: rules.files.raw.anat.nonparametric.entities
    space: optional
    description: optional

    Here, the derivative datatype rule starts by copying the raw datatype rule rules.files.raw.anat.nonparametric. It then overrides the entities portion of that rule with a new object. To extend the original entities, it again begins by referencing rules.files.raw.anat.nonparametric.entities, and adding the new entities space and description.

Expressions

Rules definitions make use of a limited language of expressions that always evaluate to true or false. These expressions may be used as selectors, determining whether a rule applies, or checks, determining whether a rule is satisfied.

Re-examining rules.checks.mri.PhasePartUnits from above:

PhasePartUnits:
  issue:
    code: PHASE_UNITS
    message: |
      Phase images (with the `part-phase` entity) must have units
      "rad" or "arbitrary".
    level: error
  selectors:
    - modality == "mri"
    - entities.part == "phase"
    - '"Units" in sidecar'
  checks:
    - intersects([sidecar.Units], ["rad", "arbitrary"])

We see expressions may contain:

  • fields such as modality, entities (which has a .part subfield), sidecar
  • String literals such as "mri", "Units" or "rad"
  • Lists containing fields or strings
  • Comparison operators such as == (equality) or in (subfield exists in field)
  • Functions such as intersects()

In fact, the full list of fields is defined in the meta.context.context object, which (currently) contains at the top level:

  • schema: access to the schema itself
  • dataset: attributes of the whole dataset
  • subject: attributes of the current subject
  • path: the full path of the current file (relative to dataset root)
  • entities: an object of entities parsed from the path
  • datatype: the datatype, parsed from the path
  • suffix: the suffix, parsed from the path
  • extension: the file extension
  • modality: the file modality, determined by datatype
  • sidecar: the metadata values, accumulated by the inheritance principle
  • associations: associated files, discovered by the inheritance principle
  • columns: the columns in the current TSV file
  • json: the contents of the current JSON file
  • gzip: the contents of the current file GZIP header
  • nifti_header: selected contents of the current NIfTI file's header
  • ome: the contents of the current OME-XML metadata
  • tiff: the contents of the current TIFF file's header

Some of these are strings, while others are nested objects. These are to be populated by an interpreter of the schema, and provide the namespace in which expressions are evaluated.

The following operators should be defined by an interpreter:

Operator Definition Example
== equality suffix == "T1w"
!= inequality entities.task != "rest"
</> less-than / greater-than sidecar.EchoTime < 0.5
<=/>= less-than-or-equal / greater-than-or-equal 0 <= 4
in object lookup, true if RHS is a subfield of LHS "Units" in sidecar
! negation, true if the following value is false, or vice versa !true == false
&& conjunction, true if both RHS and LHS are true "Units" in sidecar && sidecar.Units == "mm"
|| disjunction, true if either RHS or LHS is true a < mn || a > mx
. object query, returns value of subfield sidecar.Units
[] array/string index, returns value of Nth element (0-indexed) columns.participant_label[0]
+ numeric addition / string concatenation x + 1, stem + "suffix"
-/*// numeric operators (division coerces to float) length(array) - 2, x * 2, 1 / 2 == 0.5

The following functions should be defined by an interpreter:

Function Definition Example Note
count(arg: array, val: any) Number of elements in an array equal to val count(columns.type, "EEG") The number of times "EEG" appears in the column "type" of the current TSV file
exists(arg: str | array, rule: str) -> int Count of files in an array that exist in the dataset. String is array with length 1. Rules include "bids-uri", "dataset", "subject" and "stimuli". exists(sidecar.IntendedFor, "subject") True if all files in IntendedFor exist, relative to the subject directory.
index(arg: array, val: any) Index of first element in an array equal to val, null if not found index(["i", "j", "k"], axis) The number, from 0-2 corresponding to the string axis
intersects(a: array, b: array) -> bool true if arguments contain any shared elements intersects(dataset.modalities, ["pet", "mri"]) True if either PET or MRI data is found in dataset
allequal(a: array, b: array) -> bool true if arrays have the same length and paired elements are equal intersects(dataset.modalities, ["pet", "mri"]) True if either PET or MRI data is found in dataset
length(arg: array) -> int Number of elements in an array length(columns.onset) > 0 True if there is at least one value in the onset column
match(arg: str, pattern: str) -> bool true if arg matches the regular expression pattern (anywhere in string) match(extension, ".gz$") True if the file extension ends with .gz
max(arg: array) -> number The largest non-n/a value in an array max(columns.onset) The time of the last onset in an events.tsv file
min(arg: array) -> number The smallest non-n/a value in an array min(sidecar.SliceTiming) == 0 A check that the onset of the first slice is 0s
sorted(arg: array) -> array The sorted values of the input array sorted(sidecar.VolumeTiming) == sidecar.VolumeTiming True if sidecar.VolumeTiming is sorted
substr(arg: str, start: int, end: int) -> str The portion of the input string spanning from start position to end position substr(path, 0, length(path) - 3) path with the last three characters dropped
type(arg: Any) -> str The name of the type, including "array", "object", "null" type(datatypes) Returns "array"

The special value null

Missing values in the context object have the special value null. This value propagates through all of the above operations in a fully-defined, hopefully intuitive way. Most operations involving null simply resolve to null:

Operation Result
sidecar.MissingValue null
null.anything null
null[0] null
null && true null
null || true null
!null null
null + 1 null
null - 1 null
null * 1 null
null / 1 null
match(null, pattern) null
intersects(list, null) null
intersects(null, list) null
allequal(list, null) null
allequal(null, list) null
substr(null, 0, 1) null
substr(str, null, 1) null
substr(str, 0, null) null
length(null) null
count(null, val) null
count(list, null) null
index(null, val) null
index([0], null) null
index([], val) null
min(null) null
max(null) null
exists(null, "bids-uri") null
exists("/path", null) null

The following operators have boolean results:

Operation Result Comment
null == false false
null == true false
null != false true
null != true true
null == null true
null == 1 false Also <, >, <= and >=
"VolumeTiming" in null false

The type() function returns a string:

Operation Result
type(null) "null"

Finally, if an expression (selector or check) evaluates to null, the null will be interpreted equivalent to false. That is, a null selector will not apply the current rule, and a null check will fail.

Object files

Object files define "objects" or "terms", which are semantic descriptions of concepts used in BIDS. These reside under the object.* namespace in the schema. These files do not describe how objects of different types (for example file suffixes and file entities) interact with one another, or whether objects are required in a given dataset or file.

Overview

There are currently 12 sub-namespaces, which fall into five rough categories.

The namespaces are:

Namespace Description Group
objects.common_principles Terms that are used throughout BIDS General terms
objects.modalities Broad categories of data represented in BIDS, roughly matching recording instrument General terms
objects.entities Name-value pairs appearing in filenames Name/value terms
objects.metadata Name-value pairs appearing in JSON files Name/value terms
objects.columns Column headings and values appearing in TSV files Name/value terms
objects.datatypes Subdirectories that organize files by type (such as anat, eeg) Value terms
objects.suffixes Filename suffixes that describe the contents of the file Value terms
objects.extensions Filename component that describe the format of the file Value terms
objects.formats Terms that define the forms values (for example, in metadata) might take Formats
objects.files Files and directories that may appear at the root of a dataset Files
objects.enums Full descriptions of enumerated values used in other sub-namespaces Value terms

Because these objects vary, the contents of each namespace can vary.

Common fields to all objects:

Field Description
description A description of the term that can be understood that should not depend on particular surrounding text; may contain markdown for rendering
display_name A human-friendly name, for tools to display; may include spaces

The name/value terms groups (entities, metadata and columns) define terms where a name, when present, has a given meaning, and its value may be restricted.

These objects additionally have the field:

Field Description
name For terms that can take on multiple values (such as entities, metadata fields), the name of the term as it appears in the specification and in a dataset; must be alphanumeric; mutually exclusive with value
type The type (such as string, integer, object) of values the term describes
format The format of the term (defined in objects.formats)

Value terms groups (datatypes, suffixes, extensions) define terms where a field can take on multiple values. For example, a file has one datatype, as compared to a collection of entities.

These objects may have the fields:

Field Description
value For terms that cannot take on multiple values (for example suffixes or extensions), the string value of the term

The formats terms provide one additional field:

Field Description
pattern Regular expression validating a string rendering of a value

Value constraints

For name/value terms, the type and format fields allow constraints to be placed on the values described by the names.

Additional fields may apply to further constrain the type:

Field Description
maximum/minimum/exclusiveMinimum Value ranges for integer and number types
maxValue/minValue Value ranges for integer and number types
maxItems/minItems Size ranges for array types
enum List of accepted values for string types

Some values may be more flexible, allowing multiple possible values, or may be arrays or objects:

Field Description
anyOf A list of constraints, any of which could apply
items The array described contains values whose types are constrained
properties The object described has a given set of fields; the values of these fields may be constrained
additionalProperties The object described has constraints on its values, but not the names

On re-used objects with different definitions

In a few cases, two objects with the same name appear multiple times in the specification. When this happens, it is preferred to find a common definition, and clarify it in the rules (see below). However, in some cases, the object description and permissible values differ, and it needs to be defined as two separate objects.

Consider the following examples:

# reference column for channels.tsv files for EEG data
reference__eeg:
  name: reference
  display_name: Electrode reference
  description: |
    Name of the reference electrode(s).
    This column is not needed when it is common to all channels.
    In that case the reference electrode(s) can be specified in `*_eeg.json` as `EEGReference`).
  type: string
# reference column for channels.tsv files for iEEG data
reference__ieeg:
  name: reference
  display_name: Electrode reference
  description: |
    Specification of the reference (for example, `mastoid`, `ElectrodeName01`, `intracranial`, `CAR`, `other`, `n/a`).
    If the channel is not an electrode channel (for example, a microphone channel) use `n/a`.
  anyOf:
    - type: string
    - type: string
      enum:
        - n/a

Here, the TSV column "reference" means different things when used for EEG data, as compared to iEEG data, so two definitions are needed. Because columns use snake_case (meaning they can be expected to contain underscores), two underscores are needed to separate the column name from the string that indicates the use of the term.

The convention can be summed up in the following rules:

  1. Each specific term takes on the form <term>_<context>, where <term> is the common name that the two (or more) terms share, <context> indicates when the specific term applies.

  2. If the <term> appears in snake_case then <context> begins with an extra _.

Valid fields for definitions by sub-namespace

  • objects.common_principles

    Field Description
    display_name Human-friendly name
    description Term definition

  • objects.modalities

    Field Description
    display_name Human-friendly name
    description Term definition

  • objects.entities

    Field Description
    display_name Human-friendly name
    description Term definition
    name Key of entity, such as sub or ses
    type Type of value (always string)
    format Permissible format of values, either label or index
    enum Exclusive list of valid values, if present

    Note that descriptions should apply to all uses of the entity; if additional information applies in certain contexts, that should be written in the specification, and not the schema.

  • objects.metadata

    Field Description
    display_name Human-friendly name
    description Term definition
    name Name of field in JSON object (in CamelCase)
    unit Interpretation of numeric values
    type Type of value (one of array, string, integer, number, object or boolean)
    format Permissible format of values, from definitions in objects.formats
    enum Exclusive list of valid values, if present
    maximum Maximum for numeric values
    minimum Minimum for numeric values
    * JSON-schema fields to further constrain values

  • objects.columns

    Field Description
    display_name Human-friendly name
    description Term definition
    name Name of column in TSV file (in snake_case)
    unit Interpretation of numeric values
    type Type of value
    format Permissible format of values, from definitions in objects.formats
    pattern Regular expression constraining string values
    enum Exclusive list of valid values, if present
    maximum Maximum for numeric values
    minimum Minimum for numeric values
    * JSON-schema fields to further constrain values

  • objects.datatypes

    Field Description
    display_name Human-friendly name
    description Term definition
    value String value of datatype

  • objects.suffixes

    Field Description
    display_name Human-friendly name
    description Term definition
    value String value of suffix
    unit Interpretation of values in a data file with the given suffix
    maxValue Maximum permissible value in a data file with the given suffix
    minValue Minimum permissible value in a data file with the given suffix
    anyOf Used to describe multiple permissible units

  • objects.extensions

    Field Description
    display_name Human-friendly name
    description Term definition
    value String value of extension

  • objects.formats

    Field Description
    display_name Human-friendly name
    description Term definition
    pattern Regular expression defining format

  • objects.files

    Field Description
    display_name Human-friendly name
    description Term definition
    file_type Indicator that the file is a regular file ("regular") or directory ("directory")

  • objects.enums

    Field Description
    display_name Human-friendly name
    description Term definition
    value String value of enum

Rule files

The rules.* namespace contains most of the validatable content of the schema, apart from value constraints that can be encoded in objects.

There are several types of rule, and this section is subject to reconsolidation as patterns are found.

Core concepts

Core concepts are expressions (defined above), requirement levels and issues.

Requirement levels and severity

BIDS follows RFC 2119 and has three requirement levels: OPTIONAL, RECOMMENDED and REQUIRED. In the schema, we use optional, recommended and required.

A rule interpreter (validator) is expected to treat:

  • missing REQUIRED data/metadata as an error,
  • missing RECOMMENDED data/metadata as a warning,
  • and silently pass over missing OPTIONAL data.

BIDS also defines a level DEPRECATED, rendered in the schema as deprecated, and corresponding to a warning if the data/metadata is present.

Issues

Issues are messages intended to be communicated to a dataset curator to indicate an issue with their dataset.

They have a code and severity as well:

Field Description
code Issue identifier, such as EVENTS_TSV_MISSING
level Issue severity (warning or error)
message Message for display to a user

A level of warning corresponds to a rule in the specification that is RECOMMENDED, while a level of error corresponds to a rule that is REQUIRED.

In some cases, an issue is contained next to a level: required or level: recommended as part of a larger rule. In these cases, the level field should be omitted from the issue to avoid duplication or conflict.

Filename construction rules

A significant portion of BIDS is devoted to the naming of files, and almost all filenames consist of entities, a suffix, an extension, and a data type. Exceptions will be noted below.

rules.files contains the following subdivisions.

Namespace Description
rules.files.common.core Files and directories that reside at the top level of datasets
rules.files.common.tables Tabular metadata files that associate metadata with entities
rules.files.raw.* Raw data and metadata files that have entities, suffixes, datatypes and extensions
rules.files.deriv.* Derivative data and metadata files that have entities, suffixes, datatypes and extensions

Core files and directories

rules.files.common.core describes files that have little-to-no variability in their form. These either have a single path field, or a stem field and a list of extensions:

Field Description
level Requirement level of file, one of (optional, recommended, required, deprecated)
path Location of file, relative to dataset root; mutually exclusive with stem and extensions
stem Name of file, relative to dataset root, up to but not including the extension; mutually exclusive with path
extensions List of valid extension strings, including the initial dot (.); mutually exclusive with path

These are the entries for dataset_description.json and README:

dataset_description:
  level: required
  path: dataset_description.json
README:
  level: required
  stem: README
  extensions:
    - ''
    - .md
    - .rst
    - .txt

Here, README and README.md are both valid, while only dataset_description.json is permitted.

Tabular metadata files

rules.files.common.tables describes TSV files and their associated metadata, including participants.tsv, samples.tsv, *_sessions.tsv and *_scans.tsv. The first two use the stem field, while the latter two specify the entities used to construct the filename.

The valid fields are:

Field Description
level Requirement level of file, one of (optional, recommended, required, deprecated)
stem Name of file, relative to dataset root, up to but not including the extension; mutually exclusive with entities
entities Object where the keys are entries in objects.entities. The value is a requirement level.
extensions List of valid extension strings, including the initial dot (.)

For example:

participants:
  level: optional
  stem: participants
  extensions:
    - .tsv
    - .json
sessions:
  suffixes:
    - sessions
  extensions:
    - .tsv
    - .json
  entities:
    subject: required

Note that these files do not have a datatype, but otherwise follow the same rules as above.

BIDS filenames

rules.files.raw and rules.files.deriv contain series of related rules. These are largely grouped by datatype, but file types that appear in multiple locations may be grouped together.

The files described take the form:

[sub-<label>/][ses-<label>/]<datatype>/<entities>_<suffix><extension>

Rules have the following fields:

Field Description
suffixes List of suffixes found in objects.suffixes
extensions List of valid extension strings, including initial dot (.)
datatypes List of datatypes found in objects.datatypes
entities Object where the keys are entries in objects.entities. The value is either a requirement level or an object described by the following table.
Field Requirement level Description
level REQUIRED Requirement level of field, one of (optional, recommended, required, deprecated)
format OPTIONAL Override of entity field - Permissible format of values, either label or index
enum OPTIONAL Override of entity field - Exclusive list of valid values, if present

As an example, let us look at a (modified) part of meg.yaml:

meg:
  suffixes:
    - meg
  extensions:
    - .fif
  datatypes:
    - meg
  entities:
    subject: required
    session: optional
    task: required
    acquisition: optional
    run: optional
    processing: optional
    split: optional

crosstalk:
  suffixes:
    - meg
  extensions:
    - .fif
  datatypes:
    - meg
  entities:
    subject: required
    session: optional
    acquisition:
      level: required
      enum:
        - crosstalk

In this case, the first group has one suffix: meg. The second group has the same suffix (meg), but describes different rules for files with that suffix. While the valid extension is the same for both groups (.fif), the entities are not.

Specifically, files in the first group may have task, run, processing, and split entities, while files in the second group may not. Also, when files in the second group have the acq entity, the associated value MUST be crosstalk.

A common derivatives type is preprocessed data, where the type of the generated data is the same as the input data. BIDS Derivatives specifies that these files may be distinguished from raw data with the new entities space-<label> or desc-<label>.

This rule is encoded:

meg_meg_common:
  $ref: rules.files.raw.meg.meg
  entities:
    $ref: rules.files.raw.meg.meg.entities
    space: optional
    description: optional

When expanded, this becomes:

meg_meg_common:
  suffixes:
    - meg
  extensions:
    - .fif
  datatypes:
    - meg
  entities:
    subject: required
    session: optional
    task: required
    acquisition: optional
    run: optional
    processing: optional
    split: optional
    space: optional
    description: optional

Sidecar and tabular data rules

Tabular data and JSON sidecar files follow a similar pattern:

Name Value
JSON field value
TSV column header column values

In the specification, groups of fields/columns are described together in a table that includes the name of the field/column, the requirement level and a description. The definitions, including name and description, appear in objects.metadata, and the columns appear in objects.columns.

Here, we define YAML "tables" that can be rendered in the specification. These take the form:

RuleName:
  selectors:
    - expression1
    - expression2
  fields:
    - FieldName1:
        level: recommended
        level_addendum: required if XYZ
        description_addendum: Additional text following object description.
    - FieldName2: optional

RuleNameReq:
  selectors:
    - expression1
    - expression2
    - expression3
  fields:
    - FieldName1:
        level: required
        issue:
          code: ISSUE_NAME
          message: A description of the problem for a user

Here we show an example of two fields, one that is RECOMMENDED in most cases but REQUIRED in another, the other of which is OPTIONAL.

selectors indicate whether the current rule applies to a given file. This is not rendered in the text, but may be used by a validator. fields is an object with keys that appear in objects.metadata/objects.columns. If the value is a string, then it is a requirement level. If it is an object, then the it has the following fields

Field Requirement level Description
level REQUIRED Requirement level of field, one of (optional, recommended, required, deprecated)
level_addendum OPTIONAL Additional text to describe cases where requirement level changes
description_addendum OPTIONAL Additional text to follow the objects.metadata.<fieldname>.description
issue OPTIONAL issue object, if additional communication is warranted

The second table implements the change in the first table's level_addendum. The expression3 selector indicates the additional case where the more stringent rule is applied.

Valid fields for definitions

  1. rules.sidecars.*

    Field Description
    selectors List of expressions; any evaluating false indicate rule does not apply
    fields Object with keys that may be found in objects.metadata, values either a requirement level or an object

  2. rules.tabular_data.*

    Field Description
    selectors List of expressions; any evaluating false indicate rule does not apply
    columns Object with keys that may be found in objects.columns, values either a requirement level or an object
    initial_columns An optional list of columns that must be the first N columns of a file
    index_columns An optional list of columns that uniquely identify a row.
    additional_columns Indicates whether additional columns may be defined. One of allowed, allowed_if_defined and not_allowed.

The following tables demonstrate how mutual exclusive, required fields, may be set in rules.sidecars.*:

MRIFuncRepetitionTime:
  selectors:
    - modality == "mri"
    - datatype == "func"
    - '!("VolumeTiming" in sidecar)'
    - match(extension, "^\.nii(\.gz)?$")
  fields:
    RepetitionTime:
      level: required
      level_addendum: mutually exclusive with `VolumeTiming`

MRIFuncVolumeTiming:
  selectors:
    - modality == "mri"
    - datatype == "func"
    - '!("RepetitionTime" in sidecar)'
    - match(extension, "^\.nii(\.gz)?$")
  fields:
    VolumeTiming:
      level: required
      level_addendum: mutually exclusive with `RepetitionTime`

An additional check will be required to assert that both are not present, but these tables may be combined for rendering purposes.

Here we present an example rule in rules.tabular_data.eeg:

EEGChannels:
  selectors:
  - datatype == "eeg"
  - suffix == "channels"
  - extension == ".tsv"
  initial_columns:
  - name__channels
  - type__channels
  - units
  columns:
    name__channels: required
    type__channels: required
    units: required
    description: optional
    sampling_frequency: optional
    reference: optional
    low_cutoff: optional
    high_cutoff: optional
    notch: optional
    status: optional
    status_descriptions: optional
  additional_columns: allowed_if_defined

Checks

rules.checks can contain more complex rules. Structurally, these are similar to sidecar rules, in that they have selectors. They additionally have a checks list, and an explicit issue.

Field Description
issue Issue code object (see Issues
selectors List of expressions; any evaluating false indicate rule does not apply
checks List of expressions; any evaluating false indicate rule is violated and issue should be raised 
EventsMissing:
  issue:
    code: EVENTS_TSV_MISSING
    message: |
      Task scans should have a corresponding events.tsv file.
      If this is a resting state scan you can ignore this warning or rename the task to include the word "rest".
    level: warning # could be an error with the proper selectors, I think
  selectors:
    - '"task" in entities'
    - '!match(entities.task, "rest")'
    - suffix != "events"
  checks:
    - '"events" in associations'

Ordering rules

  • rules.entities - This file contains a list of keys into objects.entities and simply defines the order in which entities, when present, MUST appear in filenames

  • rules.common_principles - This file contains a list of terms that appear in objects.common_principles that determines the order they appear in the specification

One-off rules

  • rules.modalities - The keys in this file are the modalities, the values objects with the following field:

    Field Description
    datatypes List of datatypes mapping to modality

  • rules.dataset_metadata - These are similar to rules.sidecars.*, for JSON files at the root level. This is likely to go away in favor of other approaches.

  • rules.errors - This file describes errors that cannot be expressed in the schema. This provides common codes and language that implementing validators can use to ensure the same problems are reported to users in the same way.

Version of the schema

File SCHEMA_VERSION in the top of the directory contains a semantic version (MAJOR.MINOR.PATCH) for the schema (how it is organized). Note that while in 0. series, breaking changes are permitted without changing the MAJOR (leading) component of the version. Going forward, the 2nd, MINOR indicator should be incremented whenever schema organization introduces "breaking changes": changes which would cause existing tools reading schema to adjust their code to be able to read it again. Additions of new components to the schema should increment the last, PATCH, component of the version so that tools could selectively enable/disable loading specific components of the schema. With the release of 1.0.0 version of the schema, we expect that the MAJOR component will be incremented whenever schema organization introduces "breaking changes", MINOR - when adding new components to the schema, and PATCH - when fixing errors in existing components.

Schema publication

The BIDS Schema is compiled into a single, dereferenced object during the ReadTheDocs build of the specification. This object is published as a JSON document that can be found at /schema.json at the root of the specification. For example, the schema used to construct the 1.8.0 release of BIDS can be found at https://bids-specification.readthedocs.io/en/v1.8.0/schema.json, and the latest version that includes unreleased changes to BIDS and the schema may be found at https://bids-specification.readthedocs.io/en/latest/schema.json.

The JSON version of the schema contains schema_version and bids_version keys that identify the state of both the schema and the specification at the time it was compiled.

Metaschema

The metaschema.json file is a meta-schema that uses the JSON Schema language to formalize the allowable directories, files, fields and values of the BIDS schema, ensuring consistency across the entire schema directory. Validation of the schema is incorporated into the CI, so any changes that are inconsistent will be flagged before inclusion.