Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Policy Requirements #646

Merged
merged 71 commits into from
Sep 2, 2021
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
4402a43
Start of Requirements endpoint
schnuerle May 18, 2021
352d784
Setting up schema parts
schnuerle May 19, 2021
68f6e34
First pass at requirements metadata
schnuerle May 19, 2021
b6818c8
Requirement TOC
schnuerle May 19, 2021
b9d0e53
Added sub section schema definitions
schnuerle May 20, 2021
9c4804a
First 2 examples
schnuerle May 20, 2021
689baae
Added comma
schnuerle May 20, 2021
544e16f
Added links to Req examples
schnuerle May 20, 2021
188691e
Create agencies.csv
schnuerle May 20, 2021
4e47a17
Add agencies.csv mention
schnuerle May 20, 2021
d786abc
Added explainer for Requirement file
schnuerle May 20, 2021
e6015df
Working Group feedback changes
schnuerle May 21, 2021
dd6f4b3
Updated examples based on WG feedback
schnuerle May 22, 2021
e833f57
Special notes about required_fields
schnuerle May 22, 2021
c8b3aa3
Updated mds endpoints to make sense
schnuerle May 24, 2021
a9dea76
Added 3rd example, placeholder for 2 more
schnuerle May 24, 2021
e4c27aa
Added Provider and Other APIs examples
schnuerle May 27, 2021
d682a98
Added Agency example
schnuerle May 28, 2021
cf3bfff
Added Geography Driven Events example
schnuerle May 28, 2021
e868dba
Fixed agency_id location
schnuerle May 28, 2021
f9b7894
Added agency-language
schnuerle Jun 2, 2021
44e27d1
Added language to req examples
schnuerle Jun 2, 2021
d9c0ed4
Changed GBFS to boolean
schnuerle Jun 3, 2021
5da6e87
Updated example booleans and URLs
schnuerle Jun 3, 2021
eb0a6eb
Updated link to Policy section
schnuerle Jun 23, 2021
6c6a89d
Added optional vehicle types
schnuerle Jun 23, 2021
a2262e6
Added update frequency recommendations
schnuerle Jun 24, 2021
da6d8fa
Updated TOC
schnuerle Jun 24, 2021
78506c3
Added vehicle_type to an example
schnuerle Jun 24, 2021
05f5522
Fixed update frequency TOC conflict
schnuerle Jun 24, 2021
eb08091
Added program description and optional policy_id
schnuerle Jun 24, 2021
e3e86ea
Added descriptions and policy_ids to examples
schnuerle Jun 24, 2021
6a309f7
Typo fix
schnuerle Jun 24, 2021
30cfaca
First example converted to Programs format
schnuerle Jul 7, 2021
cbb8c11
Updated all examples with Programs structure
schnuerle Jul 8, 2021
84e86bb
Added new GBFS only example
schnuerle Jul 8, 2021
aa4fef9
Rearranged endpoint based on Programs concept
schnuerle Jul 8, 2021
683ddc6
Updated TOC
schnuerle Jul 8, 2021
5e50cc9
Fixed anchor on policy_id
schnuerle Jul 8, 2021
6ddb0a0
Transportation link updates
schnuerle Jul 8, 2021
91c0974
Renamed fields, remove policy_id
schnuerle Jul 8, 2021
ef98c64
Updated field names and removed policy_id
schnuerle Jul 8, 2021
e7115a0
Added dot notation for fields
schnuerle Jul 8, 2021
bb7d0f9
Updated dot notation in examples
schnuerle Jul 8, 2021
cef25e9
Added available_X to examples
schnuerle Jul 8, 2021
e76ca43
Added concept of available_x apis, endpoints, fields
schnuerle Jul 8, 2021
3bdc034
Updated agency URL example
schnuerle Jul 8, 2021
5589989
Typo fixes
schnuerle Jul 19, 2021
0c7da67
Added link to Agency Requirements repo
schnuerle Jul 29, 2021
380c6ee
Rearranged fields to be alphabetical
schnuerle Aug 5, 2021
2644b4c
Added possibility to support more specs than MDS/GBFS
schnuerle Aug 6, 2021
0b4b04e
Added beta 'requested' language
schnuerle Aug 6, 2021
8cda94c
Added 'disallowed_fields' to spec
schnuerle Aug 6, 2021
49a5b91
Added example for Trips with disallowed fields
schnuerle Aug 6, 2021
6a1a30f
Added an optional field to disallowed example
schnuerle Aug 6, 2021
03d298f
Added language about scaled-down MDS situations
schnuerle Aug 6, 2021
aa40dd2
Added TOC link to hosting
schnuerle Aug 6, 2021
467694c
Typo fixes
schnuerle Aug 12, 2021
0bd8ca2
Added some OMF member cities
schnuerle Aug 12, 2021
09ec011
Shortened some city URLs
schnuerle Aug 12, 2021
89ab2b1
Renamed to agency_state since not all are abbreviations
schnuerle Aug 17, 2021
611e9db
Chaging agency_uuid to agency_id
schnuerle Aug 18, 2021
cc1ec75
Changing to agency_id in examples
schnuerle Aug 18, 2021
1b680f2
Added link to beta feedback
schnuerle Aug 18, 2021
afad95e
Updated TOMP-API name
schnuerle Aug 18, 2021
14ed15b
Clarified unlisted APIs and Endpoints will not be returned
schnuerle Aug 24, 2021
e60dee6
Typo with 'between'
schnuerle Aug 24, 2021
fbf5325
Realistic max_update_interval example
schnuerle Aug 26, 2021
aed2f40
Added link from GDE to example
schnuerle Aug 26, 2021
35d8b8e
Changed all examples to monthly max update interval
schnuerle Sep 1, 2021
648b219
Disallowed fields must return no field/value
schnuerle Sep 1, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,8 @@ More than 115 cities and public agencies around the world use MDS, and it has be

Please let us know [via our website](https://www.openmobilityfoundation.org/get-in-touch/) or in the [public discussion area](https://github.com/openmobilityfoundation/mobility-data-specification/discussions) if you are an agency using MDS so we can add you to the city resource list, especially if you have published your policies or documents publicly.

To add yourself to the [agency list](/agencies.csv) and add your [Policy Requirement link](/provider.md#requirements), please let us know [via our website](https://www.openmobilityfoundation.org/get-in-touch/) or open an [Issue](https://github.com/openmobilityfoundation/mobility-data-specification/issues) or [Pull Request](https://github.com/openmobilityfoundation/mobility-data-specification/pulls).

[Top][toc]

# Providers Using MDS
Expand Down
2 changes: 2 additions & 0 deletions agencies.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
agency_name,agency_city,agency_state_abbr,agency_country_iso_code,agency_id,department_url,requirement_url
Louisville Metro,44bc31a7-464b-4ed9-b52e-8e74630826bd,Louisville,KY,US,https://louisvilleky.gov/government/public-works/dockless-find-and-ride-vehicles,https://mds.cityname.gov/requirements/1.2.0
169 changes: 156 additions & 13 deletions policy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,11 @@ This specification describes the digital relationship between _mobility as a ser
- [Update Frequency](#update-frequency)
- [Background](#background)
- [Distribution](#distribution)
- [REST Endpoints](#rest-endpoints)
- [Flat Files](#flat-files)
- [REST Endpoints](#rest-endpoints)
- [Policies](#policies)
- [Geographies](#geographies)
- [Requirements](#requirements)
- [Flat Files](#flat-files)
- [Schema](#schema)
- [Policy](#policy)
- [Rules](#rules)
Expand All @@ -25,7 +28,11 @@ This specification describes the digital relationship between _mobility as a ser
- [Messages](#messages)
- [Value URL](#value-url)
- [Order of Operations](#order-of-operations)

- [Requirement](#requirement)
- [Metadata](#requirement-metadata)
- [MDS Version](#requirement-mds-version)
- [MDS APIs](#requirement-mds-apis)

## General information

The following information applies to all `policy` API endpoints.
Expand Down Expand Up @@ -82,7 +89,7 @@ Flat files have an optional `end_date` field that will apply to the file as a wh

[Top][toc]

### REST Endpoints
## REST Endpoints

Among other use-cases, configuring a REST API allows an Agency to:

Expand All @@ -93,7 +100,7 @@ Among other use-cases, configuring a REST API allows an Agency to:

Responses must set the `Content-Type` header, as specified in the [versioning][versioning] section.

#### Responses and Error Messages
### Responses and Error Messages

The response to a client request must include a valid HTTP status code defined in the [IANA HTTP Status Code Registry][iana].

Expand All @@ -103,13 +110,13 @@ See the [Responses section][responses] for information on valid MDS response cod

Authorization is not required. An agency may decide to make this endpoint unauthenticated and public. See [Optional Authentication](/general-information.md#optional-authentication) for details.

#### Policies
### Policies

Endpoint: `/policies/{id}`
Method: `GET`
`data` Payload: `{ "policies": [] }`, an array of objects with the structure [outlined below](#policy).

##### Query Parameters
#### Query Parameters

| Name | Type | Required / Optional | Description |
| ------------ | --------- | --- | ---------------------------------------------- |
Expand All @@ -123,24 +130,34 @@ Policies will be returned in order of effective date (see schema below), with pa

`provider_id` is an implicit parameter and will be encoded in the authentication mechanism, or a complete list of policies should be produced. If the Agency decides that Provider-specific policy documents should not be shared with other Providers (e.g. punitive policy in response to violations), an Agency should filter policy objects before serving them via this endpoint.

#### Geographies
### Geographies

**Note:** see the new [Geography API](/geography#transition-from-policy) to understand the transisiton away from this endpoint, and how to support both in the MDS 1.1.0 release.
**Depreciated:** see the new [Geography API](/geography#transition-from-policy) to understand the transisiton away from this endpoint, and how to support both in a MDS 1.x.0 release.

Endpoint: `/geographies/{id}`
Method: `GET`
`data` Payload: `{ geographies: [] }`, an array of GeoJSON `Feature` objects that follow the schema [outlined here](#geography) or in [Geography](/geography#general-information).

##### Query Parameters
#### Query Parameters

| Name | Type | Required / Optional | Description |
| ------------ | --------- | --- | ---------------------------------------------- |
| `id` | UUID | Optional | If provided, returns one [Geography](/geography#general-information) object with the matching UUID; default is to return all geography objects. |

[Top][toc]

### Requirements

### Flat Files
Endpoint: `/requirements/`
Method: `GET`
`data` Payload: `{ requirements: [] }`, JSON objects that follow the schema [outlined here](#requirement).
[Beta feature](/general-information.md#beta-features): *Yes (as of 1.2.0)*.

See [Policy Requierment Examples](/policy/examples/requirements.md) for how this can be implemented.
schnuerle marked this conversation as resolved.
Show resolved Hide resolved

[Top][toc]

## Flat Files

To use flat files, policies shall be represented in two (2) files:

Expand All @@ -153,7 +170,7 @@ The publishing Agency should establish and communicate to providers how frequent

The `updated` field in the payload wrapper should be set to the time of publishing a revision, so that it is simple to identify a changed file.

#### Example `policies.json`
### Example `policies.json`

```jsonc
{
Expand All @@ -175,7 +192,7 @@ The `updated` field in the payload wrapper should be set to the time of publishi

The optional `end_date` field applies to all policies represented in the file.

#### Example `geographies.json`
### Example `geographies.json`

```jsonc
{
Expand Down Expand Up @@ -296,6 +313,8 @@ An individual `Rule` object is defined by the following fields:

### Geography

**Depreciated:** see the new [Geography API](/geography#transition-from-policy) to understand the transisiton away from this endpoint, and how to support both in a MDS 1.x.0 release.
schnuerle marked this conversation as resolved.
Show resolved Hide resolved

| Name | Type | Required / Optional | Description |
| ---------------- | --------- | --- | ----------------------------------------------------------------------------------- |
| `name` | String | Required | Name of geography |
Expand Down Expand Up @@ -365,6 +384,130 @@ If a vehicle is matched with a rule, then it _will not_ be considered in the sub

The internal mechanics of ordering are up to the Policy editing and hosting software.

[Top][toc]

### Requirement

The agency Policy Requirement file ennumerates all of the parts of MDS that an agency requires from providers, including APIs, endpoints, and optional fields, as well as information for providers about the APIs the agency is hosting. The requirements are specific to the needs and use cases of each agency, and ensures there is clarity on what data is being asked for in operating policy documents from providers, reducing the burden on both. This also allows additional public transparency and accountability around data requirements from agencies, and encourages privacy by allowing agencies to ask for only the data they need.
schnuerle marked this conversation as resolved.
Show resolved Hide resolved

This endpoint it not authenicated (ie. public), and allows the discovery of other public APIs like Geography, Policy, and Jurisdiction. The agency can host this as a file or API on their servers, on a third party server, or the OMF can host on behalf of an agency in the [agency requirements repo](#). See this [hosting guidance document](#) for more information. This requirements file can be [referenced directly](https://github.com/openmobilityfoundation/governance/blob/main/technical/OMF-MDS-Policy-Language-Guidance.md) in an agency's operating permit/policy document when discussing program data requirements.
schnuerle marked this conversation as resolved.
Show resolved Hide resolved

See [Policy Requierment Examples](/policy/examples/requirements.md) for how this can be implemented.

An agency's [Requirements](#requirements) endpoint contains a number of distinct parts, namely [metadata](#requirement-metadata) and [MDS version](#requirement-mds-version) (with sub sections on applicable providers and relevant [MDS APIs](#requirement-mds-apis)).

```jsonc
{
"metadata": {
// metadata fields per the "Requirement Metadata" section
},
"mds_versions" [
{
"version" : "[MDS VERSION NUMBER]",
"provider_ids": [
// provider id array
],
"mds_apis": [
{
"api_name" : "[MDS API]": {
// MDS endpoints, urls, optional fields
}
},
// other MDS APIs per the "Requirement MDS APIs" section
]
},
// other MDS versions per the "Requriement MDS Version" section
}
}
```
[Top][toc]

#### Requirement Metadata

Contains metadata applicable to the agency and at the top of its [Requirement](#requirement) data feed in the `metadata` section.

| Name | Type | Required / Optional | Description |
| ---------------------------- | --------------- | -------- | ----------------------------------- |
| `mds_release` | text | Required | Release of MDS that the requirements data feed aligns to, based on official MDS releases. E.g. "1.2.0" |
| `version` | integer | Required | Version of this file. Increment 1 with each modification. E.g. "3" |
| `last_updated` | [timestamp][ts] | Required | When this file `version` was last updated. E.g. "1611958740" |
| `max_update_frequency` | integer | Required | The expected maximum frequency with which this file could be updated. E.g. "P1D" |
schnuerle marked this conversation as resolved.
Show resolved Hide resolved
| `omf_review` | Enum | Required | yes/no. Was this file reviewed by OMF Staff for accuracy? E.g. "yes" |
schnuerle marked this conversation as resolved.
Show resolved Hide resolved
| `omf_review_date` | [timestamp][ts] | Optional | If `omf_review`, add timestamp. E.g. "1611958749" |
| `agency_uuid` | UUID | Required | UUID of the agency this file applies to. Must come from [agencies.csv](/agencies.csv) file. E.g. "737a9c62-c0cb-4c93-be43-271d21b784b5" |
| `agency_name` | text | Required | Name of the agency this file applies to. E.g. "Louisville Metro" |
| `agency_time_zone` | text | Required | Timezone used for dates and times across all MDS endpoints. E.g. "America/New_York" |
| `agency_currency` | text | Required | Currency used for all monetary values across all MDS endpoints. E.g. "USD" |
schnuerle marked this conversation as resolved.
Show resolved Hide resolved
| `agency_policy_website_url` | URL | Required | URL of the agency's transportation policy page. E.g. "https://www.cityname.gov/transporation/shared-devices.htm" |
| `agency_policy_document_url` | URL | Optional | URL of the agency's operating permit rules that mention data requirements. E.g. "https://www.cityname.gov/mds_data_policy.pdf" |
| `gbfs_required` | Enum | Required | yes/no. Is public GBFS required explicitly by providers? E.g. "yes" |
schnuerle marked this conversation as resolved.
Show resolved Hide resolved
| `url` | URL | Required | URL of this file. E.g. "https://mds.cityname.gov/requirements/1.2.0" |

[Top][toc]

#### Requirement MDS Version

Contains a list of providers and APIs/endpoints/fields that a version of MDS applies to in its [Requirement](#requirement) data feed. Unique combinations for MDS versions and specific providers can be defined. For example an agency can devine MDS version 1.2.0 for Provider #1 in a pilot with beta endpoints and optional fields, version 1.2.0 for other providers without beta features, and version 1.1.0 for Provider #2 with docked bikeshare.

```jsonc
// ...
"mds_versions": [
{
"version" : "[MDS VERSION NUMBER]",
"provider_ids": [
"[PROVIDER UUID]",
"[PROVIDER UUID]"
],
"mds_apis" [
{
// ...
},
// other MDS APIs
]
}
]
// ...
```

| Name | Type | Required / Optional | Description |
| ---------------------------- | -------------- | -------- | ----------------------------------- |
| `version` | text | Required | Version number of an official MDS release |
| `provider_ids` | UUID[] | Required | Array of providers that apply to this part of the requirements |

[Top][toc]

#### Requirement MDS APIs

For each combination of MDS version and provider list, you can specify the MDS APIs, endpoints, and optional fields that are required per your agency's policy. This is an array within the [Requirement MDS Version](#requirement-mds-version) `mds_apis` section in the [Requirement](#requirement) data feed.
schnuerle marked this conversation as resolved.
Show resolved Hide resolved

```jsonc
// ...
"mds_apis": [
{
"api_name" : "[MDS API]",
"url": "[ENDPOINT ROOT URL]",
"endpoints": [
{
"endpoint_name" : "[ENDPOINT NAME]",
"required_fields": [
"[FIELD NAME]",
// other field names
]
}
]
},
// other MDS APIs
]
// ...
```

| Name | Type | Required / Optional | Description |
| ----------------- | ----- | -------- | ----------------------------------- |
| `api_name` | Enum | Required | Name of the applicable MDS API: provider, agency, policy, geography, jurisdiction, metrics. At least one is required. |
| `url` | URL | Required / Optional | Location of API root URL (minus the endpoint name) if the API is unauthenticated and public. |
| `endpoints` | Array | Required | Array of endpoints required. At least one is required. |
| `endpoint_name` | Text | Required | Name of required endpoint. At least one is required. |
| `required_fields` | Array | Optional | Array of optional field names required by the agency. Can be left empty if none are required. |

[Top][toc]

Expand Down
118 changes: 118 additions & 0 deletions policy/examples/requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Policy Requirement Examples

This file presents a series of example [Policy documents](../README.md#requirement) for Agencies to use as templates.

## Table of Contents

- [Trips Only](#trips-only)
- [Vehicles Only](#vehicles-only)

## Trips Only

Version 1.1.0 for 2 providers requiring only Provider `/trips` with the optional `parking_verificaiton_url` field.
schnuerle marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"metadata": {
"mds_release": "1.2.0",
"version": "3",
"last_updated": "1611958740",
"max_update_frequency": "P1D",
"omf_review": "yes",
"omf_review_date": "1611958749",
"agency_uuid": "737a9c62-c0cb-4c93-be43-271d21b784b5",
"agency_name": "Louisville Metro",
"agency_time_zone": "America/New_York",
"agency_currency": "USD",
"agency_policy_website_url": "https:/www.cityname.gov/transporation/shared-devices.html",
"agency_policy_document_url": "https://www.cityname.gov/mds_data_policy.pdf",
"gbfs_required": "yes",
"url": "https://mds.cityname.gov/requirements/1.2.0"
},
"mds_versions": [
{
"version": "1.1.0",
"provider_ids": [
"70aa475d-1fcd-4504-b69c-2eeb2107f7be",
"2411d395-04f2-47c9-ab66-d09e9e3c3251"
],
"mds_apis": [
{
"api_name": "provider",
"endpoints": [
{
"endpoint_name" : "trips",
"required_fields": [
"parking_verification_url"
]
}
]
}
]
}
]
}
```

[Top](#table-of-contents)

## Vehicles Only

Version 1.1.0 for one provider and 1.0.0 for another provider, requiring only the Provider `/vehicles` endpoint and no optional fields, as an authenticated [alternative to GBFS](https://github.com/openmobilityfoundation/mobility-data-specification/wiki/MDS-Vehicles) for internal use.

```json
{
"metadata": {
"mds_release": "1.2.0",
"version": "2",
"last_updated": "1611958740",
"max_update_frequency": "T1M",
"omf_review": "yes",
"omf_review_date": "1611958749",
"agency_uuid": "737a9c62-c0cb-4c93-be43-271d21b784b5",
"agency_name": "Louisville Metro",
"agency_time_zone": "America/New_York",
"agency_currency": "USD",
"agency_policy_website_url": "https:/www.cityname.gov/transporation/shared-devices.html",
"agency_policy_document_url": "https://www.cityname.gov/mds_data_policy.pdf",
"gbfs_required": "yes",
"url": "https://mds.cityname.gov/requirements/1.2.0"
},
"mds_versions": [
{
"version": "1.1.0",
"provider_ids": [
"70aa475d-1fcd-4504-b69c-2eeb2107f7be"
],
"mds_apis": [
{
"api_name": "provider",
"endpoints": [
{
"endpoint_name" : "vehicles"
}
]
}
]
},
{
"version": "1.0.0",
"provider_ids": [
"2411d395-04f2-47c9-ab66-d09e9e3c3251"
],
"mds_apis": [
{
"api_name": "provider",
"endpoints": [
{
"endpoint_name" : "vehicles"
}
]
}
]
}
]
}
```

[Top](#table-of-contents)