From 6f779e761cb0cf4edea188a265e1a381876ff945 Mon Sep 17 00:00:00 2001 From: Gerardo Gonzalez <5494584+fmizzell@users.noreply.github.com> Date: Mon, 22 Aug 2022 12:06:49 -0500 Subject: [PATCH] Refactor Developer Documentation (#3621) --- Doxyfile | 8 +- Jenkinsfile | 18 +- docs_assets/css/custom.css | 16 +- docs_assets/main.md | 6 +- docs_assets/pages/1.0_start.md | 2 +- docs_assets/pages/3.1_metastore.md | 16 +- docs_assets/pages/3.2_datastore.md | 23 ++- docs_assets/pages/3.3_harvest.md | 18 +- docs_assets/pages/7.0_guide.md | 11 ++ docs_assets/pages/7.0_tutorials.md | 11 -- docs_assets/pages/8.0_developer.md | 5 + .../pages/community_code-of-conduct.md | 2 +- docs_assets/pages/dkan_dataset_info.md | 2 +- docs_assets/pages/dkan_datastore_drop.md | 2 +- docs_assets/pages/dkan_datastore_dropall.md | 2 +- docs_assets/pages/dkan_datastore_import.md | 6 +- docs_assets/pages/dkan_datastore_list.md | 2 +- docs_assets/pages/dkan_harvest_archive.md | 2 +- docs_assets/pages/dkan_harvest_deregister.md | 2 +- docs_assets/pages/dkan_harvest_info.md | 2 +- docs_assets/pages/dkan_harvest_list.md | 2 +- docs_assets/pages/dkan_harvest_publish.md | 2 +- docs_assets/pages/dkan_harvest_register.md | 2 +- docs_assets/pages/dkan_harvest_revert.md | 2 +- docs_assets/pages/dkan_harvest_run-all.md | 2 +- docs_assets/pages/dkan_harvest_run.md | 2 +- docs_assets/pages/dkan_harvest_status.md | 2 +- docs_assets/pages/dkan_metadata-form_sync.md | 2 +- .../dkan_metastore-search_rebuild-tracker.md | 2 +- docs_assets/pages/dkan_metastore_publish.md | 2 +- .../pages/dkan_sample-content_create.md | 2 +- .../pages/dkan_sample-content_remove.md | 2 +- docs_assets/pages/dkan_testusers.md | 2 +- ...ionaries.md => guide_data_dictionaries.md} | 16 +- .../{tut_dataset.md => guide_dataset.md} | 8 +- ...ettings.md => guide_datastore_settings.md} | 6 +- .../{tut_harvest.md => guide_harvest.md} | 35 ++-- ...rties.md => guide_metastore_properties.md} | 2 +- docs_junk_drawer/mod_datastore.md | 23 --- docs_junk_drawer/mod_frontend.md | 10 - docs_junk_drawer/mod_harvest.md | 177 ------------------ docs_junk_drawer/mod_metastore.md | 6 - .../modules/datastore_mysql_import/README.md | 14 +- modules/dkan_js_frontend/README.md | 12 +- modules/metastore/README.md | 21 +++ 45 files changed, 175 insertions(+), 337 deletions(-) create mode 100644 docs_assets/pages/7.0_guide.md delete mode 100644 docs_assets/pages/7.0_tutorials.md create mode 100644 docs_assets/pages/8.0_developer.md rename docs_assets/pages/{tut_data_dictionaries.md => guide_data_dictionaries.md} (71%) rename docs_assets/pages/{tut_dataset.md => guide_dataset.md} (89%) rename docs_assets/pages/{tut_datastore_settings.md => guide_datastore_settings.md} (76%) rename docs_assets/pages/{tut_harvest.md => guide_harvest.md} (54%) rename docs_assets/pages/{tut_metastore_properties.md => guide_metastore_properties.md} (89%) delete mode 100644 docs_junk_drawer/mod_datastore.md delete mode 100644 docs_junk_drawer/mod_frontend.md delete mode 100644 docs_junk_drawer/mod_harvest.md delete mode 100644 docs_junk_drawer/mod_metastore.md create mode 100644 modules/metastore/README.md diff --git a/Doxyfile b/Doxyfile index bf41e48e6a..8676d35dbb 100644 --- a/Doxyfile +++ b/Doxyfile @@ -905,7 +905,9 @@ FILE_PATTERNS = *.c \ *.ice \ *.rst \ *.json \ - *.info.yml + *.info.yml \ + *.routing.yml \ + *.services.yml # The RECURSIVE tag can be used to specify whether or not subdirectories should # be searched for input files as well. @@ -920,8 +922,8 @@ RECURSIVE = YES # Note that relative paths are relative to the directory from which doxygen is # run. -EXCLUDE = "modules/datastore/modules/datastore_mysql_import/README.md" \ - "modules/dkan_js_frontend/README.md" +#EXCLUDE = "modules/datastore/modules/datastore_mysql_import/README.md" \ +# "modules/dkan_js_frontend/README.md" # The EXCLUDE_SYMLINKS tag can be used to select whether or not files or # directories that are symbolic links (a Unix file system feature) are excluded diff --git a/Jenkinsfile b/Jenkinsfile index 3e3c085b42..041edf4a87 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -31,10 +31,10 @@ pipeline { docker container stop $i docker container rm $i done - + docker network disconnect $qa_network_id proxy || true docker network rm $qa_network_id - + sudo rm -r $WORKSPACE/* fi ''' @@ -70,11 +70,23 @@ pipeline { dktl demo dktl drush user:password admin mayisnice sudo chown -R 1000:docker $WORKSPACE/dkan-tools/vendor + dktl dkan:docs curl -fI `dktl url` ''' } } } + stage('RebuildDocs') { + steps { + sh ''' + ssh -o StrictHostKeyChecking=no root@45.33.76.242 < .contents { margin: 20px; @@ -144,10 +148,10 @@ a { border: none; } a:hover { - color: #205493; + color: #0097c9; } .contents a:visited { - color: #205493; + color: #0071bc; } #nav-tree .selected { background-image: none; diff --git a/docs_assets/main.md b/docs_assets/main.md index e7d6700e8a..5fee69cdb6 100644 --- a/docs_assets/main.md +++ b/docs_assets/main.md @@ -4,7 +4,7 @@ __DKAN__ is an open-source open-data platform inspired by [CKAN](https://ckan.or ## Structure -%Drupal allows _modules_ to extend its functionality. DKAN is a %Drupal module. +DKAN is a %Drupal module that adds data publishing functionality. Modules can have submodules; DKAN utilizes this structure to organize its internal subsystems. Information about the subsystems/components in DKAN can be found in the \ref components page. @@ -17,7 +17,7 @@ DKAN's modules and subsystems are organized around four main data functions: ### Data Management -The main function of any open data platform is to help manage data. Making data _public_ is simple, anyone can place a file in a web-accessible store these days, but making data _open_ takes a bit more work. True open data is accessible, discoverable, machine-readable, linked to other resources that provide context, published in an open format and under an open license. +The main function of any open data platform is to help manage data. Making data _public_ is simple, anyone can place a file in a web-accessible server, but making data _open_ takes a bit more work. True open data is accessible, discoverable, machine-readable, linked to other resources that provide context, published in an open format and under an open license. This is what we mean by data management: providing tools that empower data publishers to make data open, which empowers data consumers to find and use the data they need. @@ -47,4 +47,4 @@ Data in files isn't naturally searchable, but the \ref datastore module parses a DKAN is actively maintained by [CivicActions](https://civicactions.com/dkan). -To learn more about the DKAN community visit [GetDKAN.org](https://getdkan.org). +To learn more about the DKAN community visit [DKAN Discussions](https://github.com/GetDKAN/dkan/discussions). diff --git a/docs_assets/pages/1.0_start.md b/docs_assets/pages/1.0_start.md index bf02d62dee..838bf23f1b 100644 --- a/docs_assets/pages/1.0_start.md +++ b/docs_assets/pages/1.0_start.md @@ -28,7 +28,7 @@ DKAN tools bases new projects off of a [composer project](https://github.com/Get composer create-project getdkan/recommended-project my-project -Or simply create a project however you prefer and add a requirement for `getdkan/dkan`. **Do note** that a bug in Drupal core cron may cause problems with data imports, and applying [this patch](https://www.drupal.org/project/drupal/issues/3230541#comment-14206814) is highly recommended. The patch will be applied automatically if you use the recommended project. +Or simply create a project however you prefer and add a requirement for `getdkan/dkan`. **Do note** that a bug in Drupal core cron may cause problems with data imports, and applying [this patch](https://www.drupal.org/project/drupal/issues/3274931) is highly recommended. The patch will be applied automatically if you use the [recommended project](https://github.com/GetDKAN/recommended-project). ## Installing DKAN diff --git a/docs_assets/pages/3.1_metastore.md b/docs_assets/pages/3.1_metastore.md index 9d41c23a07..c1c17c58e1 100644 --- a/docs_assets/pages/3.1_metastore.md +++ b/docs_assets/pages/3.1_metastore.md @@ -1,17 +1,19 @@ -@page metastore Metastore +# Metastore {#metastore} DKAN's **Metastore** is what you use to create, retrieve, update, and delete records describing your data. These records are what we refer to as "metadata." -As a data catalog, DKAN's main goal is to help you share a collection of @ref Dataset "dataset" records. A dataset's metadata can follow virtually any _schema_, or format you want. What is important is that it points to the data you are trying to share, and that it gives useful contextual information. This usually includes when the data was released, how often it is updated and who published it, but can include details as precise as the geographic boundaries it applies to. +As a data catalog, DKAN's main goal is to help you share a collection of @ref Dataset "dataset" records. A dataset's metadata can follow virtually any _schema_, or format you want. What is important is that it points to the data you are trying to share, and that it gives useful contextual information. This usually includes when the data was released, how often it is updated and who published it, but can include details as precise as the geographic boundaries or relevant time period the data applies to. Some more details of DKAN's metastore: -* The data assets themselves (usually in the form of local files or URLs) are referred to internally in DKAN as _resources_. +* The data assets themselves (usually in the form of local files or URLs to data files) are referred to internally in DKAN as _resources_. * The structure and format of dataset metadata in DKAN are determined by a [JSON schema](https://json-schema.org/). By default, DKAN provides and utilizes the [DCAT-US metadata schema](https://resources.data.gov/resources/dcat-us/) to store datasets, but @ref ChangingYourDatasetSchema "custom schemas" can be added to the codebase to override this. * In DCAT-US, resources are placed in a sub-schema of the parent dataset called a *distribution*. +![dataset structure](https://project-open-data.cio.gov/v1.1/schema-diagram.svg) + @note - Read the documentation on @ref tut_dataset to get started adding information to the metastore. + Read the documentation on @ref guide_dataset to get started adding information to the metastore. ## Configuration @anchor ChangingYourDatasetSchema @@ -20,7 +22,7 @@ Some more details of DKAN's metastore: Replacing the dataset schema in DKAN allows you to add fields and conform to additional (or completely different) specifications. As long as you provide a valid JSON schema, any information going into the metastore will be validated against it. -To change the schema being used, copy the contents of the `schema` directory from the DKAN repo and place it in the root of your Drupal installation (src/schema). Then make any modifications necessary to the `dataset.json` file inside the `collections` directory. +To change the schema being used, copy the contents of the `schema` directory from the DKAN repo and place it in the root of your Drupal installation (src/schema). Then make any modifications necessary to the `dataset.json` file inside the `collections` directory. Note that even if you are only changing the dataset.json schema, it is important to copy ALL of the schema files as DKAN will be expecting all of the schema files to be in the same location. @warning Warning: The schema is actively used by the catalog to verify the validity of the data. Making changes to the schema after data is present in the catalog should be done with care as non-backward-compatible changes to the schema could cause issues. Look at Drupal::metastore::SchemaRetriever::findSchemaDirectory() for context. @@ -41,4 +43,6 @@ _(Note that only option #1 above has been implemented in the current version of Data dictionaries will affect the behavior of the @ref datastore. -See @ref tut_data_dictionaries for step-by-step instructions for use. \ No newline at end of file +By default, all data imported into a datastore will be stored as strings. Use the data dictionary to define which columns should be stored as dates, integers, decimals, etc. + +See @ref guide_data_dictionaries for step-by-step instructions for use. diff --git a/docs_assets/pages/3.2_datastore.md b/docs_assets/pages/3.2_datastore.md index 416e91f304..7065bb233a 100644 --- a/docs_assets/pages/3.2_datastore.md +++ b/docs_assets/pages/3.2_datastore.md @@ -1,17 +1,20 @@ -@page datastore Datastore +# Datastore {#datastore} -The DKAN **Datastore** module provides integration with the [datastore](https://github.com/GetDKAN/datastore) library. With it you are able to parse CSV files and save the data in database tables. This allows users to query the data through a public API. Note that the CSV must be in UTF-8 format to parse correctly. +The DKAN **Datastore** provides a service to parse CSV files and save the data in database tables. This allows users to query the data through a [public API](https://demo.getdkan.org/api). -When a dataset is added to the catalog, the mimetype value of each distribution object is checked to determine if the distribution is a CSV file or not. CSV distributions will be added to the import queue to be parsed and imported into the datastore. +When a dataset is added to the catalog, the mimetype value of each distribution object is checked to determine if the distribution contains a CSV file. The CSV distributions will be added to the import queue to be parsed and imported into the datastore. +- When providing a remote URL for the distribution object, be sure to include the mimetype property: `"mediaType": "text/csv"`. +- Note that the CSV must be in UTF-8 format to parse correctly. -The import will happen in batches during cron runs. The import will iterate over the file, if it does not finish, the job will be returned to the queue. Be sure to have cron running on a regular basis so that large file imports will complete. If there is a change to the metadata it will trigger a new import to the datastore. +By default, the import will happen in batches during cron runs. The import will iterate over the file, if it does not finish, the job will be returned to the queue. Be sure to have cron running on a regular basis so that large file imports will complete. You can view the status of datastore imports at `/admin/dkan/datastore/status`. -You can manually import file data into the datastore via drush with the identifier of the distribution. Use the [API](https://demo.getdkan.org/api/1/metastore/schemas/dataset/items?show-reference-ids) to get the identifier of the file you want to import. The identifier will be at ``distribution.0.data.%Ref:downloadURL.0.data.identifier`` +If you enable the @subpage mysqlimport module, the file will be imported in a single step using MySQL's native LOAD DATA function. -## Drush Commands -* @ref dkandatastorelist -* @ref dkandatastoreimport -* @ref dkandatastoredrop +If there is a change to the _distribution: downloadURL_ property, the existing datastore will be dropped and a new import will be triggered. + +## Datastore Settings + +See @ref guide_datastore_settings for configuration information. ## Data dictionary integration @@ -21,4 +24,4 @@ If you are using DKAN Data Dictionaries on your site, the datastore will refer t 2. Create a new job in the queue to alter the new table in the database to match its columns to the field types defined in the dictionary, and change the column description/comment in the database to match the "title" in the dictionary. 3. Report the status of the alter job once it has run. -See @ref metastore and @ref tut_data_dictionaries for more on how to use this feature. \ No newline at end of file +See @ref metastore and @ref guide_data_dictionaries for more on how to use this feature. diff --git a/docs_assets/pages/3.3_harvest.md b/docs_assets/pages/3.3_harvest.md index 62b660dda1..d635591345 100644 --- a/docs_assets/pages/3.3_harvest.md +++ b/docs_assets/pages/3.3_harvest.md @@ -1,19 +1,7 @@ -@page harvest Harvest +# Harvest {#harvest} -The DKAN **Harvest** module provides integration with the [harvest](https://github.com/GetDKAN/harvest) library. This enables you to use the public feed or API of another data portal and import items from that portal’s catalog into your own. +The DKAN **Harvest** module enables you to use the public feed or API of another data portal and import items from that portal’s catalog into your own. For example, [Data.gov](https://data.gov/) harvests all of its datasets from the [data.json](https://project-open-data.cio.gov/v1.1/schema/) files of [hundreds of U.S. federal, state and local data portals](https://catalog.data.gov/harvest). -A [harvest](glossary.html#term_harvest) is the execution of a [Harvest Plan](glossary.html#term_harvestplan). - -## Drush Commands -* @ref dkanharvestregister -* @ref dkanharvestderegister -* @ref dkanharvestrun -* @ref dkanharvestrunall -* @ref dkanharvestrevert -* @ref dkanharveststatus -* @ref dkanharvestinfo -* @ref dkanharvestlist -* @ref dkanharvestpublish -* @ref dkanharvestarchive +See @ref guide_harvest diff --git a/docs_assets/pages/7.0_guide.md b/docs_assets/pages/7.0_guide.md new file mode 100644 index 0000000000..9662fc2e3a --- /dev/null +++ b/docs_assets/pages/7.0_guide.md @@ -0,0 +1,11 @@ +@page guide User Guide + +@subpage guide_dataset + +@subpage guide_metastore_properties + +@subpage guide_datastore_settings + +@subpage guide_harvest + +@subpage guide_data_dictionaries diff --git a/docs_assets/pages/7.0_tutorials.md b/docs_assets/pages/7.0_tutorials.md deleted file mode 100644 index f85a934c5a..0000000000 --- a/docs_assets/pages/7.0_tutorials.md +++ /dev/null @@ -1,11 +0,0 @@ -@page tutorials Tutorials - -@subpage tut_dataset - -@subpage tut_metastore_properties - -@subpage tut_datastore_settings - -@subpage tut_harvest - -@subpage tut_data_dictionaries \ No newline at end of file diff --git a/docs_assets/pages/8.0_developer.md b/docs_assets/pages/8.0_developer.md new file mode 100644 index 0000000000..bb36bc38c2 --- /dev/null +++ b/docs_assets/pages/8.0_developer.md @@ -0,0 +1,5 @@ +@page developer Developer + +@subpage dev_metastore + +@subpage dev_frontend diff --git a/docs_assets/pages/community_code-of-conduct.md b/docs_assets/pages/community_code-of-conduct.md index f64c3e3198..c7bdbfc578 100644 --- a/docs_assets/pages/community_code-of-conduct.md +++ b/docs_assets/pages/community_code-of-conduct.md @@ -1,4 +1,4 @@ -@page conduct Code of conduct +# Code of conduct {#conduct} ## Purpose diff --git a/docs_assets/pages/dkan_dataset_info.md b/docs_assets/pages/dkan_dataset_info.md index f7fd45a924..1aa52d8fd3 100644 --- a/docs_assets/pages/dkan_dataset_info.md +++ b/docs_assets/pages/dkan_dataset_info.md @@ -1,4 +1,4 @@ -@page dkandatasetinfo dkan:dataset-info +# dkan:dataset-info {#dkandatasetinfo} View information about a dataset. diff --git a/docs_assets/pages/dkan_datastore_drop.md b/docs_assets/pages/dkan_datastore_drop.md index ef9084ada7..dafd0898a0 100644 --- a/docs_assets/pages/dkan_datastore_drop.md +++ b/docs_assets/pages/dkan_datastore_drop.md @@ -1,4 +1,4 @@ -@page dkandatastoredrop dkan:datastore:drop +# dkan:datastore:drop {#dkandatastoredrop} Drop a datastore. diff --git a/docs_assets/pages/dkan_datastore_dropall.md b/docs_assets/pages/dkan_datastore_dropall.md index f3d7e4db2b..211845b300 100644 --- a/docs_assets/pages/dkan_datastore_dropall.md +++ b/docs_assets/pages/dkan_datastore_dropall.md @@ -1,3 +1,3 @@ -@page dkandatastoredropall dkan:datastore:drop-all +# dkan:datastore:drop-all {#dkandatastoredropall} Drop a ALL datastore tables. diff --git a/docs_assets/pages/dkan_datastore_import.md b/docs_assets/pages/dkan_datastore_import.md index 01bef162e4..d64bc6c793 100644 --- a/docs_assets/pages/dkan_datastore_import.md +++ b/docs_assets/pages/dkan_datastore_import.md @@ -1,6 +1,8 @@ -@page dkandatastoreimport dkan:datastore:import +# dkan:datastore:import {#dkandatastoreimport} -Import a datastore. +You can manually import file data into the datastore with the identifier of the distribution. There are two main ways to get the distribution uuid: +- Use the [API](https://demo.getdkan.org/api/1/metastore/schemas/dataset/items?show-reference-ids) to get the identifier of the file you want to import. The identifier will be at ``distribution.0.data.%Ref:downloadURL.0.data.identifier`` +- Use @ref dkandatasetinfo #### Arguments diff --git a/docs_assets/pages/dkan_datastore_list.md b/docs_assets/pages/dkan_datastore_list.md index 13e836fa80..b351b2dbc4 100644 --- a/docs_assets/pages/dkan_datastore_list.md +++ b/docs_assets/pages/dkan_datastore_list.md @@ -1,4 +1,4 @@ -@page dkandatastorelist dkan:datastore:list +# dkan:datastore:list {#dkandatastorelist} List information about all datastores. diff --git a/docs_assets/pages/dkan_harvest_archive.md b/docs_assets/pages/dkan_harvest_archive.md index 1c7a8d33a4..8273a8b28c 100644 --- a/docs_assets/pages/dkan_harvest_archive.md +++ b/docs_assets/pages/dkan_harvest_archive.md @@ -1,4 +1,4 @@ -@page dkanharvestarchive dkan:harvest:archive +# dkan:harvest:archive {#dkanharvestarchive} Archives (unpublish) harvested entities. diff --git a/docs_assets/pages/dkan_harvest_deregister.md b/docs_assets/pages/dkan_harvest_deregister.md index 0ea3dd6262..a06416af11 100644 --- a/docs_assets/pages/dkan_harvest_deregister.md +++ b/docs_assets/pages/dkan_harvest_deregister.md @@ -1,4 +1,4 @@ -@page dkanharvestderegister dkan:harvest:deregister +# dkan:harvest:deregister {#dkanharvestderegister} Deregister a harvest. diff --git a/docs_assets/pages/dkan_harvest_info.md b/docs_assets/pages/dkan_harvest_info.md index 8eb7d1d681..2596af6d95 100644 --- a/docs_assets/pages/dkan_harvest_info.md +++ b/docs_assets/pages/dkan_harvest_info.md @@ -1,4 +1,4 @@ -@page dkanharvestinfo dkan:harvest:info +# dkan:harvest:info {#dkanharvestinfo} Give information about a previous harvest run. diff --git a/docs_assets/pages/dkan_harvest_list.md b/docs_assets/pages/dkan_harvest_list.md index c09dfe9f7c..9cbe7a4dca 100644 --- a/docs_assets/pages/dkan_harvest_list.md +++ b/docs_assets/pages/dkan_harvest_list.md @@ -1,4 +1,4 @@ -@page dkanharvestlist dkan:harvest:list +# dkan:harvest:list {#dkanharvestlist} List available harvests. diff --git a/docs_assets/pages/dkan_harvest_publish.md b/docs_assets/pages/dkan_harvest_publish.md index 4cfaa578d0..c0a6b77c4b 100644 --- a/docs_assets/pages/dkan_harvest_publish.md +++ b/docs_assets/pages/dkan_harvest_publish.md @@ -1,4 +1,4 @@ -@page dkanharvestpublish dkan:harvest:publish +# dkan:harvest:publish {#dkanharvestpublish} Publishes harvested entities. diff --git a/docs_assets/pages/dkan_harvest_register.md b/docs_assets/pages/dkan_harvest_register.md index fece76fdbb..a7961e26dd 100644 --- a/docs_assets/pages/dkan_harvest_register.md +++ b/docs_assets/pages/dkan_harvest_register.md @@ -1,4 +1,4 @@ -@page dkanharvestregister dkan:harvest:register +# dkan:harvest:register {#dkanharvestregister} Register a new harvest. diff --git a/docs_assets/pages/dkan_harvest_revert.md b/docs_assets/pages/dkan_harvest_revert.md index 1858a7f84f..c09d66d1e5 100644 --- a/docs_assets/pages/dkan_harvest_revert.md +++ b/docs_assets/pages/dkan_harvest_revert.md @@ -1,4 +1,4 @@ -@page dkanharvestrevert dkan:harvest:revert +# dkan:harvest:revert {#dkanharvestrevert} Revert a harvest, i.e. remove harvested entities and unpublish orhpaned keywords, themes, and distributions. diff --git a/docs_assets/pages/dkan_harvest_run-all.md b/docs_assets/pages/dkan_harvest_run-all.md index 25217bb99a..88893c8ea5 100644 --- a/docs_assets/pages/dkan_harvest_run-all.md +++ b/docs_assets/pages/dkan_harvest_run-all.md @@ -1,4 +1,4 @@ -@page dkanharvestrunall dkan:harvest:run-all +# dkan:harvest:run-all {#dkanharvestrunall} Run all pending harvests. diff --git a/docs_assets/pages/dkan_harvest_run.md b/docs_assets/pages/dkan_harvest_run.md index 1adae98335..885ce13dc3 100644 --- a/docs_assets/pages/dkan_harvest_run.md +++ b/docs_assets/pages/dkan_harvest_run.md @@ -1,4 +1,4 @@ -@page dkanharvestrun dkan:harvest:run +# dkan:harvest:run {#dkanharvestrun} Run a harvest. diff --git a/docs_assets/pages/dkan_harvest_status.md b/docs_assets/pages/dkan_harvest_status.md index 3f4735d462..822f1b21be 100644 --- a/docs_assets/pages/dkan_harvest_status.md +++ b/docs_assets/pages/dkan_harvest_status.md @@ -1,4 +1,4 @@ -@page dkanharveststatus dkan:harvest:status +# dkan:harvest:status {#dkanharveststatus} Show status of of a particular harvest run. diff --git a/docs_assets/pages/dkan_metadata-form_sync.md b/docs_assets/pages/dkan_metadata-form_sync.md index 947b908cce..c32f33acbc 100644 --- a/docs_assets/pages/dkan_metadata-form_sync.md +++ b/docs_assets/pages/dkan_metadata-form_sync.md @@ -1,4 +1,4 @@ -@page dkanmetadataformsync dkan:metadata-form:sync +# dkan:metadata-form:sync {#dkanmetadataformsync} Synchronize the module with the React app. diff --git a/docs_assets/pages/dkan_metastore-search_rebuild-tracker.md b/docs_assets/pages/dkan_metastore-search_rebuild-tracker.md index 63a723cca0..4315ff454e 100644 --- a/docs_assets/pages/dkan_metastore-search_rebuild-tracker.md +++ b/docs_assets/pages/dkan_metastore-search_rebuild-tracker.md @@ -1,3 +1,3 @@ -@page dkanmetastoresearchrebuildtracker dkan:metastore-search:rebuild-tracker +# dkan:metastore-search:rebuild-tracker {#dkanmetastoresearchrebuildtracker} Rebuild the search api tracker for the dkan index. diff --git a/docs_assets/pages/dkan_metastore_publish.md b/docs_assets/pages/dkan_metastore_publish.md index 7a14cf6030..e5689519c9 100644 --- a/docs_assets/pages/dkan_metastore_publish.md +++ b/docs_assets/pages/dkan_metastore_publish.md @@ -1,4 +1,4 @@ -@page dkanmetastorepublish dkan:metastore:publish +# dkan:metastore:publish {#dkanmetastorepublish} Publish the latest version of a dataset. diff --git a/docs_assets/pages/dkan_sample-content_create.md b/docs_assets/pages/dkan_sample-content_create.md index e72c83ee39..18f53a0013 100644 --- a/docs_assets/pages/dkan_sample-content_create.md +++ b/docs_assets/pages/dkan_sample-content_create.md @@ -1,3 +1,3 @@ -@page dkansamplecontentcreate dkan:sample-content:create +# dkan:sample-content:create {#dkansamplecontentcreate} Create sample content. diff --git a/docs_assets/pages/dkan_sample-content_remove.md b/docs_assets/pages/dkan_sample-content_remove.md index fa614f1347..3f1b265010 100644 --- a/docs_assets/pages/dkan_sample-content_remove.md +++ b/docs_assets/pages/dkan_sample-content_remove.md @@ -1,3 +1,3 @@ -@page dkansamplecontentremove dkan:sample-content:remove +# dkan:sample-content:remove {#dkansamplecontentremove} Remove sample content. diff --git a/docs_assets/pages/dkan_testusers.md b/docs_assets/pages/dkan_testusers.md index 23948dcdfa..75fe7e53ce 100644 --- a/docs_assets/pages/dkan_testusers.md +++ b/docs_assets/pages/dkan_testusers.md @@ -1,4 +1,4 @@ -@page dkantestusers dkan:qa-users-create +# dkan:qa-users-create {#dkantestusers} DKAN ships with two test users: testapiuser and testadmin. diff --git a/docs_assets/pages/tut_data_dictionaries.md b/docs_assets/pages/guide_data_dictionaries.md similarity index 71% rename from docs_assets/pages/tut_data_dictionaries.md rename to docs_assets/pages/guide_data_dictionaries.md index 5057ac6bb9..3eb0278c6c 100644 --- a/docs_assets/pages/tut_data_dictionaries.md +++ b/docs_assets/pages/guide_data_dictionaries.md @@ -1,8 +1,9 @@ -@page tut_data_dictionaries How to use Data Dictionaries +# How to use Data Dictionaries {#guide_data_dictionaries} ## Tutorial I: Catalog-wide data dictionary -The simplest way to use data dictionaries on your site is to create one for the entire catalog. To do this, let's first create a new dictionary using the API (a GUI process for this is not yet available). We are going to take our list of fields from one of the sample datasets that ships with DKAN, "Data on bike lanes in Florida." +### Creating a data dictionary via the API +The simplest way to use data dictionaries on your site is to create one for the entire catalog. To do this, let's first create a new dictionary using the API. We are going to take our list of fields from one of the sample datasets that ships with DKAN, "Data on bike lanes in Florida." ```http POST http://mydomain.com/api/1/metastore/schemas/data-dictionary/items @@ -66,4 +67,13 @@ drush -y config:set metastore.settings data_dictionary_mode 1 drush -y config:set metastore.settings data_dictionary_sitewide 7fd6bb1f-2752-54de-9a33-81ce2ea0feb2 ``` -Both of these settings will be available in the admin UI soon. \ No newline at end of file +### Creating a data dictionary via the UI +1. Log in as an administrator. +2. From the DKAN menu, select Data Dictionary -> Create. +3. Enter a name for your data dictionary that will serve as its identifier. +4. Define the fields for your data dictionary +5. Click the "Save" button. +6. From the DKAN menu, seclect Data Dictionary -> Settings. +7. Select "Sitewide" from the Dictionary Mode options. +8. Type in the name of the data-dictionary you created in step 3. +9. Click the "Save configuration" button. diff --git a/docs_assets/pages/tut_dataset.md b/docs_assets/pages/guide_dataset.md similarity index 89% rename from docs_assets/pages/tut_dataset.md rename to docs_assets/pages/guide_dataset.md index 146ce2e87d..7731ede40c 100644 --- a/docs_assets/pages/tut_dataset.md +++ b/docs_assets/pages/guide_dataset.md @@ -1,4 +1,4 @@ -@page tut_dataset How to add a Dataset +# How to add a Dataset {#guide_dataset} ## API @@ -27,8 +27,10 @@ Authorization: Basic admin:admin 1. Log in to the site. 2. Navigate to Admin > DKAN > Datasets. 3. Click the "+ Add new dataset" button. -4. Use the _Download URL_ field to enter a url to your file or upload a local file. -5. Fill in the form and click "submit". +4. Use the Distribution _Download URL_ field to enter a url to your file or upload a local file. +5. Fill in the form with as much descriptive information as you can to make it discoverable. +6. Click "submit". +7. Run cron to start the import. ## Demo site content Running cron will import the csv files to the datstore. diff --git a/docs_assets/pages/tut_datastore_settings.md b/docs_assets/pages/guide_datastore_settings.md similarity index 76% rename from docs_assets/pages/tut_datastore_settings.md rename to docs_assets/pages/guide_datastore_settings.md index 70cc427848..48bff5f47a 100644 --- a/docs_assets/pages/tut_datastore_settings.md +++ b/docs_assets/pages/guide_datastore_settings.md @@ -1,10 +1,12 @@ -@page tut_datastore_settings How to configure datastore settings. +# How to configure datastore settings {#guide_datastore_settings} -Datastore settings can be found at `admin/dkan/datastore` +Datastore settings can be found at `/admin/dkan/datastore` ## Rows limit This sets the maximum number of rows the datastore endpoints can return in a single request. The default is 500 rows. Caution: setting too high can lead to timeouts or memory issues, values above 20,000 not recommended. It is advised to run load tests to determine how high you can safely set this value. +Use the *limit* and *offset* parameters to iterate through result sets that are larger than the row limit when running queries against the datastore API. + ## Datastore triggering properties By default the only way trigger a "refresh" to an existing datastore is a change to the data file name. If the data in your file is updated but file name remains the same, you can select other "triggers" to ensure that the old datastore is dropped and a new one is generated. diff --git a/docs_assets/pages/tut_harvest.md b/docs_assets/pages/guide_harvest.md similarity index 54% rename from docs_assets/pages/tut_harvest.md rename to docs_assets/pages/guide_harvest.md index 11fe4fc030..a12417716a 100644 --- a/docs_assets/pages/tut_harvest.md +++ b/docs_assets/pages/guide_harvest.md @@ -1,16 +1,12 @@ -@page tut_harvest How to create a harvest +# How to create a harvest {#guide_harvest} Use drush commands to [harvest](glossary.html#term_harvest) data into your catalog. ## Register a harvest -1. Register a new [Harvest Plan](glossary.html#term_harvestplan). - - Create a unique name as the **identifier** - - Provide the **extract** object with type and uri values: type being the class that matches the data structure, default is "\\Harvest\\ETL\\Extract\\DataJson") - - Provide the full URI for the data source (such as `http://example.com/data.json` or `file://source/data.json`) - - Provide the **load** object that defines the type of content you want to create, default is: "\\Drupal\\harvest\\Load\\Dataset" - -2. If you would also like to make changes to the data you are harvesting, you can create custom **transforms** that will modify the data before saving to your catalog. Add multiple transforms as an array. + Register a new [Harvest Plan](glossary.html#term_harvestplan). + - Create a unique name as the **identifier** of your harvest + - Provide the full URI for the data source **Example** @@ -18,19 +14,22 @@ Use drush commands to [harvest](glossary.html#term_harvest) data into your catal You can view a list of all registered harvest plans with @ref dkanharvestlist - ### How to create transforms - @par - Transforms allow you to modify what you are harvesting. [Click here](https://github.com/GetDKAN/socrata_harvest) to see an example of how you can create a custom module to add a transform class. - @par - **Example with a transform item** - - drush dkan:harvest:register --identifier=myHarvestId --extract-uri=http://example.com/data.json --transform="\\Drupal\\custom_module\\Transform\\CustomTransform" - ## Run the harvest -Once you have registered a harvest source, run the import, passing in the identifier as an arguement + Once you have registered a harvest source, run the import, passing in the identifier as an arguement - drush dkan:harvest:run example + drush dkan:harvest:run myHarvestId ## View the status of the harvest Navigate to `admin/dkan/harvest` to view the status of the extraction, the date the harvest was run, and the number of datasets that were added by the harvest. By clicking on the harvest ID, you will also see specific information about each dataset, and the status of the datastore import. + +## Transforms +If you would also like to make changes to the data you are harvesting, you can create custom **transforms** that will modify the data before saving it to your catalog. Add multiple transforms as an array. + + ### How to create transforms + @par + Transforms allow you to modify what you are harvesting. [Click here](https://github.com/GetDKAN/socrata_harvest) to see an example of how you can create a custom module to add a transform class. + @par + **Example with a transform item** + + drush dkan:harvest:register --identifier=myHarvestId --extract-uri=http://example.com/data.json --transform="\\Drupal\\custom_module\\Transform\\CustomTransform" diff --git a/docs_assets/pages/tut_metastore_properties.md b/docs_assets/pages/guide_metastore_properties.md similarity index 89% rename from docs_assets/pages/tut_metastore_properties.md rename to docs_assets/pages/guide_metastore_properties.md index 896d098e24..86ccf2e304 100644 --- a/docs_assets/pages/tut_metastore_properties.md +++ b/docs_assets/pages/guide_metastore_properties.md @@ -1,4 +1,4 @@ -@page tut_metastore_properties How to add or remove entity generation for schema sub properties +# How to add or remove entity generation for schema sub properties {#guide_metastore_properties} When you create a dataset, additional data nodes will be created for specific sub-elements of the dataset as well, the default properties are: publisher, theme, keywords, and distribution. These data nodes will provide unique reference ids for the sub-elements and can be accesssed via an API endpoint. Learn more [here](https://demo.getdkan.org/api). diff --git a/docs_junk_drawer/mod_datastore.md b/docs_junk_drawer/mod_datastore.md deleted file mode 100644 index 194c07c086..0000000000 --- a/docs_junk_drawer/mod_datastore.md +++ /dev/null @@ -1,23 +0,0 @@ -## Datastore API - -Your data is now available via the Datastore API! -`api/1/datastore/sql?query=` - -**Parameters** -- resource_identifier (mixed) – identifier of the resource to be searched against -- properties (mixed) – array or string of matching conditions to select -- conditions (string) - retrieve only objects with properties of certain values -- sort (string) – comma separated field names with ordering -- order (string) – comma separated field names -- limit (int) – maximum number of rows to return -- offset (int) – offset this number of rows -- count (boolean) - mark query as a count query - -**Example** - -``` -http://domain.com/api/1/datastore/sql?query=[SELECT * FROM ${resource_identifier}][WHERE state = "OK"][ORDER BY county ASC][LIMIT 5 OFFSET 100] -``` - -**Configuration** -If your server's resources are limited you can adjust the default number of rows returned with the configuration page located at `admin/config/dkan/sql-endpoint`. diff --git a/docs_junk_drawer/mod_frontend.md b/docs_junk_drawer/mod_frontend.md deleted file mode 100644 index 07dd878ba0..0000000000 --- a/docs_junk_drawer/mod_frontend.md +++ /dev/null @@ -1,10 +0,0 @@ -@page frontend Decoupled Frontend - -## DKAN Frontend -This is an integration module that allows the React app driving the frontend to be embedded in Drupal. It will set the front page to use ``/home`` to serve the app. - -The page routes are defined in `frontend/config/install/frontend.config.yml` - -## [Data Catalog App](https://github.com/GetDKAN/data-catalog-app) - -This is a React app that will use the [Data Catalog Components](https://github.com/GetDKAN/data-catalog-components) library to build the frontend. It will serve as a starting point for your own customizations. If you are **not** using DKAN Tools, follow the instructions on the [README file](https://github.com/GetDKAN/data-catalog-app/blob/master/README.md) for manual installation. diff --git a/docs_junk_drawer/mod_harvest.md b/docs_junk_drawer/mod_harvest.md deleted file mode 100644 index 0169b06123..0000000000 --- a/docs_junk_drawer/mod_harvest.md +++ /dev/null @@ -1,177 +0,0 @@ -@page harvest Harvest - -The DKAN **Harvest** module provides integration with the [harvest](https://github.com/GetDKAN/harvest) library. This enables you to use the public feed or API of another data portal and import items from that portal’s catalog into your own. - -For example, [Data.gov](https://data.gov/) harvests all of its datasets from the [data.json](https://project-open-data.cio.gov/v1.1/schema/) files of [hundreds of U.S. federal, state and local data portals](https://catalog.data.gov/harvest). - - A "harvest" is the execution of a [harvest plan](#harvest-plan). - -## Drush Commands - -| Command | Args | Notes | -| -- | -- | -- | -| dkan:harvest:list | n/a | Lists avaialble harvests. | -| dkan:harvest:register | $config | Register a new harvest, file saved to the *dkan_harvest/plans* directory. | -| dkan:harvest:deregister | $identifier | Deletes the cached plan file. | -| dkan:harvest:cache | $identifier | This will fetch the source data, apply the source configuration and cache the data to a local file. | -| dkan:harvest:run | $identifier | This will harvest the current cache for the selected sources. | -| dkan:harvest:revert | $identifier | Reverts harvest. | - - -

Harvest Plan schema

-The harvest plan is the configuration used to import data into your catalog. -\ref https://github.com/GetDKAN/harvest/blob/master/schema/schema.json - - -\code{.json} - { - "$schema": "http://json-schema.org/draft-07/schema#", - "$id": "harvest-plan", - "type": "object", - "title": "Harvest Plan", - "required": [ - "identifier", - "extract", - "load" - ], - "properties": { - "identifier": { - "type": "string", - "title": "The plan's identifier", - "pattern": "^(.*)$" - }, - "extract": { - "type": "object", - "title": "Extract", - "required": [ - "type", - "uri" - ], - "properties": { - "type": { - "type": "string", - "title": "Class utilized to extract the data from the source." - }, - "uri": { - "type": "string", - "title": "The URL or Location of the Source", - "examples": [ - "http://demo.getdkan.com/data.json" - ] - } - } - }, - "transforms": { - "type": "array", - "title": "The Transforms for the Harvest", - "additionalProperties": false, - "items": { - "type": "string" - } - }, - "load": { - "type": "object", - "title": "The Load settings for the Harvest", - "required": [ - "type" - ], - "properties": { - "type": { - "type": "string", - "title": "Class utilized to load the harvested data." - } - } - } - } -} -\endcode - -## Setup - -Currently there is no UI for creating or running a harvest, use the drush commands above to harvest data into your catalog. - -1. Register a new harvest source, passing in the harvest plan configuration as JSON, wrapped in single quotes, do not add spaces between elements: - - Create a unique name as the **identifier** - - Provide the **extract** object with type and uri values: type being the class that matches the data structure, (most likely "\\Harvest\\ETL\\Extract\\DataJson"), and the full URI for the source endpoint (such as `http://source/data.json` or `file://source/data.json`) - - Provide the **load** object that defines the type of content you want to create, most likely datasets, so use: "\\Drupal\\harvest\\Load\\Dataset" - -2. If you would also like to make changes to the data you are harvesting, you can create custom **transforms** that will modify the data before saving to your catalog. Add multiple transforms as an array. [Learn more here](#transforms). - -**Example** -\code{.bash} -drush dkan-harvest:register '{"identifier":"example","extract":{"type":"\\Harvest\\ETL\\Extract\\DataJson","uri":"https://source/data.json"},"transforms":[],"load":{"type":"\\Drupal\\harvest\\Load\\Dataset"}}' -\endcode - -## Run the harvest -Once you have registered a harvest source, run the import, passing in the identifier as an arguement `drush dkan-harvest:run example` - -

How to create transforms

-Transforms allow you to modify what you are harvesting. -Let's say you want to harvest from a Socrata catalog and only want to import the distributions with csv files. [Create a custom module](https://www.drupal.org/docs/8/creating-custom-modules) with the following structure: custom_module/src/Harvest/Transform/Socrata.php - -\code{.php} -identifier; - $item->identifier = $this->getIdentifier($identifier); - - // Add a keyword when keywords are null. - if (empty($item->keyword)) { - $item->keyword = ['No keywords provided']; - } - - // Add a description if null. - if (empty($item->description)) { - $item->description = 'No description provided'; - } - - // Add titles for csv distributions. - if ($item->distribution) { - foreach ($item->distribution as $key => $dist) { - if ($dist->mediaType != "text/csv") { - unset($item->distribution[$key]); - } - else { - $dist->title = "{$item->identifier}.csv"; - $item->distribution[$key] = $dist; - } - } - } - - return $item; - } - - /** - * Private. - * - * Convert the url identifier to a non-url based identifier. - */ - private function getIdentifier($identifier) { - $path = parse_url($identifier, PHP_URL_PATH); - $path = str_replace('/api/views/', "", $path); - return $path; - } -} -\endcode - -**Example with new transform** - -\code{.bash} -drush dkan-harvest:register '{"identifier":"example","extract":{"type":"\\Harvest\\ETL\\Extract\\DataJson","uri":"https://source/data.json"},"transforms":["\\Drupal\\custom_module\\Transform\\Socrata"],"load":{"type":"\\Drupal\\harvest\\Load\\Dataset"}}' -\endcode diff --git a/docs_junk_drawer/mod_metastore.md b/docs_junk_drawer/mod_metastore.md deleted file mode 100644 index ae702f04d7..0000000000 --- a/docs_junk_drawer/mod_metastore.md +++ /dev/null @@ -1,6 +0,0 @@ -**Sample schema files** - -- [JSON - Minimum Requirements](https://project-open-data.cio.gov/v1.1/examples/catalog-sample.json) -- [JSON - With Extended Fields](https://project-open-data.cio.gov/v1.1/examples/catalog-sample-extended.json) - -![dataset structure](https://project-open-data.cio.gov/v1.1/schema-diagram.svg) diff --git a/modules/datastore/modules/datastore_mysql_import/README.md b/modules/datastore/modules/datastore_mysql_import/README.md index 0424fa0c85..89172586f2 100644 --- a/modules/datastore/modules/datastore_mysql_import/README.md +++ b/modules/datastore/modules/datastore_mysql_import/README.md @@ -1,12 +1,8 @@ -# DKAN Datastore MySQL Import +# DKAN Datastore MySQL Import {#mysqlimport} -This is an **expiremental** module to import CSVs into the DKAN datastore using -MySQL's native LOAD DATA function, similar to running a `mysqlimport` -command. It is over 5000% faster than DKAN's default datastore importer, but -requires a MySQL database and that the Drupal DB user have permissions to use -`LOAD DATA LOCAL INFILE`. +This module will import CSVs into the DKAN datastore using MySQL's native LOAD DATA function, similar to running a `mysqlimport` command. It is over 5000% faster than DKAN's default batch datastore importer, but **requires** a MySQL database and that the %Drupal DB user have permissions to use +`LOAD DATA LOCAL INFILE`. It also does not currently support the import of .tsv files. -Better test coverage, and a refactored import system that is more easily -swappable/extensible coming soon! +Better test coverage, and a refactored import system that is more easily swappable/extensible coming soon! -To use, simply enable datastore_mysql_import and clear your cache. \ No newline at end of file +To use, simply enable _datastore_mysql_import_ and clear your cache. diff --git a/modules/dkan_js_frontend/README.md b/modules/dkan_js_frontend/README.md index 9862651efa..83a0c6785a 100644 --- a/modules/dkan_js_frontend/README.md +++ b/modules/dkan_js_frontend/README.md @@ -1,6 +1,6 @@ -# DKAN JS Frontend Module +# DKAN JS Frontend Module {#dev_frontend} -This module provides a way to access a decoupled JavaScript frontend at defined paths. +This is an integration module that provides a way to access a decoupled JavaScript frontend at defined paths. ## Defining Paths The paths used are defined by the `dkan_js_frontend.config` yml. In the routes array add any path you want this module to use with a string structured like `unique_path_name,/the_path`. The first part is used by Drupal to store paths and the second is the actual path the JS frontend will show at. @@ -9,12 +9,16 @@ The paths used are defined by the `dkan_js_frontend.config` yml. In the routes a If the Drupal [Simple XML sitemap module](https://www.drupal.org/project/simple_sitemap) is installed, the DKAN JS Frontend module will automatically add static routes and dataset routes listed in the `dkan_js_frontend.config` yml to the default sitemap. ## JS/CSS -The module assumes Create React App (CRA) has been loaded into the `/src/frontend` folder of the site. This can be changed by updating the `dkan_js_frontend.config` keys of `css_folder` and `js_folder` with the new directory path. +The module assumes Create React App (CRA) has been loaded into the `/src/frontend` folder of the site. This can be changed by updating the `dkan_js_frontend.config` keys of `css_folder` and `js_folder` with the new directory path. The code will glob all files in the folders specified and attach them to any route/path that has been defined. The glob functionality should allow you to get around issues like CRA's hash in file names. The JS/CSS is directly attached to the page template provided, so the generated `index.html` file from CRA will not be used so all header updates will need to be made in Drupal and not the public files provided by the JS framework. +## [Data Catalog App](https://github.com/GetDKAN/data-catalog-app#readme) + +This is a React app that will use the [Data Catalog Components](https://github.com/GetDKAN/data-catalog-components) library to build the frontend. It will serve as a starting point for your own customizations. If you are **not** using DKAN Tools, follow the instructions on the [README file](https://github.com/GetDKAN/data-catalog-app/blob/master/README.md) for manual installation. + ## Tips If you are setting the JS frontend as the main frontend for your site, like on the demo DKAN site, you will want to do the following: -* Turn off Bartik and enable Stark as your frontend theme. This will remove any issues of the Bartik CSS mixing with the JS frontend CSS. +* Turn off Bartik and enable Stark as your frontend theme. This will remove any issues of the Bartik CSS mixing with the JS frontend CSS. * Set the `404` and `home` paths for the site to `/home`. This will make it so whenever Drupal returns a page not found it will load the JS frontend and visiting the root url the JS site will be loaded instead of a default Drupal node. diff --git a/modules/metastore/README.md b/modules/metastore/README.md new file mode 100644 index 0000000000..33e787850d --- /dev/null +++ b/modules/metastore/README.md @@ -0,0 +1,21 @@ +# Metastore {#dev_metastore} + +The metastore's job is to ingest data in JSON format, validate the JSON data against a JSON Schema, and to store the valid data. + +As we take ownership of the management of these data, the metastore also provides ways to retrieve, update and delete the data. + +## Entrypoints + +There are 3 main ways to interact with the metastore module: +- Drush commands +- Endpoints +- Services + +### Drush commands +To see the Drush commands that are made available by this module, look at the Drupal::metastore::Commands::MetastoreCommands class. + +### Endpoints +To see the endpoints that are made available by this module, check the metastore.routing.yml file. + +### Services +For development, we want to interact with the code/classes directly. Drupal's dependency injection mechanism makes it fairly easy to get a fully-built class/service injected into your own classes in your own module. To see the services available reference the metastore.services.yml file.