Host a collection of example datasets.xml snippets somewhere?

[MathewBiddle]

Sometimes I get questions about how to host a specific type of data in ERDDAP and I typically respond with an example datasets.xml snippet.

I'm wondering if we could collate a collection of dataset.xml snippets that serve as examples for how to serve various EDD types.

The one I'm working on now is sharing media files and how it can require two datasets to be hosted. One via EDDTableFromFileNames and one EDDTableFromAscii. It would be nice to have a reference I could point people to.

maybe adding to https://github.com/ERDDAP/erddapContent/blob/main/content/erddap/datasets.xml is an approach? Or just adding more to the docs? Or, something exists already?

[rmendels]

@MathewBiddle I think that is a great idea and what Chris has started to try to do at

https://erddap.github.io/docs/category/admin-tips

Note also there is a similar section for Users.

[MathewBiddle]

Fantastic! I'll check out how to add a subpage.

Putting this here so I don't loose it.

How did PMEL share these image data https://data.pmel.noaa.gov/pmel/erddap/tabledap/northpole_webcam_full.html??

Matt's understanding: They are using ERDDAP's ability to host media files. They serve all the images as a dataset using EDDTableFromFileNames (this dataset), then they created these csv files to generate the northpole_webcam_full dataset, which includes the additional data and url references to the source images, using the fileAccessBaseUrl and fileAccessSuffix to build the url attributes in datasets.xml.

[ChrisJohnNOAA]

The pages in admin-tips are from markdown files here:
https://github.com/ERDDAP/erddap.github.io/tree/main/docs/server-admin/admin-tips

If we want to have a catalog of dataset examples we could also make a new directory.

There's also a lot of information in the datasets page (https://github.com/ERDDAP/erddap.github.io/blob/main/docs/server-admin/datasets.md) but that is large and I don't want to add examples to that (I'd rather break that down to smaller pages).

[srstsavage]

+1 to having a new directory specifically for dataset fragment examples, and also +1 to the general idea of having a library of dataset fragment examples

[rmendels]

@MathewBiddle I also wrote to the ERDDAP™ Google Group awhile back as to how we are sharing the FED habitat videos:

https://coastwatch.pfeg.noaa.gov/erddap/search/index.html?page=1&itemsPerPage=1000&searchFor=HAGE

I can probably dig that up again.

[MathewBiddle]

Just the datasets.xml snippet for those two datasets would be great! It helps folks configuring something new if there is an example to work with.

[rmendels]

@ChrisJohnNOAA When you set it up how you want, and with whatever structure you want, I can add the HAGE video example. The question is if you want to set it up so that there is also a README to go with eh snippet to give some explanation of what it is doing and how it works.

[ChrisJohnNOAA]

I think dataset examples are going to be extremely useful for admins, and should likely have a prominent space in the documentation. I'm going to propose we add a new folder under server-admin (a sibling to the admin-tips folder).

I'd like to try to standardize the examples to make it easy for folks to find what they're looking for. I think each example should be its own file. I'm going to propose a starting point template for this below. I'm guessing it's missing stuff, so please give feedback on it. In particular I think we may want more structure for the description block, but I'm not sure what yet.

---

Title of dataset example

Description of the example, including when you might want to use this approach. This should probably include dataset type, data source information (like compatible file types), and description of the data.

Set up

Description of any steps needed to set the example up (hosting data on AWS, etc...).

Example

Link to a live dataset.

XML snippet in a code block

[MathewBiddle]

If it helps, below is what I did for the ERDDAP copy demonstration webpage. Fairly similar to your proposed structure.

https://mathewbiddle.github.io/erddap_copy/introduction.html#using-erddap-to-link-to-an-existing-erddap-dataset

[ChrisJohnNOAA]

I've made a pull request with the first dataset example file.

ERDDAP/erddap.github.io#81

Comments welcome and more examples welcome.

[rmendels]

@ChrisJohnNOAA @MathewBiddle Since the first example is more of my unintelligible scrawl, I think it better that others edit or comment as needed. If I can get out from under a bunch of stuff, I will write up something about sideByside, where there are two options on how you add that child datasets, and which you use depends a lot on the structure of the datasets, and what order you aggregate in matters. For example, if each child dataset is itself aggregating netcdf files, you need to do that aggregation first, while I have seen people do the opposite and that causes problems. Sort of like do I aggregate rows or columns, if that makes any sense.