Filter datasets included in data.json

[dafeder]

CIVIC-3430

User Story

As a site manager, I need to filter the datasets included in data.json, so that I can have dataset content on my site that does not get federated to data.gov

Desctiption

Many data portals publish datasets harvested from multiple state and city organizations. However, these datasets should not be included in the data.json feed that gets reported to data.gov.

Add a page to the admin UI that lets a site manager toggle groups from appearing in the data.json. If a dataset is not in the list of groups defined in the settings, do not add it to the data.json file.

An alternate strategy would be to filter or exclude by arbitrary POD value pairs (for instance, to include only datasets with a certain bureauCode or publisher.name).

• Decide on one strategy or another before beginning work

The available filtering options should allow for an easy way to QA that the correct datasets are being included in the data.json

Acceptance Criteria

• Site managers have a UI where they can filter the datasets included in the data.json feed by
  group/publisher
• Admins have a UI where they can see
  the total number of datasets on the site
  the number of datasets marked to be included in the data.json feed
• The data.json feed reflects the correct datasets from filters

[dafeder]

We have some working code for this in another repo that would fufill the first option described above (organic groups-based filtering). For now we're going to clean that up and move it into DKAN repo to complete this story - we may revisit later if we see a need for finer-grained filtering.

[dafeder]

(Filtering on data.json schema presents a number of problems as well, might require rendering all datasets and then removing filtered ones, which would not be very efficient)

[dafeder]

Here is a gist that sort of anonymizes the custom code we did for a client: https://gist.github.com/dafeder/a4f036e717a63a05cb5a05787db31d14

[kducharm]

This ticket will be used to re-build DKAN once open_data_schema_map module is updated, ticket: https://waffle.io/NuCivic/dkan_management/cards/59ce9dfa45643200124b88a1 @kimwdavidson