Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(elasticsearch): Add endpoint for getting the size of timeseries indices #8265

Merged
merged 11 commits into from
Jun 21, 2023

Conversation

iprentic
Copy link
Contributor

@iprentic iprentic commented Jun 20, 2023

Example output against quickstart:

curl -i -X POST http://localhost:8080/operations\?action\=getIndexSizes | tail -n1 | jq

{
  "value": {
    "indexSizes": [
      {
        "aspectName": "datahubingestionrunsummary",
        "sizeMb": 0.208,
        "indexName": "datajob_datahubingestionrunsummaryaspect_v1",
        "entityName": "datajob"
      },
      {
        "aspectName": "dashboardusagestatistics",
        "sizeMb": 0.208,
        "indexName": "dashboard_dashboardusagestatisticsaspect_v1",
        "entityName": "dashboard"
      },
      {
        "aspectName": "chartusagestatistics",
        "sizeMb": 0.208,
        "indexName": "chart_chartusagestatisticsaspect_v1",
        "entityName": "chart"
      },
      {
        "aspectName": "datasetprofile",
        "sizeMb": 0.208,
        "indexName": "dataset_datasetprofileaspect_v1",
        "entityName": "dataset"
      },
      {
        "aspectName": "datahubingestioncheckpoint",
        "sizeMb": 0.208,
        "indexName": "datajob_datahubingestioncheckpointaspect_v1",
        "entityName": "datajob"
      },
      {
        "aspectName": "assertionrunevent",
        "sizeMb": 0.208,
        "indexName": "assertion_assertionruneventaspect_v1",
        "entityName": "assertion"
      },
      {
        "aspectName": "dataprocessinstancerunevent",
        "sizeMb": 0.208,
        "indexName": "dataprocessinstance_dataprocessinstanceruneventaspect_v1",
        "entityName": "dataprocessinstance"
      },
      {
        "aspectName": "operation",
        "sizeMb": 0.208,
        "indexName": "dataset_operationaspect_v1",
        "entityName": "dataset"
      },
      {
        "aspectName": "datasetusagestatistics",
        "sizeMb": 0.208,
        "indexName": "dataset_datasetusagestatisticsaspect_v1",
        "entityName": "dataset"
      }
    ]
  }
}

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the product PR or Issue related to the DataHub UI/UX label Jun 20, 2023
@RequiredArgsConstructor
public class GetIndexSizesResolver implements DataFetcher<CompletableFuture<GetIndexSizesResult>> {

private final TimeseriesAspectService _timeseriesAspectService;
Copy link
Collaborator

@RyanHolstien RyanHolstien Jun 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we only interested in TimeSeries? I think for the operational endpoint it would be useful to support both. Maybe TimeSeries is enough for just the MVP though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This endpoint is specifically for timeseries indices, yes

@iprentic iprentic enabled auto-merge (squash) June 21, 2023 16:37
@iprentic iprentic merged commit 3a0b2ed into master Jun 21, 2023
47 of 48 checks passed
@iprentic iprentic deleted the nd-size-endpoint branch June 21, 2023 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product PR or Issue related to the DataHub UI/UX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants