Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model Monitoring] Add TDEngine Connector #5592

Merged
merged 24 commits into from
May 22, 2024

Conversation

Eyal-Danieli
Copy link
Member

@Eyal-Danieli Eyal-Danieli commented May 19, 2024

Following #5210 , this PR implements TDEngine connector as a supported TSDB solution for storing model monitoring real time data.

How to define a new tdengine connection
To define a new TDEngine connection, the user can pass tsdb_connection through project.set_model_monitoring_credentials(tsdb_connection="taosws://...").

Using one of the above suggested approaches, the TDEngine connection will be stored as a k8s secret under the project and the relevant workers (stream, writer).

How the TDEngine is implemented

  • The new connector is defined under mlrun/model_monitoring/db/tsdb/tdengine/tdengine_connector.py. The main connector object called TDEngineConnector and it implements the required functionality as defined in the connector class TSDBConnector.
  • The implementation is based on the recommended TDEngine websocket connection. We used the TDEngine official connector taosws.
  • The supertables schemas and the queries are defined under mlrun/model_monitoring/db/tsdb/tdengine/schemas.py. Each supertable should include a name, columns, tags, and a database. At the moment we have 3 different tables under the same default database - mlrun_model_monitoring. The schema design of each supertable is described below.
  • The supertables are created on the 1st api call to establish the model monitoring infrastructure. Note that it only happens once because the supertables are similar accross different projects.

tdengine_predictions_170

tdengine_app_result_170_V2

tdengine_metrics_170_V2

A related JIRA: https://iguazio.atlassian.net/browse/ML-6261

@Eyal-Danieli Eyal-Danieli marked this pull request as draft May 19, 2024 15:16
@Eyal-Danieli Eyal-Danieli marked this pull request as ready for review May 20, 2024 07:23
jond01
jond01 previously requested changes May 20, 2024
Copy link
Member

@jond01 jond01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because of the confusing defaults

mlrun/model_monitoring/db/tsdb/tdengine/schemas.py Outdated Show resolved Hide resolved
dependencies.py Outdated Show resolved Hide resolved
@jond01 jond01 dismissed their stale review May 21, 2024 06:41

The changes were made

mlrun/projects/project.py Outdated Show resolved Hide resolved
@assaf758 assaf758 merged commit 2e01eb2 into mlrun:development May 22, 2024
12 checks passed
rokatyy pushed a commit to rokatyy/mlrun that referenced this pull request May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants