Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Get and GetStats should list transforms that have a task but no config #80955

Closed
hendrikmuhs opened this issue Nov 23, 2021 · 2 comments
Assignees
Labels
>enhancement :ml/Transform Transform Team:ML Meta label for the ML team

Comments

@hendrikmuhs
Copy link
Contributor

If a transform has no config document but still has a task it becomes invisible, both GET and _stats do not list the them. This situation can happen if the documents got deleted manually or the whole transform internal index got deleted. System indices will prevent this to some extent, however it would be nice to handle such a corner case.

It is possible to force delete or stop an invisible transform, but you need to know they exist and you need the name.

We should improve the above and list invisible transforms in GET and _stats (if not in both, at least in 1 of them.)

Requirements:

  • ensure a invisible transform is listed after creation of the transform and deletion of its config (hint: we have an integration test for this for delete/stop)
  • the invisible transform should not be listed next to good ones, but somehow marked as broken (HLRC lists "invalid_transforms", maybe something similar)
  • ensure interaction in the transform UI keeps working, it should be possible to delete the transform, but not start/update it (might require kibana changes, create issue once we have a design)

Hint:

  • TransformNodes.findPersistentTasks as used for _stop
@elasticmachine elasticmachine added the Team:ML Meta label for the ML team label Nov 23, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@przemekwitek
Copy link
Contributor

I consider this done from a backend perspective.

We should improve the above and list invisible transforms in GET and _stats (if not in both, at least in 1 of them.)

It's done for GET but not for _stats.

The example response from GET when there exists a transform task without config is:

{
  "count" : 0,
  "transforms" : [ ],
  "errors" : [
    {
      "type" : "dangling_task",
      "reason" : "Found task for transform [cont-t-1], but no configuration for it. To delete this transform use DELETE with force=true."
    }
  ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :ml/Transform Transform Team:ML Meta label for the ML team
Projects
None yet
Development

No branches or pull requests

3 participants