New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
_reindex from remote and its bugs #22027
Comments
Also, docs are way far from usable: "source": {
"index": "metricbeat-*"
},
"dest": {
"index": "metricbeat"
},
"script": {
"lang": "painless",
"inline": "ctx._index = 'metricbeat-' + (ctx._index.substring('metricbeat-'.length(), ctx._index.length())) + '-1'"
} You're assigning index to itself, basically. this wont' work. maybe you meant update: |
Did you try it ? I just did and it works like a charm. It basically does what you're asking below:
Yes it's possible with what you call "dirty scripting". Sorry if you don't like scripts but that's the way it works.
Yes it doesn't work if you invent new syntax, what works is what's documented. I'll ignore the last part of your comment since I understand your frustration but please note that being aggressive doesn't solve anything ;) Regarding the hang, I can reproduce if the source node is not responding. I tested with the source node down and in that case the reindex blocks and there is no way to access the hanging task in the destination node. |
.tasks / _tasks (why 2 of them?) API is unusable, reread my comment -- it just doesn't work, even for tasks that completed fine. As regarding the main problem, yes, I tried specifying the script exactly as per doc. It doesn't work, it's trying to log to "metricbeat" , not to "metricbeat-2016-12-07". I can provide access to our cluster so that you can try it. |
@celesteking to echo @jimczi's comment - this isn't your first issue where you get really aggressive. Seriously, instead of telling us how shit it all is, just point out the problems you're having and we'll try to help you. The aggression just makes us want to look at the other 1,000 open issue instead of yours.
|
According to REST guidelines, you should've used I will stop logging issues from now on (or helping in any way) and will probably switch over another tool for log storage. This thing is not production ready. Period. |
@jimczi, do you know if the reindex was running on the source node? I can imagine a situation where you start a reindex, and then shoot the node that the reindex was running on before it finishes. The task get action notices that the node is no longer running and looks in the tasks index. If it doesn't find the task it then it reports that error message. And it won't find it because the reindex didn't complete before the node left. I think I need to add a better error message to that. As to the question of multi-source, multi-destination: this comes a fair bit but I think the script solution is fine. The reason you might not want to do this at all is that you probably want to manage the process of creating each of the sub-indexes so you have progress and an easy way to pick up where you left off and things like that. If you do want to do it you can use the script. It tested on every build so it is going to work. |
@jimczi and I talked - trying to reindex from remote from a node that refuses the connection indeed hangs. The reindex process is still in the tasks API and can be found with |
If you try to close the rest client inside one of its callbacks then it blocks itself. The thread pool switches the status to one that requests a shutdown and then waits for the pool to shutdown. When another thread attempts to honor the shutdown request it waits for all the threads in the pool to finish what they are working on. Thus thread a is waiting on thread b while thread b is waiting on thread a. It isn't quite that simple, but it is close. Relates to elastic#22027
Improves the error message returned when looking up a task that belongs to a node that is no longer part of the cluster. The new error message tells the user that the node isn't part of the cluster. This is useful because if you start a task and the node goes down there isn't a record of the task at all. This hints to the user that the task might have died with the node. Relates to elastic#22027
If you try to close the rest client inside one of its callbacks then it blocks itself. The thread pool switches the status to one that requests a shutdown and then waits for the pool to shutdown. When another thread attempts to honor the shutdown request it waits for all the threads in the pool to finish what they are working on. Thus thread a is waiting on thread b while thread b is waiting on thread a. It isn't quite that simple, but it is close. Relates to #22027
If you try to close the rest client inside one of its callbacks then it blocks itself. The thread pool switches the status to one that requests a shutdown and then waits for the pool to shutdown. When another thread attempts to honor the shutdown request it waits for all the threads in the pool to finish what they are working on. Thus thread a is waiting on thread b while thread b is waiting on thread a. It isn't quite that simple, but it is close. Relates to #22027
If you try to close the rest client inside one of its callbacks then it blocks itself. The thread pool switches the status to one that requests a shutdown and then waits for the pool to shutdown. When another thread attempts to honor the shutdown request it waits for all the threads in the pool to finish what they are working on. Thus thread a is waiting on thread b while thread b is waiting on thread a. It isn't quite that simple, but it is close. Relates to #22027
Improves the error message returned when looking up a task that belongs to a node that is no longer part of the cluster. The new error message tells the user that the node isn't part of the cluster. This is useful because if you start a task and the node goes down there isn't a record of the task at all. This hints to the user that the task might have died with the node. Relates to #22027
Improves the error message returned when looking up a task that belongs to a node that is no longer part of the cluster. The new error message tells the user that the node isn't part of the cluster. This is useful because if you start a task and the node goes down there isn't a record of the task at all. This hints to the user that the task might have died with the node. Relates to #22027
I merged #22062 just now to improve the error message when the node isn't part of the cluster any more.
I merged #22061 this morning to fix the hang. |
I'm trying to mass-reindex from remote. As elasticdump is "unsupported", I was trying to use so-called "reindex from remote" feature. It failed in following aspekts:
index: blah-*-2016-*
. It looks like this isn't possible without some dirty scripting. It would be wonderful if all this would happen under the hood.Next, I was trying to use the documented API.
It should've been found as per docs.
It's also unclear what's going on, was the task stalled, hung, connecting? Why is it taking so long without any movement? I'm on v5.0.2. Thanks.
The text was updated successfully, but these errors were encountered: