Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IndicesStoreIntegrationIT#testIndexCleanup fails on NFS #24453

Closed
danielmitterdorfer opened this issue May 3, 2017 · 0 comments · Fixed by #24911
Closed

IndicesStoreIntegrationIT#testIndexCleanup fails on NFS #24453

danielmitterdorfer opened this issue May 3, 2017 · 0 comments · Fixed by #24911
Assignees
@dakrone
Labels
:Core/Infra/Core Core issues without another label >test Issues or PRs that are addressing/adding tests >test-failure Triaged test failures from CI

Comments

@danielmitterdorfer
Copy link
Member

Failure link: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+nfs/233/console

Error output:

java.lang.IllegalArgumentException: [move_allocation] can't move 0, from {node_t1}{sEwZOe4QT_iXMWWYqi7v5g}{k4rxPpO9SXKeDpEIpAzpAw}{127.0.0.1}{127.0.0.1:9621}, to {node_t3}{5X9WxIQJRXGDfEet6eE3wQ}{AsBEE3NCR92Ol2ruRDpGMQ}{127.0.0.1}{127.0.0.1:9623}, since its not allowed, reason: [YES(shard has no previous failures)][YES(primary shard for this replica is already active)][YES(explicitly ignoring any disabling of allocation due to manual allocation commands via the reroute API)][YES(target node version [6.0.0-alpha1] is the same or newer than source node version [6.0.0-alpha1])][YES(the shard is not being snapshotted)][YES(node passes include/exclude/require filters)][YES(the shard does not exist on the same node)][NO(the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=1b], having less than the minimum required [1b] free space, actual free: [-1.6E10b])][YES(below shard recovery limit of outgoing: [0 < 4] incoming: [0 < 2])][YES(total shard limits are disabled: [index: -1, cluster: -1] <= 0)][YES(allocation awareness is not enabled, set cluster setting [cluster.routing.allocation.awareness.attributes] to enable it)]

the problem is:

the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=1b], having less than the minimum required [1b] free space, actual free: [-1.6E10b].

This is very likely related to #23093 (due to https://bugs.openjdk.java.net/browse/JDK-8162520).
@danielmitterdorfer danielmitterdorfer added :Core/Infra/Core Core issues without another label >test-failure Triaged test failures from CI >test Issues or PRs that are addressing/adding tests labels May 3, 2017
@dakrone dakrone mentioned this issue May 26, 2017
Merged
dakrone added a commit to dakrone/elasticsearch that referenced this issue May 26, 2017
@dakrone
Adjust available and free bytes to be non-negative on huge FSes
23fb36c 
In elastic#23093 we made a change so that total bytes for a filesystem would not be a
negative value when the total bytes were > Long.MAX_VALUE.

This fixes elastic#24453 which had a related issue where `available` and `free` bytes
could also be so large that they were negative. These will now return
`Long.MAX_VALUE` for the bytes if the JDK returns a negative value.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dakrone dakrone

Labels
:Core/Infra/Core Core issues without another label >test Issues or PRs that are addressing/adding tests >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Adjust available and free bytes to be non-negative on huge FSes dakrone/elasticsearch
2 participants
@dakrone @danielmitterdorfer