Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flood stage exceeded is incorrectly logged #26201

Closed
PhaedrusTheGreek opened this issue Aug 14, 2017 · 3 comments · Fixed by #26204
Closed

Flood stage exceeded is incorrectly logged #26201

PhaedrusTheGreek opened this issue Aug 14, 2017 · 3 comments · Fixed by #26204

Comments

@PhaedrusTheGreek
Copy link
Contributor

PhaedrusTheGreek commented Aug 14, 2017

Tested on

  • ES 6.0.0-beta1
  • Mac OSX Sierra

Node is reporting flood stage exceeded:

[2017-08-14T09:29:37,167][WARN ][o.e.c.r.a.DiskThresholdMonitor] [Lq-N3tZ] flood stage disk watermark [95%] exceeded on [Lq-N3tZ8QY6B648xAA8H0g][Lq-N3tZ][/Users/jay/Workspace/elasticsearch-6.0.0-beta1/data/nodes/0] free: 11.1gb[9.9%], all indices on this node will marked read-only

But the math doesn't add up (flood stage is 95% and there's 9.9% free disk space

And indexing is still allowed, even on indices created before the flood stage log message appears.

{
    "_index": "foo",
    "_type": "bar",
    "_id": "7",
    "_version": 1,
    "result": "created",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 0,
    "_primary_term": 1
}
@PhaedrusTheGreek PhaedrusTheGreek changed the title writing can continue after flood stage is exceeded Flood stage exceeded is incorrectly logged Aug 14, 2017
@jasontedor
Copy link
Member

Is this a clean install, are you sure that you did not set the flood state setting to a disk space free instead of a disk percentage free? Can you show the output of GET /_cluster/settings?include_defaults=true&filter_path=**.flood_stage&pretty=true and GET /_nodes/settings?filter_path=**.flood_stage&pretty=true?

@PhaedrusTheGreek
Copy link
Contributor Author

Seems that this line is comparing with the high watermark accidentally

@jasontedor
Copy link
Member

You're right: #26204

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants