Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indices not following rotation policy #3268

Closed
freebsdsa opened this issue Dec 28, 2016 · 7 comments
Closed

Indices not following rotation policy #3268

freebsdsa opened this issue Dec 28, 2016 · 7 comments

Comments

@freebsdsa
Copy link

I am testing Graylog 2.2 Beta and am using a stream to pull certain message types into its own ES indices so that we can have a different retention policy. I noticed that the new indices are not rotating per day as I have set it up to do, but the default ones rotate just fine.

Both indices show the same setup index time and P1D.

  • Graylog Version: 2.2.0 beta 2
  • Elasticsearch Version: 2.4.1
  • MongoDB Version: 3.2.11
  • Operating System: Centos 7.2
@bernd
Copy link
Member

bernd commented Dec 30, 2016

Thanks for testing the beta release! We will take a look at this asap.

@bernd bernd self-assigned this Jan 2, 2017
@bernd bernd added the to-verify label Jan 2, 2017
@bernd bernd added this to the 2.2.0 milestone Jan 2, 2017
@bernd
Copy link
Member

bernd commented Jan 2, 2017

@freebsdsa I am unable to reproduce this with the current beta.3 snapshot. Can you please send us your Graylog server log and the output of the following command executed on your server?

curl -u admin:<your-admin-password> 127.0.0.1:9000/api/system/indices/index_sets?pretty=true

Thanks!

@freebsdsa
Copy link
Author

freebsdsa commented Jan 3, 2017

Here is the output from that command.

{
  "total" : 2,
  "index_sets" : [ {
    "id" : "585195e6fbc89a3d0a58fed6",
    "title" : "Default index set",
    "description" : "The Graylog default index set",
    "index_prefix" : "graylog",
    "shards" : 18,
    "replicas" : 0,
    "rotation_strategy_class" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy",
    "rotation_strategy" : {
      "type" : "org.graylog2.indexer.rotation.strategies.AutoValue_TimeBasedRotationStrategyConfig",
      "type" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig",
      "rotation_period" : "P1D"
    },
    "retention_strategy_class" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy",
    "retention_strategy" : {
      "type" : "org.graylog2.indexer.retention.strategies.AutoValue_DeletionRetentionStrategyConfig",
      "type" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig",
      "max_number_of_indices" : 365
    },
    "creation_date" : "2016-12-14T18:56:38.916Z",
    "writable" : true,
    "default" : true
  }, {
    "id" : "58543f87fbc89a24547bf1ad",
    "title" : "Event Logs",
    "description" : "Gateway Event Logs",
    "index_prefix" : "eventlog",
    "shards" : 18,
    "replicas" : 0,
    "rotation_strategy_class" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy",
    "rotation_strategy" : {
      "type" : "org.graylog2.indexer.rotation.strategies.AutoValue_TimeBasedRotationStrategyConfig",
      "type" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig",
      "rotation_period" : "P1D"
    },
    "retention_strategy_class" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy",
    "retention_strategy" : {
      "type" : "org.graylog2.indexer.retention.strategies.AutoValue_DeletionRetentionStrategyConfig",
      "type" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig",
      "max_number_of_indices" : 300
    },
    "creation_date" : "2016-12-16T19:24:55.83Z",
    "writable" : true,
    "default" : false
  } ]
}

I also attached the full server log.
server.log.zip

@bernd
Copy link
Member

bernd commented Jan 4, 2017

@freebsdsa Thanks for the logs and the index set data!

The index set configuration looks good but I can't find anything in the logs that indicates an error with the index rotation. Since I am unable to reproduce this with the current master, please update to beta.3 or wait for beta.4 which will be release this week.

Please let us know if it works once you updated, thanks! I will leave this ticket open.

@tokred
Copy link

tokred commented Jan 4, 2017

I can confirm this issue on beta.3!

I have created two additional index sets which do not get rotated:

{
  "total" : 3,
  "index_sets" : [ {
    "id" : "586502faf4db16270516cdc5",
    "title" : "Default index set",
    "description" : "The Graylog default index set",
    "index_prefix" : "graylog",
    "shards" : 4,
    "replicas" : 1,
    "rotation_strategy_class" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy",
    "rotation_strategy" : {
      "type" : "org.graylog2.indexer.rotation.strategies.AutoValue_TimeBasedRotationStrategyConfig",
      "type" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig",
      "rotation_period" : "P1D"
    },
    "retention_strategy_class" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy",
    "retention_strategy" : {
      "type" : "org.graylog2.indexer.retention.strategies.AutoValue_DeletionRetentionStrategyConfig",
      "type" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig",
      "max_number_of_indices" : 30
    },
    "creation_date" : "2016-12-29T12:35:06.8Z",
    "index_analyzer" : "whitespace",
    "index_optimization_max_num_segments" : 1,
    "index_optimization_disabled" : false,
    "writable" : true,
    "default" : true
  }, {
    "id" : "58650bc7f4db16270516d76b",
    "title" : "Long Time Index Set",
    "description" : "Long Time Index Set",
    "index_prefix" : "gllong",
    "shards" : 4,
    "replicas" : 1,
    "rotation_strategy_class" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy",
    "rotation_strategy" : {
      "type" : "org.graylog2.indexer.rotation.strategies.AutoValue_TimeBasedRotationStrategyConfig",
      "type" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig",
      "rotation_period" : "P1D"
    },
    "retention_strategy_class" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy",
    "retention_strategy" : {
      "type" : "org.graylog2.indexer.retention.strategies.AutoValue_DeletionRetentionStrategyConfig",
      "type" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig",
      "max_number_of_indices" : 90
    },
    "creation_date" : "2016-12-29T13:12:39.765Z",
    "index_analyzer" : "standard",
    "index_optimization_max_num_segments" : 1,
    "index_optimization_disabled" : false,
    "writable" : true,
    "default" : false
  }, {
    "id" : "58650b01f4db16270516d695",
    "title" : "Short Time Index Set",
    "description" : "Short Time Index Set",
    "index_prefix" : "glshort",
    "shards" : 4,
    "replicas" : 1,
    "rotation_strategy_class" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategy",
    "rotation_strategy" : {
      "type" : "org.graylog2.indexer.rotation.strategies.AutoValue_TimeBasedRotationStrategyConfig",
      "type" : "org.graylog2.indexer.rotation.strategies.TimeBasedRotationStrategyConfig",
      "rotation_period" : "P1D"
    },
    "retention_strategy_class" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy",
    "retention_strategy" : {
      "type" : "org.graylog2.indexer.retention.strategies.AutoValue_DeletionRetentionStrategyConfig",
      "type" : "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig",
      "max_number_of_indices" : 7
    },
    "creation_date" : "2016-12-29T13:09:21.725Z",
    "index_analyzer" : "standard",
    "index_optimization_max_num_segments" : 1,
    "index_optimization_disabled" : false,
    "writable" : true,
    "default" : false
  } ]
}

Please advise on how I can provide additional info in order to support.

Best regards

@joschi joschi self-assigned this Jan 5, 2017
@joschi
Copy link
Contributor

joschi commented Jan 5, 2017

I couldn't reproduce this on current master either.

@freebsdsa @tokred Make sure that you've assigned the correct index sets to your streams and retry with the latest beta.

Graylog 2.2.0-beta.4 should be released today. If the problem still exists there for you, please add a comment in this issue. Until then, I'll close this as neither @bernd nor I could reproduce it.

@joschi
Copy link
Contributor

joschi commented Jan 5, 2017

Aaand open again. Looks like this is a problem with TimeBasedRotationStrategy specifically.

We'll try to fix it for the next beta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants