Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lookup module not registered with overlord #5727

Closed
2PacIsAlive opened this issue May 1, 2018 · 9 comments
Closed

Lookup module not registered with overlord #5727

2PacIsAlive opened this issue May 1, 2018 · 9 comments
Labels
Area - Batch Ingestion Feature stale

Comments

@2PacIsAlive
Copy link

2PacIsAlive commented May 1, 2018 •
edited
Loading

Currently, the lookup module is only registered with peons, historicals, and brokers.

This makes it impossible to use lookups in a reindexing task filter.

Here is an example reindexing task spec that showcases the failure:

{
  "type": "index_hadoop",
  "spec": {
    "dataSchema": {
      "dataSource": "REDACTED",
      "ignoreWhenNoSegments": true,
      "parser": {
        "type": "string",
        "parseSpec": {
          "format": "json",
          "timestampSpec": {
            "column": "timestamp",
            "format": "auto"
          },
          "dimensionsSpec": {
            "dimensions": [
              "example",
              "timestamp"
            ],
            "dimensionExclusions": [],
            "spatialDimensions": []
          }
        }
      },
      "metricsSpec": [
        {
          "type": "count",
          "name": "count"
        }
      ],
      "granularitySpec": {
        "segmentGranularity": "DAY",
        "queryGranularity": "NONE",
        "intervals": [
          "2017-04-22T16:52:31.373Z/2018-04-23T14:52:31.373Z"
        ]
      }
    },
    "ioConfig": {
      "type": "hadoop",
      "inputSpec": {
        "type": "dataSource",
        "ingestionSpec": {
          "dataSource": "REDACTED",
          "intervals": [
            "2017-04-22T16:52:31.373Z/2018-04-23T14:52:31.373Z"
          ],
          "filter": {
            "type": "or",
            "fields": [
              {
                "type": "selector",
                "dimension": "example",
                "value": "example_lookup_value",
                "extractionFn": {
                  "type": "registeredLookup",
                  "lookup": "example_lookup",
                  "retainMissingValue": true
                }
              },
              {
                "type": "not",
                "field": {
                  "type": "or",
                  "fields": [
                    {
                      "type": "selector",
                      "dimension": "example",
                      "value": "123"
                    },
                    {
                      "type": "selector",
                      "dimension": "example",
                      "value": "456"
                    }
                  ]
                }
              }
            ]
          }
        },
        "dataSource": "REDACTED"
      }
    },
    "tuningConfig": {
      "type": "hadoop",
      "jobProperties": {
        "fs.s3.awsAccessKeyId": "REDACTED",
        "fs.s3.awsSecretAccessKey": "REDACTED",
        "fs.s3n.awsAccessKeyId": "REDACTED",
        "fs.s3n.awsSecretAccessKey": "REDACTED",
        "fs.s3.impl": "org.apache.hadoop.fs.s3native.NativeS3FileSystem",
        "fs.s3n.impl": "org.apache.hadoop.fs.s3native.NativeS3FileSystem",
        "mapreduce.job.classloader": "true",
        "mapreduce.job.classloader.system.classes": "-javax.validation.,java.,javax.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop."
      },
      "forceExtendableShardSpecs": true
    }
  },
  "hadoopDependencyCoordinates": [
    "org.apache.hadoop:hadoop-client:2.7.1"
  ]
}

This task will produce the following error:

2018-04-23T21:29:51,442 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_REDACTED_2018-04-23T21:29:43.610Z, type=index_hadoop, dataSource=REDACTED}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:211) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:176) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_162]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_162]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_162]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:208) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	... 7 more
Caused by: java.lang.IllegalArgumentException: Guice configuration errors:

1) Could not find a suitable constructor in io.druid.query.lookup.LookupConfig. Classes must have either one (and only one) constructor annotated with @Inject or a zero-argument constructor that is not private.
  at io.druid.query.lookup.LookupConfig.class(LookupConfig.java:39)
  while locating io.druid.query.lookup.LookupConfig
    for the 1st parameter of io.druid.query.lookup.LookupReferencesManager.<init>(LookupReferencesManager.java:69)
  while locating io.druid.query.lookup.LookupReferencesManager

1 error (through reference chain: java.util.ArrayList[0])
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2774) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700) ~[jackson-databind-2.4.6.jar:2.4.6]
	at io.druid.indexer.HadoopDruidIndexerConfig.<init>(HadoopDruidIndexerConfig.java:228) ~[druid-indexing-hadoop-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexer.HadoopDruidIndexerConfig.fromSpec(HadoopDruidIndexerConfig.java:130) ~[druid-indexing-hadoop-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessing.runTask(HadoopIndexTask.java:297) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:208) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	... 7 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Guice configuration errors:

1) Could not find a suitable constructor in io.druid.query.lookup.LookupConfig. Classes must have either one (and only one) constructor annotated with @Inject or a zero-argument constructor that is not private.
  at io.druid.query.lookup.LookupConfig.class(LookupConfig.java:39)
  while locating io.druid.query.lookup.LookupConfig
    for the 1st parameter of io.druid.query.lookup.LookupReferencesManager.<init>(LookupReferencesManager.java:69)
  while locating io.druid.query.lookup.LookupReferencesManager

1 error (through reference chain: java.util.ArrayList[0])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:189) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:249) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:206) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:536) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700) ~[jackson-databind-2.4.6.jar:2.4.6]
	at io.druid.indexer.HadoopDruidIndexerConfig.<init>(HadoopDruidIndexerConfig.java:228) ~[druid-indexing-hadoop-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexer.HadoopDruidIndexerConfig.fromSpec(HadoopDruidIndexerConfig.java:130) ~[druid-indexing-hadoop-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessing.runTask(HadoopIndexTask.java:297) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:208) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	... 7 more
Caused by: com.google.inject.ConfigurationException: Guice configuration errors:

1) Could not find a suitable constructor in io.druid.query.lookup.LookupConfig. Classes must have either one (and only one) constructor annotated with @Inject or a zero-argument constructor that is not private.
  at io.druid.query.lookup.LookupConfig.class(LookupConfig.java:39)
  while locating io.druid.query.lookup.LookupConfig
    for the 1st parameter of io.druid.query.lookup.LookupReferencesManager.<init>(LookupReferencesManager.java:69)
  while locating io.druid.query.lookup.LookupReferencesManager

1 error
	at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1045) ~[guice-4.1.0.jar:?]
	at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050) ~[guice-4.1.0.jar:?]
	at io.druid.guice.GuiceInjectableValues.findInjectableValue(GuiceInjectableValues.java:48) ~[druid-common-0.10.0-iap5.jar:0.10.0-iap5]
	at com.fasterxml.jackson.databind.DeserializationContext.findInjectableValue(DeserializationContext.java:293) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.impl.PropertyValueBuffer.inject(PropertyValueBuffer.java:66) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.startBuilding(PropertyBasedCreator.java:160) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:331) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:536) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:234) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:206) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:536) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:156) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:126) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769) ~[jackson-databind-2.4.6.jar:2.4.6]
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700) ~[jackson-databind-2.4.6.jar:2.4.6]
	at io.druid.indexer.HadoopDruidIndexerConfig.<init>(HadoopDruidIndexerConfig.java:228) ~[druid-indexing-hadoop-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexer.HadoopDruidIndexerConfig.fromSpec(HadoopDruidIndexerConfig.java:130) ~[druid-indexing-hadoop-0.10.0-iap5.jar:0.10.0-iap5]
	at io.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessing.runTask(HadoopIndexTask.java:297) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:208) ~[druid-indexing-service-0.10.0-iap5.jar:0.10.0-iap5]
	... 7 more
2018-04-23T21:29:51,450 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_hadoop_REDACTED_2018-04-23T21:29:43.610Z] status changed to [FAILED].
2018-04-23T21:29:51,453 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
  "id" : "index_hadoop_REDACTED_2018-04-23T21:29:43.610Z",
  "status" : "FAILED",
  "duration" : 2838
}

It might be sufficient to add a line like this to the overlord startup file, but I have not tested this
@drcrallen
Copy link
Contributor

I think you are correct, would you be able to do a PR?

@2PacIsAlive
Copy link
Author

👍Sure thing I will do that this weekend

@jihoonson
Copy link
Contributor

I think this isn't a bug but a missing feature. I'll change the label.

@jihoonson jihoonson added Feature and removed Bug labels Sep 17, 2018
@scrawfor
Copy link
Contributor

I'm taking a stab at this. In my first attempt I added the new LookupModule() to the CliOverlord.java file as @2PacIsAlive suggested. This got me further, but the middle manager wouldn't run any tasks and threw this error.

com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class org.apache.druid.indexing.kafka.KafkaIndexTask] value failed: function 'lookup' is not defined.

So I added new LookupModule() to CliMiddleManager.java as well. This worked as the tasks get created and executed, and lookups are mapping data.

However, it also attempts to load lookup values on the middle manager, which throws out of memory errors

2019-02-13T13:24:56,227 ERROR [NamespaceExtractionCacheManager-1] org.apache.druid.server.lookup.namespace.cache.CacheScheduler - Failed to update namespace [JdbcExtractionNamespace{connectorConfig=DbConnectorConfig{cr
eateTables=true, connectURI='Redacted', user='Redacted', passwordProvider=org.apache.druid.metadata.DefaultPasswordProvider}, table='Redacted', keyC
olumn='Redacted', valueColumn='Redacted', tsColumn='null', filter='null', pollPeriod=P1D}] : org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl@6fc24ef7
org.skife.jdbi.v2.exceptions.CallbackFailedException: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: org.postgresql.util.PSQLException: Ran out of memory retrieving query results. [statement:"Redacted", located:"Redacted", rewritten:"Redacted", arguments:{ positional:{}, named:{}, finder:[]}]
        at org.apache.druid.server.lookup.namespace.JdbcCacheGenerator.generateCache(JdbcCacheGenerator.java:77) ~[druid-lookups-cached-global-0.13.0-incubating.jar:0.13.0-incubating]
        at org.apache.druid.server.lookup.namespace.JdbcCacheGenerator.generateCache(JdbcCacheGenerator.java:49) ~[druid-lookups-cached-global-0.13.0-incubating.jar:0.13.0-incubating]
        at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl.tryUpdateCache(CacheScheduler.java:228) [druid-lookups-cached-global-0.13.0-incubating.jar:0.13.0-incubating]
        at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl.updateCache(CacheScheduler.java:207) [druid-lookups-cached-global-0.13.0-incubating.jar:0.13.0-incubating]
        at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl.access$600(CacheScheduler.java:143) [druid-lookups-cached-global-0.13.0-incubating.jar:0.13.0-incubating]
        at org.apache.druid.server.lookup.namespace.cache.CacheScheduler$EntryImpl$2.run(CacheScheduler.java:189) [druid-lookups-cached-global-0.13.0-incubating.jar:0.13.0-incubating]
Caused by: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: org.postgresql.util.PSQLException: Ran out of memory retrieving query results. [statement:"Redacted", located:"Redacted", rewritten:"Redacted", arguments:{ posi
tional:{}, named:{}, finder:[]}]
        at org.apache.druid.server.lookup.namespace.JdbcCacheGenerator$1.withHandle(JdbcCacheGenerator.java:99) ~[?:?]
        at org.apache.druid.server.lookup.namespace.JdbcCacheGenerator$1.withHandle(JdbcCacheGenerator.java:79) ~[?:?]
        at org.apache.druid.server.lookup.namespace.JdbcCacheGenerator$1.withHandle(JdbcCacheGenerator.java:99) ~[?:?]
        at org.apache.druid.server.lookup.namespace.JdbcCacheGenerator$1.withHandle(JdbcCacheGenerator.java:79) ~[?:?]

Does anyone with a better understanding of the architecture have any ideas on how we could get the MiddleManager to recognize the lookup syntax without attempting to load them? We can't set druid.lookup.enableLookupSyncOnStartup=false because the middle manager will pass that property to the peon right?

For reference, my Indexing Task Spec:

{
  "type": "kafka",
   "dataSchema": {
     "dataSource": "REDACTED",
     "parser": {
       "type": "string",
       "parseSpec": {
         "format": "json",
         "timestampSpec": {
           "column": "timestamp",
           "format": "iso"
         },    
         "dimensionsSpec": {
           "dimensions": [
             "user_type",
             "client_id",
           ]
         }
       }
     },
     "metricsSpec": [
       {
         "type": "hyperUnique",
         "name": "unique_views",
         "fieldName": "view_id"
       },
     ],
     "transformSpec": {
       "transforms": [
         {
           "type": "expression",
           "name": "user_type",
           "expression": "lookup(client_id, 'user_type_lookup')"
         },
       ],
       "filter": {
         "type": "and",
         "fields": [
           {
             "type": "not",
             "field": {
               "type": "in",
               "dimension": "profile",
               "values": ["REDACTED"]
             }
           }          
         ]
       }
     },
     "granularitySpec": {
       "type": "uniform",
       "segmentGranularity": "DAY",
       "queryGranularity": "NONE",
       "rollup": true
     }
   },
  "tuningConfig": {
    "type": "kafka",
    "maxRowsInMemory": 5000,
    "maxRowsPerSegment": 5000000,
    "intermediatePersistPeriod": "PT2M",
    "resetOffsetAutomatically": true
  },
  "ioConfig": {
    "topic": "REDACTED",
    "consumerProperties": {
      "bootstrap.servers": "REDACTED"
    },
    "useEarliestOffset": false,
    "skipOffsetGaps": true
  }
}

@jihoonson
Copy link
Contributor

How about setting druid.lookup.enableLookupSyncOnStartup = false for MMs but adding -Ddruid.lookup.enableLookupSyncOnStartup=true to druid.indexer.runner.javaOpts for tasks? Does it work?

@scrawfor
Copy link
Contributor

scrawfor commented Feb 14, 2019 •
edited
Loading

Unfortunately it doesn't. It prevents lookups from loading on both the middle manager and the peons. The option is passed, but it's passed first, and then overridden by the configuration.

Job Log:

[LookupConfig{snapshotWorkingDir='', enableLookupSyncOnStartup=false, numLookupLoadingThreads=4, coordinatorFetchRetries=3, lookupStartRetries=3, coordinatorRetryDelay=60000}]
...
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Ddruid.lookup.enableLookupSyncOnStartup=true
...
2019-02-14T14:14:37,380 INFO [main] org.apache.druid.cli.CliPeon - * druid.lookup.enableLookupSyncOnStartup: false  
...

@scrawfor scrawfor mentioned this issue Feb 15, 2019
Closed
@scrawfor
Copy link
Contributor

scrawfor commented Feb 15, 2019 •
edited
Loading

I found the magic settings to prevent the lookups from loading on the middle manager.

# Disable lookups on middleManager, but enable on peon.
druid.lookup.enableLookupSyncOnStartup=false
druid.indexer.fork.property.druid.lookup.enableLookupSyncOnStartup=true

Created #7082

@clintropolis clintropolis mentioned this issue Mar 12, 2019
Merged
@FaxianZhao FaxianZhao mentioned this issue Mar 20, 2019
Closed
@clintropolis clintropolis mentioned this issue May 22, 2019
Closed
@stale
Copy link

stale bot commented Feb 11, 2020

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

@stale stale bot added the stale label Feb 11, 2020
@stale
Copy link

stale bot commented Mar 10, 2020

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

@stale stale bot closed this as completed Mar 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area - Batch Ingestion Feature stale
Projects
None yet
Milestone
No milestone
4 participants
@jihoonson @2PacIsAlive @scrawfor @drcrallen