Skip to content
This repository has been archived by the owner on Jan 29, 2022. It is now read-only.

FAQ

Jump to bottom
Luke Lovett edited this page Oct 13, 2015 · 9 revisions

Frequently Asked questions:

What types of shard keys are supported by the ShardChunkMongoSplitter?

The ShardChunkMongoSplitter does not support a hashed _id index, since MongoDB will instead use the unhashed _id index to create the splits, resulting in all data on one split. All other index types are supported, including hashed indexes on fields that are not _id.

java.lang.IncompatibleClassChangeError: Found class XXX, but interface was expected

There are two common causes of this Exception:

  1. The version of the Hadoop Connector that you're running isn't compatible with your version of Hadoop. Since version 1.3, the Hadoop Connector is compatible with all versions of Hadoop, but this isn't true for versions prior to 1.3. Make sure you have the right version.

  2. You're using a different version of the MongoDB Java Driver than what the connector expects. You either need to switch to the version that the connector expects (in either build.gradle or project/MongoHadoopBuild.scala, depending on the version of the connector that you have), or change the version in that file to match the one you have.

Are UUIDs supported by the connector?

As of 1.3.3, the Hadoop Connector supports UUIDs generated by the MongoDB Java Driver, or any UUID made to be compatible with the Java driver (an example in the Python driver). For more details, see JAVA-403 in the MongoDB Jira project.

I get a ClassNotFoundException/NoClassDefFoundError when using the connector. What do I do?

This Exception usually indicates that the connector or Java driver jars aren't being picked up correctly. Consult this checklist for easy debugging:

  • mongo-hadoop-core.jar and mongo-java-driver.jar are on Hadoop's CLASSPATH and Hive/Pig/Spark's CLASSPATH.

  • Application-specific jars are on Hadoop's CLASSPATH (e.g., mongo-hadoop-hive.jar) and Hive/Pig/Spark's CLASSPATH.

  • You have at least 3.0.0+ of the MongoDB Java Driver. Make sure that you have the "uber" jar (called mongo-java-driver.jar, not mongodb-driver.jar).

  • You do not have other Java driver artifacts on any of your CLASSPATHs. For example:

    • mongodb-driver.jar
    • bson.jar
    • mongodb-driver-core.jar
Clone this wiki locally