Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why is FourMcTextInputFormat not an InputFormat? #50

Closed
robmaz opened this issue Feb 25, 2021 · 1 comment
Closed

why is FourMcTextInputFormat not an InputFormat? #50

robmaz opened this issue Feb 25, 2021 · 1 comment

Comments

@robmaz
Copy link

robmaz commented Feb 25, 2021

So I try to run a streaming job with a .4mc compressed input like

> hadoop jar $hadoop_streaming_jar -libjars $hadoop_4mc_jar -input txt.4mc -inputformat com.hadoop.mapreduce.FourMcTextInputFormat -output /test/out -mapper mapper.sh -reducer reducer.sh

and get an error:

Exception in thread "main" java.lang.RuntimeException: class com.hadoop.mapreduce.FourMcTextInputFormat not org.apache.hadoop.mapred.InputFormat

But

  [4mc-2.2.0]$ grep extends java/hadoop-4mc/src/main/java/com/hadoop/mapreduce/FourMcInputFormat.java
   public abstract class FourMcInputFormat<K, V> extends FileInputFormat<K, V> {

which as per the API doc in turn extends InputFormat:

  org.apache.hadoop.mapred
  Class FileInputFormat<K,V>
  java.lang.Object
  org.apache.hadoop.mapred.FileInputFormat<K,V>
  All Implemented Interfaces:
  InputFormat<K,V>

So why does this not work? Is this not supposed to work?

@robmaz
Copy link
Author

robmaz commented Mar 3, 2021

Ok, wrong API doc entry. Apparently it's a org.apache.hadoop.mapreduce.InputFormat, not a org.apache.hadoop.mapred.InputFormat as the streaming interface expects ... I guess that makes it a duplicate of #17.

@robmaz robmaz closed this as completed Mar 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant