Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow raw json to be used without parsing #9

Closed
costin opened this issue Mar 27, 2013 · 7 comments
Closed

allow raw json to be used without parsing #9

costin opened this issue Mar 27, 2013 · 7 comments

Comments

@costin
Copy link
Member

costin commented Mar 27, 2013

There are cases where the raw json might be used as input or sent as output (without transforming it into an object).
This should be supported either by trying to automatically identify the time or by looking at a flag.

@ash211
Copy link
Contributor

ash211 commented Apr 9, 2013

I'd appreciate this for my use cases -- deserializing JSON in Scala is much easier than the nested MapWritable objects, especially with the JVM's type erasure.

@costin
Copy link
Member Author

costin commented Apr 9, 2013

Note: The Writable class will still be used whether you see them or not because the data needs to be serialized/deserialized by the Hadoop infrastructure

@tzolov
Copy link

tzolov commented Apr 15, 2013

I have an use case that needs such feature as well.
Can't we use org.apache.hadoop.io.Text to wrap the raw json?

@costin costin mentioned this issue Aug 16, 2013
@Downchuck
Copy link

How about using ByteWritable (seems like this proposal stalled on Text) and then in the addToIndex use scratchPad.setBytes? I've noticed on some pull requests in other bug reports that there have been design considerations stopping acceptance. Is there a particular design consideration that would get in the way of a patch here?

@Downchuck
Copy link

Any updates on this bug -- is this something that's just waiting on a pull request?

@Downchuck
Copy link

This bug seems to be stagnant -- issue #75 refers to the Pig interface, this issue is only about the MapReduce interface.

costin added a commit that referenced this issue Jan 16, 2014
add support for indexing raw json without any type conversion
currently only the M/R module has been tested properly
the data is passed as is, without any transformation
if needed, the raw document can be inspected to extra 'id' information

relates #9
fixes #75
fixes #126
costin added a commit that referenced this issue Jan 17, 2014
costin added a commit that referenced this issue Jan 17, 2014
costin added a commit that referenced this issue Jan 17, 2014
costin added a commit that referenced this issue Jan 17, 2014
costin added a commit that referenced this issue Jan 17, 2014
costin added a commit that referenced this issue Jan 17, 2014
costin added a commit that referenced this issue Jan 20, 2014
@costin
Copy link
Member Author

costin commented Feb 6, 2014

Supported in master and 1.3 M2

@costin costin closed this as completed Feb 6, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants