Skip to content
This repository was archived by the owner on Apr 5, 2022. It is now read-only.

Conversation

mminella
Copy link
Contributor

Updated pre-packaged jobs to allow a delimiter to be injected where appropriate.

  • Updated configuration of filejdbc, hdfsjdbc, hdfsmongodb, and jdbchdfs
    jobs to allow for the delimiter to be injected
  • Added BatchJobFiledDelimiterOptionMixin to provide metadata around
    new delimiter option
  • Updated jdbc_tests script to test the processing of a job using a tab
    as a delimiter.

Notes:

  1. unescape does not belong in BatchJobFieldDelimiterOptionMixin, but I don't have a better place to put it right now (not familiar enough with the XD structure to know where to put a utility class like that…and it really belongs in Spring Core IMHO).
  2. I know sed works differently from OS X to Linux. I haven't tested the Linux version of the commas_to_tabs function. I'm running on the assumption that Linux supports the \t notation for a tab character (which OS X does not).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forgot to use tabs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grrr..no. Result of switching to IntelliJ from Eclipse. Still working out the configs for everything. I'll update.

where apropriate
* Updated configuration of filejdbc, hdfsjdbc, hdfsmongodb, and jdbchdfs
 jobs to allow for the delimiter to be injected
* Added BatchJobFiledDelimiterOptionMixin to provide metadata around
 new delimiter option
* Updated jdbc_tests script to test the processing of a job using a tab
 as a delimiter.
@markfisher markfisher changed the title XD-1536: Add ability to inject delimiter on pre-packaged jobs that deal with files. [DO NOT MERGE YET] XD-1536: Add ability to inject delimiter on pre-packaged jobs that deal with files. Apr 14, 2014
@ericbottard ericbottard changed the title [DO NOT MERGE YET] XD-1536: Add ability to inject delimiter on pre-packaged jobs that deal with files. [pending SHL-139] XD-1536: Add ability to inject delimiter on pre-packaged jobs that deal with files. Apr 18, 2014
@markpollack
Copy link
Contributor

I'll look into the shell issue and get XD back on spring-shell snapshots.

@markfisher markfisher assigned pperalta and markpollack and unassigned pperalta May 13, 2014
@ericbottard
Copy link
Contributor

Merged with escaping support directly in spring-shell 8c2d0eb

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging this pull request may close these issues.

5 participants