[BEAM-2079] Support TextIO as SQL source/sink#2942
[BEAM-2079] Support TextIO as SQL source/sink#2942xumingming wants to merge 4 commits intoapache:DSL_SQLfrom
Conversation
|
Retest this please. |
|
Retest this please |
|
R: @xumingmin |
| CSVParser parser = csvFormat.parse(reader); | ||
| rawRecord = parser.getRecords().get(0); | ||
| } catch (IOException e) { | ||
| LOG.error("error record: " + str, e); |
There was a problem hiding this comment.
throw exception here to report error? same for L76
There was a problem hiding this comment.
I was thinking that some error records should not fail the whole job. But lets be strict for now.
| row.addField(idx, raw); | ||
| break; | ||
| default: | ||
| row.addField(idx, raw); |
There was a problem hiding this comment.
throw BeamSqlUnsupportedException instead, row.addField(idx, raw); always fail here.
| } | ||
| printer.println(); | ||
| } catch (IOException e) { | ||
| LOG.error("invalid record: " + row, e); |
| public void processElement(ProcessContext ctx) { | ||
| String str = ctx.element(); | ||
|
|
||
| StringReader reader = new StringReader(str); |
There was a problem hiding this comment.
need to close StringReader at the end of method;
|
Let's use |
|
Test updated. |
|
R: @jbonofre |
|
LGTM |
|
Let me take a look. |
|
I merged on |
4 participants
Support TextIO as SQL source/sink