Datax---transformer的替换换行符与emoji的方法

[AronChung]

第一种，直接在parameter中获取字段做过滤，但这种方式很难测试，\n不知该如何转义，最终无法实现替换转义

"transformer":[{
"name": "dx_groovy",
"parameter":
{"code": "Column column = record.getColumn(16);\ndef str = column.asString();\nif(str == null){return null;}\ndef newStr=null;\nnewStr = str.replaceAll(\"\\n\",\"\\n\");\nrecord.setColumn(16, new StringColumn(newStr));\nreturn record;",
"extraPackage":[]}
}]

第二种，在GroovyTransformerStaticUtil.java类中写静态方法

public class GroovyTransformerStaticUtil  {

    public static String hiveColumnEscaped(String str) {
        if(str == null) return null;

        String result = StringUtils.replace(str, "\n", "\\n");


        result = StringUtils.replace(result, "\1", "\\1");
        result = StringUtils.replace(result, "\2", "\\2");
        result = StringUtils.replace(result, "\3", "\\3");
        return result;
    }

    public static Record hiveRowEscaped(Record record) {
        int length = record.getColumnNumber();

        for( int i = 0; i < length; i++){
            String result = "";
            Column column = record.getColumn(i);
            String str = column.asString();
            if(str == null) return record;

            result = StringUtils.replace(str, "\n", "\\n");


            result = StringUtils.replace(result, "\1", "\\1");
            result = StringUtils.replace(result, "\2", "\\2");
            result = StringUtils.replace(result, "\3", "\\3");

            record.setColumn(i, new StringColumn(result));

        }
        return record;
    }
}

然后再parameter中引用：

"transformer":[{
"name": "dx_groovy",
"parameter":
{"code": "Column column = record.getColumn(16);\ndef str = column.asString();\ndef newStr=null;\nnewStr = hiveEscaped(str);\nrecord.setColumn(16, new StringColumn(newStr));\nreturn record;",
"extraPackage":[]}
}]

由于这种替换单个字段的方式需要指定column，所以最好采取第三种引用方式，即使用代码中的hiveRowEscaped():

"transformer":[{
"name": "dx_groovy",
"parameter":
{"code": "Record finalRecord = hiveRowEscaped(record);\nreturn finalRecord;",
"extraPackage":[]}
}]