Add support for quotes in DelimitedLineAggregator

[spring-projects-issues]

Doug Breaux opened BATCH-2463 and commented

I can't imagine why org.springframework.batch.item.file.transform.DelimitedLineAggregator doesn't already support a quote character like the delimited tokenizer does. I can see that the StringUtils class being used doesn't provide this capability either, so I'm guessing it just wasn't trivially easy to add, but it seems like a necessary capability for working with delimited files.

I know I need it and am going to have to implement it some other way instead. (As far as I can tell, I can't simply extend an existing Batch class to add a quote delimiter around an individual field value as needed.)

---

2 votes, 9 watchers

[spring-projects-issues]

Kiichi Kuramoto commented

I agree in this issue, too.

CSV format is not standardized as an official specification,
I think that the RFC4180 is defact standard in general.

It is described as below to RFC 4180（Common Format and MIME Type for CSV Files）.（No.5,6）

• Each field may or may not be enclosed in double quotes.
• Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double-quotes.

In addition, these are supported in some libraries.

• SuperCSV
  http://super-csv.github.io/super-csv/examples_reading.html
  Please refer to sample code.

• OpenCSV
  http://opencsv.sourceforge.net/

Handling quoted entries with embedded carriage returns (ie entries that span multiple lines).

Also, as another one of the reasons, FlatFileItemReader corresponds to the reading of delimited file the fields are enclosed in double-quotes.

[spring-projects-issues]

Doug Breaux commented

Any update?

[desprez]

If it help i wrote this class based on BeanWrapperFieldExtractor

public class CSVQuotingBeanWrapperFieldExtractor<T> implements FieldExtractor<T>, InitializingBean {

		private String[] names;

		/**
		 * @param names field names to be extracted by the {@link #extract(Object)}
		 *              method.
		 */
		public void setNames(final String[] names) {
			Assert.notNull(names, "Names must be non-null");
			this.names = Arrays.asList(names).toArray(new String[names.length]);
		}

		/**
		 * @see org.springframework.batch.item.file.transform.FieldExtractor#extract(java.lang.Object)
		 */
		@Override
		public Object[] extract(final T item) {
			final List<Object> values = new ArrayList<>();

			final BeanWrapper bw = new BeanWrapperImpl(item);
			for (final String propertyName : this.names) {
				if (bw.getPropertyType(propertyName).isAssignableFrom(String.class)) {
					values.add(doublequoteIfString(bw.getPropertyValue(propertyName)));
				} else {
					values.add(bw.getPropertyValue(propertyName));
				}
			}
			return values.toArray();
		}

		@Override
		public void afterPropertiesSet() {
			Assert.notNull(names, "The 'names' property must be set.");
		}

		private String quote(final String str) {
			return str != null ? "\"" + str + "\"" : null;
		}

		private Object doublequoteIfString(final Object obj) {
			return obj instanceof String ? quote((String) obj) : obj;
		}
}