Add the ability to write CSV files. #42

Closed
thyzzv opened this Issue Feb 25, 2014 · 5 comments

Comments

Projects
None yet
4 participants
@thyzzv

thyzzv commented Feb 25, 2014

Please add the ability to also write CSV files.

@jnash67

This comment has been minimized.

Show comment
Hide comment
@jnash67

jnash67 May 2, 2014

I would like to second this. Currently I use CSVeed in my project and also have the an old 2.4-SNAPSHOT version of OpenCSV just for bean writing. Would like to switch entirely to CSVeed. The main class I use in the no longer updated OpenCSV is BeanToCsv which should be relatively easy to adapt to the CSVeed framework. Here is that class. fyi, the dateformat conversion was not in the original code, I added that. The opencsv license is the same apache 2.0 license as csveed.

package au.com.bytecode.opencsv.bean;

/**
 Copyright 2007 Kyle Miller.

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 */
import au.com.bytecode.opencsv.CSVWriter;

import java.beans.IntrospectionException;
import java.beans.PropertyDescriptor;
import java.io.Writer;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

/**
 * Allows to export Java beans content to a new CSV spreadsheet file.
 *
 * @author Kali <kali.tystrit@gmail.com>
 */
public class BeanToCsv<T> {

    DateFormat df = new SimpleDateFormat(("dd-MM-yyyy"));

public BeanToCsv() {
}

public boolean write(MappingStrategy<T> mapper, Writer writer,
 List<?> objects) {
return write(mapper, new CSVWriter(writer), objects);
}

public boolean write(MappingStrategy<T> mapper, CSVWriter csv,
 List<?> objects) {
if (objects == null || objects.isEmpty())
return false;

try {
csv.writeNext(processHeader(mapper));
List<Method> getters = findGetters(mapper);
for (Object obj : objects) {
String[] line = processObject(getters, obj);
csv.writeNext(line);
}
return true;
} catch (Exception e) {
throw new RuntimeException("Error writing CSV !", e);
}
}

protected String[] processHeader(MappingStrategy<T> mapper)
throws IntrospectionException {
List<String> values = new ArrayList<String>();
int i = 0;
PropertyDescriptor prop = mapper.findDescriptor(i);
while (prop != null) {
values.add(prop.getName());
i++;
prop = mapper.findDescriptor(i);
}
return values.toArray(new String[0]);
}

protected String[] processObject(List<Method> getters, Object bean)
throws IntrospectionException, IllegalArgumentException,
IllegalAccessException, InvocationTargetException {
List<String> values = new ArrayList<String>();
// retrieve bean values
for (Method getter : getters) {
Object value = getter.invoke(bean, (Object[]) null);
if (value == null) {
values.add("null");
} else {
    if (Date.class.isAssignableFrom(value.getClass())) {
                    values.add(df.format(value));
    } else {
        values.add(value.toString());
    }

}
}
return values.toArray(new String[0]);
}

/**
 * Build getters list from provided mapper.
 */
private List<Method> findGetters(MappingStrategy<T> mapper)
throws IntrospectionException {
int i = 0;
PropertyDescriptor prop = mapper.findDescriptor(i);
// build getters methods list
List<Method> readers = new ArrayList<Method>();
while (prop != null) {
readers.add(prop.getReadMethod());
i++;
prop = mapper.findDescriptor(i);
}
return readers;
}
}

jnash67 commented May 2, 2014

I would like to second this. Currently I use CSVeed in my project and also have the an old 2.4-SNAPSHOT version of OpenCSV just for bean writing. Would like to switch entirely to CSVeed. The main class I use in the no longer updated OpenCSV is BeanToCsv which should be relatively easy to adapt to the CSVeed framework. Here is that class. fyi, the dateformat conversion was not in the original code, I added that. The opencsv license is the same apache 2.0 license as csveed.

package au.com.bytecode.opencsv.bean;

/**
 Copyright 2007 Kyle Miller.

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 */
import au.com.bytecode.opencsv.CSVWriter;

import java.beans.IntrospectionException;
import java.beans.PropertyDescriptor;
import java.io.Writer;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

/**
 * Allows to export Java beans content to a new CSV spreadsheet file.
 *
 * @author Kali &lt;kali.tystrit@gmail.com&gt;
 */
public class BeanToCsv<T> {

    DateFormat df = new SimpleDateFormat(("dd-MM-yyyy"));

public BeanToCsv() {
}

public boolean write(MappingStrategy<T> mapper, Writer writer,
 List<?> objects) {
return write(mapper, new CSVWriter(writer), objects);
}

public boolean write(MappingStrategy<T> mapper, CSVWriter csv,
 List<?> objects) {
if (objects == null || objects.isEmpty())
return false;

try {
csv.writeNext(processHeader(mapper));
List<Method> getters = findGetters(mapper);
for (Object obj : objects) {
String[] line = processObject(getters, obj);
csv.writeNext(line);
}
return true;
} catch (Exception e) {
throw new RuntimeException("Error writing CSV !", e);
}
}

protected String[] processHeader(MappingStrategy<T> mapper)
throws IntrospectionException {
List<String> values = new ArrayList<String>();
int i = 0;
PropertyDescriptor prop = mapper.findDescriptor(i);
while (prop != null) {
values.add(prop.getName());
i++;
prop = mapper.findDescriptor(i);
}
return values.toArray(new String[0]);
}

protected String[] processObject(List<Method> getters, Object bean)
throws IntrospectionException, IllegalArgumentException,
IllegalAccessException, InvocationTargetException {
List<String> values = new ArrayList<String>();
// retrieve bean values
for (Method getter : getters) {
Object value = getter.invoke(bean, (Object[]) null);
if (value == null) {
values.add("null");
} else {
    if (Date.class.isAssignableFrom(value.getClass())) {
                    values.add(df.format(value));
    } else {
        values.add(value.toString());
    }

}
}
return values.toArray(new String[0]);
}

/**
 * Build getters list from provided mapper.
 */
private List<Method> findGetters(MappingStrategy<T> mapper)
throws IntrospectionException {
int i = 0;
PropertyDescriptor prop = mapper.findDescriptor(i);
// build getters methods list
List<Method> readers = new ArrayList<Method>();
while (prop != null) {
readers.add(prop.getReadMethod());
i++;
prop = mapper.findDescriptor(i);
}
return readers;
}
}
@robert-bor

This comment has been minimized.

Show comment
Hide comment
@robert-bor

robert-bor May 5, 2014

Owner

This functionality is in much hotter in demand than I realized. It's next on the list.

Owner

robert-bor commented May 5, 2014

This functionality is in much hotter in demand than I realized. It's next on the list.

robert-bor added a commit that referenced this issue May 5, 2014

robert-bor added a commit that referenced this issue May 5, 2014

robert-bor added a commit that referenced this issue May 6, 2014

Issue #42 big rename of CsvReader to CsvClient in order to facilitate…
… CsvWriter as well, inclusion of row write-methods in facade as well.

robert-bor added a commit that referenced this issue May 6, 2014

robert-bor added a commit that referenced this issue May 6, 2014

@robert-bor robert-bor added this to the v0.4.0 milestone May 6, 2014

@robert-bor robert-bor closed this May 6, 2014

@robert-bor

This comment has been minimized.

Show comment
Hide comment
@robert-bor

robert-bor May 6, 2014

Owner

v0.4.0 has been released, containing basic write functionality. There are a couple of things that are handy to have, like dynamic column support, the ability to declare whether quotes are required for cells, the ability to declare a custom order etc.

Perhaps you can have a look and let me know what you think of the write-ability of CSVeed?

Owner

robert-bor commented May 6, 2014

v0.4.0 has been released, containing basic write functionality. There are a couple of things that are handy to have, like dynamic column support, the ability to declare whether quotes are required for cells, the ability to declare a custom order etc.

Perhaps you can have a look and let me know what you think of the write-ability of CSVeed?

@jnash67

This comment has been minimized.

Show comment
Hide comment
@jnash67

jnash67 Jun 20, 2014

Sorry for late reply. The opportunity only now came up to revisit the CSV writing part of the code. I spent today trying to integrate the new version, with only partial success.

I'm getting a NullPointerException in the constructor of HeaderImpl.

public HeaderImpl(Line row) {
    this.header = row;
    Column currentColumn = new Column();
    for (String headerCell : header) {
        this.indexToName.put(currentColumn, headerCell);
        this.nameToIndex.put(headerCell.toLowerCase(), currentColumn);
        currentColumn = currentColumn.nextColumn();
    }
}

I suspect the issue is that the loop for

(String headerCell : header) {

is iterating through every property picked up by introspection, and ignoring the fact that properties may have been manually mapped. For reading purposes, I generate I create my BeanInstructions (formerly BeanReaderInstructions) as follows:

private static <T extends DisplayFriendly> BeanInstructions getListedPropertiesBeanInstructions(Class<T> clazz,
        Collection<String> pids) {
    BeanInstructions bi = new BeanInstructionsImpl(clazz);
    bi.setMapper(TolerantColumnNameMapper.class);
    for (String pid : pids) {
        try {
            bi.mapColumnNameToProperty(pid, pid);
        } catch (CsvException ce) {
            ce.printStackTrace();
        }
    }
    return bi;
}

TolerantColumnNameMapper is a ColumnNameMapper that tolaterates missing column names when reading. If I have an old zip file and I've added a field to a Bean, it just gives a warning when the zip file is missing something as opposed to throwing an exception and terminating out of the entire reading process:

import org.csveed.api.Header;
import org.csveed.bean.ColumnNameMapper;
import org.csveed.common.Column;
import org.csveed.report.CsvException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class TolerantColumnNameMapper<T> extends ColumnNameMapper<T> {
    private final static Logger LOGGER = LoggerFactory.getLogger(TolerantColumnNameMapper.class);

    @Override
    protected void checkKey(Header header, Column key) {
        try {
            header.getIndex(key.getColumnName());
        } catch (CsvException err) {
            LOGGER.warn("In class " + this.beanInstructions.getBeanClass().getSimpleName() + " -- This is probably a new " +
                    "field.  Issue will go away when CSV file is persisted.");
        }
    }
}

I'm not familiar enough with the framework to fix how the header loop and possibly other write loops should work. So until this is fixed, I'll have to stick with the old csv writing code.

On a slightly separate note, I noticed you made

String toString(K value) throws Exception;

a required part of the Converter interface. No issues with that except you should probably provide a default implementation in AbstractConverter since it will usually be value.toString() anyway. So for now, I'm using EasierAbstractConverter as follows instead of AbstractConverter:

import org.csveed.bean.conversion.AbstractConverter;

public abstract class EasierAbstractConverter<K> extends AbstractConverter<K> {

    public EasierAbstractConverter(Class<K> clazz) {
        super(clazz);
    }

    @Override
    public String toString(K value) throws Exception {
        return value.toString();
    }
}

jnash67 commented Jun 20, 2014

Sorry for late reply. The opportunity only now came up to revisit the CSV writing part of the code. I spent today trying to integrate the new version, with only partial success.

I'm getting a NullPointerException in the constructor of HeaderImpl.

public HeaderImpl(Line row) {
    this.header = row;
    Column currentColumn = new Column();
    for (String headerCell : header) {
        this.indexToName.put(currentColumn, headerCell);
        this.nameToIndex.put(headerCell.toLowerCase(), currentColumn);
        currentColumn = currentColumn.nextColumn();
    }
}

I suspect the issue is that the loop for

(String headerCell : header) {

is iterating through every property picked up by introspection, and ignoring the fact that properties may have been manually mapped. For reading purposes, I generate I create my BeanInstructions (formerly BeanReaderInstructions) as follows:

private static <T extends DisplayFriendly> BeanInstructions getListedPropertiesBeanInstructions(Class<T> clazz,
        Collection<String> pids) {
    BeanInstructions bi = new BeanInstructionsImpl(clazz);
    bi.setMapper(TolerantColumnNameMapper.class);
    for (String pid : pids) {
        try {
            bi.mapColumnNameToProperty(pid, pid);
        } catch (CsvException ce) {
            ce.printStackTrace();
        }
    }
    return bi;
}

TolerantColumnNameMapper is a ColumnNameMapper that tolaterates missing column names when reading. If I have an old zip file and I've added a field to a Bean, it just gives a warning when the zip file is missing something as opposed to throwing an exception and terminating out of the entire reading process:

import org.csveed.api.Header;
import org.csveed.bean.ColumnNameMapper;
import org.csveed.common.Column;
import org.csveed.report.CsvException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class TolerantColumnNameMapper<T> extends ColumnNameMapper<T> {
    private final static Logger LOGGER = LoggerFactory.getLogger(TolerantColumnNameMapper.class);

    @Override
    protected void checkKey(Header header, Column key) {
        try {
            header.getIndex(key.getColumnName());
        } catch (CsvException err) {
            LOGGER.warn("In class " + this.beanInstructions.getBeanClass().getSimpleName() + " -- This is probably a new " +
                    "field.  Issue will go away when CSV file is persisted.");
        }
    }
}

I'm not familiar enough with the framework to fix how the header loop and possibly other write loops should work. So until this is fixed, I'll have to stick with the old csv writing code.

On a slightly separate note, I noticed you made

String toString(K value) throws Exception;

a required part of the Converter interface. No issues with that except you should probably provide a default implementation in AbstractConverter since it will usually be value.toString() anyway. So for now, I'm using EasierAbstractConverter as follows instead of AbstractConverter:

import org.csveed.bean.conversion.AbstractConverter;

public abstract class EasierAbstractConverter<K> extends AbstractConverter<K> {

    public EasierAbstractConverter(Class<K> clazz) {
        super(clazz);
    }

    @Override
    public String toString(K value) throws Exception {
        return value.toString();
    }
}
@mparaz

This comment has been minimized.

Show comment
Hide comment
@mparaz

mparaz Dec 10, 2014

In RowWriterImpl.writeEOL():

writer.write(rowInstructions.getEndOfLine())

but the getEndOfLine() is:

return this.symbolMapping.getFirstMappedCharacter(EncounteredSymbol.EOL_SYMBOL)

which is only the first character - the \r of \r\n

Thanks.

mparaz commented Dec 10, 2014

In RowWriterImpl.writeEOL():

writer.write(rowInstructions.getEndOfLine())

but the getEndOfLine() is:

return this.symbolMapping.getFirstMappedCharacter(EncounteredSymbol.EOL_SYMBOL)

which is only the first character - the \r of \r\n

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment