Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make the maxRowSample parameter public configurable #408

Closed
nihaoo opened this issue Jul 29, 2020 · 3 comments
Closed

make the maxRowSample parameter public configurable #408

nihaoo opened this issue Jul 29, 2020 · 3 comments
Assignees
Milestone

Comments

@nihaoo
Copy link

nihaoo commented Jul 29, 2020

Look like the maxRowSample parameter is hardcoded to 20 in the com.univocity.parsers.csv.CsvParser.java:
(
return new CsvFormatDetector(20, settings, whitespaceRangeStart) {
)

protected final InputAnalysisProcess getInputAnalysisProcess() {
	if (settings.isDelimiterDetectionEnabled() || settings.isQuoteDetectionEnabled()) {
		return new CsvFormatDetector(20, settings, whitespaceRangeStart) {
			@Override
			void apply(char delimiter, char quote, char quoteEscape) {
				if (settings.isDelimiterDetectionEnabled()) {
					CsvParser.this.delimiter = delimiter;
					CsvParser.this.delimiters[0] = delimiter;

Could we make maxRowSample a public configurable parameter? we would like to increase the value of maxRowLines to increase the delimiter detection accuracy.

@tiddman
Copy link

tiddman commented Aug 11, 2020

@jbax This issue is unfortunately a deal breaker for us because we have found errors in the delimiter detection in production using only 20 lines, whereas increasing this to i.e. 100 works. Because of where this is hard coded our only option is to fork the code and build our own jar which we'd really rather avoid if possible. Can you give any indication about timeline or priority for this change? Thanks so much.

jbax added a commit that referenced this issue Aug 13, 2020
…s to use when trying to detect the CSV format (#408)
@jbax
Copy link
Member

jbax commented Aug 13, 2020

Done, I'll release a new version later today. Thank you for using our parsers!

@jbax jbax closed this as completed Aug 13, 2020
@tiddman
Copy link

tiddman commented Aug 17, 2020

@jbax Outstanding! Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants