Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config support for legacy quote escaping in LOAD CSV #8903

Merged
merged 2 commits into from Feb 28, 2017

Conversation

craigtaverner
Copy link
Contributor

@craigtaverner craigtaverner commented Feb 27, 2017

changelog: Added a new config setting dbms.import.csv.legacy_quote_escaping to select whether to conform to the standard https://tools.ietf.org/html/rfc4180 for interpreting escaped quotation characters in CSV files loaded using LOAD CSV. Setting this to false will use the standard, interpreting repeated quotes '""' as a single in-lined quote, while true will use the legacy convention originally supported in Neo4j 3.0 and 3.1, allowing a backslash to include quotes in-lined in fields.

@@ -37,6 +37,7 @@ case class LogicalPlanningContext(planContext: PlanContext,
useErrorsOverWarnings: Boolean = false,
errorIfShortestPathFallbackUsedAtRuntime: Boolean = false,
errorIfShortestPathHasCommonNodesAtRuntime: Boolean = true,
legacyCsvQuoteEscaping: Boolean = true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of places where there's a default value specified here. Could that be collected so that when if flips, which it should do at some point, it can be flipped in one place?

Copy link
Member

@tinwelint tinwelint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grand!

@xiatian90s
Copy link

Hi, I am using the Neo4j Windows desktop version 3.1.4 and I tried to set the dbms.import.csv.legacy_quote_escaping to false. However, I can not find this variable in my C:\Users\txia\AppData\Roaming\Neo4j Community Edition\neo4j.conf.

Here is what my neo4j.conf looks like(to attach it, I change it to .txt):
neo4j.txt

Any idea? Thanks!

@craigtaverner
Copy link
Contributor Author

@xiatian90s you should be able to just add it as a line to the end of your neo4j.conf file.

dbms.import.csv.legacy_quote_escaping=false

@aamanlamba
Copy link

Hi, what is the default value for dbms.import.csv.legacy_quote_escaping if it is not specified in the config file?

@tinwelint
Copy link
Member

It's true by default, but may be flipped to false as default in a later version

@upidea
Copy link

upidea commented Nov 22, 2017

This is a bad solution. I think don't ask data to change for data is always big.
http://grokbase.com/t/gg/neo4j/14b230yqm3/csv-import-problems-w-double-quotes

Proposed Solution:
Include a TEXTQUALIFIER *true/false *option so that the default rule for
assuming double quotes around values can be disabled more easily and also
assume that content between the FIELDTERMINIATOR are qualified by the
FIELDTERMINIATOR that is specified by the user in the header of the CSV
import cypher.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants