New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TokenizerConverter doesn't handle mixed quoted and unquoted k/v pairs #1083

Closed
joschi opened this Issue Mar 31, 2015 · 0 comments

Comments

Projects
None yet
1 participant
@joschi
Contributor

joschi commented Mar 31, 2015

The TokenizerConverter currently only handles quoted k/v pairs or unquoted k/v pairs, but not a mix of both.

For example the string otters in k1="v1" k2=v2 more otters results in only k1:v1 being extracted but k2:v2 being ignored. The correct result would be k1:v1 and k2:v2 being extracted.

@joschi joschi added inputs bug labels Mar 31, 2015

@joschi joschi added this to the 1.1.0 milestone Mar 31, 2015

joschi added a commit that referenced this issue Mar 31, 2015

Allow mixed plain and quoted k/v pairs in TokenizerConverter
TokenizerConverter previously only extracted either plain k/v pairs ('k1=v1') or quoted
k/v pairs ('k2="v2"') in strings but not both (the logic was skewed towards quoted k/v
pairs).

The new implementation allows a mix of both styles ('k1=v1 k2="v2"') and also allows
quoted keys ('"k1"=v1 "k2"="v2"') and retains whitespace characters inside quoted keys and
values.

Fixes #1083

joschi added a commit that referenced this issue Mar 31, 2015

Allow mixed plain and quoted k/v pairs in TokenizerConverter
TokenizerConverter previously only extracted either plain k/v pairs ('k1=v1') or quoted
k/v pairs ('k2="v2"') in strings but not both (the logic was skewed towards quoted k/v
pairs).

The new implementation allows a mix of both styles ('k1=v1 k2="v2"') and retains whitespace
characters inside quoted values.

Fixes #1083

joschi added a commit that referenced this issue Mar 31, 2015

Allow mixed plain and quoted k/v pairs in TokenizerConverter
TokenizerConverter previously only extracted either plain k/v pairs ('k1=v1') or quoted
k/v pairs ('k2="v2"') in strings but not both (the logic was skewed towards quoted k/v
pairs).

The new implementation allows a mix of both styles ('k1=v1 k2="v2"') and retains whitespace
characters inside quoted values.

Fixes #1083

(cherry picked from commit 2ee5cb8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment