Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] What is and how to escape "RepIdx" in .csv file when converting to Json #71

Closed
SummerSun opened this Issue Mar 19, 2019 · 5 comments

Comments

Projects
None yet
2 participants
@SummerSun
Copy link

commented Mar 19, 2019

I have to apologize first that this is not an issue but a question and you do a GREAT JOB of this package!

My question is I want to do the same thing with C#, but when read in the first line, I got so many "RepIdx". I searched around but have no clue what it is.

I uploaded my file and tried the conversion in you WebSite, it works like a charm! I tried to debug into the source code, no escaping related to the "RepIdx" kind of string. Could you please provide some hint here?

@martindrapeau

This comment has been minimized.

Copy link
Owner

commented Mar 19, 2019

Hi,
I'm not sure where you saw RepIdx. Could you provide a screenshot here?
--Martin

@SummerSun

This comment has been minimized.

Copy link
Author

commented Mar 20, 2019

@martindrapeau Thanks for replying!

This is the the first line of the source csv file I got:

"RecordId","DataModelId","ADDRESS_LINE1","ADDRESS_LINE2","ADDRESS_LINE3","ADDRESS_LINE4","POSTCODE","CITY","COUNTRY","COUNTRY_ISO_CODE","LATITUDE","LONGITUDE","RepIdx","COUNTRY_REGION","RepIdx","COUNTRY_REGION_TYPE","NUTS1","NUTS2","NUTS3","WORLD_REGION","US_STATE","COUNTY","ADDRESS_TYPE","RepIdx","PHONE_NUMBER","RepIdx","FAX_NUMBER","RepIdx","WEBSITE","RepIdx","EMAIL","BVD_ID_NUMBER","BVD_ACCOUNT_NUMBER","RepIdx","NATIONAL_ID","RepIdx","NATIONAL_ID_LABEL","RepIdx","NATIONAL_ID_TYPE","RepIdx","TRADE_REGISTER_NUMBER","RepIdx","VAT_NUMBER","EUROPEAN_VAT_NUMBER","ECB_FVC_ID","ECB_MFI_ID","TIN","LEI","RepIdx","STATISTICAL_NUMBER","RepIdx","COMPANY_ID_NUMBER","RepIdx","NATIONAL_ID_PREVIOUS","RepIdx","NATIONAL_ID_DATE_PREVIOUS","RepIdx","INFORMATION_PROVIDER_ID","RepIdx","INFORMATION_PROVIDER_ID_LABEL","TICKER","ISIN","RepIdx","PREVIOUS_NAME","RepIdx","PREVIOUS_NAME_DATE","RepIdx","AKA_NAME"

There are so many fields with same name "RepIdx". And this is the result from your tool

[
  {
    "RecordId": "***",
    "DataModelId": "***",
    "ADDRESS_LINE1": "***",
    "ADDRESS_LINE2": "",
    "ADDRESS_LINE3": "",
    "ADDRESS_LINE4": "",
    "POSTCODE": 11416,
    "CITY": "RIYADH",
    "COUNTRY": "***",
    "COUNTRY_ISO_CODE": "SA",
    "LATITUDE": "",
    "LONGITUDE": "",
    **"RepIdx": 0,**
    "COUNTRY_REGION": "",
    "COUNTRY_REGION_TYPE": "",
    "NUTS1": "",
    "NUTS2": "",
    "NUTS3": "",
    "WORLD_REGION": "Middle East",
    "US_STATE": "",
    "COUNTY": "",
    "ADDRESS_TYPE": "***",
    "PHONE_NUMBER": "***",
    "FAX_NUMBER": "***",
    "WEBSITE": "***",
    "EMAIL": "***",
    "BVD_ID_NUMBER": "***",
    "BVD_ACCOUNT_NUMBER": "***",
    "NATIONAL_ID": 1010158683,
    "NATIONAL_ID_LABEL": "***",
    "NATIONAL_ID_TYPE": "***",
    "TRADE_REGISTER_NUMBER": ***,
    "VAT_NUMBER": "",
    "EUROPEAN_VAT_NUMBER": "",
    "ECB_FVC_ID": "",
    "ECB_MFI_ID": "",
    "TIN": ***,
    "LEI": "***",
    "STATISTICAL_NUMBER": "",
    "COMPANY_ID_NUMBER": "***",
    "NATIONAL_ID_PREVIOUS": "",
    "NATIONAL_ID_DATE_PREVIOUS": "",
    "INFORMATION_PROVIDER_ID": "***",
    "INFORMATION_PROVIDER_ID_LABEL": "***",
    "TICKER": ***,
    "ISIN": "***",
    "PREVIOUS_NAME": "",
    "PREVIOUS_NAME_DATE": "",
    "AKA_NAME": "***"
  }
]

Everything works so good with your tool, there is no duplicate key named "RepIdx". I am wondering, how did you get rid of the field "RepIdx"?

@martindrapeau

This comment has been minimized.

Copy link
Owner

commented Mar 20, 2019

Hi Summer,

Your CSV has duplicated keys RepIdx throughout. Currently CSVJSON collapses duplicate keys. That is indeed a bug. It should either report an error and ask you to make them unique, or, even better, it should make them unique itself.

Let me find an automatic solution to this problem.

--Martin

martindrapeau added a commit that referenced this issue Mar 20, 2019

@martindrapeau

This comment has been minimized.

Copy link
Owner

commented Mar 20, 2019

Ok, the issue is now resolved. Here is the testcase demonstrating the fix. Notice that '__#' is added when duplicate column headers are detected.
https://csvjson.com/csv2json/d5c3451fcf0b6e1a1d70a4ece2b6424c

Give it another go with your CSV file. Let me know if everything is resolved.

@SummerSun

This comment has been minimized.

Copy link
Author

commented Mar 22, 2019

Confirmed that it is resolved, I thought it was a feature 😂😂😂

Anyway, thank you! Really good package!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.