Skip to content
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.

Tap doesn't clean special characters in headers and hence fails #202

Open
Atif8Ted opened this issue Feb 9, 2023 · 0 comments
Open

Tap doesn't clean special characters in headers and hence fails #202

Atif8Ted opened this issue Feb 9, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@Atif8Ted
Copy link

Atif8Ted commented Feb 9, 2023

Describe the bug
A clear and concise description of what the bug is.
The Tap has no functionality to handle junk characters/special characters in headers ( that is not visible to naked eyes) . Also it doesn't remove spaces in between headers ("column name" -> column_name) to make it more suitable for databases and it doesn't remove the trailing white spaces in headers. These all are very crucial since some third-party apps exports there data directly to s3 and we don't have much say in controlling the headers. and as per the standard column names should not contain spaces or special character.

To Reproduce
Steps to reproduce the behavior:

  1. Prepare the data as '...' put non-ascii values or junk values in colunm name for example "\ufeffmessage_id" and try to load it with passing key_properties as ["message_id"], it will fail since it would not be able to find message_id
  2. Run the command '....'
  3. See error

Expected behavior
A clear and concise description of what you expected to happen.
if the column has special character, it should be removed and spaces should be replaced with _,
ex: "\ufeffmessage_id" -> "message_id"
"message id" -> "message_id"
message%%id-> message_id
Screenshots
If applicable, add screenshots to help explain your problem.

Your environment

  • Version of tap: [e.g. 2.0.0]
  • Version of python [e.g. 3.8]

Additional context
Add any other context about the problem here.

@Atif8Ted Atif8Ted added the bug Something isn't working label Feb 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant