Destination S3: Allow special chars for table_prefix

[m-ronchi]

Hi,
I am trying to write to S3 destination with keys formatted as static_prefix/source=<namespace>/object=<table_name>/<filename>, like this:

aitbyte is normalizing the path to only contain letters and unserscores, resulting in paths like this:
static_prefix/source_<namespace>/object_<table_name>/<filename>

configured paths should not be modified

[marcosmarxm]

thanks @m-ronchi. I know the use of = when you partitioned your data using parquet format. https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines probably is possible to do. @tuliren tagging you here see some opinion.

[etsybaev]

Hi @m-ronchi .

Many thanks for your ticket.
Quick question: are you interested in "=" symbols only or in some other symbols as well?

AWS guide proposes to URL encode these symbols just in case. Would it be ok for your case?
https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines

Many of those would make a URL to file invalid without URL encoding

Could you please confirm that an example on capture below is what you're requesting for?

Thanks

[m-ronchi]

Hi,
basically, I wanted to be able to configure an AWS Glue crawler to create a partitioned spectrum schema.

other than the = character (mandatory to be able to partition), I may need the other characters, e.g. .../date=2022-03-15/...

I haven't tested url encoding

yes, the screenshot is what I have on my spectrum sources

[etsybaev]

#11173