Search manipulation without normalization

[bungle]

What is the problem this feature will solve?

We (Kong) experience some issues with Ada forcibly normalising data. While I agree that it is in general good thing to do and expected, and according to W3C spec, this causes some issues.

For example sometimes our Gateway needs to remove a single search parameter from the request url. This is most of the time good, but for some it causes issues. We would like to have mode/function or something to just remove the search parameter that we want to remove and leave rest intact. Issues that we have seen with just removing a single search parameter:

• incoming %20 gets converted to +.
• some get encoded, e.g. , when original search parameters had it non-encoded gets encoded when we remove different search parameter and get the final search string.

What is great with Ada btw. is that at least it retains the original order of parameters when manipulating them!

I am using removal of search parameter as an example here, but this may be useful in other manipulations as well. E.g. adding a new search parameter, and keep rest in their original form. Perhaps outside search parameters too.

I do understand that this may be a stretch to Ada, but it would probably make it a bit more general purpose (or granular) without violating too much anything. Please close this if you consider this something that Ada should not do.

What is the feature you are proposing to solve the problem?

Some possibilities are:

1. provide API that tells positions where parameter starts and ends (indices) on incoming url (so that we can remove that on our own easily with simple string manipulation). Possibility for same parameter appearing multiple times can be solved by calling this multiple times (and always returning the left most indices).
2. provide APIs, parameters, options that do non-changing removals etc.
3. provide a way to disable normalization

What alternatives have you considered?

Alternative for us is to implement our own parsing logic to do this or fork / patch Ada (or refuse to change the behavior).

[anonrig]

hey @bungle, have you tried the url search params api? it gives you almost all of the things you've asked.

[bungle]

@anonrig thanks for the reply, currently I am limited to ada_c API and with that it seems not available. Once I feed string to ada_parse_search_params all I can get out of it is a normalized string (I want to remove search parameter, but not normalize the final search string that I get out of the lib with ada_search_params_to_string). Am I missing something? I guess it is possible in C++ API (couldn't though figure out how)?

[anonrig]

For example sometimes our Gateway needs to remove a single search parameter from the request url.

We can add a searchParams attribute to Ada, which will lazily convert the search string to a URLSearchParams attribute, while keeping the URLSearchParams instance in sync with the search attribute upon manipulation. This would solve this problem.

We would like to have mode/function or something to just remove the search parameter that we want to remove and leave rest intact.

url.search_params().remove("key") would solve this as well.

I am using removal of search parameter as an example here, but this may be useful in other manipulations as well. E.g. adding a new search parameter, and keep rest in their original form. Perhaps outside search parameters too.

URLSearchParams has these methods as well, but I'm not following which particular code is normalizing your data, and how can I reproduce this?

Once I feed string to ada_parse_search_params all I can get out of it is a normalized string.

The easiest way to add this by introducing a similar method to URLSearchParams like into_raw_string() which has a similar functionality to to_string but without normalization. Would this solve your problem?

[bungle]

The example in pseudo C is something like this:

params = ada_parse_search_params("a=%20&b=remove&c=2", 18);
ada_search_params_remove(params, "b", 1);
res = ada_search_params_to_string(params)

The res will be a=+&c=2 (if I recall correctly), what I want or am after is a=%20&c=2.

[bungle]

The easiest way to add this by introducing a similar method to URLSearchParams like into_raw_string() which has a similar functionality to to_string but without normalization. Would this solve your problem?

I think it would.

[anonrig]

The easiest way to add this by introducing a similar method to URLSearchParams like into_raw_string() which has a similar functionality to to_string but without normalization. Would this solve your problem?

I think it would.

I'm glad to hear that! I'll try to do my best to open a PR this week but any help is welcome

[bungle]

I'm glad to hear that! I'll try to do my best to open a PR this week but any help is welcome

Yes, I will follow closely, and if you open a branch/pr I can confirm then if it solves my issue. Thank you.

[anonrig]

Opened a PR: #1023

Let me know if this is what you wanted.