Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mapToQuery changes #364

Merged
merged 13 commits into from Oct 8, 2022
Merged

mapToQuery changes #364

merged 13 commits into from Oct 8, 2022

Conversation

techouse
Copy link
Collaborator

@techouse techouse commented Oct 5, 2022

This PR will require a bit of discussion:

  • I changed how empty strings and null are encoded into the query string.

    I've never seen query parameters taken out if their values were empty or null if they were provided to the method encoding the map and I think it's up to the user to clean them up explicitly as most APIs simply ignore them.
    For comparison, Python's urllib.parse.urlencode employs the same logic.

    // before
    print(mapToQuery({'foo': ''})); // prints ''
    // after
    print(mapToQuery({'foo': ''})); // prints 'foo='
    
    // before
    print(mapToQuery({'foo': null})); // prints ''
    // after
    print(mapToQuery({'foo': null})); // prints 'foo='
  • I changed how empty strings and null inside Lists are encoded into query strings.

    Since these values here are packed into a List I'm not sure why they should be encoded, so I removed them (essentially the reverse of the bullet point above which targets standalone values). A query like this is more or less used for filtering by multiple IDs or equivalent where an empty param might actually be detrimental.

    // before
    print(mapToQuery('foo': ['', 'baz', 'etc'])); // prints 'foo=&foo=baz&foo=etc'
    // after
    print(mapToQuery('foo': ['', 'baz', 'etc'])); // prints 'foo=baz&foo=etc'
    
    // before
    print(mapToQuery('foo': [null, 'baz', 'etc'])); // prints 'foo=&foo=baz&foo=etc'
    // after
    print(mapToQuery('foo': [null, 'baz', 'etc'])); // prints 'foo=baz&foo=etc'
  • I made _normalizeValue null-safe. Now it will encode null to '' rather than explode violently 😅

  • I added the option to use square brackets [ ] as an encoded query list and map separator.

    Currently, they are hardcoded to dots ., and it's already been expressed that more flexibility is needed QueryMap buildUrl  #301.
    So I've added an optional Method annotation bool useBrackets.

    // using this input in both instances
    final Map<String, dynamic> input = {
       'bar': 'baz',
       'zap': 'abc',
       'etc': {
         'abc': 'def',
         'ghi': 'jkl',
         'mno': {
           'opq': 'rst',
           'uvw': 'xyz',
           'list': ['a', 123, false],
         },
       },
     };
    
    // the default way using dots as query map separators
    @Get(path: '/map_query_param')
    Future<Response<String>> getUsingMapQueryParam(
      @Query('value') Map<String, dynamic> value,
    );
    
    // the above produces a URL like this (the line breaks are there just for readability)
    //
    // /test/map_query_param
    // ?value.bar=baz
    // &value.zap=abc
    // &value.etc.abc=def
    // &value.etc.ghi=jkl
    // &value.etc.mno.opq=rst
    // &value.etc.mno.uvw=xyz
    // &value.etc.mno.list=a
    // &value.etc.mno.list=123
    // &value.etc.mno.list=false
    
    // specifying queryMapSeparator: 'brackets'
    @Get(path: '/map_query_param_with_brackets', useBrackets: true)
    Future<Response<String>> getUsingMapQueryParamWithBrackets(
      @Query('value') Map<String, dynamic> value,
    );
    
    // this produces a URL like this (the line breaks are there just for readability)
    //
    // /test/map_query_param_with_brackets
    // ?value%5Bbar%5D=baz
    // &value%5Bzap%5D=abc
    // &value%5Betc%5D%5Babc%5D=def
    // &value%5Betc%5D%5Bghi%5D=jkl
    // &value%5Betc%5D%5Bmno%5D%5Bopq%5D=rst
    // &value%5Betc%5D%5Bmno%5D%5Buvw%5D=xyz
    // &value%5Betc%5D%5Bmno%5D%5Blist%5D%5B%5D=a
    // &value%5Betc%5D%5Bmno%5D%5Blist%5D%5B%5D=123
    // &value%5Betc%5D%5Bmno%5D%5Blist%5D%5B%5D=false
    //
    // URI decoded this reads as:
    //
    // /test/map_query_param_with_brackets
    // ?value[bar]=baz
    // &value[zap]=abc
    // &value[etc][abc]=def
    // &value[etc][ghi]=jkl
    // &value[etc][mno][opq]=rst
    // &value[etc][mno][uvw]=xyz
    // &value[etc][mno][list][]=a
    // &value[etc][mno][list][]=123
    // &value[etc][mno][list][]=false 

And of course, I've added a tonne of tests ✅

Check chopper/tests/utils_test.dart for more examples

@techouse techouse added the enhancement New feature or request label Oct 5, 2022
@JEuler
Copy link
Collaborator

JEuler commented Oct 8, 2022

Oh my god, this is really cool! Thank you so much, I agree that we should not ignore the null params silently, and should just pass it as it is.

I saw your request about "hey man release to master" - let's do this after that PR merge. Do you have any plans to add more code here? :)

@techouse
Copy link
Collaborator Author

techouse commented Oct 8, 2022

Do you have any plans to add more code here? :)

I think one more thing.... Gimme 15min

@techouse techouse marked this pull request as draft October 8, 2022 13:37
@techouse techouse marked this pull request as ready for review October 8, 2022 14:23
@techouse
Copy link
Collaborator Author

techouse commented Oct 8, 2022

Done. I added the option to also use brackets in lists, e.g

hxxp://path/to/script?foo[]=123&foo[]=456&foo[]=789

As that is also one of the ways some backends parse lists (this is the standard in PHP, for example).

@techouse techouse requested a review from JEuler October 8, 2022 14:51
@JEuler JEuler merged commit d74790c into lejard-h:develop Oct 8, 2022
@JEuler
Copy link
Collaborator

JEuler commented Oct 8, 2022

I will prepare the PR to master soon, maybe today, maybe tomorrow morning. Thank you very much for contributing! Need to add you to the author list!

@techouse techouse deleted the query-string-tests branch October 8, 2022 16:15
JEuler pushed a commit that referenced this pull request Oct 8, 2022
JEuler added a commit that referenced this pull request Oct 15, 2022
* Fix Header Option Casting (#260)

Co-authored-by: Ivan Terekhin <i.terhin@gmail.com>

* Fix for #259 (#263)

* 4.0.1 fixes (#264)

* analyzer dependency upgraded (#296)

* fix(generator): fix PartValueFile value not nullable if arg is (#288) (#293)

* Chopper generator release 4.0.2 (#297)

* fix: fix this.body cast of null value when response body is null (#291) (#292)

* Interpolation fixes (#275)

* encodeQueryComponent now encodeComponent (#278)

* Prevent double call on token refreshment (#276)

* Fixes for #309 #308 (#310)

* Remove new keyword from interceptors.md (#312)

* Analyzer upgrade (#320)

Co-authored-by: István Juhos <stewemetal@gmail.com>

* Add unnecessary_brace_in_string_interps to lint ignores (#317)

* Extend pragma to quiet the linter (#318)

Co-authored-by: Ivan Terekhin <i.terhin@gmail.com>

* Fix converter getting called twice if using an authenticator with a JsonConverter on the request (#324)

* migrate example to nullsafety (#331)

* Resolve problem in main_json_serializable example (#328)

* Add @FiledMap @PartMap @PartFileMap (#335)

Co-authored-by: Meysam Karimi <mysmartapply.it4@gmail.com>

* Upgrade of analyzer (#340)

* Fix nullable QueryMap fails to compile (#344)

* Change return type of decodeJson to FutureOr in order to be able to support compute() (#345)

* Migrate from pedantic to lints ^2.0.0 with lints/recommended.yaml (#349)

* Version bumped for release (#352)

* Revert analyzer to ^4.1.0 and silence linters for Element.enclosingElement (#354)

* [chopper_generator] Update analyzer to ^4.4.0 and code_builde to ^4.3.0 and migrate deprecated code (#358)

* Add Makefiles to streamline development (#357)

* Add Bug Report Github issue template (#359)

* [chopper_generator] Add types to the generated variables (#360)

* Provide an example using an Isolate Worker Pool with Squadron (#361)

* mapToQuery changes (#364)

* Version bumped / changelog update (#367)

* Request extends http.BaseRequest (#370)

* Exclude null query vars by default and add new @method annotation includeNullQueryVars (#372)

* 5.1.0 (dev) (#373)

Co-authored-by: Ivan Terekhin <231950+JEuler@users.noreply.github.com>

Co-authored-by: Youssef Raafat <youssefraafatnasry@gmail.com>
Co-authored-by: luis901101 <luis901101@gmail.com>
Co-authored-by: melvspace <ratealt@gmail.com>
Co-authored-by: Michal Šrůtek <35694712+michalsrutek@users.noreply.github.com>
Co-authored-by: István Juhos <stewemetal@gmail.com>
Co-authored-by: Andre <andre.lipke@gmail.com>
Co-authored-by: John Wimer <john@wimer.org>
Co-authored-by: Max Röhrl <max.roehrl11@gmail.com>
Co-authored-by: ipcjs <gipcjs@gmail.com>
Co-authored-by: ibadin <exbatek@gmail.com>
Co-authored-by: Meysam Karimi <31154534+meysam1717@users.noreply.github.com>
Co-authored-by: Meysam Karimi <mysmartapply.it4@gmail.com>
Co-authored-by: Klemen Tusar <techouse@gmail.com>
Co-authored-by: Klemen Tusar <k.tusar@cmcmarkets.com>
Co-authored-by: Ivan Terekhin <231950+JEuler@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants