Support for Vec in Query?

[davidpdrsn]

Discussed in #433

^{Originally posted by Vesafary October 29, 2021}
Hi!
I've been struggling with this for a while, and didn't get it to work.
Looking through the docs and examples also didn't give me the answer.

What I'm looking for is support for one of the following:

<url>?id=1&id=2&id=3
<url>?id=[1, 2, 3]

I've tried several options like

struct IdQuery {
    ids: Vec<uuid::Uuid>,
}
async fn get_ids(filter: Query<IdQuery>)

struct IdQuery {
    id: uuid::Uuid,
}
async fn get_ids(filter: Query<Vec<IdQuery>>)

Of course I can implement it myself using the RawQuery extractor, but it seems like fairly basic usage so I was wondering whether I did something wrong.

Cheers!

[davidpdrsn]

Oh your right! axum uses serde_urlencoded for Query which it seems doesn't support sequences nox/serde_urlencoded#85. I didn't realize this, thats quite a bummer.

I did a very quick test with serde_qs which is mentioned in the issue and that works:

struct Qs<T>(T);

#[axum::async_trait]
impl<T> FromRequest for Qs<T>
where
    T: serde::de::DeserializeOwned,
{
    type Rejection = Infallible;

    async fn from_request(req: &mut RequestParts) -> Result<Self, Self::Rejection> {
        // TODO: error handling
        let query = req.uri().query().unwrap();
        Ok(Self(serde_qs::from_str(query).unwrap()))
    }
}

async fn handler(Qs(ids): Qs<Input>) {
    dbg!(ids);
}

#[derive(serde::Deserialize, Debug)]
struct Input {
    ids: Vec<i32>,
}

Now I'm wondering if axum should just change to serde_qs instead of serde_urlencoded 🤔

I'm gonna convert this into an issue.

[davidpdrsn]

I'm gonna do some digging and compare serde_urlencoded to serde_qs and figure out if it makes sense to switch.

Some notes:

• It would seem there is no standard way to serialize nested types which appears to be the reason reqwest uses serde_urlencoded.
• warp also uses serde_urlencoded and doesn't seem to have any plans to switch.

[davidpdrsn]

From looking around I think sticking with serde_urlencoded is the safest move. Given that the workaround is just

struct Qs<T>(T);

#[axum::async_trait]
impl<T> FromRequest for Qs<T>
where
    T: serde::de::DeserializeOwned,
{
    type Rejection = Infallible;

    async fn from_request(req: &mut RequestParts) -> Result<Self, Self::Rejection> {
        // TODO: error handling
        let query = req.uri().query().unwrap();
        Ok(Self(serde_qs::from_str(query).unwrap()))
    }
}

thats probably gonna be fine.

I'll close this issue for now, but feel free to reopen if there are more thoughts/comments.

[jplatte]

FWIW I'm maintaining a patched version of serde_urlencoded with support for id=a&id=b&id=c as part of the Ruma project. Would be happy to maintain it separately if there is interest.

id=[1,2,3] already works with serde_urlencoded if you add a small helper function along the lines of

fn deserialize_array<'de, D, T>(deserializer: D) -> Result<Vec<T>, D::Error>
where
    D: Deserializer<'de>,
    T: Deserialize<'de>,
{
    let s = String::deserialize(deserializer)?;
    let v = todo!("parse the bracketed comma-separated items")
    Ok(v)
}

and use it with #[serde(deserialize_with = "deserialize_arrray")].

[ezracelli]

For the curious, the patch of serde_urlencoded mentioned above is now maintained as a separate crate: https://crates.io/crates/serde_html_form

Other relevant issues in serde_urlencoded with its maintainer's reasoning as to why this isn't natively supported:

• Serialize Vec<T> nox/serde_urlencoded#6
• Arrays support nox/serde_urlencoded#75

[jplatte]

Yes, and we have Query and Form extractors in axum-extra that use it.

[flo-at]

id=[1,2,3] already works with serde_urlencoded if you add a small helper function along the lines of

fn deserialize_array<'de, D, T>(deserializer: D) -> Result<Vec<T>, D::Error>
where
    D: Deserializer<'de>,
    T: Deserialize<'de>,
{
    let s = String::deserialize(deserializer)?;
    let v = todo!("parse the bracketed comma-separated items")
    Ok(v)
}

and use it with #[serde(deserialize_with = "deserialize_arrray")].

This does not work for strings though. At least I haven't found a way to make it work. The problem is that at that point, the string s is already url-decoded, so any [, ], or , in the array element string makes it impossible to decide which comma is a delimiter and which is part of the string element.

The workaround with the custom Extractor that uses serde_qs is not perfect either because it's not possible to apply this to only one field but only the whole struct afaik.

What's left is:

• escaping the delimiters inside the element strings
• base64(url) encode the element strings

If you guys see a better way to make this work please help me out!
Cheers.

[jplatte]

@flo-at can you describe your use case in a bit more detail? Have you seen the Query extractor in axum-extra?

[flo-at]

@jplatte After some fiddling I found a working solution that is alright. It's a combination of the axum_extra::extract::Query extractor and a #[serde(default, deserialize_with(..))] attribute, like this:

#[derive(serde::Deserialize)]
struct SearchArgs {
    #[serde(default, deserialize_with = "filter_vec_deserialize")]
    filters : Option<Vec<Filter>>,
}

filter_vec_deserlialize then uses let opt = Option::<Vec<String>>::deserialize(deserializer)?; to get the individual strings that I handle myself. This internally uses the Query extractor from axum_extra.

This way I can use my custom deserializer on the inner type Filter and let serde_html_form handle the Vec<String> part.
Thanks for the hint!

Edit: Just in case someone else stumbles across this: If you use the option I posted above, you'll realize that only vectors will work. If there is just a single parameter in the URL, it will fail to deserialize that into a Vec. You can either use serde_with::OneOrMany or a minimal implementation like this to fix this:

#[derive(Deserialize)]      
#[serde(untagged)]      
enum OneOrMany<T> {         
    One(T),                                      
    Many(Vec<T>),      
}                               

Edit2: With the current implementation, it works a lot better if you leave out the Option<_> and treat an empty Vec<_> as None. See jplatte/serde_html_form#2.

[arcstur]

Using Form in the axum-extra crate solved it for me really quickly.

[joebnb]

Using Form in the axum-extra crate solved it for me really quickly.

https://docs.rs/axum-extra/0.9.3/axum_extra/extract/struct.Query.html