Handling multi-valued fields?

[ibrewster]

How does this library handle multi-valued fields? For example, if the input form allows for the entry of an arbitrary number of "people", each of which has a field height, weight, etc?

If I understand the source code correctly, as data comes in it is simply appended to an array (for ValueTarget()), which is joined to a single bytes string when value is called, which would leave me with something like 14022036 if there were three weight values submitted (140,220,36), with no way to know where to split the values.

I could make a custom subclass that doesn't join the array, but that would only work if I could be sure each "chunk" was a full, discrete, value, which I do not believe is the case...

[ibrewster]

Looking at the source code a bit more, it looks like perhaps I could utilize the "on_finish" function to split the individual values? I'm assuming that will be called each time a value is "stored" to the target. Is that an appropriate use of the function?

[siddhantgoel]

Do you have an example of what the request body looks like on the server side in such cases?

Depending on what the server sees, I would also probably suggest some variation of ValueTarget. If there's a fixed delimiter that's known before hand, you could watch for it inside on_data_received and adapt the values collection. But an example would be helpful before I can recommend something.

[ibrewster]

Sure, I'll put together a minimal working example. It's a fairly common case, however: for example checkboxes in a HTML form where you could have one or more checked, and should get a list of checked values as a result. Said form is designed by simply giving each checkbox in the set the same "name" attribute. (see https://html5-tutorial.net/forms/checkboxes/ under the section of "multiple choices", for example).

Like I said though, I'll go ahead and put together a minimal working example so you can see not only the HTML, but also what the server gets in the body. May take a couple of days.

[ibrewster]

So using the example HTML from the link above, I get a very simple request body (as retrieved by calling flask.request.stream.read():

favorite_pet=Cats&favorite_pet=Dogs&favorite_pet=Birds

Assuming I check all three boxes. Obviously if I only check two of them, then I only get two values, not all three as shown here :-)

Of course, my actual form is much more complicated, including file uploads and many more fields - I can flesh out this example if needed to more closely match the actual use case. However, this should (I think) demonstrate the portion of the body related to multi-valued fields.

The code I used to test this, if interested, is the following:

import flask

app = flask.Flask(__name__)


@app.route('/')
def index():
    html = """
    <form method="post" action="/testPost">      
    <fieldset>      
        <legend>What is Your Favorite Pet?</legend>      
        <input type="checkbox" name="favorite_pet" value="Cats">Cats<br>      
        <input type="checkbox" name="favorite_pet" value="Dogs">Dogs<br>      
        <input type="checkbox" name="favorite_pet" value="Birds">Birds<br>      
        <br>      
        <input type="submit" value="Submit now" />      
    </fieldset>      
</form>
    """
    return html


@app.route('/testPost', methods = ["POST"])
def test_post():
    body = flask.request.stream.read()
    print(body)
    return ''


app.run()

[ibrewster]

I realized just now that the body content is a bit different if using multipart/form-data encoding on the form (as will be the case if uploading any files), so I modified the HTML portion as such:

    <form method="post" action="/testPost", enctype="multipart/form-data">      
        <legend>What is Your Favorite Pet?</legend>
        <input type="checkbox" name="favorite_pet" value="Cats">Cats<br>      
        <input type="checkbox" name="favorite_pet" value="Dogs">Dogs<br>      
        <input type="checkbox" name="favorite_pet" value="Birds">Birds<br>      
        <br>
        Upload a photo of your pet: <input type=file name=fileUpload>
        <br>
        <input type="submit" value="Submit now" />
</form>

with the following resulting body:

b'------WebKitFormBoundaryM31StAfyAYukZZNw\r\nContent-Disposition: form-data; name="favorite_pet"\r\n\r\nCats\r\n------WebKitFormBoundaryM31StAfyAYukZZNw\r\nContent-Disposition: form-data; name="favorite_pet"\r\n\r\nBirds\r\n------WebKitFormBoundaryM31StAfyAYukZZNw\r\nContent-Disposition: form-data; name="fileUpload"; filename=""\r\nContent-Type: application/octet-stream\r\n\r\n\r\n------WebKitFormBoundaryM31StAfyAYukZZNw--\r\n'

So, basically it appears to treat each "instance" of the field as a separate field, which just happens to have the same name.

[siddhantgoel]

Interesting. Thanks for sending the form body.

One solution could be to implement a target that can "collect" values? Something along the lines of ValueCollectorTarget. Similar to ValueTarget but if you're expecting multiple values for a given field then we put all of them inside a in-memory list or something inside the class.

[ibrewster]

Agreed. The question is at what point to "collect" the individual values - presumably you can't do it at the "on_data_received" level, because a chunk may only be a partial value (presumably). As such, I'm thinking the on_finish function is the correct location, but as I don't know what triggers that function, I'm not completely confidant in that solution. I have implemented this that appears to work in testing:

class ListTarget(BaseTarget):
    """ValueTarget stores the input in an in-memory list of bytes.
    This is useful in case you'd like to have the value contained in an
    in-memory string.
    """

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self._temp_value = []
        self._values = []

    def on_data_received(self, chunk: bytes):
        self._temp_value.append(chunk)
        
    def on_finish(self):
        value = b''.join(self._temp_value)
        self._values.append(value)
        
        self._temp_value = []

    @property
    def value(self):
        return self._values 

... but as I mentioned, not being sure what triggers the on_finish function means I'm not sure that won't break if, for example, the value is long enough to span multiple chunks.

[siddhantgoel]

on_finish is called by target.finish which in turn is called by Part.finish which signifies the end of a "part" of the form-data encoded bytes. So when on_finish is called, we should be done with a single part. So it should be the right place to do the value collection.

Although we should also be able to use the data you provided in the earlier comment and write a test case for testing such a ListTarget.

[ibrewster]

Agreed. I can try to get around to writing such a test case in the next week or two, depending on how things go at work.

[siddhantgoel]

Awesome, thanks!