improve falcon form data read part

[kemingy]

@yedpodtrzitko It depends what you are trying to achieve. Do you want to buffer whole files in memory? Or do you want to spool them to temp files like some other frameworks do?

Falcon even puts a cap on a maximum amount of data that can be referenced this way (await part.get_data()) in order to avoid surprises such as running out of memory.

Use await part.stream.read() to read the whole part as a bytestring, or await part.stream.pipe(async_file), or read by chunks, and store the result somewhere. You'll probably need to introduce some new object type to hold these attributes.

Originally posted by @vytas7 in #225 (comment)

[kemingy]

Keeping two releases is hard 😢

Will release v1 and improve this later.

[vytas7]

@kemingy I'm happy to help or at least advise, but could you shed some light on Spectree's design wrt my questions above. What type of data from a multipart form is typically sent to the validator?

[kemingy]

@kemingy I'm happy to help or at least advise, but could you shed some light on Spectree's design wrt my questions above. What type of data from a multipart form is typically sent to the validator?

Thanks! Here is the current implementation for the form file type:

spectree/spectree/models.py

Lines 173 to 192 in 3deaf00

	class BaseFile:
	"""
	An uploaded file included as part of the request data.
	"""
	@classmethod
	def __get_validators__(cls):
	# one or more validators may be yielded which will be called in the
	# order to validate the input, each validator will receive as an input
	# the value returned from the previous validator
	yield cls.validate
	@classmethod
	def __modify_schema__(cls, field_schema: Dict[str, Any]) -> None:
	field_schema.update(format="binary", type="string")
	@classmethod
	def validate(cls, value: Any):
	# https://github.com/luolingchun/flask-openapi3/blob/master/flask_openapi3/models/file.py
	return value

In my option, I would prefer to leave the large form body to falcon (since it's already there). I feel SpecTree doesn't really "validate" the body. I wonder if SpecTree should provide a struct that when users try to read it, users actually read from falcon. Buffering it to a temp file also looks good.

[vytas7]

I see. Falcon offers parsing on-the-fly without buffering anything anywhere, so spooling to a temp file should be built in the Spectree plugin, if this is desired. Because iterating to the next part will force the stream to be read through (and discarded if the data is not used).

Alternatively one could potentially pause iteration at the part of interest, and hand that off to Spectree, since from what I've seen validation doesn't support multiple files, or does it? But not sure if such approach wouldn't cause confusion either.