-
Notifications
You must be signed in to change notification settings - Fork 325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: Add support for functional tests to run in mint. #547
Conversation
505c764
to
5302134
Compare
tests/functional/tests.py
Outdated
# return empty bytes to indicate EOF. | ||
return b'' | ||
|
||
data = b''.join([b'o' for i in range(amt)]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this doesn't look like random data.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad..local change is not pushed.
tests/functional/tests.py
Outdated
metadata={'x-amz-storage-class': 'STANDARD_IA'}) | ||
|
||
# Fput a large file. | ||
print("Upload a largfile through multipart upload") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: s/largfile/largefile
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
@harshavardhana does this also address #548 ? |
Yes it does @nitisht |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested with play server. LGTM.
Changed a bunch of things, added more randomness as asked by @krisis . Please take a look again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We will be using streams of generated data for functional tests run with minio/mint. With that in mind, doesn't it make sense to use LimitedRandomReader based streams unconditionally in tests?
tests/functional/tests.py
Outdated
|
||
# make translation table from 0..255 to 97..122 | ||
bal = [c.encode('ascii') for c in ascii_lowercase] | ||
data = b''.join([bal[int(random() * 26)] for _ in range(amt)]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a consequence to the previous comment, we should compute the remaining bytes to be returned.
amt = min(amt, self._limit - self._offset_location)
tests/functional/tests.py
Outdated
How much of the content to read. | ||
""" | ||
# If offset is bigger than size. Treat it as EOF return here. | ||
if self._offset_location + amt > self._limit: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should return empty bytes (b''
) only when self._offset_location == self._limit
. E.g, when
self._limit = 10
, self._offset = 9
and amt = 5
, we should return the last byte and not b''
.
For path APIs we need local files. Streaming is only for generating content for streaming APIs |
@harshavardhana IIUC, we are running file based object operations (like FPutObject) inside mint environment and stream-based object operations outside of mint environment. Shouldn't we run all kinds of operations in both environments? |
Yes ideally, but there seems to be some confusing aspects floating around on the nature of functional tests themselves. Should be used to test SDK or server and what is their rightful place. Updated the PR as well to provide mint as a sub-set behavior only meant to be used as data dir location. If not we auto-generate this data such that some APIs are not skipped on regular testing without mint. |
This PR is needed to support multiple environments in which the following test script will be run. Re-purpose the limit reader and make it a random reader.
This PR is needed to support multiple environments in
which the following test script will be run.