Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: Add support for functional tests to run in mint. #547

Merged
merged 1 commit into from
Jul 6, 2017

Conversation

harshavardhana
Copy link
Member

This PR is needed to support multiple environments in
which the following test script will be run.

@harshavardhana
Copy link
Member Author

harshavardhana commented Jul 4, 2017

This is ready for review @krisis @nitisht @vadmeste with mint related changes.

@deekoder deekoder requested review from nitisht and krisis July 4, 2017 13:32
# return empty bytes to indicate EOF.
return b''

data = b''.join([b'o' for i in range(amt)])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't look like random data.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad..local change is not pushed.

metadata={'x-amz-storage-class': 'STANDARD_IA'})

# Fput a large file.
print("Upload a largfile through multipart upload")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: s/largfile/largefile

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@nitisht
Copy link
Contributor

nitisht commented Jul 4, 2017

@harshavardhana does this also address #548 ?

@harshavardhana
Copy link
Member Author

@harshavardhana does this also address #548 ?

Yes it does @nitisht

Copy link
Contributor

@nitisht nitisht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with play server. LGTM.

@harshavardhana
Copy link
Member Author

Changed a bunch of things, added more randomness as asked by @krisis . Please take a look again.

Copy link
Member

@krisis krisis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will be using streams of generated data for functional tests run with minio/mint. With that in mind, doesn't it make sense to use LimitedRandomReader based streams unconditionally in tests?


# make translation table from 0..255 to 97..122
bal = [c.encode('ascii') for c in ascii_lowercase]
data = b''.join([bal[int(random() * 26)] for _ in range(amt)])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a consequence to the previous comment, we should compute the remaining bytes to be returned.
amt = min(amt, self._limit - self._offset_location)

How much of the content to read.
"""
# If offset is bigger than size. Treat it as EOF return here.
if self._offset_location + amt > self._limit:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should return empty bytes (b'') only when self._offset_location == self._limit. E.g, when
self._limit = 10, self._offset = 9 and amt = 5, we should return the last byte and not b''.

@harshavardhana
Copy link
Member Author

For path APIs we need local files. Streaming is only for generating content for streaming APIs

@krisis
Copy link
Member

krisis commented Jul 5, 2017

@harshavardhana IIUC, we are running file based object operations (like FPutObject) inside mint environment and stream-based object operations outside of mint environment. Shouldn't we run all kinds of operations in both environments?

@harshavardhana
Copy link
Member Author

@harshavardhana IIUC, we are running file based object operations (like FPutObject) inside mint environment and stream-based object operations outside of mint environment. Shouldn't we run all kinds of operations in both environments?

Yes ideally, but there seems to be some confusing aspects floating around on the nature of functional tests themselves. Should be used to test SDK or server and what is their rightful place.

Updated the PR as well to provide mint as a sub-set behavior only meant to be used as data dir location. If not we auto-generate this data such that some APIs are not skipped on regular testing without mint.

This PR is needed to support multiple environments in
which the following test script will be run.

Re-purpose the limit reader and make it a random reader.
@deekoder deekoder merged commit 2ce227c into minio:master Jul 6, 2017
@harshavardhana harshavardhana deleted the mint branch July 6, 2017 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants