Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate uploaded files #1974

Merged
merged 9 commits into from
Oct 22, 2016
Merged

Conversation

josenavas
Copy link
Contributor

@josenavas josenavas commented Oct 20, 2016

Fixes #1085

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) to 91.304% when pulling bc88df5 on josenavas:verify-biom into 66d5e4b on biocore:master.

@josenavas
Copy link
Contributor Author

Yay!! Can I have a review in here?

while redis_info['status_msg'] == 'Running':
sleep(0.05)
redis_info = loads(r_client.get(job_id))
sleep(0.05)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it just me, or is this pattern being copied and pasted?

redis_info = loads(r_client.get(job_id))
while redis_info['status_msg'] == 'Running':
sleep(0.05)
redis_info = loads(r_client.get(job_id))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is but not sure what the solutions could be as the code is duplicated in two different tests files, do you have an idea of how to solve it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd build up a library of testing functions in a separate python script to store these sorts of functions.

But its definitely not blocking - if you guys think this is a good idea, I guess this can be raised as a issue to refactor later.

redis_info = loads(r_client.get(job_id))
while redis_info['status_msg'] == 'Running':
sleep(0.05)
redis_info = loads(r_client.get(job_id))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.03%) to 91.342% when pulling 8ff5115 on josenavas:verify-biom into 66d5e4b on biocore:master.

@josenavas
Copy link
Contributor Author

@ElDeveloper @antgonza available for a quick review/merge?

@ElDeveloper
Copy link
Member

@josenavas, I'm reviewing this right now.

Copy link
Member

@ElDeveloper ElDeveloper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josenavas, this looks good, I have some minor comments. Let me know what you think and we can merge right away since tests are passing already.

job = ProcessingJob.create(
User(user_id),
Parameters.load(command, values_dict={
'template': prep_template_id,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super minor but I think this could be prep.id instead of needing this new variable, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prep_template_id is a parameter to this function, so there is no new variable created.

from qiita_db.processing_job import ProcessingJob


def wait_for_prep_information_job(prep_id, raise_if_none=True):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why this function is not documented?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops I forgot! Thanks!

res = r_client.get('prep_template_%d' % prep_id)

if raise_if_none:
assert res is not None, "unexpectedly None"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be overcautious, but what about raising an actual AssertionError, when we use assert we run the risk that if python is being executed in an "optimized" mode, these statements will be ignored. I suggest this be changed to:

if raise_if_none and res is None
    raise AssertionError('Unexpectedly None')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a really good point - changed!

Copy link
Contributor Author

@josenavas josenavas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ElDeveloper - I've addressed your comments.

from qiita_db.processing_job import ProcessingJob


def wait_for_prep_information_job(prep_id, raise_if_none=True):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops I forgot! Thanks!

res = r_client.get('prep_template_%d' % prep_id)

if raise_if_none:
assert res is not None, "unexpectedly None"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a really good point - changed!

job = ProcessingJob.create(
User(user_id),
Parameters.load(command, values_dict={
'template': prep_template_id,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prep_template_id is a parameter to this function, so there is no new variable created.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.04%) to 91.327% when pulling f99b501 on josenavas:verify-biom into 66d5e4b on biocore:master.

@ElDeveloper ElDeveloper merged commit 33ff464 into qiita-spots:master Oct 22, 2016
@ElDeveloper
Copy link
Member

Thanks @josenavas!

@ElDeveloper
Copy link
Member

@josenavas, GitHub seems to be down from UCSD campus :(
https://twitter.com/githubstatus (as well as Twitter).

On (Oct-21-16|10:20), Jose Navas wrote:

@ElDeveloper @antgonza available for a quick review/merge?

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#1974 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants