Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

postgres 9.5->13.4 #3144

Merged
merged 16 commits into from Oct 20, 2021
Merged

postgres 9.5->13.4 #3144

merged 16 commits into from Oct 20, 2021

Conversation

antgonza
Copy link
Member

@antgonza antgonza commented Sep 9, 2021

No description provided.

@antgonza antgonza changed the title postgres 9.5->13.4 [WIP] postgres 9.5->13.4 Sep 9, 2021
Copy link
Member Author

@antgonza antgonza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some general comments to help with review.

@@ -184,7 +184,12 @@ def __init__(self, id_):

with qdb.sql_connection.TRN:
self._check_subclass()
if not self._check_id(id_):
try:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main error with the new version pgsql-13.4 is that it actually verifies that the uuids are uuids (vs other kind of strings) and it raises a specific error. However, this is not used in qiita and I don't see this being used in the near future + most tests in the code actually use random strings as identifiers so it is better to simply try to check the id here and if it fails (for whatever reason) report that the id doesn't exist.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know, see suggestion below for being specific on the exception

@@ -837,7 +837,7 @@ def _common_extend_steps(self, md_template):

# check that we are within the limit of number of samples
ms = self.max_samples()
nsamples = len(existing_samples) + len(new_samples) + len(self)
nsamples = len(existing_samples) + len(new_samples)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is "interesting" not sure how but here we are actually adding twice the existing samples: (1) len(existing_samples) and (2) len(self).

@@ -60,6 +60,8 @@ def get(self):
continue

for job in cmd.processing_jobs:
if job.hidden:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not showing hidden jobs for admins

@@ -42,7 +42,7 @@ def test_samples_detail(self):
obs = _sample_details(qiita_db.study.Study(1),
['1.SKD7.640191', 'doesnotexist'])
self.assertEqual(len(obs), len(exp))
self.assertEqual(obs, exp)
self.assertCountEqual(obs, exp)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remember in python3 this actually tests the list/array values and not only the count.

@@ -291,7 +291,7 @@ def delete_sample_or_column(job):


def _delete_analysis_artifacts(analysis):
aids = [a.id for a in analysis.artifacts]
aids = list(set([a.id for a in analysis.artifacts]))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

making sure we only get one.

@antgonza antgonza changed the title [WIP] postgres 9.5->13.4 postgres 9.5->13.4 Oct 14, 2021
@coveralls
Copy link

coveralls commented Oct 15, 2021

Coverage Status

Coverage decreased (-0.03%) to 91.093% when pulling 30b39cb on antgonza:pgsql-13.4 into 80ea238 on qiita-spots:dev.

# making sure the analysis, first thing to delete, still exists
self.assertTrue(Analysis.exists(1))

# delete everything from the EBI submissions and the processing job so
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is actually extremely complicated because the "populate.sql" reuses files/values so when the code rolls back, it tries to overwrite the same files and it goes 🍌. Additionally, we are testing the study delete in a new study below. Thus, removing this section.

@antgonza antgonza requested a review from wasade October 15, 2021 15:14
qiita_db/base.py Outdated
if not self._check_id(id_):
try:
_id = self._check_id(id_)
except Exception:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
except Exception:
except psycopg2.errors.InvalidTextRepresentation:

@@ -837,7 +837,7 @@ def _common_extend_steps(self, md_template):

# check that we are within the limit of number of samples
ms = self.max_samples()
nsamples = len(existing_samples) + len(new_samples) + len(self)
nsamples = len(curr_samples) + len(new_samples)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nsamples should be the total of samples a new file will have: current+new; however, the line before was adding existing_samples [the samples already in the prep that are in the new list] + the new samples + len(self) [the total number of samples in the prep].

@@ -155,8 +155,8 @@ def test_get_valid(self):
exp = {'is_public': False,
'has_sample_information': True,
'sample_information_has_warnings': False,
'preparations': [{'id': 1, 'has_artifact': True},
{'id': 2, 'has_artifact': True}]
'preparations': [{'id': 2, 'has_artifact': True},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the test is sensitive to order from the database, would it make sense to sort observed and expected based on a key?

@antgonza
Copy link
Member Author

@wasade; I think this is ready - thank you for your reviews.

@wasade wasade merged commit 22b0f4f into qiita-spots:dev Oct 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants