Issue #26 test cases and uri code updated #31

DaasDaham · 2020-03-02T23:13:06Z

Tested with all existing test cases (all passed) and added new test cases.
List of changes:

Call _verify_srametadb(sqlite_file) function before calling super() to ensure first that the file is right since this function also calls BASEdb.
Changed sqlite.connect() to use uri syntax which can be used to control details of the newly created database connection.
Added a check to replace all the '?' characters in input as they were causing the input to be truncated at the first '?' character, this might be happening because the uri syntax also uses ? to start mode declaration.

saketkc · 2020-03-03T19:11:27Z

pysradb/basedb.py

-        self.db = sqlite3.connect(self.sqlite_file)
+        # Originally sqlite3.connect(self.sqlite_file)
+        self.sqlite_file = self.sqlite_file.replace('?','')
+        self.db = sqlite3.connect('file:{}?mode=rw'.format(self.sqlite_file), uri=True)


Any particular reason to have the rw mode and not just r?

sqlite3.connect() has rwc (read-write-create) mode and it gives both read-write permission to user as well as creates the file if it is not present. I changed it to rw mode so that it does not create a new file if it does not exist and to keep the database permissions same as the original command. If we want to give only read permission I'll change it to ro, please confirm.

We do not need the write/create permission. The file is downloaded from an online resource. If it does not exist, it should just throw an error. Probably it is simpler to just check os.path.isfile(input_file)?

Thanks for the clarification. I'll change the mode to ro. Actually, it does throw an error in case no file is found sqlite3.OperationalError: unable to open database file which is caught in _verify_srametadb() function so there is no need for a separate check.

saketkc · 2020-03-03T19:13:39Z

pysradb/basedb.py

@@ -24,7 +25,9 @@ def __init__(self, sqlite_file):

    def open(self):
        """Open sqlite connection."""
-        self.db = sqlite3.connect(self.sqlite_file)
+        # Originally sqlite3.connect(self.sqlite_file)
+        self.sqlite_file = self.sqlite_file.replace('?','')


Is there a need to replace '?'?

The '?' character was causing the input to be truncated after '?' in uri syntax. But now since you have clarified that user does not provide a wrong path, I'll remove this check.

saketkc · 2020-03-03T19:15:10Z

tests/_test_sradb.py

+    wrong_path = os.path.join(os.getcwd(), "data", "SRAmet")
+    try:
+        db = SRAdb(path)
+    except:


This should explicitly mention the Exception it is catching.

saketkc · 2020-03-03T19:15:58Z

tests/_test_sradb.py

+        db = SRAdb(path)
+    except:
+        pass
+    assert os.path.isfile(wrong_path) == False


This is redundant test (as is user provided wrong path) and can be removed.

Alright, thanks for clarifying. I will remove it.
I'll add other test cases then

saketkc · 2020-03-03T19:16:16Z

tests/_test_sradb.py

+        db = SRAdb(path)
+    except:
+        pass
+    assert os.path.isfile(path) == False


Redundant check, can be removed.

saketkc · 2020-03-03T19:16:28Z

tests/_test_sradb.py

+    path = os.path.join(os.getcwd(), "data", "{}".format(wrong_filename))
+    try:
+        db = SRAdb(path)
+    except:


Requires explicit Exception.

saketkc · 2020-03-03T19:16:55Z

tests/_test_sradb.py

+        pass
+    try:
+        db = SRAdb(wrongfile_path)
+        assert False


Not sure what is being tested here..

This function is to check if it throws an error in case a wrong filename was provided by the user and the wrong filename already existed. It should throw error sqlite3.OperationalError: no such table: metaInfo.
However, since we are assuming that the user provides correct input then this test is also redundant.

saketkc · 2020-03-03T19:17:19Z

tests/_test_sradb.py

+        db = SRAdb(wrongfile_path)
+        assert False
+    except Exception as e:
+        assert True


What use case is this test case handling?

saketkc · 2020-03-03T19:19:50Z

Thanks @DaasDaham for your contribution! I have left inline reviews. _test_sradb.py would not run currently. It needs to be moved to test_sradb.py if it is to be detected by pytest. The problem however with that is the SRAmetadb.sqlite.gz can sometimes take a long time to be decompressed and hence the tests often fail on travis. I haven't tested it for a while, so its definitely worth a shot.

DaasDaham · 2020-03-05T01:05:26Z

Thanks @DaasDaham for your contribution! I have left inline reviews. _test_sradb.py would not run currently. It needs to be moved to test_sradb.py if it is to be detected by pytest. The problem however with that is the SRAmetadb.sqlite.gz can sometimes take a long time to be decompressed and hence the tests often fail on travis. I haven't tested it for a while, so its definitely worth a shot.

Yes travis is throwing no space left error while extracting SRAmetadb.sqlite.gz. Can this be resolved?

saketkc · 2020-03-05T05:10:24Z

Hi @DaasDaham, I cannot think of an alternate to test the SRAmetadb mode. For now I would suggest, that we move back test_sradb.py to _test_sradb.py and add a new test_sradb.py with the only test being the one which throws an error if the file is either not present or not a valid sqlite file.

Thanks once again for your contribution!

DaasDaham · 2020-03-05T20:48:48Z

Hi @saketkc I have made all the required changes. Please have a look

DaasDaham · 2020-03-08T15:56:28Z

Hi @DaasDaham, I cannot think of an alternate to test the SRAmetadb mode. For now I would suggest, that we move back test_sradb.py to _test_sradb.py and add a new test_sradb.py with the only test being the one which throws an error if the file is either not present or not a valid sqlite file.

Thanks once again for your contribution!

Hi @saketkc, I haven't heard from you regarding this issue, I have made the changes that you mentioned in my latest commit. Please have a look at it.

saketkc · 2020-03-07T20:37:33Z

tests/test_sradb.py

+    path = 'SRAmetadb.sqlite' 
+    try:
+        db = SRAdb(path)
+        assert False


Why assert False?

This is because while testing, SRAmetadb.sqlite file won't exist in the current directory, hence if it succeeds in establishing a connection, then it would be wrong, hence the assert False.

Ummm, but an error would be raised before hand and hence this statement is practically unreachable?

Yes, practically it shouldn't reach it.
This statement is just for edge case in case it makes a connection...

I guess it should be removed then...

Done, I've removed it in the latest commit.
But travis is failing to build even though pytest passes all checks. It may be because of the black --check statement.

saketkc · 2020-03-07T20:38:42Z

tests/test_sradb.py

+    try:
+        db = SRAdb(path)
+        assert False
+    except SystemExit:


Can you help me understand which case would a SystemExit error arise?

In case if a file does not exist, sqlite3.connect() (line 27) in basedb.py will throw an Operational error which will be caught in except: statement (line 152 in sradb.py) which will eventually lead to a sys.exit(1) on line 157 in sradb.py. This is the SystemExit that my function would catch.

removed assert False statement

saketkc · 2020-03-10T06:25:18Z

@DaasDaham can you run black . and recommit? The tests are failing because of the formatting.

codecov · 2020-03-10T06:51:57Z

Codecov Report

Merging #31 into master will increase coverage by 14.84%.
The diff coverage is 59.62%.

@@             Coverage Diff             @@
##           master      #31       +/-   ##
===========================================
+ Coverage   24.17%   39.02%   +14.84%     
===========================================
  Files           8        5        -3     
  Lines        1187      984      -203     
===========================================
+ Hits          287      384       +97     
+ Misses        900      600      -300

Impacted Files	Coverage Δ
pysradb/cli.py	`0.00% <0.00%> (ø)`
pysradb/download.py	`22.41% <28.00%> (+4.63%)`	⬆️
pysradb/basedb.py	`41.86% <50.00%> (-55.76%)`	⬇️
pysradb/sraweb.py	`81.66% <81.75%> (ø)`
pysradb/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99f738e...783e017. Read the comment docs.

DaasDaham · 2020-03-10T06:55:23Z

@DaasDaham can you run black . and recommit? The tests are failing because of the formatting.

Thanks for the help! , it builds successfully now

saketkc · 2020-03-10T07:07:21Z

Thanks for your contribution @DaasDaham! Are you okay with your name being added in CONTRIBUTORS?

DaasDaham · 2020-03-10T07:57:10Z

Thanks for your contribution @DaasDaham! Are you okay with your name being added in CONTRIBUTORS?

Sure, thanks!.
Just one thing, this latest commit introduced a change in the print statement on line 46 in cli.py. I am really sorry for this, I have opened a pull request to revert it back to original print statement #33

saketkc · 2020-03-22T18:21:01Z

Hi @DaasDaham, are you okay with me adding you to the AUTHORS list?

DaasDaham added 4 commits March 1, 2020 14:32

Solves Issue_26 using uri filenames.

33920be

Merge remote-tracking branch 'upstream/master' into issue_7

c5c29c4

Issue saketkc#26 test_case added

b7be4d4

Issue saketkc#26 test_case added updated

a21b62d

DaasDaham mentioned this pull request Mar 2, 2020

Solves Issue #26 using uri filenames. #27

Closed

saketkc requested changes Mar 3, 2020

View reviewed changes

DaasDaham added 2 commits March 4, 2020 23:19

test cases updated

7d9db56

test cases updated

f781f1d

test_sradb.py changes

952a5c5

saketkc reviewed Mar 8, 2020

View reviewed changes

updated test_not_valid_file() in test_sradb.py

a26a89a

removed assert False statement

DaasDaham added 3 commits March 10, 2020 12:16

black

ec87aaa

merge

66e2215

reformatted with black

783e017

saketkc merged commit babd8f6 into saketkc:master Mar 10, 2020

saketkc mentioned this pull request Mar 10, 2020

Python Api #25

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue #26 test cases and uri code updated #31

Issue #26 test cases and uri code updated #31

DaasDaham commented Mar 2, 2020 •

edited

saketkc Mar 3, 2020

DaasDaham Mar 3, 2020

saketkc Mar 3, 2020

DaasDaham Mar 3, 2020

saketkc Mar 3, 2020

DaasDaham Mar 3, 2020

saketkc Mar 3, 2020

saketkc Mar 3, 2020

DaasDaham Mar 3, 2020

saketkc Mar 3, 2020

saketkc Mar 3, 2020

saketkc Mar 3, 2020

DaasDaham Mar 3, 2020

saketkc Mar 3, 2020

saketkc commented Mar 3, 2020

DaasDaham commented Mar 5, 2020

saketkc commented Mar 5, 2020

DaasDaham commented Mar 5, 2020

DaasDaham commented Mar 8, 2020

saketkc Mar 7, 2020

DaasDaham Mar 9, 2020

saketkc Mar 9, 2020

DaasDaham Mar 9, 2020

saketkc Mar 9, 2020

DaasDaham Mar 10, 2020

saketkc Mar 7, 2020

DaasDaham Mar 9, 2020

saketkc commented Mar 10, 2020

codecov bot commented Mar 10, 2020 •

edited

DaasDaham commented Mar 10, 2020

saketkc commented Mar 10, 2020

DaasDaham commented Mar 10, 2020

saketkc commented Mar 22, 2020

Issue #26 test cases and uri code updated #31

Issue #26 test cases and uri code updated #31

Conversation

DaasDaham commented Mar 2, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saketkc commented Mar 3, 2020

DaasDaham commented Mar 5, 2020

saketkc commented Mar 5, 2020

DaasDaham commented Mar 5, 2020

DaasDaham commented Mar 8, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saketkc commented Mar 10, 2020

codecov bot commented Mar 10, 2020 • edited

Codecov Report

DaasDaham commented Mar 10, 2020

saketkc commented Mar 10, 2020

DaasDaham commented Mar 10, 2020

saketkc commented Mar 22, 2020

DaasDaham commented Mar 2, 2020 •

edited

codecov bot commented Mar 10, 2020 •

edited