Check if folder already exists by checking its actual name, not the name of the entry to -d flag #2425

gmotzespina · 2019-09-03T06:07:29Z

Removes folder names based on it's name not on data_name

jcohenadad · 2019-09-03T19:54:54Z

scripts/sct_download_data.py

@@ -127,24 +128,46 @@ def main(args=None):
    data_name = arguments['-d']
    verbose = int(arguments.get('-v'))
    sct.init_sct(log_level=verbose, update=True)  # Update log level
-    dest_folder = arguments.get('-o', os.path.abspath(os.curdir))
+    dest_tmp_folder = arguments.get('-o', os.path.abspath(os.curdir)+'/tmp')


"/" doesn't work in Windows, and what if user already has a local folder called "./tmp/"?
there are better ways to generate temp folders, see class sct_utils.TempFolder

Oh ok, nice one, looking at it.

jcohenadad · 2019-09-03T19:55:48Z

scripts/sct_download_data.py

+    # unzip
+    unzip(tmp_file, dest_tmp_folder, verbose)
+    data_extracted_name = data_name
+    if dest_tmp_folder != os.path.abspath(os.curdir)+'/bin':


Because it is the path where the installation requirements are downloaded.

sct_download_data is not only used for installing binaries (look at the help)

jcohenadad

Travis failing: https://travis-ci.org/neuropoly/spinalcordtoolbox/jobs/580121506#L1089

likely because of problem when downloading the data

jcohenadad · 2019-09-10T16:03:16Z

scripts/sct_download_data.py

-                              'https://www.neuro.polymtl.ca/_media/downloads/sct/20180613_deepseg_lesion_models.zip'],
-        'c2c3_disc_models': ['https://osf.io/t97ap/?action=download',
-                            'https://www.neuro.polymtl.ca/_media/downloads/sct/20190117_c2c3_disc_models.zip']
+        "sct_example_data": [


@gmotzespina please see the convention: https://github.com/neuropoly/spinalcordtoolbox/wiki/Single-or-double-quotes

unless you have a good reason for going with double quote?

I started testing out Black, and it changed my file to double quotes, since it is kind of the black standard. I'll revert all of this changes before sending the request again.

While checking out black I also figured out that using double quotes instead of single might be a better option since we can write stuff like this: "it's a test" otherwise, while using single quotes we would need to write the same text this way: 'it\'s a test'.

What do you think about it?

the double quote argument is indeed a good argument, but we could also make the argument that with single quote, we can do that: 'When using this option, enter parameter "n"'. Also, single quotes makes the code lighter.

Anyway, regardless the decision, this should be mixed with this PR, which is about fixing a specific problem, not changing code cosmetic. Otherwise, it makes it difficult for me to review your code.

so: open a separate issue regarding code cosmetic, and we will address it specifically

Right, I didn't thought about the options.

Yeah, I'll revert those changes once the PR is ready, they just got mixed because of Black, but it was not intended to have those changes in this PR.

jcohenadad · 2019-09-10T16:06:57Z

scripts/sct_download_data.py

-        sct.rmtree(data_name)
+    # Extracting the binaries doesn't require removing the previously created directories
+    # since binaries are files, not directories
+    if "binaries" in data_name:


will break if compressed files don't have the term "binaries" in it. Why not simply:

check if the uncompressed zip gives you a folder

check if the folder already exists

if it does, remove it

Yeah, I figured out that this would happen, I'm currently working on a fix. to have this done in a cleaner way.

jcohenadad · 2019-09-10T16:07:43Z

scripts/sct_download_data.py

+        for folder in dirs:
+            folders.append(os.path.join(root, folder))
+    data_extracted_path = folders[0]
+    directories = data_extracted_path.split("/")


python has built-in modules to do such manipulations which will break with windows (when, one day, SCT will be compatible with windows...)

…xist

temp folder was already displayed by the previous line of code

jcohenadad

👍

…ame_2390 Check if folder already exists by checking its actual name, not the name of the entry to -d flag Former-commit-id: da80ab466bdefc3f979d6a1991735f1533838b6a [formerly 23c02c9 [formerly 8746c6f]] Former-commit-id: f0c6acc3c6e40757a255a2fa097c065acaa29f79 Former-commit-id: 3eb4950

…ame_2390 Check if folder already exists by checking its actual name, not the name of the entry to -d flag Former-commit-id: 8746c6f

gmotzespina self-assigned this Sep 3, 2019

gmotzespina added the card:WORK_IN_PROCESS label Sep 3, 2019

gmotzespina added this to the 4.0.1 milestone Sep 3, 2019

gmotzespina added card:TO_REVIEW and removed card:WORK_IN_PROCESS labels Sep 3, 2019

gmotzespina requested a review from jcohenadad September 3, 2019 11:39

jcohenadad reviewed Sep 3, 2019

View reviewed changes

jcohenadad requested changes Sep 3, 2019

View reviewed changes

gmotzespina added card:WORK_IN_PROCESS and removed card:TO_REVIEW labels Sep 4, 2019

gmotzespina changed the title ~~Remove folder based on name not on data_name~~ WIP Remove folder based on name not on data_name Sep 4, 2019

jcohenadad reviewed Sep 10, 2019

View reviewed changes

Download data on temporary folders and remove if the folder already e…

7914530

…xist

gmotzespina force-pushed the gmo/folder_removal_based_on_name_2390 branch from 0e910c6 to 7914530 Compare September 10, 2019 17:37

Add shutil import

ef0f5e1

gmotzespina added card:TO_REVIEW and removed card:WORK_IN_PROCESS labels Sep 10, 2019

gmotzespina requested a review from jcohenadad September 10, 2019 21:06

jcohenadad modified the milestones: 4.0.1, 4.0.3 Sep 12, 2019

jcohenadad added 3 commits September 12, 2019 22:14

Merge branch 'master' into gmo/folder_removal_based_on_name_2390

5615ab6

sct_download_data: Changed entry course_beijing19

6e316ab

sct_download_data: Removed print

0ac703d

temp folder was already displayed by the previous line of code

jcohenadad approved these changes Sep 13, 2019

View reviewed changes

jcohenadad changed the title ~~WIP Remove folder based on name not on data_name~~ Remove folder based on its name, not on data_name Sep 13, 2019

jcohenadad changed the title ~~Remove folder based on its name, not on data_name~~ Check if folder already exists by checking its actual name, not the name of the entry to -d flag Sep 13, 2019

jcohenadad removed the card:TO_REVIEW label Sep 13, 2019

jcohenadad added enhancement category: improves performance/results of an existing feature sct_download_data context: labels Sep 13, 2019

gmotzespina merged commit 8746c6f into master Sep 13, 2019

This was referenced Sep 20, 2019

Error when copying unzipped files if destination folder already exists #2448

Closed

Fixed that appeared when copying unzipped files if destination files already exists #2449

Merged

jcohenadad pushed a commit that referenced this pull request Dec 18, 2019

Merge pull request #2425 from neuropoly/gmo/folder_removal_based_on_n…

23c02c9

…ame_2390 Check if folder already exists by checking its actual name, not the name of the entry to -d flag Former-commit-id: 8746c6f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check if folder already exists by checking its actual name, not the name of the entry to -d flag #2425

Check if folder already exists by checking its actual name, not the name of the entry to -d flag #2425

gmotzespina commented Sep 3, 2019 •

edited

Loading

jcohenadad Sep 3, 2019

gmotzespina Sep 4, 2019

jcohenadad Sep 3, 2019

gmotzespina Sep 4, 2019

jcohenadad Sep 4, 2019

jcohenadad left a comment

jcohenadad Sep 10, 2019

gmotzespina Sep 10, 2019

jcohenadad Sep 10, 2019

gmotzespina Sep 10, 2019

jcohenadad Sep 10, 2019

gmotzespina Sep 10, 2019

jcohenadad Sep 10, 2019

jcohenadad left a comment

Check if folder already exists by checking its actual name, not the name of the entry to -d flag #2425

Check if folder already exists by checking its actual name, not the name of the entry to -d flag #2425

Conversation

gmotzespina commented Sep 3, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcohenadad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcohenadad left a comment

Choose a reason for hiding this comment

gmotzespina commented Sep 3, 2019 •

edited

Loading