Only if Modified Recipe to then run Zip or other methods

[TMLWinston]

The base problem that causes this request:
I have a Download task that downloads multiple files to a directory. These files are huge and and I am able to get them to download to a subdirectory using the Download tasks and the check if modified is great. The problem is the "Extraction" phase of the download the Zip is always executing.

I am looking for an easy way to keep the easy of use of the multiple file download with a prevention of unzipping if a file is not modified.

Current Example:

task temp_download_example(type: Download) {
    String base = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.base_filename.toString()
    String file1 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file1_filename.toString()
    String file2 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file2_filename.toString()
    String file3 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file3_filename.toString()
    String file4 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file4_filename.toString()
    String file5 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file5_filename.toString()
    String file6 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file6_filename.toString()
    String file7 = gradle.ext.nexusUrl.toString() + gradle.ext.repo_directory.toString() +  gradle.ext.file7_filename.toString()
    String destdir = getRootDir().toString() + "/" + gradle.ext.file_version.toString() + "/"
    src([
            base.toString(),
            file1.toString(),
            file2.toString(),
            file3.toString(),
            file4.toString(),
            file5.toString(),
            file6.toString(),
            file7.toString(),
    ])
    dest destdir
    onlyIfModified true
    retries 5
    acceptAnyCertificate true
    doLast {
        System.out.println("Extracting Files")

        copy {
            from zipTree(destdir + gradle.ext.base_filename.toString())
            from zipTree(destdir + gradle.ext.file1_filename.toString())
            from zipTree(destdir + gradle.ext.file2_filename.toString())
            from zipTree(destdir + gradle.ext.file3_filename.toString())
            from zipTree(destdir + gradle.ext.file4_filename.toString())
            from zipTree(destdir + gradle.ext.file5_filename.toString())
            from zipTree(destdir + gradle.ext.file6_filename.toString())
            from zipTree(destdir + gradle.ext.file7_filename.toString())
            into rootProject.projectDir.toString()
        }
    }
}

Downloading and unzipping of small sets of files can be prevented individually. I have to do the copy after the download to make sure the files exist.

The request is a FLAG to prevent secondary doLasts, etc.. after based on if a file is modeled.

If there is a recipe then I apologize, I have not found one.

[michel-kraemer]

The issue here is that you're extracting the files in a doLast block. Gradle cannot cache the block's output so it will always execute it again. You have to define a Copy task instead. Try this for example:

task downloadZipFile(type: Download) {
    src([
        'https://github.com/michel-kraemer/gradle-download-task/archive/refs/tags/5.3.0.zip',
        'https://github.com/michel-kraemer/gradle-download-task/archive/refs/tags/5.2.1.zip',
        'https://github.com/michel-kraemer/gradle-download-task/archive/refs/tags/5.2.0.zip'
    ])
    dest buildDir
}

task downloadAndUnzipFile(dependsOn: downloadZipFile, type: Copy) {
    for (f in downloadZipFile.outputs.files) {
        from zipTree(f)
    }
    into new File(buildDir, "extracted")
}

defaultTasks 'downloadAndUnzipFile'

Let me know if this helps, and if so, please close the issue. Thanks.

[TMLWinston]

The recipe you provided has the same issues as the original setup.

I modified the downloadZipFiles to put:
onlyIfModified true
retries 5
acceptAnyCertificate true

I believe the behavior I am looking for is that the downloadAndUnzipFile would not be supplied a file to unzip if the file was not re-downloaded. The variable: downloadZipFile.outputs.files could be only supplied with the actual files when they are actually downloaded again.

As a default task, unzipping the data alone is over 4 minutes. Both methods above stop the download if the file exists already and is not modified. That saves 30 plus minutes in our remote work environments of today.

Great Product.

[michel-kraemer]

OK. So, do I understand you correctly that you were able to solve the issue?

[TMLWinston]

No, The feature request is to prevent the unzip task to even run if the file is already downloaded and not modified. The current functionality is to return the files as downloaded even if they had not been downloaded due to not being modified. That causes the zip copy functions to activate which causes a delay that is not needed after initial download.

[michel-kraemer]

Interesting. For me, it displays UP-TO-DATE on the console for the unzip task, but I guess Gradle still has to traverse all files inside the ZIP file to compare them with the files in your target directory in order to determine if the task is up to date or not, which also takes some time. If you want to skip that too then you can simply add the following line to the downloadAndUnzipFile task:

onlyIf { downloadZipFile.didWork }

This way, the task will only be executed if the download task was executed too.

[TMLWinston]

That helped a lot! That worked.