Keep Source

[Stratus3D]

Summary

• Add support for a download callback script in plugins. This script will eventually be required.
• Add support for a always_keep_download configuration option.
• Add support for a --keep-download flag for the install command.

Fixes: #74

This PR addresses the oldest open issue in our issue tracker! 🎉

Other Information

This PR is the first of two PRs to add this keep source/download management functionality. The next PR will add three new commands to asdf:

  asdf download <name> <version>           Download a specific version of a package
  asdf download list [<name>]              List all downloads, optionally filter by package name
  asdf download rm <name> <version>        Remove the download for a specific version of a package

[jthegedus]

Is there an intent to skip the download if the specific version is already present?

[jthegedus]

Another question, what is the plan for a plugin that has a bin/download script but the user is on an older version of asdf that is unaware of the download script? Should plugin authors just ask them to update or do something more elaborate?

[Stratus3D]

@jthegedus yes that is the intent. Although that reminds me I also may implement a --force-redownload flag for the install command as well. Although I suppose the new asdf download rm <tool> <version> command (not yet added to this PR) would do the same thing.

[Stratus3D]

Another question, what is the plan for a plugin that has a bin/download script but the user is on an older version of asdf that is unaware of the download script?

Good question. I didn't think about that. I was implementing this as something that would be backward compatible with old plugins. I guess this would not result in new plugins working with older versions of asdf.

[Stratus3D]

Actually, plugins can be backwards compatible with the current version of asdf. All the install script needs to do is check for the presence of the ASDF_DOWNLOAD_PATH. If it is present assume the source has already been downloaded to that directory. Otherwise assume an older version of asdf and download the source to a temp directory before installing.

[Stratus3D]

I am going to update my asdf-lua plugin with support for this as a test.

[jthegedus]

Is this what you meant @Stratus3D ? Using in asdf-firebase and asdf-gcloud

# bin/install
    if [ -z "${ASDF_DOWNLOAD_PATH:-}" ]; then
        tmp_download_dir=$(mktemp -d -t 'asdf_${plugin_name}_XXXXXX')
        trap 'rm -rf "${tmp_download_dir}"' EXIT
        printf "run download script for older versions of asdf\\n"
        export ASDF_DOWNLOAD_PATH="${tmp_download_dir}"

        # download
        bash "$(dirname "$0")/download"
    fi

[Stratus3D]

@jthegedus yes. Exactly.

Hoping to finish this PR this week.

[Stratus3D]

This PR is finally ready for review @asdf-vm/core !

[jthegedus]

Awesome, I'll give it a test with my plugins this weekend 💯

[Stratus3D]

Had a chance to test this yet @jthegedus ?

[jthegedus]

Sorry mate, not yet. Will try do so tomorrow. Been slammed with work 😫

[jthegedus]

Tested this out, works a treat 👌

One more question though, with asdf plugin remove <tool>, should the default behaviour remove the cached downloads or not? Perhaps a flag for asdf plugin remove <tool> --remove-downloads to cleanup after a plugin? Or the inverse asdf plugin remove <tool> --keep-downloads? Or make it remove the download dir of the plugin depending on the value of the keep_downloads flag?

[Stratus3D]

@jthegedus good question. At first I was thinking not remove the downloads directory when the plugin is uninstalled. But I think problems could arise if a different plugin with the same name were installed. The downloaded files might differ between plugins and cause problems. I think it's best to avoid this by always removing downloads when a plugin is removed.

[jthegedus]

I think problems could arise if a different plugin with the same name were installed

Perhaps making the download path be the GitHub username and repo name, like

https://github.com/jthegedus/asdf-firebase

to

jthegedus-asdf-firebase

would be another option.

Removing downloads when plugin is removed is easier

[Stratus3D]

That does get around the issue of the downloads being available to a plugin that works differently, but it would make the asdf downloads command that I am going to add more complicated. The API I have in mind is:

asdf download-list [<tool>]
asdf download <tool> <version>
asdf download-rm [<tool> <version>],[--all]

If we had to take into account the tool name and some additional plugin identifier like username, these commands would be more complicated. Maybe it is worth it, I am not sure.

[Stratus3D]

With the way I use asdf, I think I've removed plugins maybe 3-4 times over the last 4 years of using it. Others may use it differently, but to me it doesn't seem to be a common action.

[jthegedus]

With the way I use asdf, I think I've removed plugins maybe 3-4 times over the last 4 years of using it. Others may use it differently, but to me it doesn't seem to be a common action.

I would agree that it is not common.

asdf downloads...

Do we need these additional commands? I think removing all downloads on plugin removal as the default behaviour would be enough no? I get list and rm, but why have asdf download <tool> <version>?

[Stratus3D]

Do we need these additional commands? I think removing all downloads on plugin removal as the default behaviour would be enough no? I get list and rm, but why have asdf download ?

Good question. I assumed it would make sense to allow the user to download a version first before installing if they wanted to apply manual patches the source code before installing. Having an asdf download <tool> <version> command makes that easier. asdf download ls would show the paths to downloads if the user wanted to modify source code at a later time. And asdf download rm would make it easy to remove files from a download that went bad (e.g. download succeeded but a plugin or server bug results in unusable files being downloaded). Maybe an asdf download command isn't all that important. I'm open to not having it if you think that would be better.