Plugin manager should validate the SHA, where possible #12750

clintongormley · 2015-08-09T08:21:40Z

The plugin manager should perform SHA sum validation when available. If there is a shasum file in a specific location with a specific naming pattern, we can verify the sum after download.

spinscale · 2015-08-12T10:14:30Z

Implementation detail: When running mvn install the creation of the checksums results simply in .md5 and .sha1 suffixes appended to the file...

Impl: We simply should GET that URL as well... http://.../analysis-kuromoji.zip.sha1

If checksum file exists, compare the checksum with the downloaded file. If these differ, exit. If these match, print a message telling that checksums were ok
If checksum does not exist, behave as usual, but print a warning, that no checksum comparison has happened, but proceed to install the plugin

Not sure if we need command line parameters like --ignore-invalid-checksums or --abort-without-checksums for now.

dakrone · 2015-08-13T21:04:55Z

@spinscale I'm looking at this, it looks like we don't actually provide sha1 or md5 files for our own plugins? I am just curious if there's an already-available plugin I can use for manual testing of this.

dakrone · 2015-08-13T21:28:53Z

Also, I guess nevermind on our own plugins, I can run a local server and test it with a URL that way, I was just curious if we already hosted the checksum files.

spinscale · 2015-08-13T21:40:18Z

no, this is since 2.0...

When a plugin is downloaded, this change additionally tries to download `${pluginurl}.sha1` and verify the SHA1 checksum for the file. If no .sha1 file is found, it tries `${pluginurl}.md5`. Note that if neither checksum file is found, a notice is printed but the plugin can still be installed. If the checksum check fails, the plugin install is aborted. Example output if no checksums are available: ``` bin/plugin install elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT -> Installing elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT... Trying http://download.elastic.co/elasticsearch/elasticsearch-analysis-icu/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip ... Downloading .....................................DONE Verifying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip checksums if available ... NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify) ``` Example output if checksums are available: ``` bin/plugin install elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT -> Installing elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT... Trying http://download.elastic.co/elasticsearch/elasticsearch-analysis-icu/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip ... Downloading .....................................DONE Verifying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip checksums if available ... Downloading .DONE ``` Example output if checksums fail: ``` bin/plugin install elasticsearch/elasticsearch-analysis-kuromoji/2.5.0 -url http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip -> Installing elasticsearch/elasticsearch-analysis-kuromoji/2.5.0... Trying http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip ... Downloading .............................................DONE Verifying http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip checksums if available ... Downloading .DONE ERROR: incorrect hash, file hash: [dbdc9c2cd32782054497a21fbdcae3ca1ff23c80], expected: [dbdc9c2cd32782054497a21fbdcae3ca1ff23c80-bad] ``` Resolves elastic#12750

When a plugin is downloaded, this change additionally tries to download `${pluginurl}.sha1` and verify the SHA1 checksum for the file. If no .sha1 file is found, it tries `${pluginurl}.md5`. Note that if neither checksum file is found, a notice is printed but the plugin can still be installed. If the checksum check fails, the plugin install is aborted. Example output if no checksums are available: ``` bin/plugin install elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT -> Installing elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT... Trying http://download.elastic.co/elasticsearch/elasticsearch-analysis-icu/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip ... Downloading .....................................DONE Verifying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip checksums if available ... NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify) ``` Example output if checksums are available: ``` bin/plugin install elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT -> Installing elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT... Trying http://download.elastic.co/elasticsearch/elasticsearch-analysis-icu/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-analysis-icu/2.6.0-SNAPSHOT/elasticsearch-analysis-icu-2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/2.6.0-SNAPSHOT.zip ... Trying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip ... Downloading .....................................DONE Verifying https://github.com/elasticsearch/elasticsearch-analysis-icu/archive/master.zip checksums if available ... Downloading .DONE ``` Example output if checksums fail: ``` bin/plugin install elasticsearch/elasticsearch-analysis-kuromoji/2.5.0 -url http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip -> Installing elasticsearch/elasticsearch-analysis-kuromoji/2.5.0... Trying http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip ... Downloading .............................................DONE Verifying http://localhost:8000/elasticsearch-analysis-kuromoji-2.5.0.zip checksums if available ... Downloading .DONE ERROR: incorrect hash, file hash: [dbdc9c2cd32782054497a21fbdcae3ca1ff23c80], expected: [dbdc9c2cd32782054497a21fbdcae3ca1ff23c80-bad] ``` Resolves #12750

clintongormley added >enhancement discuss :Core/Infra/Plugins Plugin API and infrastructure v2.0.0 labels Aug 9, 2015

clintongormley mentioned this issue Aug 10, 2015

Testing and packaging changes for 2.0 #12768

Closed

16 tasks

spinscale removed the discuss label Aug 12, 2015

clintongormley assigned dakrone Aug 12, 2015

dakrone mentioned this issue Aug 14, 2015

Validate checksums for plugins if available #12888

Merged

dakrone closed this as completed in #12888 Aug 14, 2015

clintongormley added v2.0.0-beta2 v2.0.0-beta1 and removed v2.0.0 v2.0.0-beta2 labels Sep 14, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plugin manager should validate the SHA, where possible #12750

Plugin manager should validate the SHA, where possible #12750

clintongormley commented Aug 9, 2015

spinscale commented Aug 12, 2015

dakrone commented Aug 13, 2015

dakrone commented Aug 13, 2015

spinscale commented Aug 13, 2015

Plugin manager should validate the SHA, where possible #12750

Plugin manager should validate the SHA, where possible #12750

Comments

clintongormley commented Aug 9, 2015

spinscale commented Aug 12, 2015

dakrone commented Aug 13, 2015

dakrone commented Aug 13, 2015

spinscale commented Aug 13, 2015