[Enhancement][Distribution] Support generating extracted dirs of binaries alongside tar/deb/rpm......

[peterzhuamazon]

Similar to opensearch-project/OpenSearch#2571.

We want core dashboards to be able to have extracted binaries, not only tar/deb/rpm that is packaged already.

Tar example, currently support in OS:

# Run this task:
./gradlew distribution:archives:linux-tar:build

# We get:
~/OpenSearch/distribution/archives/linux-tar/build/distributions
opensearch-min-1.3.0-linux-x64.tar.gz

~/OpenSearch/distribution/archives/linux-tar/build/tar-extracted/opensearch-1.3.0
bin  config  jdk  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.md

Rpm example, currently support in OS:

# Run this task:
./gradlew distribution:packages:rpm:build

# We get:
~/OpenSearch/distribution/packages/rpm/build/distributions
opensearch-min-1.3.0.x86_64.rpm

~/OpenSearch/distribution/packages/rpm/build/extracted/rpm-extracted
etc  usr  var

~/OpenSearch/distribution/packages/rpm/build/extracted/rpm-extracted/usr/share/opensearch
bin  jdk  lib  LICENSE.txt  modules  NOTICE.txt  plugins  README.md

We hope dashboards core can also support this.

Thanks.

[bbarani]

@seraphjiang @kavilla Can you please look in to this issue? This is a blocker for creating RPM artifacts.

[kavilla]

@bbarani, @peterzhuamazon

Could you provide the reason at why this is a blocker for RPM artifacts? Can the build repo not support it during the assemble workflow? Seems like for distributions of linux x64 and arm64 that has been the standard and the rpm artifact is available extract.

[bbarani]

@kavilla We are working with core team on a new process for OpenSearch to directly take the extracted files, install plugins, then use tar/rpmbuild/zip to create the final bundle. This requires core to provide the extracted files directly, as this is the logical process to proceed.

The old process is a messy way to extract the core tar/rpm/zip then package them back, especially in rpm, the cpio utility we are using is not able to identify symlink and cause a lot of error in the extraction process.

More info can be found here: opensearch-project/OpenSearch#2517

We would like to follow the new process for the Dashboards in order to use the common RPM generation code to support dashboards package as well.

CC: @CEHENKLE @seanneumann @seraphjiang

[bbarani]

We are facing some issues in our new plan. @peterzhuamazon will update this thread with more details soon

[kavilla]

I see it's originally based on the issue from OpenSearch. Does OpenSearch Dashboards have a same issue? Also in the issue there is a long term and short term plan. To unblock, for OpenSearch Dashboards could we go with the short term plan and then when a maintainer has some availability we can assign? Otherwise, we could provide guidance.