Ruby Layers not properly packaged

[jkahn117]

Description:

When using SAM to build a Ruby layer, packaging of Gems is incorrect. Gems are bundled in ruby /lib/vendor/bundle/ruby/2.7.0 using the default bundler workflow, these Gems are not picked up by the runtime.

Per documentation, GEM_PATH (ruby/gems/2.7.0) should be used. Shared / custom code is packaged properly to RUBYLIB (ruby/lib).

Steps to reproduce the issue:

1. Create new SAM application.
2. Add Layer to template:

SAMBuildLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: SAMBuild
      CompatibleRuntimes:
        - ruby2.7
    Metadata:
      BuildMethod: ruby2.7

3. Add directory SAMBuild containing a Gemfile with valid contents.
4. Run sam build command.
5. Review results and/or test with a sample function.

Observed result:

Result of build (.aws-sam/build) contains SAMBuildLayer/ruby/lib/vendor/bundle/ruby/2.7.0/gems with bundled Gems. When attempting to run sample function, Gem is not found.

Expected result:

Gem should be packaged in ruby/gems/2.7.0 and available to function when executed. I was able to build using a Makefile:

MakeBuildLayer:
  Type: AWS::Serverless::LayerVersion
  Properties:
    ContentUri: MakeBuild
    CompatibleRuntimes:
      - ruby2.7
  Metadata:
    BuildMethod: makefile

And Makefile in MakeBuild directory:

build-MakeBuildLayer:
  bundle install --path=$(ARTIFACTS_DIR)
  rm -rf $(ARTIFACTS_DIR)/ruby/2.7.0/cache && rm -rf $(ARTIFACTS_DIR)/ruby/2.7.0/bin
  mkdir $(ARTIFACTS_DIR)/ruby/gems
  mv $(ARTIFACTS_DIR)/ruby/2.7.0 $(ARTIFACTS_DIR)/ruby/gems

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

SAM CLI, version 0.52.0

[jorian]

Having this issue. Are you still using the Makefile workaround, @jkahn117 ?

Edit: found your blog, really helpful 👍

[davegallant]

I ran into the same issue today.

The only way I was able to get the above workaround to work on SAM 1.24.0 was to adjust the above Makefile with:

RUBY_VERSION = 2.7.0

mkfile_path := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))

build-MakeBuildLayer:
	bundle install --path=$(mkfile_path)
	rm -rf $(mkfile_path)/ruby/$(RUBY_VERSION)/cache && rm -rf $(mkfile_path)/ruby/$(RUBY_VERSION)/bin
	mkdir -p $(mkfile_path)/ruby/gems
	mv $(mkfile_path)/ruby/$(RUBY_VERSION) $(mkfile_path)/ruby/gems

Otherwise, it would package up the ContentUri and nothing would be there.

[ryanjfrizzell]

+1 here for those who need some postgres (pg gem) love...based on @davegallant and @jkahn117 's work above

SHELL := /bin/bash
PGVERSION := 11.10
BP := $(ARTIFACTS_DIR)
PGSOURCE := postgresql-$(PGVERSION)
.PHONY: build-RubyLayer
.ONESHELL:

build-RubyPgLayer:

	yum install -y amazon-linux-extras tree
	yum makecache
	yum install -y postgresql-devel postgresql-libs
	curl -O https://ftp.postgresql.org/pub/source/v11.10/postgresql-11.10.tar.gz
	tar -xvzf $(PGSOURCE).tar.gz
	(cd $(PGSOURCE) && ./configure --prefix=$(BP) --without-readline && $(MAKE) -C . MAKELEVEL=0 all && $(MAKE) install)
	bundle config --local build.pg --with-pg-config=$(BP)/bin/pg_config
	bundle config --local silence_root_warning true
	bundle config --local path $(ARTIFACTS_DIR)
	bundle install
	mkdir $(ARTIFACTS_DIR)/ruby/gems
	mv $(ARTIFACTS_DIR)/ruby/2.7.0 $(ARTIFACTS_DIR)/ruby/gems