Yum does not have perl-Bundle-LWP available for installation

[samkit-jain]

Full traceback

Unhandled exception during build: Yum does not have perl-Bundle-LWP available for installation
Traceback (most recent call last):
  File "/opt/aws/bin/cfn-init", line 171, in <module>
    worklog.build(metadata, configSets)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 129, in build
    Contractor(metadata).build(configSets, self)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 530, in build
    self.run_config(config, worklog)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 542, in run_config
    CloudFormationCarpenter(config, self._auth_config).build(worklog)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 229, in build
    changes['packages'][manager] = CloudFormationCarpenter._packageTools[manager]().apply(packages, self._auth_config)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/rpm_tools.py", line 74, in apply
    raise ToolError("Yum does not have %s available for installation" % pkg_spec)
ToolError: Yum does not have perl-Bundle-LWP available for installation

This happens when using the script https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/resource-configuration/autoscaling-memory-utlization.config on the platform Docker running on 64bit Amazon Linux 2/3.0.0.

Running yum search on the instance results in

$ yum search all perl-Bundle-LWP
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Warning: No matches found for: perl-Bundle-LWP
No matches found

What is the fix/workaround?

[samkit-jain]

I tried removing perl-Bundle-LWP from the packages list and no error came this time but I couldn't see the metrics on CloudWatch. Checking the logs in the /var/log/cwpump.log, I found

Can't locate Digest/SHA.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . /opt/cloudwatch/aws-scripts-mon) at /opt/cloudwatch/aws-scripts-mon/AwsSignatureV4.pm line 23.
BEGIN failed--compilation aborted at /opt/cloudwatch/aws-scripts-mon/AwsSignatureV4.pm line 23.
Compilation failed in require at /opt/cloudwatch/aws-scripts-mon/CloudWatchClient.pm line 21.
BEGIN failed--compilation aborted at /opt/cloudwatch/aws-scripts-mon/CloudWatchClient.pm line 21.
Compilation failed in require at /opt/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl line 86.
BEGIN failed--compilation aborted at /opt/cloudwatch/aws-scripts-mon/mon-put-instance-data.pl line 86.

and as of https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html#mon-scripts-perl_prereq, perl-Digest-SHA.x86_64, is required as a package dependency. So, I added it to the packages making it look like

packages:
  yum:
    perl-DateTime: []
    perl-Sys-Syslog: []
    perl-LWP-Protocol-https: []
    perl-Switch: []
    perl-URI: []
    perl-Digest-SHA.x86_64: []

and now I can see the metrics on CloudWatch. I am not sure that the issue was because of removing perl-Bundle-LWP or it was always there because of missing perl-Digest-SHA.x86_64.

[dankhen]

@samkit-jain Thanks. I'll check with the Elastic Beanstalk service team and attempt to fix the example.

[joeyparis]

@dankhen Has this example been fixed anywhere? I am attempting to run a similar script (https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-memory-monitoring/) and am running into the same issues as above.

[r3na]

Any update here? got the same problem here.
`2020/07/06 15:09:05.085594 [ERROR] Error occurred during build: Yum does not have perl-Bundle-LWP available for installation

2020/07/06 15:09:05.085882 [ERROR] An error occurred during execution of command [app-deploy] - [PreBuildEbExtension]. Stop running the command. Error: EbExtension build failed. Please refer to /var/log/cfn-init.log for more details. `

[dankhen]

Update: This perl bundle issue is now being investigated by the Elastic Beanstalk team as a potential bug. We'll hold on fixing the example until we have results.

[dhruvg]

@dankhen thanks for the update. I notice this issue is marked closed. So, how would you recommend we follow the issue so we know when it is fixed? Will you update this issue?

[dankhen]

@dhruvg This repo is for documentation examples. The issue isn't likely in docs, and it's now in the hands of the Beanstalk platform team, so the docs team isn't tracking it. Unfortunately this repo isn't the place to track it.
We'll announce a fix in the platform update release notes, if one is communicated to us.

[sudeepmalepati]

Hello
i am facing the same issue regarding amazon Linx 2. What is the workaround for this issue. Basically my target is to send amazon linux 2 metrics to cloudwatch which is hosted in the Elastic beanstalk.
Where this page is not at all working for amazon Linux 2 (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-cw.html). Is there any way?

[dharmendrakac]

Please do that to fix this issue for Amazon Linux 2 based platforms, replace the perl-Bundle-LWP yum package in the preceding configuration file with perl-Digest-SHA.

[sudeepmalepati]

yes, I have tried in that way but no use @dharmendraka
But I have fixed my issue by using a cloud watch agent. So I have written a .ebextensoins that will install a cloud watch agent and stream logs and custom metrics to cloud watch.
Create a .config file under the .ebextensions and use the following script.

 files:
  "/cw.sh" :
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/bash
      sudo yum install amazon-cloudwatch-agent -y
      sudo amazon-linux-extras install collectd -y
      echo '{
              "agent": {
                      "metrics_collection_interval": 60,
                      "run_as_user": "root"
              },
              "logs": {
                      "logs_collected": {
                              "files": {
                                      "collect_list": [
                                              {
                                                      "file_path": "/var/log/php-fpm/error.log",
                                                      "log_group_name": "php-fpm-errorlogs",
                                                      "log_stream_name": "{instance_id}"
                                              }
                                      ]
                              }
                      }
              },
              "metrics": {
                      "append_dimensions": {
                              "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
                              "ImageId": "${aws:ImageId}",
                              "InstanceId": "${aws:InstanceId}",
                              "InstanceType": "${aws:InstanceType}"
                      },
                      "metrics_collected": {
                              "collectd": {
                                      "metrics_aggregation_interval": 60
                              },
                              "disk": {
                                      "measurement": [
                                              "used_percent"
                                      ],
                                      "metrics_collection_interval": 60,
                                      "resources": [
                                              "*"
                                      ]
                              },
                              "mem": {
                                      "measurement": [
                                              "mem_used_percent"
                                      ],
                                      "metrics_collection_interval": 60
                              },
                              "statsd": {
                                      "metrics_aggregation_interval": 60,
                                      "metrics_collection_interval": 10,
                                      "service_address": ":8125"
                              }
                      }
              }
              }' > "/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-config.json"
      cd /opt/aws/amazon-cloudwatch-agent/etc/
      sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-config.json -s


commands:
  command block:
    command: "sudo bash cw.sh"
    cwd: "/"

[sandwormusmc]

This is causing my Elastic Beanstalk environment to enter an infinite loop and continuously restart the Docker daemon, causing images to fail to download with a cryptic "Unexpected EOF" error. I finally found this issue report, and it seems that it is causing cfn-init to loop, and basically makes my Elastic Beanstalk environment unusable. I was hoping for a smooth upgrade from the previous, deprecated platform to this, so I wonder how many other customers this is affecting.

[ec2-user@ip-172-31-45-207 ~]$ cat /var/log/cfn-init.log
2021-08-09 15:39:11,528 [ERROR] Unhandled exception during build: Yum does not have perl-Bundle-LWP available for installation
Traceback (most recent call last):
File "/opt/aws/bin/cfn-init", line 176, in
worklog.build(metadata, configSets)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 135, in build
Contractor(metadata).build(configSets, self)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 561, in build
self.run_config(config, worklog)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 573, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/construction.py", line 238, in build
self._auth_config)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/rpm_tools.py", line 76, in apply
raise ToolError("Yum does not have %s available for installation" % pkg_spec)
cfnbootstrap.construction_errors.ToolError: Yum does not have perl-Bundle-LWP available for installation

---

And then it continues to run _OnInstanceReboot and other stages again:

2021-08-09 15:44:18,135 [INFO] -----------------------Starting build-----------------------
2021-08-09 15:44:18,140 [INFO] Running configSets: _OnInstanceReboot
2021-08-09 15:44:18,143 [INFO] Running configSet _OnInstanceReboot
2021-08-09 15:44:18,145 [INFO] Running config AWSEBBaseConfig
2021-08-09 15:44:18,326 [INFO] Command clearbackupfiles succeeded
2021-08-09 15:44:18,328 [INFO] ConfigSets completed
2021-08-09 15:44:18,328 [INFO] -----------------------Build complete-----------------------
2021-08-09 15:44:20,039 [INFO] -----------------------Starting build-----------------------
2021-08-09 15:44:20,044 [INFO] Running configSets: Infra-EmbeddedPreBuild
2021-08-09 15:44:20,047 [INFO] Running configSet Infra-EmbeddedPreBuild
2021-08-09 15:44:20,049 [INFO] Running config prebuild_0_FPElasticBeanstalk
2021-08-09 15:44:20,052 [INFO] Test failed with code 1
2021-08-09 15:44:20,055 [INFO] Test failed with code 1
2021-08-09 15:44:20,057 [INFO] Test failed with code 1
2021-08-09 15:44:20,060 [INFO] Test failed with code 1
2021-08-09 15:44:20,063 [INFO] Command 05_flag_swapmem succeeded
2021-08-09 15:44:20,066 [INFO] Running config prebuild_1_FPElasticBeanstalk

[sandwormusmc]

I would suggest this gotcha be documented here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/docker-multicontainer-migration.html

Definitely bit me and had me scratching my head for hours...

[tschaub]

But I have fixed my issue by using a cloud watch agent. So I have written a .ebextensoins that will install a cloud watch agent and stream logs and custom metrics to cloud watch.
Create a .config file under the .ebextensions and use the following script.

@sudeepmalepati - In my case, it looks like CloudWatch agent is already installed and configured. I see a /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml file on my EC2 instances already. I'm hoping to find a way to customize the metrics that are sent (I'd like to include mem_used_percent). Has anyone found a way to extend the existing CloudWatch agent configuration? All the documentation I've found is for deprecated methods that appear to no longer work.

[ldodds]

This issue was closed here by @dankhen with a note to say that this isn't an issue for the documentation. But this repo contains an example that is incorrect so this bug affects this repo as well as the documentation.

I've left feedback on the documentation page, but could we get an update for this repo too? Its still a problem.