Deploying the BIG-IP VE in Azure - Fails

[hondocalrissian]

Describe the bug

onBoarder script fails with the following error "Cannot set db variable (restjavad.useextramb = true)."

Expected behavior

Deployment to complete and have a working environment configured per the ARM template deployed

Current behavior

Deployment error is displayed in Azure Resource manager with the complete error being:

2023-07-25T19:28:30.842Z [20846]: info: Configuration file: /config/cloud/runtime-init.conf
2023-07-25T19:28:30.860Z [20846]: info: Validating provided declaration
2023-07-25T19:28:30.856Z [20846]: info: Processing controls parameters
2023-07-25T19:28:30.914Z [20846]: info: Successfully validated declaration
2023-07-25T19:28:30.917Z [20846]: info: Executing custom pre_onboard_enabled commands
2023-07-25T19:28:30.929Z [20846]: info: Executing inline shell command: /usr/bin/setdb provision.extramb 1000
2023-07-25T19:28:31.046Z [20846]: info: Shell command: /usr/bin/setdb provision.extramb 1000 execution completed; response:
2023-07-25T19:28:31.050Z [20846]: info: Executing inline shell command: /usr/bin/setdb restjavad.useextramb true
2023-07-25T19:28:31.157Z [20846]: error: Command failed: /usr/bin/setdb restjavad.useextramb true
Cannot set db variable (restjavad.useextramb = true).

2023-07-25T19:28:31.158Z [20846]: info: Sending F5 Teem report for failure case.
2023-07-25T19:28:41.846Z [20846]: info: {"id":"98ad5c92-e50e-4f42-c256a986cb3c","product":"BIG-IP","cpuCount":8,"diskSize":41984,"memoryInMb":32176,"version":"15.1.9.1","nicCount":3,"regKey":"TYQMM-LOPNP-VXQMW-GPQCP-VDJRGXU","platformId":"Z100","hostname":"bigip1","management":"10.X.X.X/24","provisionedModules":{"ltm":"nominal"},"installedPackages":{},"environment":{"pythonVersion":"Python 2.7.5","pythonVersionDetailed":"2.7.5 (default, Jul 11 2023, 10:14:43) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]","nodeVersion":"v6.9.1","libraries":{"ssh":"OpenSSH_7.4p1, OpenSSL 1.0.2s-fips 28 May 2019"}}}
2023-07-25T19:28:51.561Z [20846]: info: F5 Teem report was successfully sent for failure case.
2023-07-25T19:28:51.562Z [20846]: info: Command failed: /usr/bin/setdb restjavad.useextramb true
Cannot set db variable (restjavad.useextramb = true).

Possible solution

I suspect that this issue may be version related. The version attempting to be deployed is 15.1.9.1

Steps to reproduce

Use the Azure deploy button at the link https://github.com/F5Networks/f5-azure-arm-templates-v2/tree/main/examples/failover

Fill out the template info as requested

Screenshots

Context

I am trying to deploy a HA pair per the Azure deploy template with 3 NIC's.
Attempting to replicate a pre-existing environment in one Azure region to another.

Your Environment

• Version used: Deploying version":"15.1.9.1

[mikeshimkus]

@starkeyjhonnold Hi, the template link you shared is from the v1 ARM template project, have you tried the v2 failover template?

[hondocalrissian]

Sorry. My fault. I copy and pasted the wrong link. I corrected the comment.
This error is seen with the v2 template.

I have been through the process several times and it fails at the same location everytime.

[mikeshimkus]

I created internal issue #288 to track updating the templates to use the correct db variables for newer versions of BIG-IP.

In the meantime, please try again. I made a change to the template in the default artifact location that should allow you to deploy.

[hondocalrissian]

@shyawnkarim

Good day.
I tried again several minutes ago and am getting the same error in the onBoarder portion.
Take a look and let me know if I can get around this error or not.

Here is the error output.
<=
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"VMExtensionProvisioningError","message":"VM has reported a failure when processing extension 'onboarder' (publisher 'Microsoft.Azure.Extensions' and type 'CustomScript'). Error message: "Enable failed: failed to execute command: command terminated with exit status=1\n[stdout]\ninit-azure/src/lib/cloud/cloudFactory.d.ts\n./f5-bigip-runtime-init-azure/src/lib/cloud/cloudFactory.js\n./f5-bigip-runtime-init-azure/src/lib/cloud/gcp\n./f5-bigip-runtime-init-azure/src/lib/cloud/gcp/cloudClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/cloud/gcp/cloudClient.js\n./f5-bigip-runtime-init-azure/src/lib/logger.d.ts\n./f5-bigip-runtime-init-azure/src/lib/logger.js\n./f5-bigip-runtime-init-azure/src/lib/provider\n./f5-bigip-runtime-init-azure/src/lib/provider/secret\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/hashicorpVaultClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/hashicorpVaultClient.js\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/index.d.ts\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/index.js\n./f5-bigip-runtime-init-azure/src/lib/resolver\n./f5-bigip-runtime-init-azure/src/lib/resolver/index.d.ts\n./f5-bigip-runtime-init-azure/src/lib/resolver/index.js\n./f5-bigip-runtime-init-azure/src/lib/resolver/resolverClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/resolver/resolverClient.js\n./f5-bigip-runtime-init-azure/src/lib/telemetry\n./f5-bigip-runtime-init-azure/src/lib/telemetry/index.d.ts\n./f5-bigip-runtime-init-azure/src/lib/telemetry/index.js\n./f5-bigip-runtime-init-azure/src/lib/telemetry/telemetryClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/telemetry/telemetryClient.js\n./f5-bigip-runtime-init-azure/src/lib/utils.d.ts\n./f5-bigip-runtime-init-azure/src/lib/utils.js\n./f5-bigip-runtime-init-azure/src/lib/validator.d.ts\n./f5-bigip-runtime-init-azure/src/lib/validator.js\n./f5-bigip-runtime-init-azure/src/schema\n./f5-bigip-runtime-init-azure/src/schema/base_schema.json\n./f5-bigip-runtime-init-azure/src/version\n74618 blocks\n2023-07-28T10:08:22 - Getting lastest AT metadata at https://cdn.f5.com/product/cloudsolutions/f5-extension-metadata/latest/metadata.json\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 46195 100 46195 0 0 129k 0 --:--:-- --:--:-- --:--:-- 130k\n2023-07-28T10:08:24 - Creating command utility.\n2023-07-28T10:08:24 - RPM installation is completed.\n2023-07-28T17:08:25.147Z [20756]: info: Configuration file: /config/cloud/runtime-init.conf\n2023-07-28T17:08:25.158Z [20756]: info: Processing controls parameters\n2023-07-28T17:08:25.159Z [20756]: info: Validating provided declaration\n2023-07-28T17:08:25.212Z [20756]: info: Successfully validated declaration\n2023-07-28T17:08:25.212Z [20756]: info: Executing custom pre_onboard_enabled commands\n2023-07-28T17:08:25.223Z [20756]: info: Executing inline shell command: /usr/bin/setdb provision.extramb 1000\n2023-07-28T17:08:25.339Z [20756]: info: Shell command: /usr/bin/setdb provision.extramb 1000 execution completed; response: \n2023-07-28T17:08:25.343Z [20756]: info: Executing inline shell command: /usr/bin/setdb restjavad.useextramb true\n2023-07-28T17:08:25.448Z [20756]: error: Command failed: /usr/bin/setdb restjavad.useextramb true\nCannot set db variable (restjavad.useextramb = true).\n\n2023-07-28T17:08:25.453Z [20756]: info: Sending F5 Teem report for failure case.\n2023-07-28T17:08:36.180Z [20756]: info: {"id":"e47c88b7-fedf-5d4c-ff53eaa57c10","product":"BIG-IP","cpuCount":8,"diskSize":41984,"memoryInMb":32176,"version":"15.1.9.1","nicCount":3,"regKey":"TYQMM-LOPNP-VXQMW-GPQCP-VDJRGXU","platformId":"Z100","hostname":"bigip1","management":"10.X.X.X/24","provisionedModules":{"ltm":"nominal"},"installedPackages":{},"environment":{"pythonVersion":"Python 2.7.5","pythonVersionDetailed":"2.7.5 (default, Jul 11 2023, 10:14:43) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]","nodeVersion":"v6.9.1","libraries":{"ssh":"OpenSSH_7.4p1, OpenSSL 1.0.2s-fips 28 May 2019"}}}\n2023-07-28T17:08:46.714Z [20756]: info: F5 Teem report was successfully sent for failure case.\n2023-07-28T17:08:46.716Z [20756]: info: Command failed: /usr/bin/setdb restjavad.useextramb true\nCannot set db variable (restjavad.useextramb = true).\n\n\n[stderr]\n". More information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot. "}]}
<=

[mikeshimkus]

Can you confirm that the contentVersion of the template you're using is 2.8.0.0? Example:

f5-azure-arm-templates-v2/examples/failover/azuredeploy.json

Line 3 in 002470a

"contentVersion": "2.8.0.0",

What value did you use for the templateBaseUrl parameter?

[hondocalrissian]

Thanks for the follow-up. It seems the Azure Deploy in the Existing stack on the Github page is still using version 2.7.0.0. I will see what I can do to use version 2.8.0.0. If the Azure Deploy button link can be updated that would be great too! "https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FF5Networks%2Ff5-azure-arm-templates-v2%2Fv2.7.0.0%2Fexamples%2Ffailover%2Fazuredeploy-existing-network.json” Let me know Thanks!

…

On Aug 1, 2023, at 10:11 AM, Mike Shimkus ***@***.***> wrote: Can you confirm that the contentVersion of the template you're using is 2.8.0.0? Example: https://github.com/F5Networks/f5-azure-arm-templates-v2/blob/002470aceb1192b0524495e5e393a18d958d4453/examples/failover/azuredeploy.json#L3 What value did you use for the templateBaseUrl parameter? — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A5Y6YTYYOVPL7WEWIHI7LM3XTEMD5ANCNFSM6AAAAAA2Y2OC3M>. You are receiving this because you were mentioned.

[hondocalrissian]

@mikeshimkus @shyawnkarim

I retried with this link and verified that it was using version 2.8.0.0
https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FF5Networks%2Ff5-azure-arm-templates-v2%2Fv2.8.0.0%2Fexamples%2Ffailover%2Fazuredeploy-existing-network.json

Same error:
2023-08-01T17:42:26.179Z [25751]: info: Command failed: /usr/bin/setdb restjavad.useextramb true
Cannot set db variable (restjavad.useextramb = true).

[hondocalrissian]

@shyawnkarim @mikeshimkus
I redeployed again using the 2.8.0.0 template and using "f5-networks:f5-big-ip-good:f5-bigip-virtual-edition-200m-good-hourly:15.1.801000".
It worked without issue.
My previous deployments were using a later version of 15.1.
"f5-networks:f5-big-ip-good:f5-bigip-virtual-edition-200m-good-hourly:15.1.901000"

15.1.8.1 is listed as a validated version for the runtime-init.
https://github.com/F5Networks/f5-bigip-runtime-init#validated-big-ip-versions.

It seems a validated version for the runtime-init is the key. . .

[shyawnkarim]

Closing. This issue was resolved with Release 2.9.0.0.