Issue importing full json WAF policy with v1.19.0 - resource bigip_waf_policy

[amolari]

Environment

• TMOS/Bigip Version: 17.1.0.1
• Terraform Version: 1.3.9
• Terraform bigip provider Version: 1.19.0

Summary

I could import the same WAF policy json successfully on v1.18.0
When running the same code with 1.19.0 I have an error during the apply.
I do see the policy being imported on the BIG-IP, though.

Steps To Reproduce

Steps to reproduce the behavior:
1.
resource "bigip_waf_policy" "test_for_TF_l" {
name = "test_for_TF_l"
template_name = "POLICY_TEMPLATE_BLANK"
policy_import_json = file("./Common_test_for_TF_l.json")
}

The file Common_test_for_TF_l.json is a full-json policy.

Expected Behavior

WAF policy import successful, without any TF error

Actual Behavior

2023-08-30T10:54:00.018+0200 [DEBUG] provider.terraform-provider-bigip_v1.19.0: Import policy:{Filename:test_for_TF_l.json FullPath:/Common/test_for_TF_l Policy:{FullPath:/Common/test_for_TF_l}}: timestamp=2023-08-30T10:54:00.018+0200
2023-08-30T10:54:00.033+0200 [INFO] provider.terraform-provider-bigip_v1.19.0: AWAF Import policy TaskID :WPGdpF7_i-ePYQM54FUShg: timestamp=2023-08-30T10:54:00.033+0200
bigip_waf_policy.test_for_TF_l: Still creating... [10s elapsed]
2023-08-30T10:54:10.066+0200 [ERROR] provider.terraform-provider-bigip_v1.19.0: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR tf_provider_addr=provider tf_resource_type=bigip_waf_policy tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_summary="error retrieving waf policy : [ERROR] WafPolicy: test_for_TF_l on partition Common not found" tf_proto_version=5.3 tf_req_id=478fca96-cc0a-c39c-3be2-482e31709a4e @module=sdk.proto timestamp=2023-08-30T10:54:10.065+0200
2023-08-30T10:54:10.069+0200 [ERROR] vertex "bigip_waf_policy.test_for_TF_l" error: error retrieving waf policy : [ERROR] WafPolicy: test_for_TF_l on partition Common not found
╷
│ Error: error retrieving waf policy : [ERROR] WafPolicy: test_for_TF_l on partition Common not found
│
│ with bigip_waf_policy.test_for_TF_l,
│ on waf_policy_import.tf line 1, in resource "bigip_waf_policy" "test_for_TF_l":
│ 1: resource "bigip_waf_policy" "test_for_TF_l" {
│
╵
2023-08-30T10:54:10.081+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2023-08-30T10:54:10.085+0200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/f5networks/bigip/1.19.0/linux_amd64/terraform-provider-bigip_v1.19.0 pid=7644
2023-08-30T10:54:10.085+0200 [DEBUG] provider: plugin exited

[pgouband]

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1289.

[pgouband]

Hi @amolari,

The issue is not reproductible in our lab.
The error is about the policy not available in Common partition. Can you check your configuration and your environment?

[amolari]

Hi @pgouband

• the exact same code/policy works without issue with 1.18.0 (between my 2 tests I really just change the version of the provider, the rest is identical)
• the policy is imported and then there is the TF error. On the BIGIP:

# list asm policy test_for_TF_l all-properties
asm policy test_for_TF_l {
    app-service none
    blocking-mode disabled
    description "test policy configured through the UI"
    encoding iso-8859-1
    inactive
    parent-policy none
    partition Common
    policy-builder disabled
    policy-template POLICY_TEMPLATE_BLANK
    policy-type security
    virtual-servers none
}

[amolari]

Hi @pgouband
I've found something: if I change the policy json file to have applicationLanguage=utf-8 then I can import it without an error.
Could this PR be the issue: #851 ?

[pgouband]

Hi @amolari,

A fix will be included in our next release.

[amolari]

@RavinderReddyF5 @pgouband
Doing some testing with v1.20.0 and there is an issue (error undetected).

I have a basic policy (full json export) which the only setting set (from Fondamental template) is the Application language, set to "iso-8859-1".
I can import (1st time) the policy successfully using this TF code:

resource "bigip_waf_policy" "test_basic_pol_iso" {
  name = "test_basic_pol_iso"
  template_name = "POLICY_TEMPLATE_BLANK"
  policy_import_json = file("./mytests/test_basic_pol_iso.json")
}

Everything's fine until I modify my json file (simple change such as description). The TF apply is successful but the policy is not updated. There is an issue for non-utf8 policies (I've seen it on the UI) and I've just opened F5 support case 00491134.

But the point is that the TF apply doesn't detect the error and return a successful status.
What we see in the bigip asm logs:
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Import Policy Task Import Policy Task (1697630924.505415) [add]: Target Policy was set to /Common/test_basic_pol_iso. Filename was set to test_basic_pol_iso.json. { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Import Policy Task Import Policy Task (1697630924.505415) [update]: Status was set to STARTED. Execution Start Time was set to 1697630924. { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Apply Policy Task Apply Policy Task (1697630924.532803) [add]: Target Policy was set to /Common/test_basic_pol_iso. { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630924.572459) [add]: Target Policy was set to /Common/test_basic_pol_iso. Result Inline was set to 1. { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch crit g_server_rpc_handler_async.pl[11273]: 01310027:2: ASM subsystem error (asm_config_server.pl,F5::ImportExportPolicy::Base::fatal_error): Failed action: Imported and replaced policies have different encodings.
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch crit g_server_rpc_handler_async.pl[11273]: 01310027:2: ASM subsystem error (asm_config_server.pl,F5::ImportExportPolicy::Base::fatal_error): InternalError - import_policy failed: fatal: Failed action: Imported and replaced policies have different encodings.
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Import Policy Task Import Policy Task (1697630924.505415) [update]: Status was set to FAILURE. End Time was set to 1697630925. Message was set to InternalError - import_policy failed: fatal: Failed action: Imported and replaced policies have different encodings. . { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:44 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: Per-invocation log rate exceeded; throttling.
Oct 18 14:08:49 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: Resuming log processing at this invocation; held 4 messages.
Oct 18 14:08:49 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630929.628154) [add]: Target Policy was set to /Common/test_basic_pol_iso. Result Inline was set to 1. { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:49 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630929.628154) [update]: Status was set to STARTED. Execution Start Time was set to 1697630929. { audit: username = admin, client IP = 172.31.10.111 }
Oct 18 14:08:50 bigip-h1-g3t.techlab.ch info perl[14435]: 01310053:6: ASMConfig change: Export Policy Task Export Policy Task (1697630929.628154) [update]: File Size was set to 57586. Status was set to COMPLETED. End Time was set to 1697630930. Message was set to Policy '/Common/test_basic_pol_iso' was successfully exported.. { audit: username = admin, client IP = 172.31.10.111 }

TF apply should return an error not a change applied successfully.

[pgouband]

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1349.

[pgouband]

Hi @amolari,

A WAF policy can't be replaced when the application language is different so the behavior is by design.

[RavinderReddyF5]

hi @amolari please open new issue if still problem seen