[azure logs] Add processor version options to Raw Events integration (v1) #15900

zmoog · 2025-11-07T11:57:00Z

Proposed commit message

Add the processor version option to the Raw Events integration v1. With these new options, users can switch from processor v1 (current default) to the processor v2.

This change anticipate the switch to v2 as default processor, see #15703 for context and more details.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
~~I have verified that any added dashboard complies with Kibana's Dashboard good practices~~

How to test this PR locally

Install the Raw Events integration (v1)
Open "Advanced settings", and set Processor version to v2

Screenshots

elastic-vault-github-plugin-prod · 2025-11-07T12:17:20Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

packages/azure/data_stream/activitylogs/elasticsearch/ingest_pipeline/default.yml

muthu-mps

@zmoog - changelog entry is missing, change looks good to me otherwise!

packages/azure/data_stream/eventhub/manifest.yml

kaiyan-sheng

It looks good to me! Lets add a changelog and it's good to go!

zmoog · 2025-11-13T21:13:36Z

It looks good to me! Lets add a changelog and it's good to go!

Adding changelog and running a final test before merging.

elasticmachine · 2025-11-13T21:26:32Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

zmoog · 2025-11-13T21:28:40Z

@elastic/security-service-integrations, could you take a look at this PR when you have time?

efd6

A link to the background in the commit message would be helpful.

packages/azure/data_stream/eventhub/manifest.yml

Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>

# Conflicts: # packages/azure/changelog.yml # packages/azure/manifest.yml

efd6

This is waiting for obs-ds-hosted-services, but LGTM.

gizas · 2025-11-18T11:35:23Z

packages/azure/data_stream/eventhub/agent/stream/stream.yml.hbs

+{{#if processor_update_interval}}
+processor_update_interval: {{processor_update_interval}}
+{{/if}}
+{{#if processor_start_position}}


Is it possible to add sth like if processor_version}==v2 then processor_start_position?

Same also for below fields that are only realted to v2

I also added a couple of checklist items for this in the next issue #15703, so I can do the same across all the integrations using this input.

It seems that the built-in helper #if only supports checking if a value is empty to skip rendering a block:

From https://handlebarsjs.com/guide/builtin-helpers.html#if:

You can use the if helper to conditionally render a block. If its argument returns false, undefined, null, "", 0, or [], Handlebars will not render the block.

This problem should auto-resolve soon as we are switching to processor v2 as default and dropping v1 after that.

packages/azure/data_stream/eventhub/manifest.yml

Updated all the occurrences of the following config options: - `processor_version` - `processor_start_position`

zmoog

Switched to type: select UI element in Azure Logs and Custom Azure Logs integrations.

The built-in helper #if doesn't seem to support expressions though.

packages/azure/data_stream/eventhub/manifest.yml

zmoog · 2025-11-18T21:50:13Z

packages/azure/data_stream/eventhub/agent/stream/stream.yml.hbs

+{{#if processor_update_interval}}
+processor_update_interval: {{processor_update_interval}}
+{{/if}}
+{{#if processor_start_position}}


I also added a couple of checklist items for this in the next issue #15703, so I can do the same across all the integrations using this input.

zmoog · 2025-11-18T22:07:08Z

packages/azure/data_stream/eventhub/agent/stream/stream.yml.hbs

+{{#if processor_update_interval}}
+processor_update_interval: {{processor_update_interval}}
+{{/if}}
+{{#if processor_start_position}}


It seems that the built-in helper #if only supports checking if a value is empty to skip rendering a block:

From https://handlebarsjs.com/guide/builtin-helpers.html#if:

You can use the if helper to conditionally render a block. If its argument returns false, undefined, null, "", 0, or [], Handlebars will not render the block.

This problem should auto-resolve soon as we are switching to processor v2 as default and dropping v1 after that.

elasticmachine · 2025-11-18T22:43:25Z

💚 Build Succeeded

Buildkite Build
Commit: a243fc2

History

💚 Build #34252 succeeded b494949
💚 Build #34232 succeeded 648b982
💚 Build #34119 succeeded 27ab064
💚 Build #34083 succeeded e764712
💚 Build #33834 succeeded d5e9f6a
💚 Build #33824 succeeded 63f959e

cc @zmoog

elastic-vault-github-plugin-prod · 2025-11-19T13:38:18Z

Package azure - 1.30.0 containing this change is available at https://epr.elastic.co/package/azure/1.30.0/

elastic-vault-github-plugin-prod · 2025-11-19T13:38:19Z

Package azure_logs - 0.4.2 containing this change is available at https://epr.elastic.co/package/azure_logs/0.4.2/

…(v1) (elastic#15900) Add the processor version option to the Raw Events integration v1. With these new options, users can switch from processor v1 (current default) to the processor v2. This change anticipate the switch to v2 as default processor.

zmoog self-assigned this Nov 7, 2025

zmoog added Integration:azure Azure Logs Team:obs-ds-hosted-services Observability Hosted Services team [elastic/obs-ds-hosted-services] labels Nov 7, 2025

zmoog marked this pull request as ready for review November 7, 2025 12:09

zmoog requested review from a team as code owners November 7, 2025 12:09

muthu-mps reviewed Nov 7, 2025

View reviewed changes

packages/azure/data_stream/activitylogs/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved

andrewkroh added the Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] label Nov 7, 2025

zmoog force-pushed the integrations-raw-events-v2 branch 2 times, most recently from 1bc4092 to d5e9f6a Compare November 7, 2025 14:24

muthu-mps approved these changes Nov 7, 2025

View reviewed changes

kaiyan-sheng reviewed Nov 7, 2025

View reviewed changes

packages/azure/data_stream/eventhub/manifest.yml Show resolved Hide resolved

kaiyan-sheng reviewed Nov 7, 2025

View reviewed changes

zmoog requested a review from a team as a code owner November 13, 2025 21:12

zmoog added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Nov 13, 2025

zmoog force-pushed the integrations-raw-events-v2 branch from b4ffb98 to 27ab064 Compare November 13, 2025 21:29

zmoog enabled auto-merge (squash) November 13, 2025 21:29

efd6 reviewed Nov 17, 2025

View reviewed changes

packages/azure/data_stream/eventhub/manifest.yml Show resolved Hide resolved

zmoog and others added 4 commits November 17, 2025 22:45

Add processor version options

a5c04e8

Update packages/azure/data_stream/eventhub/manifest.yml

12ae1ea

Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>

Update changelog

efb01d3

# Conflicts: # packages/azure/changelog.yml # packages/azure/manifest.yml

Switch from type: text to type: select

b494949

zmoog force-pushed the integrations-raw-events-v2 branch from 648b982 to b494949 Compare November 17, 2025 21:46

efd6 approved these changes Nov 17, 2025

View reviewed changes

gizas reviewed Nov 18, 2025

View reviewed changes

packages/azure/data_stream/eventhub/manifest.yml Show resolved Hide resolved

zmoog added 2 commits November 18, 2025 23:18

Switch from type:text to type:select

ce8984e

Updated all the occurrences of the following config options: - `processor_version` - `processor_start_position`

Update changelog

a243fc2

zmoog commented Nov 18, 2025

View reviewed changes

zmoog requested review from gizas and kaiyan-sheng November 19, 2025 09:14

andrewkroh added the Integration:azure_logs Custom Azure Logs label Nov 19, 2025

gizas approved these changes Nov 19, 2025

View reviewed changes

zmoog merged commit 2d1bf6b into main Nov 19, 2025
7 checks passed

zmoog deleted the integrations-raw-events-v2 branch November 19, 2025 13:16

andrewkroh added the enhancement New feature or request label Nov 25, 2025

[azure logs] Add processor version options to Raw Events integration (v1) #15900

[azure logs] Add processor version options to Raw Events integration (v1) #15900

Uh oh!

Conversation

zmoog commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Screenshots

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Uh oh!

Uh oh!

muthu-mps left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kaiyan-sheng left a comment

Choose a reason for hiding this comment

Uh oh!

zmoog commented Nov 13, 2025

Uh oh!

elasticmachine commented Nov 13, 2025

Uh oh!

zmoog commented Nov 13, 2025

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

gizas Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

zmoog Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

zmoog Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zmoog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zmoog Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

zmoog Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Nov 18, 2025

💚 Build Succeeded

History

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 19, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

zmoog commented Nov 7, 2025 •

edited

Loading

elastic-vault-github-plugin-prod bot commented Nov 7, 2025 •

edited

Loading