[microsoft_sqlserver,system] Add ECS error.code mapping

[ishleenk17]

• Bug

What does this PR do?

This PR fixes the type conflicts for error.message and error.code for datastreams in system and mssql package

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Closes [System] Type Conflict Error for error.code and error.message fields #6766

[muthu-mps]

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

[ishleenk17]

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

They will have to upgrade to this version if they think it's a deal breaker for them.

[muthu-mps]

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

They will have to upgrade to this version if they think it's a deal breaker for them.

Does it requires removing the conflicting field and re-indexing the existing data ? Are we considering this as a breaking change ? Otherwise the changes looks good !

[ishleenk17]

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

They will have to upgrade to this version if they think it's a deal breaker for them.

Does it requires removing the conflicting field and re-indexing the existing data ? Are we considering this as a breaking change ? Otherwise the changes looks good !

Yes, I think this would be categorized as a breaking change since it's going to be a mapping change.
@andrewkroh: How do we handle such a scenario?
do we document them?

[andrewkroh]

I would categorize this as a bug. ECS was not adhered here and the default ES dynamic mappings prevailed.

When the new package version is installed Fleet will _rollover the backing data stream index so that the new mappings are applied. From a data indexing point of view there should be no issues.

[ishleenk17]

I would categorize this as a bug. ECS was not adhered here and the default ES dynamic mappings prevailed.

When the new package version is installed Fleet will _rollover the backing data stream index so that the new mappings are applied. From a data indexing point of view there should be no issues.

Thanks for the clarity @andrewkroh

[muthu-mps]

@andrewkroh - I am trying to understand what happens when we try to search or perform aggregations against the data after the new change.

Will the type conflict still exists ?

Observations :

• Applied an aggregation query against the datastream which has a type of long in older indices and with type of keyword in newer indices after _rollover.
• Getting two different type for same field error.

Correct me if I am wrong,

• This happens because we are searching for the data from older indices which has a different field type.
• This can be avoided either by searching for the data from the newer index only or by re-indexing the older index with newer mapping.

[andrewkroh]

That is correct. Any functions that require knowing the type can have issues. Queries without aggs (like Discover) should work. The options for resolution are reindexing, deleting, closing, or adding a runtime field in the old indices to shadow the field with the correct type.

[ishleenk17]

If we go by this document, I think we should consider this as a bug fix and not a breaking change.
There are lesser chances that the error.code field will be used in aggregation queries.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-07-17T04:44:54.518+0000

• Duration: 18 min 39 sec

Test stats 🧪

Test	Results
Failed	0
Passed	165
Skipped	0
Total	165

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (5/5)	💚
Files	100.0% (6/6)	💚 2.778
Classes	100.0% (6/6)	💚 2.778
Methods	70.27% (78/111)	👎 -21.883
Lines	100.0% (4062/4062)	💚 8.852
Conditionals	100.0% (0/0)	💚

[elasticmachine]

Package microsoft_sqlserver - 2.1.1 containing this change is available at https://epr.elastic.co/search?package=microsoft_sqlserver

[elasticmachine]

Package system - 1.36.2 containing this change is available at https://epr.elastic.co/search?package=system