Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

disable custom metrics #1260

Merged
merged 15 commits into from
May 23, 2024
Merged

disable custom metrics #1260

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
7ce4bb3
disable custom metrics by default
May 21, 2024
1d4cb67
missed changes
May 21, 2024
116e554
changes for windows
May 21, 2024
8ca95c2
fix bugs
May 21, 2024
c10a7e6
change name
May 22, 2024
bee02a5
remove markers
May 22, 2024
784dc76
improve custom metrics deprecation logic
May 22, 2024
60be9ec
remove unnecessary function
May 22, 2024
6fe6ee7
update win scripts
May 22, 2024
5325d5e
fix bug
May 22, 2024
0457e70
remove extra changes
May 22, 2024
28f7654
update enablement conditions
May 23, 2024
5b26090
fix bug
May 23, 2024
adc85fd
add kube-cm as well
May 23, 2024
35c6af6
fix config issues
May 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion build/common/installer/scripts/fluent-bit-conf-customizer.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ def substituteFluentBitPlaceHolders
multilineLogging = ENV["AZMON_MULTILINE_ENABLED"]
stacktraceLanguages = ENV["AZMON_MULTILINE_LANGUAGES"]
resourceOptimizationEnabled = ENV["AZMON_RESOURCE_OPTIMIZATION_ENABLED"]
enableCustomMetrics = ENV["ENABLE_CUSTOM_METRICS"]
windowsFluentBitDisabled = ENV["AZMON_WINDOWS_FLUENT_BIT_DISABLED"]
kubernetesMetadataCollection = ENV["AZMON_KUBERNETES_METADATA_ENABLED"]
annotationBasedLogFiltering = ENV["AZMON_ANNOTATION_BASED_LOG_FILTERING"]
Expand Down Expand Up @@ -194,7 +195,10 @@ def substituteFluentBitPlaceHolders

new_contents = substituteMultiline(multilineLogging, stacktraceLanguages, new_contents)

if !@isWindows || (@isWindows && (!windowsFluentBitDisabled.nil? && windowsFluentBitDisabled.to_s.downcase == "false"))
# Valid resource optimization scenarios
# if Linux and Custom Metrics not enabled
# or if Windows and Fluent Bit is not disabled
if (!@isWindows && (enableCustomMetrics.nil? || enableCustomMetrics.to_s.downcase == "false")) || (@isWindows && (!windowsFluentBitDisabled.nil? && windowsFluentBitDisabled.to_s.downcase == "false"))
new_contents = substituteResourceOptimization(resourceOptimizationEnabled, new_contents)
end
File.open(@fluent_bit_conf_path, "w") { |file| file.puts new_contents }
Expand Down
8 changes: 7 additions & 1 deletion build/common/installer/scripts/tomlparser-agent-config.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,10 +384,16 @@ def populateSettingValuesFromConfigMap(parsedConfig)
end
end

enable_custom_metrics = ENV["ENABLE_CUSTOM_METRICS"]
if !enable_custom_metrics.nil? && enable_custom_metrics.to_s.downcase == "true"
@resource_optimization_enabled = false
puts "Resource Optimization disabled since custom metrics is enabled"
end

windows_fluent_bit_config = parsedConfig[:agent_settings][:windows_fluent_bit]
if !windows_fluent_bit_config.nil?
windows_fluent_bit_disabled = windows_fluent_bit_config[:disabled]
if !windows_fluent_bit_disabled.nil? && windows_fluent_bit_disabled.downcase == "false"
if !windows_fluent_bit_disabled.nil? && windows_fluent_bit_disabled.to_s.downcase == "false"
@windows_fluent_bit_disabled = false
end
puts "Using config map value: AZMON_WINDOWS_FLUENT_BIT_DISABLED = #{@windows_fluent_bit_disabled}"
Expand Down
17 changes: 17 additions & 0 deletions build/common/installer/scripts/tomlparser-common-agent-config.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
@disableTelemetry = false
@logEnableKubernetesMetadataCacheTTLSeconds = 60
pfrcks marked this conversation as resolved.
Show resolved Hide resolved
@enableHighLogScaleMode = false
@enableCustomMetrics = false
ganga1980 marked this conversation as resolved.
Show resolved Hide resolved

def is_windows?
return !@os_type.nil? && !@os_type.empty? && @os_type.strip.casecmp("windows") == 0
Expand Down Expand Up @@ -81,6 +82,14 @@ def populateSettingValuesFromConfigMap(parsedConfig)
end
end

pfrcks marked this conversation as resolved.
Show resolved Hide resolved
if !parsedConfig.nil? && !parsedConfig[:agent_settings].nil?
custom_metrics = parsedConfig[:agent_settings][:custom_metrics]
if !custom_metrics.nil? && !custom_metrics[:enabled].nil?
@enableCustomMetrics = custom_metrics[:enabled]
puts "Using config map value: enabled = #{@enableCustomMetrics} for custom metrics"
end
end

rescue => errorStr
puts "config::error:Exception while reading config settings for agent configuration setting - #{errorStr}, using defaults"
end
Expand Down Expand Up @@ -117,6 +126,11 @@ def get_command_windows(env_variable_name, env_variable_value)
file.write(commands)
end

if @enableCustomMetrics
commands = get_command_windows("ENABLE_CUSTOM_METRICS", @enableCustomMetrics)
file.write(commands)
end

commands = get_command_windows("AZMON_KUBERNETES_METADATA_CACHE_TTL_SECONDS", @logEnableKubernetesMetadataCacheTTLSeconds)
file.write(commands)
# Close file after writing all environment variables
Expand All @@ -136,6 +150,9 @@ def get_command_windows(env_variable_name, env_variable_value)
if @enableHighLogScaleMode
file.write("export ENABLE_HIGH_LOG_SCALE_MODE=#{@enableHighLogScaleMode}\n")
end
if @enableCustomMetrics
file.write("export ENABLE_CUSTOM_METRICS=#{@enableCustomMetrics}\n")
end
file.write("export AZMON_KUBERNETES_METADATA_CACHE_TTL_SECONDS=#{@logEnableKubernetesMetadataCacheTTLSeconds}\n")
# Close file after writing all environment variables
file.close
Expand Down
141 changes: 141 additions & 0 deletions build/linux/installer/conf/container-cm.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Fluentd config file for OMS Docker - container components (non kubeAPI)

# Forward port 25225 for container logs
# gangams - not used and get ridoff after confirming safe to remove
<source>
@type forward
port 25225
bind 127.0.0.1
</source>

# MDM metrics from telegraf
<source>
@type tcp
tag oms.mdm.container.perf.telegraf.*
bind 0.0.0.0
port 25228
format json
</source>

# Container inventory
<source>
@type containerinventory
tag oneagent.containerInsights.CONTAINER_INVENTORY_BLOB
run_interval 60
@log_level info
</source>

#cadvisor perf
<source>
@type cadvisor_perf
tag oneagent.containerInsights.LINUX_PERF_BLOB
run_interval 60
@log_level info
</source>

#custom_metrics_mdm filter plugin
<filter mdm.cadvisorperf**>
@type cadvisor2mdm
metrics_to_collect cpuUsageNanoCores,memoryWorkingSetBytes,memoryRssBytes,pvUsedBytes
@log_level info
</filter>

<filter oms.mdm.container.perf.telegraf**>
@type telegraf2mdm
@log_level info
</filter>

#containerinventory
<match **CONTAINER_INVENTORY_BLOB**>
@type forward
@log_level info
send_timeout 30
connect_timeout 30
heartbeat_type none
<server>
host 0.0.0.0
port "#{ENV['MDSD_FLUENT_SOCKET_PORT']}"
</server>
<buffer>
@type file
path /var/opt/microsoft/docker-cimprov/state/containerinventory*.buffer
overflow_action drop_oldest_chunk
chunk_limit_size 4m
queue_limit_length 20
flush_interval 20s
retry_max_times 10
retry_wait 5s
retry_max_interval 5m
flush_thread_count 5
</buffer>
keepalive true
</match>

#cadvisorperf
<match **LINUX_PERF_BLOB**>
@type forward
@log_level info
send_timeout 30
connect_timeout 30
heartbeat_type none
<server>
host 0.0.0.0
port "#{ENV['MDSD_FLUENT_SOCKET_PORT']}"
</server>
<buffer>
@type file
path /var/opt/microsoft/docker-cimprov/state/cadvisorperf*.buffer
overflow_action drop_oldest_chunk
chunk_limit_size 4m
queue_limit_length 20
flush_interval 20s
retry_max_times 10
retry_wait 5s
retry_max_interval 5m
flush_thread_count 5
</buffer>
keepalive true
</match>

<match mdm.cadvisorperf** oms.mdm.container.perf.telegraf**>
@type mdm
@log_level info
<buffer>
@type file
path /var/opt/microsoft/docker-cimprov/state/out_mdm_cdvisorperf*.buffer
overflow_action drop_oldest_chunk
chunk_limit_size 4m
flush_interval 20s
retry_max_times 10
retry_wait 5s
retry_max_interval 5m
flush_thread_count 5
</buffer>
retry_mdm_post_wait_minutes 30
</match>

#InsightsMetrics
<match **INSIGHTS_METRICS_BLOB**>
@type forward
@log_level info
send_timeout 30
connect_timeout 30
heartbeat_type none
<server>
host 0.0.0.0
port "#{ENV['MDSD_FLUENT_SOCKET_PORT']}"
</server>
<buffer>
@type file
path /var/opt/microsoft/docker-cimprov/state/insightsmetrics*.buffer
overflow_action drop_oldest_chunk
chunk_limit_size 4m
queue_limit_length 20
flush_interval 20s
retry_max_times 10
retry_wait 5s
retry_max_interval 5m
flush_thread_count 5
</buffer>
keepalive true
</match>
38 changes: 0 additions & 38 deletions build/linux/installer/conf/container.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,6 @@
bind 127.0.0.1
</source>

# MDM metrics from telegraf
<source>
@type tcp
tag oms.mdm.container.perf.telegraf.*
bind 0.0.0.0
port 25228
format json
</source>

# Container inventory
<source>
@type containerinventory
Expand All @@ -33,18 +24,6 @@
@log_level info
</source>

#custom_metrics_mdm filter plugin
<filter mdm.cadvisorperf**>
@type cadvisor2mdm
metrics_to_collect cpuUsageNanoCores,memoryWorkingSetBytes,memoryRssBytes,pvUsedBytes
@log_level info
</filter>

<filter oms.mdm.container.perf.telegraf**>
@type telegraf2mdm
@log_level info
</filter>

#containerinventory
<match **CONTAINER_INVENTORY_BLOB**>
@type forward
Expand Down Expand Up @@ -97,23 +76,6 @@
keepalive true
</match>

<match mdm.cadvisorperf** oms.mdm.container.perf.telegraf**>
@type mdm
@log_level info
<buffer>
@type file
path /var/opt/microsoft/docker-cimprov/state/out_mdm_cdvisorperf*.buffer
overflow_action drop_oldest_chunk
chunk_limit_size 4m
flush_interval 20s
retry_max_times 10
retry_wait 5s
retry_max_interval 5m
flush_thread_count 5
</buffer>
retry_mdm_post_wait_minutes 30
</match>

#InsightsMetrics
<match **INSIGHTS_METRICS_BLOB**>
@type forward
Expand Down
Loading
Loading