Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for aws filter plugin #173

Merged
merged 1 commit into from Nov 27, 2021
Merged

Conversation

wenchajun
Copy link
Member

@wenchajun wenchajun commented Nov 25, 2021

What this PR dose?

Adds support for https://docs.fluentbit.io/manual/pipeline/filters/aws-metadata filter plugin

Why we need it?

Fluentbit supports a variety of plug-ins, but not all fluentbit-operator adaptations. More plug-ins will be supported in the future, see the fields supported by the Filter plug-in

Field Description Scheme Default
imds_version Specify which version of the instance metadata service to use. Valid values are 'v1' or 'v2'. string v2
az The availability zone; for example, "us-east-1a". bool true
ec2_instance_id The EC2 instance ID. bool true
ec2_instance_type The EC2 instance type. bool false
private_ip The EC2 instance private ip. bool false
ami_id The EC2 instance image id. bool false
account_id The account ID for current EC2 instance. bool false
hostname The hostname for current EC2 instance. bool false
vpc_id The VPC ID for current EC2 instance. bool false

The final results

The configuration is written to secret
1637847906

Signed-off-by: chengdehao dehaocheng@yunify.com

@wanjunlei wanjunlei self-requested a review November 26, 2021 01:25
// The availability zone; for example, "us-east-1a". Default is true.
Az *bool `json:"az,omitempty"`
//The EC2 instance ID.Default is true.
Ec2InstanceId *bool `json:"ec2InstanceId,omitempty"`
Copy link
Collaborator

@wanjunlei wanjunlei Nov 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ec2InstanceId => EC2InstanceID
Ec2InstanceType => EC2InstanceType
PrivateIp => PrivateIP
AmiId => AmiID
AccountId => AccountID
VpcId => VpcID

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed

// +kubebuilder:object:generate:=true

// The AWS Filter Enriches logs with AWS Metadata. Currently the plugin adds the EC2 instance ID and availability zone to log records.
type Aws struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aws => AWS

@@ -61,6 +61,8 @@ type FilterItem struct {
Throttle *filter.Throttle `json:"throttle,omitempty"`
// RewriteTag defines a RewriteTag configuration.
RewriteTag *filter.RewriteTag `json:"rewriteTag,omitempty"`
//Aws defines a Aws configuration.
Aws *filter.Aws `json:"aws,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aws => AWS

// The availability zone; for example, "us-east-1a". Default is true.
Az *bool `json:"az,omitempty"`
//The EC2 instance ID.Default is true.
EC2InstanceID *bool `json:"ec2InstanceId,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ec2InstanceId => ec2InstanceID
privateIp => privateIP
amiId => amiID
accountId => accountID
vpcId => vpcID

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I overlooked that

// +kubebuilder:validation:Enum:=v1;v2
ImdsVersion string `json:"imdsVersion,omitempty"`
// The availability zone; for example, "us-east-1a". Default is true.
Az *bool `json:"az,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Az could be changed to AZ too

Signed-off-by: chengdehao <dehaocheng@yunify.com>
@benjaminhuo benjaminhuo merged commit f49d349 into fluent:master Nov 27, 2021
@wenchajun wenchajun deleted the awsfilter branch November 29, 2021 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants