Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Logstash crashes with env NODE_NAME #7450

Closed
kaisecheng opened this issue Jan 9, 2024 · 0 comments · Fixed by #7475
Closed

Logstash crashes with env NODE_NAME #7450

kaisecheng opened this issue Jan 9, 2024 · 0 comments · Fixed by #7475
Assignees
@kaisecheng
Labels
>bug Something isn't working :logstash

Comments

@kaisecheng
Copy link
Contributor

kaisecheng commented Jan 9, 2024 •
edited
Loading

Proposal

Use case. Why is this important?
Logstash container crashes before log4j be able to write logstash-plain.log.

Bug Report

What did you do?

apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: logstash-sample
spec:
  count: 1
  version: 8.11.1
  podTemplate:
    spec:
      containers:
        - name: logstash
          env:
            - name: "NODE_NAME"
              value: "Crash!"

What did you expect to see?

Logstash starts without error

What did you see instead? Under which circumstances?

Logstash crashes with CrashLoopBackOff

$ kubectl get pod

NAME                      READY   STATUS     RESTARTS     AGE
logstash-sample-ls-0      0/1     Error      1 (2s ago)   3s

$ kubectl logs -f logstash-sample-ls-0

Defaulted container "logstash" out of: logstash, logstash-internal-init-config (init)
2024/01/09 12:08:42 Setting 'node.name' from environment.
2024/01/09 12:08:42 error: open /usr/share/logstash/config/logstash.yml: read-only file system

Environment

  • ECK version:

    Logstash technical preview. ECK 2.8

Root cause

This is an issue relates to logstash image startup process env2yaml, which overwrites the logstash.yml regardless of the existing settings in the file. The source of logstash.yml is from Secret which is readonly mapping to mount path. Changing ln -sf to cp could solve the issue.
@kaisecheng kaisecheng added >bug Something isn't working :logstash labels Jan 9, 2024
@kaisecheng kaisecheng mentioned this issue Jan 17, 2024
Merged
pebrc pushed a commit that referenced this issue Jan 25, 2024
@kaisecheng
Fix Logstash crashes with env (#7475)
7ab8da0 
Fixed: #7450

Logstash crashes when env variable name is in the list of
[env2yaml](https://github.com/elastic/logstash/blob/main/docker/data/logstash/env2yaml/env2yaml.go#L50-L155)
This PR changes the config init container to copy the `logstash.yml` to
`config` Volume to allow updating the file.

The following resource should start without error
```yaml
apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: logstash-sample
spec:
  count: 1
  version: 8.11.1
  podTemplate:
    spec:
      containers:
        - name: logstash
          env:
            - name: "NODE_NAME"
              value: "No_Crash!"
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kaisecheng kaisecheng

Labels
>bug Something isn't working :logstash
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix Logstash crashes with env kaisecheng/cloud-on-k8s
1 participant
@kaisecheng