Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

podman events missbehaving when there are zero events #15688

Closed
Luap99 opened this issue Sep 8, 2022 · 2 comments · Fixed by #15717
Closed

podman events missbehaving when there are zero events #15688

Luap99 opened this issue Sep 8, 2022 · 2 comments · Fixed by #15717
Assignees
Labels
In Progress This issue is actively being worked by the assignee, please do not work on this at this time. kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@Luap99
Copy link
Member

Luap99 commented Sep 8, 2022

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

podman events --stream=false is used for many test to make sure that we do not hang in the tests.
However as I found out while working on #15673 that --events-backend=file hangs if the event log file does not exists.
For --events-backend=none it also hangs for some reason? I would expect an error since there are no events.
Lastly with --events-backend=journald it just errors out right away when there are no events.

Steps to reproduce the issue:
file, as rootless:

$ rm $XDG_RUNTIME_DIR/libpod/tmp/events/events.log
$ podman events --events-backend=file --stream=false <--just hangs until you create events or create the file

journald:
first empty the journal

$ sudo journalctl --rotate
$ sudo journalctl --vacuum-time=1s
$ podman events --events-backend=journald --stream=false
Error: failed to get journal cursor: failed to get cursor: cannot assign requested address
$ podman events --events-backend=journald <--same error without stream=false
Error: failed to get journal cursor: failed to get cursor: cannot assign requested address

none:

$ podman events --events-backend=none --stream=false  <-- hangs without error
$ podman events --events-backend=none  <-- also hangs without error

Describe the results you expected:
The file backend with --stream=false should not hang as this can/will break tests.

The journald backend just not fail when there are no events.

The podman events with none backend should just error, there is no reason to ever run this command.

Output of podman version:

latest main
@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Sep 8, 2022
@edsantiago
Copy link
Collaborator

Suggestion please: when podman is hanging, please print the path to the log file. (Correction: preferably before podman hangs).

@Luap99
Copy link
Member Author

Luap99 commented Sep 9, 2022

Suggestion please: when podman is hanging, please print the path to the log file. (Correction: preferably before podman hangs).

you mean on debug or info log level?

@Luap99 Luap99 self-assigned this Sep 9, 2022
@Luap99 Luap99 added the In Progress This issue is actively being worked by the assignee, please do not work on this at this time. label Sep 9, 2022
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 9, 2022
podman --event-backend events --stream=false should never hang. The
problem is that our tail library will wait for the file to be created
which makes sense when we do not run with --stream=false. To fix this we
can just always create the file when the logger is iinitialized. This
would also help to report errors early on in case the file is not
accessible.

Fixes part one from containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 9, 2022
podman --events-backend none events should return with an error since it
will never be able to actually list events.

Fixes part three of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 9, 2022
Currently podman events will just fail with `Error: failed to get journal
cursor: failed to get cursor: cannot assign requested address` when the
journal contains zero podman events.

The problem is that we are using the journal accesors wrong. There is no
need to call GetCursor() and compare them manually. The Next() return an
integer which tells if it moved to the next or not. This means the we can
remove GetCursor() which would fail when there is no entry.

This also includes another bug fix. Previously the logic called Next()
twice for the first entry which caused us to miss the first entry.

To reproduce this issue you can run the following commands:
```
sudo journalctl --rotate
sudo journalctl --vacuum-time=1s
```
Note that this will delete the full journal.

Now run podman events and it fails but with this patch it works.
Now generate a single event, i.e. podman pull alpine, and run
podman events --until 1s.

I am not sure how to get a reliable test into CI, I really do not want
to delete the journal and developer or CI systems.

Fixes second part of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 9, 2022
podman --events-backend file events --stream=false should never hang. The
problem is that our tail library will wait for the file to be created
which makes sense when we do not run with --stream=false. To fix this we
can just always create the file when the logger is initialized. This
would also help to report errors early on in case the file is not
accessible.

Fixes part one from containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 9, 2022
podman --events-backend none events should return with an error since it
will never be able to actually list events.

Fixes part three of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 9, 2022
Currently podman events will just fail with `Error: failed to get journal
cursor: failed to get cursor: cannot assign requested address` when the
journal contains zero podman events.

The problem is that we are using the journal accessors wrong. There is no
need to call GetCursor() and compare them manually. The Next() return an
integer which tells if it moved to the next or not. This means the we can
remove GetCursor() which would fail when there is no entry.

This also includes another bug fix. Previously the logic called Next()
twice for the first entry which caused us to miss the first entry.

To reproduce this issue you can run the following commands:
```
sudo journalctl --rotate
sudo journalctl --vacuum-time=1s
```
Note that this will delete the full journal.

Now run podman events and it fails but with this patch it works.
Now generate a single event, i.e. podman pull alpine, and run
podman events --until 1s.

I am not sure how to get a reliable test into CI, I really do not want
to delete the journal and developer or CI systems.

Fixes second part of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
podman --events-backend file events --stream=false should never hang. The
problem is that our tail library will wait for the file to be created
which makes sense when we do not run with --stream=false. To fix this we
can just always create the file when the logger is initialized. This
would also help to report errors early on in case the file is not
accessible.

Fixes part one from containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
podman --events-backend none events should return with an error since it
will never be able to actually list events.

Fixes part three of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
Currently podman events will just fail with `Error: failed to get journal
cursor: failed to get cursor: cannot assign requested address` when the
journal contains zero podman events.

The problem is that we are using the journal accessors wrong. There is no
need to call GetCursor() and compare them manually. The Next() return an
integer which tells if it moved to the next or not. This means the we can
remove GetCursor() which would fail when there is no entry.

This also includes another bug fix. Previously the logic called Next()
twice for the first entry which caused us to miss the first entry.

To reproduce this issue you can run the following commands:
```
sudo journalctl --rotate
sudo journalctl --vacuum-time=1s
```
Note that this will delete the full journal.

Now run podman events and it fails but with this patch it works.
Now generate a single event, i.e. podman pull alpine, and run
podman events --until 1s.

I am not sure how to get a reliable test into CI, I really do not want
to delete the journal and developer or CI systems.

Fixes second part of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
podman --events-backend none events should return with an error since it
will never be able to actually list events.

Fixes part three of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
Currently podman events will just fail with `Error: failed to get journal
cursor: failed to get cursor: cannot assign requested address` when the
journal contains zero podman events.

The problem is that we are using the journal accessors wrong. There is no
need to call GetCursor() and compare them manually. The Next() return an
integer which tells if it moved to the next or not. This means the we can
remove GetCursor() which would fail when there is no entry.

This also includes another bug fix. Previously the logic called Next()
twice for the first entry which caused us to miss the first entry.

To reproduce this issue you can run the following commands:
```
sudo journalctl --rotate
sudo journalctl --vacuum-time=1s
```
Note that this will delete the full journal.

Now run podman events and it fails but with this patch it works.
Now generate a single event, i.e. podman pull alpine, and run
podman events --until 1s.

I am not sure how to get a reliable test into CI, I really do not want
to delete the journal and developer or CI systems.

Fixes second part of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
podman --events-backend file events --stream=false should never hang. The
problem is that our tail library will wait for the file to be created
which makes sense when we do not run with --stream=false. To fix this we
can just always create the file when the logger is initialized. This
would also help to report errors early on in case the file is not
accessible.

Fixes part one from containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
podman --events-backend none events should return with an error since it
will never be able to actually list events.

Fixes part three of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/libpod that referenced this issue Sep 12, 2022
Currently podman events will just fail with `Error: failed to get journal
cursor: failed to get cursor: cannot assign requested address` when the
journal contains zero podman events.

The problem is that we are using the journal accessors wrong. There is no
need to call GetCursor() and compare them manually. The Next() return an
integer which tells if it moved to the next or not. This means the we can
remove GetCursor() which would fail when there is no entry.

This also includes another bug fix. Previously the logic called Next()
twice for the first entry which caused us to miss the first entry.

To reproduce this issue you can run the following commands:
```
sudo journalctl --rotate
sudo journalctl --vacuum-time=1s
```
Note that this will delete the full journal.

Now run podman events and it fails but with this patch it works.
Now generate a single event, i.e. podman pull alpine, and run
podman events --until 1s.

I am not sure how to get a reliable test into CI, I really do not want
to delete the journal and developer or CI systems.

Fixes second part of containers#15688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 16, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
In Progress This issue is actively being worked by the assignee, please do not work on this at this time. kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants