Log lines for an active step are inaccessible #886
Comments
|
It looks like this is a KI, but I can't find a canonical bug tracking it: |
|
👋 Actions engineer here So I wouldn't classify this as a bug, but it's just a missing feature. When a step is running, you can do "inspect element", and see that there is a active websocket that is streaming in new output from the runner. When a step is completed though, a GET request is made to our service that downloads all the logs for step in one go. If a step is in-progress and a user refreshes a page or first loads a page while it is in-progress, the logs that were already streamed to the websocket are effectively gone until the step is completed and we can fetch everything using an internal GET api. The solution for us is to somehow cache existing logs that have already been streamed and if a user refreshes a step that is in-progress, fetch the cached logs and display that + append new logs that are coming in through the websocket. We'll eventually add this feature (this would be really really nice to have), but at the moment we just haven't been able to pick this up. |
|
This would be great to have because some steps can take a while, or in fact due to error get stuck for a good half hour - and you're sitting there wondering if you should cancel it or not. It comes up often as a problem during development of your workflow! |
|
I believe this is on our backlog. |
|
yeah this would be nice to have 😃 |
TingluoHuang
added
Actions Feature
|
Hi! This seems like a possibly useful addition. However, suggestions for GitHub Actions functionality belong in the GitHub feedback forum, where the product managers and engineers will be able to review them. This repository is really only focused on the runner application, which is a very small component of GitHub Actions. We do have this opened on our backlog, so I'm going to close this issue here. Thanks again! |
|
This is basic CI functionality, so it's a bug. I don't know how you could classify it as a nice to have feature. Sounds like you aren't in the trenches having to diagnose builds much |
|
I'm suffering this right now :p. Update: 1hr 42min, make your bets! |
|
I agree with the thought process that this is not nice to have but a must-have. If you're working on debugging a prod deploy that has gone south, you may not be able to cancel the build, but rather you must let it complete the rollback steps. This means you lose valuable time and information by needing to wait... |
|
@ethomson can we get a link to the backlog Item in this thread? |
|
As others have commented, I don't see this as a nice-to-have but rather, an elemental feature. A link to the backlog feature would be nice! |
|
@TingluoHuang @konradpabjan any update for this issue/bug? |
|
Not a fan of bumping for the sake of it, but yes, +1, this is not a "nice to have" but a fundamental feature that should have been part of initial design requirements for a CI solution - if not realtime, at least near-realtime access to logging of job outputs is crucial |
|
I'm also going to heavily +1 this. It was mentioned that a discussion would need to be opened to express desires for this feature/bugfix to be resolved sooner than later if it's on the backlog. I didn't see any when I searched, so I've created this discussion here: |
|
?? why did this get closed? I think this is not a "nice to have" feature, but rather a "missing" feature. it's been almost two years and I can't still see the log in real time.. |
|
this is not released yet, they seem making a progress for this issue behind the scene.
|
|
@0xF4D3C0D3 do you believe these are related to this issue? They seem to be about getting the actual Worker logs into our hands, rather than the step its executing. I may be misreading. |
I've been following this for awhile I've always understood it to be the step it's executing. Currently the logs often don't show up until after the step finishes. Can be a long time and you have no idea if your step is hung or not |
Same, this is currently defeating my transition from Jenkins to Actions. By my comment I meant that these commits recently made by @AvaStancu may not be related to this issue here, but aiming to help people wanting to watch/debug the runner logs itself. Unfortunately, the Issues related to those commits are in a private repo and we can't get context. |
|
I don't think this is being worked on right now... |
|
This is definitely what I would call a bug and it's rather annoying to not be able to get an idea of where a step has reached particularly when debugging very long-running steps. |
|
Hey, we want to follow up here from GitHub Actions engineering. First and foremost just wanted to let everyone know that we hear you. We're very aware that this is a big ask and one of the big pain points with actions. Over recent months there's big architectural changes happening as part of larger efforts to set us up for many larger goals; this being one of them. The services and the key pinnings behind those changes are out in production and being phased in. We hope to realize this as part of those changes soon. Even though the issue is closed (since it's really not primarily a runner issue) we wanted to follow up since this is high activity. Thanks for the (very clear) feedback here. :) |
|
About three months later and I'm still waiting for a practical solution. It's really a pain when you have PR-triggered test jobs that run for over an hour producing logs from time to time, but there is no way to have a look at what has been printed so far - unless you capture all log from the beginning. |
Anyone have an idea of what the ACTIVE and OPEN issue is for tracking the acclaimed changes that would remediate this? So far we are all here watching and waiting on closed issues (this one and #891 ) - doesn't bode well for me trying to have optimism on issues that have no priority or have been deemed completed 😞 |
|
Thanks @bryanmacfarlane and @yacaovsnc for the transparency, that is good news 🙂 |
|
Hi! Are we getting close to onboarding people to this? Liberty would love to get in on this ASAP |
|
Same, this is making some of our deployment processes effectively a black box |
@yacaovsnc, now that it is early September, are you able to provide an updated ETA? |
|
if you want to try it out, then email me. my email is my full name (alias) and that's at github dot com. I can hook you up. We finished scale testing and had a bug bash. We're fixing a few bugs 🤞 so hopefully a few days. I'm the messenger here and relaying back to the teams so 🙏 ❤️ |
|
^ Amazing news! Us folks at aptos-labs would love to try it out if you can include us as well (sent an email too) |
|
Any chance of getting this enabled on the duckdb and duckdblabs orgs? We have several long running jobs that would really benefit from this |
|
👋 Hi all - Actions PM here. I wanted to thank everyone for their interest in testing out the feature. We have a good amount of customers in the test group already so we won't be taking on anymore for now. I'll reach back out if we open it again. |
|
Very excited to see this feature. |
|
@andrewakim do we have any updates here? |
|
👋 Hi all - Appreciate the patience. The team is still working on the feature and addressing a few issues we came across. I'll set a reminder for myself to post an update in 2-3 weeks and I'll hopefully have a bit more detail around timelines by then. |
|
As promised, just wanted to share a quick update. We had to shift gears to focus on a few other priorities that came up in the last few weeks but we're planning on returning to this work later in the quarter. I'll keep sending out updates as things develop. |
|
👋 Hi, @andrewakim @bryanmacfarlane What is the timeline looking like for this feature to be released? |
|
👋 Hi everyone - Good news! We're planning on rolling this feature out slowly starting in a few weeks. We'll begin to ramp up the rollout in the new year. Thanks for your patience! |
|
following... |
|
asking for this feature is like asking Microsoft to give away % of their current revenue, literally.. the runner billing per minutes spent, duuh 😅 Look: so there's a GET request "in the end when stage has been completed", which means they record the logs into some storage anyway, in the process, while the amazing websocket is "only streaming to browser". So letting the users access that "full log from current storage - whatever was cached so far" is a piece of cake (we're all developers here, and we know it 💯 ). The reasons why it would last 2.5 years to "fix" or implement the feature could be bandwidth cost (more GET requests in total, while stage is not yet completed) which is miniscule since we can stream the naughty websocket all day and watch logs anyway, even if we missed the critical first hundred lines - so bandwidth is not an issue. Or it could be the cost we all pay for the github runners per minute billed, and we obviously will spend more minutes waiting for stages to complete 🧐. Kinda ridiculous but since there are so many other pros to github, most of us keeps playing along like we don't understand what really goes on with this bug over all these years 😆 . Enjoy that runners billing revenue, Microsoft! 🚀💰. Well we can't blame the engineering team, this is business 🤷♂️. EDIT: * 3 years to fix 🙊 🙉 |
|
@Dmitry1987 Did you bother to read even just 2 comments back before you posted your diatribe? TL;DR: The fix for this is being rolled out. |
Do you bother to understand I don't care if the fix is out or not, I'm talking about the past 3 years that it's been dragged obviously? I honestly don't care since I'll stream the stdout of our scripts to another storage from which we'll simply display it in any quick and dirty node.js/vue.js dashboard (well, here's your feature right there 🤣 I don't need to wait for it 3 years, and it proves that it is an easy fix, and the real reason was a business decision to milk more runner minutes, 💯 ). As I said, we're not Grandmas with flowers, we're all developers and we understand how things work. So far the handling of this "bug" was subpar and we do our conclusions. For example, Azure already "enjoys" the effect of community sentiment (when engineers tell the management to avoid using Azure) your business decisions should take this into account since it affects the long term customer base. I don't buy that it was a difficult or expensive feature to roll out during 3 years 🤷♂️ |
|
Part of GitHub Roadmap github/roadmap#839, since a week
I'm wondering how this rollout will hit my custom actions runner, tracking this ca. 1 year. Due to implied protocol swap from azure pipelines like dotnet backend to a new golang based backend |
Bzzzzt. That's a 404. |
|
My apologies @brianjmurrell this post was in error. Please keep watching the roadmap for updates github/roadmap#839! |
|
Interestingly, I noticed today that I have a beta label on the workflow UI with a link to https://github.com/orgs/community/discussions/89879. A sample run with the beta label showing for me is https://github.com/ChristopherHX/github-act-runner/actions/runs/7618673328/job/20750292224 And I could see old log lines in the UI of running steps on GitHub Hosted Runners even after pressing f5, which is good. UPDATE - 23 Jan 2024 23:28 CEST
https://github.com/orgs/community/discussions/89879#discussioncomment-8224894 |
|
Hi, this still seems to be an issue. Correct? |
|
If you have "beta" badge on your run then log lines should update, if not
then it's still old behavior
…On Thu, Apr 25, 2024, 11:12 AM lure8225 ***@***.***> wrote:
Hi, this still seems to be an issue. Correct?
If I have a runner and watch the steps live I will get the streaming
connection and see the step output. If I open the workflow run after the
step has started and before the step has finished I will se nothing as the
output is not fetched via get yet but there is also no new output streamed
—
Reply to this email directly, view it on GitHub
<#886 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMUMHJ7QK6G7IG7BLZ3R5TY7EMMHAVCNFSM4VN3E7U2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBXG42TENBVGY4A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Thanks for the instant reply. I'm using a selfhosted runner via ARC. Will this issue also be resolved for these when this gets out of beta? |
|
I'm using a self hosted runner as well and have beta badge. I think it's
like a feature flag
…On Thu, Apr 25, 2024, 11:21 AM lure8225 ***@***.***> wrote:
Thanks for the instant reply. I'm using a selfhosted runner via ARC. Will
this issue also be resolved for these when this gets out of beta?
—
Reply to this email directly, view it on GitHub
<#886 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMUMHLGZHMCXHZ3I2KC6DTY7ENOJAVCNFSM4VN3E7U2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBXG42TMMJXHA2Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Looks like this has been released! 🎉 |
and others
Describe the bug
When opening the "run" page for an active job you can see:
The log lines for the active steps will then start streaming in, but the prior lines for that step are not there, and there appears to be no way to see them until the step completes.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
All of the log lines that were emit before I loaded the page should be visible, then new ones stream in afterwards.
The text was updated successfully, but these errors were encountered: