Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CoreCLR's CNTRL_SHUTDOWN_EVENT handler prevents graceful exit of services during system shutdown #83093

Open
gkapellmann opened this issue Mar 7, 2023 · 33 comments

Comments

@gkapellmann
Copy link

gkapellmann commented Mar 7, 2023

Description

I am not sure if this belongs here, but this is a new weird behaviour I have been fighting with lately.

I have a service, where the 'Main_Worker' is a 'IHostedService' and a 'BackgroundService'. I have overriden the 'StopAsync' function and the only thing it really does is write to the Windows Event Log a message and save a txt file with some data.

While I was working with this structure in .NET 6, it all worked preatty well. Then I upgraded to .Net 7, and it seems that now 'StopAsync' is not triggering when I turn of the PC by a normal Power -> Shutdown or if I use 'Process.Start("cmd", @"/c shutdown -s -t 0");'

When in .NET 6, both ways it worked without problems.

Reproduction Steps

public async override Task<Task> StopAsync(CancellationToken cancellationToken) { Log.Text("Stopping Worker_Main service..."); logger.LogWarning("Test Service is Stopping..."); await Log.SaveCurrentLog(); return base.StopAsync(cancellationToken); }

That is my StopAsync function, it used to work like this with no problems. But now, even if you take away the await line, not even the logger is appearing.

Expected behavior

At least I would expect to see the logged message in the Windows Event Logger when the PC is turned on again.

Actual behavior

No trace of the message being logged or the file being written. The service probably closes, yet I have no idea if it is in a forced mode or if it is actually being closed in a gracefull way.

Regression?

No response

Known Workarounds

I do not have workarounds at the moment.

Configuration

Have tested in Windows 10 & 11
Both are x64

Other information

No response

@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Mar 7, 2023
@ghost
Copy link

ghost commented Mar 7, 2023

Tagging subscribers to this area: @dotnet/area-extensions-hosting
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

I am not sure if this belongs here, but this is a new weird behaviour I have been fighting with lately.

I have a service, where the 'Main_Worker' is a 'IHostedService' and a 'BackgroundService'. I have overriden the 'StopAsync' function and the only thing it really does is write to the Windows Event Log a message and save a txt file with some data.

While I was working with this structure in .NET 6, it all worked preatty well. Then I upgraded to .Net 7, and it seems that now 'StopAsync' is not triggering when I turn of the PC by a normal Power -> Shutdown or if I use 'Process.Start("cmd", @"/c shutdown -s -t 0");'

When in .NET 6, both ways it worked without problems.

Reproduction Steps

public async override Task<Task> StopAsync(CancellationToken cancellationToken) { Log.Text("Stopping Worker_Main service..."); logger.LogWarning("Test Service is Stopping..."); await Log.SaveCurrentLog(); return base.StopAsync(cancellationToken); }

That is my StopAsync function, it used to work like this with no problems. But now, even if you take away the await line, not even the logger is appearing.

Expected behavior

At least I would expect to see the logged message in the Windows Event Logger when the PC is turned on again.

Actual behavior

No trace of the message being logged or the file being written. The service probably closes, yet I have no idea if it is in a forced mode or if it is actually being closed in a gracefull way.

Regression?

No response

Known Workarounds

I do not have workarounds at the moment.

Configuration

Have tested in Windows 10 & 11
Both are x64

Other information

No response

Author: gkapellmann
Assignees: -
Labels:

untriaged, area-Extensions-Hosting

Milestone: -

@gkapellmann
Copy link
Author

As a matter of fact, just to give a bit more of context, here is a test I have been doing:

lifetime.ApplicationStarted.Register(() => logger.LogInformation("ApplicationStarted Done...")); lifetime.ApplicationStopping.Register(() => logger.LogInformation("ApplicationStopping Done...")); lifetime.ApplicationStopped.Register(() => logger.LogInformation("ApplicationStopped Done..."));

ApplicationStarted appears in the event viewer perfectly well. But the other two are not. This when restaring or shutdown.

But if I stop gracefully the service manually, both messageds appear fine.

@buyaa-n
Copy link
Contributor

buyaa-n commented Mar 13, 2023

Looks similar to #62579 and #81274, @gkapellmann could you try the workaround mentioned and let us know if that helps your issue? The workaround is for unblocking you for short term, if you could let us know how it goes, it would be a useful data for to the team to find appropriate fix.

@buyaa-n buyaa-n added this to the Future milestone Mar 13, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Mar 13, 2023
@buyaa-n buyaa-n added the needs-author-action An issue or pull request that requires more info or actions from the author. label Mar 13, 2023
@ghost
Copy link

ghost commented Mar 13, 2023

This issue has been marked needs-author-action and may be missing some important information.

@ericstj
Copy link
Member

ericstj commented Mar 15, 2023

This one is actually different than the error 1067 issue. In that case the service process was exiting before completing ServiceBase's OnStop method which is what notified SCM that the service cleanly stopped. In that case it would run all ApplicationStopping and ApplicationStopped.

In this case the machine is shutting down. I was looking at a similar issue internally and found that the problem was actually due to #41101. During a shutdown a service will first receive a CNTRL_SHUTDOWN_EVENT. The default is to do nothing for a service process per https://learn.microsoft.com/en-us/windows/console/handlerroutine#remarks, but that was changed for .NET processes in #41101.

I was able to workaround this by adding ConsoleControlHandler that cancels other handlers when seeing CNTRL_SHUTDOWN_EVENT - stopping the runtime's handler from shutting down the process and letting the service exit normally. In .NET 6.0 you can do this with a one-liner:

    using var registration = PosixSignalRegistration.Create(PosixSignal.SIGTERM, (context) => context.Cancel = true);

Just make sure that registration stays alive for the entire duration of the service - I put it in the Main() routine.

We should look into adding something like this to ServiceBase -- or having the runtime try to avoid handling CNTRL_SHUTDOWN_EVENT for services.

@ericstj ericstj added bug and removed needs-author-action An issue or pull request that requires more info or actions from the author. labels Mar 15, 2023
@ericstj ericstj modified the milestones: Future, 8.0.0 Mar 15, 2023
@ghost
Copy link

ghost commented Mar 15, 2023

Tagging subscribers to this area: @dotnet/area-system-serviceprocess
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

I am not sure if this belongs here, but this is a new weird behaviour I have been fighting with lately.

I have a service, where the 'Main_Worker' is a 'IHostedService' and a 'BackgroundService'. I have overriden the 'StopAsync' function and the only thing it really does is write to the Windows Event Log a message and save a txt file with some data.

While I was working with this structure in .NET 6, it all worked preatty well. Then I upgraded to .Net 7, and it seems that now 'StopAsync' is not triggering when I turn of the PC by a normal Power -> Shutdown or if I use 'Process.Start("cmd", @"/c shutdown -s -t 0");'

When in .NET 6, both ways it worked without problems.

Reproduction Steps

public async override Task<Task> StopAsync(CancellationToken cancellationToken) { Log.Text("Stopping Worker_Main service..."); logger.LogWarning("Test Service is Stopping..."); await Log.SaveCurrentLog(); return base.StopAsync(cancellationToken); }

That is my StopAsync function, it used to work like this with no problems. But now, even if you take away the await line, not even the logger is appearing.

Expected behavior

At least I would expect to see the logged message in the Windows Event Logger when the PC is turned on again.

Actual behavior

No trace of the message being logged or the file being written. The service probably closes, yet I have no idea if it is in a forced mode or if it is actually being closed in a gracefull way.

Regression?

No response

Known Workarounds

I do not have workarounds at the moment.

Configuration

Have tested in Windows 10 & 11
Both are x64

Other information

No response

Author: gkapellmann
Assignees: -
Labels:

bug, area-System.ServiceProcess, regression-from-last-release

Milestone: 8.0.0

@ericstj
Copy link
Member

ericstj commented Mar 15, 2023

Ideally we don't do this only in ServiceBase since that wouldn't fix cases where folks were implementing services themselves without using ServiceBase. Instead we should try to replicate whatever the OS does in the default control handler to distinguish between normal console processes and services.

@gkapellmann
Copy link
Author

Hello @ericstj, thank you for your answers!

I have not tested your workaround in .NET 6.0, but I did test wit with a small example in .NET 7.0 and is not working for me. As a matter of fact neither the StopAsync or StartAsync functions are getting triggered.

@ericstj
Copy link
Member

ericstj commented Mar 16, 2023

@gkapellmann if StartAsync is not triggered then you are having a problem with even starting up your service. Are you able to share your example?

@mangod9
Copy link
Member

mangod9 commented Mar 16, 2023

@janvorli as FYI..

@gkapellmann
Copy link
Author

@ericstj Sure, no problem, I have created another repo with my very simple service example here.

It is based in .NET 7.0, when running you should be ableto see the events in the event manager, then shutting the PC down and then on again and I am not able to see the start async or stop async messages, but the main executeasyc events are there properly.

@ericstj
Copy link
Member

ericstj commented Mar 17, 2023

@gkapellmann I took a look at your repro. I was able to find out what was wrong by using Time-Travel debugging and examining the behavior at shutdown.

The PosixSignalRegistration workaround helped in that it did prevent the runtime from disabling exception handling, but the problem was there was still an unhandled exception in the logger:

(3/17/2023 9:24:59 AM) .ctor> Constructor...
(3/17/2023 9:24:59 AM) StartAsync> StartAsync...
(3/17/2023 9:24:59 AM) ExecuteAsync> Start of ExecuteAsync...
(3/17/2023 9:25:07 AM) StopAsync> StopAsync...
(3/17/2023 9:25:07 AM) StopAsync> System.AggregateException: An error occurred while writing to logger(s). (The interface is unknown.)
 ---> System.ComponentModel.Win32Exception (1717): The interface is unknown.
   at System.Diagnostics.EventLogInternal.InternalWriteEvent(UInt32 eventID, UInt16 category, EventLogEntryType type, String[] strings, Byte[] rawData, String currentMachineName)
   at System.Diagnostics.EventLogInternal.WriteEvent(EventInstance instance, Byte[] data, Object[] values)
   at System.Diagnostics.EventLog.WriteEvent(EventInstance instance, Byte[] data, Object[] values)
   at System.Diagnostics.EventLog.WriteEvent(EventInstance instance, Object[] values)
   at Microsoft.Extensions.Logging.EventLog.WindowsEventLog.WriteEntry(String message, EventLogEntryType type, Int32 eventID, Int16 category)
   at Microsoft.Extensions.Logging.EventLog.EventLogLogger.WriteMessage(String message, EventLogEntryType eventLogEntryType, Int32 eventId)
   at Microsoft.Extensions.Logging.EventLog.EventLogLogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.Logger.<Log>g__LoggerLog|13_0[TState](LogLevel logLevel, EventId eventId, ILogger logger, Exception exception, Func`3 formatter, List`1& exceptions, TState& state)
   --- End of inner exception stack trace ---
   at Microsoft.Extensions.Logging.Logger.ThrowLoggingError(List`1 exceptions)
   at Microsoft.Extensions.Logging.Logger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.Logger`1.Microsoft.Extensions.Logging.ILogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.LoggerExtensions.Log(ILogger logger, LogLevel logLevel, EventId eventId, Exception exception, String message, Object[] args)
   at Microsoft.Extensions.Logging.LoggerExtensions.Log(ILogger logger, LogLevel logLevel, String message, Object[] args)
   at Microsoft.Extensions.Logging.LoggerExtensions.LogInformation(ILogger logger, String message, Object[] args)
   at TestService.Worker.StopAsync(CancellationToken cancellationToken) in C:\scratch\SimpleWinService_dotnet\VeroTest-WinService\Worker.cs:line 45

This was preventing you from running your SaveCurrentLog method. I was able to workaround that by wrapping the call to _logger.LogInformation in your stopasync with a try/catch.

I'll take a deeper look at this to see why it's happening. Could be that this is normal during shutdown. Could be that a handle got disposed out from under the logger.

@ericstj
Copy link
Member

ericstj commented Mar 17, 2023

Looks like those unhandled exceptions can also occur during the ApplicationStopping and ApplicationStopped events where the logger is also called. There are also cases where the framework library will log:

Logger.LogInformation("Application is shutting down...");

I saw that the handle was still valid - so not a case of being disposed while still logging. The very first failure to write to the event log fails with RPC_S_SERVER_UNAVAILABLE at this stack:

00 0000003f`631bdd08 00007ffd`f7b9fe7c     ntdll!RtlRaiseException
01 0000003f`631bdd10 00007ffd`f8067c60     KERNELBASE!RaiseException+0x6c
02 0000003f`631bddf0 00007ffd`f8067c24     RPCRT4!RpcpRaiseException+0x34
03 0000003f`631bde20 00007ffd`f8067c06     RPCRT4!RpcRaiseException+0x14
04 0000003f`631bde50 00007ffd`f80f121a     RPCRT4!NdrSendReceive+0x66
05 0000003f`631bde80 00007ffd`f80f069b     RPCRT4!NdrpClientCall3+0x6aa
06 0000003f`631be1a0 00007ffd`f8d88cc0     RPCRT4!NdrClientCall3+0xeb
07 0000003f`631be530 00007ffd`f8d88991     ADVAPI32!ElfReportEventW+0x254
08 0000003f`631be6b0 00007ffd`47cfad54     ADVAPI32!ReportEventW+0x171
09 0000003f`631be760 00007ffd`47cfab9d     0x00007ffd`47cfad54
0a 0000003f`631be840 00007ffd`47cfa20b     System_Diagnostics_EventLog!Interop.Advapi32.ReportEvent+0xed [/_/src/libraries/System.Diagnostics.EventLog/src/Microsoft.Interop.LibraryImportGenerator/Microsoft.Interop.LibraryImportGenerator/LibraryImports.g.cs @ 2239] 
0b 0000003f`631be8f0 00007ffd`47d01aba     System_Diagnostics_EventLog!System.Diagnostics.EventLogInternal.InternalWriteEvent+0x2cb [/_/src/libraries/System.Diagnostics.EventLog/src/System/Diagnostics/EventLogInternal.cs @ 1383] 
0c 0000003f`631be9b0 00007ffd`47d01816     System_Diagnostics_EventLog!System.Diagnostics.EventLogInternal.WriteEvent+0x27a [/_/src/libraries/System.Diagnostics.EventLog/src/System/Diagnostics/EventLogInternal.cs @ 1341] 
0d 0000003f`631bea90 00007ffd`47d017bb     System_Diagnostics_EventLog!System.Diagnostics.EventLog.WriteEvent+0x36 [/_/src/libraries/System.Diagnostics.EventLog/src/System/Diagnostics/EventLog.cs @ 1035] 
0e 0000003f`631beac0 00007ffd`47d0127d     System_Diagnostics_EventLog!System.Diagnostics.EventLog.WriteEvent+0x2b [/_/src/libraries/System.Diagnostics.EventLog/src/System/Diagnostics/EventLog.cs @ 1030] 
0f 0000003f`631beaf0 00007ffd`47d00f10     Microsoft_Extensions_Logging_EventLog!Microsoft.Extensions.Logging.EventLog.WindowsEventLog.WriteEntry+0xcd [/_/src/libraries/Microsoft.Extensions.Logging.EventLog/src/WindowsEventLog.cs @ 35] 
10 0000003f`631beb90 00007ffd`47d008c8     Microsoft_Extensions_Logging_EventLog!Microsoft.Extensions.Logging.EventLog.EventLogLogger.WriteMessage+0xe0 [/_/src/libraries/Microsoft.Extensions.Logging.EventLog/src/EventLogLogger.cs @ 178] 
11 0000003f`631bed20 00007ffd`47cfc3ab     Microsoft_Extensions_Logging_EventLog!Microsoft.Extensions.Logging.EventLog.EventLogLogger.Log<Microsoft.Extensions.Logging.FormattedLogValues>+0x418 [/_/src/libraries/Microsoft.Extensions.Logging.EventLog/src/EventLogLogger.cs @ 120] 

So it looks to me like the machine is just shutting down the other services that would be needed to write to the event log. I tried adding service dependencies on the EventLog and RpcSs services to the test service, but it didn't seem to help.

@ericstj
Copy link
Member

ericstj commented Mar 17, 2023

I tried retargeting this service to net6.0 and noticed the exact same problem, so I'm not so sure writing to the eventlog during shutdown has ever worked correctly here.

@gkapellmann
Copy link
Author

It is interesting, I am going to look back in my repo history.

In the end the real priority is to have SaveCurrentLog working, the event logger is just for debugging, and I guess that what you say regarding the services makes sense. I did get the saved logs before, could it be that the order of the services shutting down is semi-random?

@ericstj
Copy link
Member

ericstj commented Mar 21, 2023

I did get the saved logs before, could it be that the order of the services shutting down is semi-random?

It could just be timing that's machine dependent. I'm reproducing on a clean VM without many other services so that might mean that all the other services shut down very fast. If you happen to have a repro that shows things working consistently, I can have a look and try to explain the difference if I can repro in my environment. Another thing you could do to see if you were getting lucky with timing is to take your working repro and add delay (for instance, in StopAsync before calling the logger) to see if it will fail.

Probably we should add documentation to the EventLog log provider that it's not safe to use during shutdown and we might want to go and add catch statements around calls to the logger that happen during shutdown. That feels like a pitfall in that logger cc @dotnet/area-extensions-logging

@gkapellmann
Copy link
Author

Hello @ericstj

I have been a bit busy but managed to do some tests. I guess I was getting lucky, I did a complete cleanup of the service and re-installed. Seems to behave different with every instalation.

But then, what I have tried is to do a loop that awaits for the stoppingToken from ExecuteAsync, then try to do a EventLog, followed by a "SaveCurrentLog" action. But I dont get a log file either here.

There are multiple issues here:
1.- For some reason when running as a service, the log txt file is not being written, but when running as a normal app (double click), the log txt file is written.
2.- In the image below and in the txt file log you can see the second problem. Execute Async doesnt finish totally by the time that StopAsync happens. Not allowing for the last section of EsecuteAsync to execute? I am a bit confused of what is the expected workflow of these events tbh.

image

In my head I see StartAsync first, followed by ExecuteAsync. When the stopping is requested, StopAsync is triggered, but that shouldnt prevent ExecuteAsync to finish completely, right?

I have done this updates in the repo, if you want to test this changes.

@ericstj
Copy link
Member

ericstj commented Apr 3, 2023

For some reason when running as a service, the log txt file is not being written, but when running as a normal app (double click), the log txt file is written.

Double check the directory. Services run with a working directory of c:\windows\system32.

When the stopping is requested, StopAsync is triggered, but that shouldnt prevent ExecuteAsync to finish completely, right?

ExecuteAsync is called by StartAsync which receives a cancellationToken that will be cancelled when the application is stopped.

using var combinedCancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, _applicationLifetime.ApplicationStopping);
CancellationToken combinedCancellationToken = combinedCancellationTokenSource.Token;
await _hostLifetime.WaitForStartAsync(combinedCancellationToken).ConfigureAwait(false);
combinedCancellationToken.ThrowIfCancellationRequested();
_hostedServices = Services.GetRequiredService<IEnumerable<IHostedService>>();
foreach (IHostedService hostedService in _hostedServices)
{
// Fire IHostedService.Start
await hostedService.StartAsync(combinedCancellationToken).ConfigureAwait(false);

There is no guarantee that StopAsync waits for any completion of ExecuteAsync. It looks to me like your ExecuteAsync method is just polling the cancellation token - so no cancellation registration is guaranteed to run -- and nothing else would wait on ExecuteAsync exiting gracefully.

@gkapellmann
Copy link
Author

Right, so I think that things have gone really clearer for me with this.

Part of the problem is that I was trying to use the backgroundservice in a way its not meant to be used.

So:

  • If ExecuteAsync has a loop, it not necessarily will be fully finished when the service is ending.
  • StopAsync is called at the end, but whatever happens in it might already be unloaded or unavailable.
  • StopAsync shouldn't have long tasks in it.

I have modified how I do the logging, and seems to be working fine now.

Thanks a lot @ericstj for your help through this.

If we all are ok, I am good to close this issue.

@ericstj
Copy link
Member

ericstj commented Apr 17, 2023

This issue did help us discover the shutdown problem with CoreCLR's CNTRL_SHUTDOWN_EVENT which I mentioned here: #83093 (comment).

We've seen other reports of this same problem, so I'd like to keep this issue open to track a fix for that in CoreCLR. @janvorli was looking into a fix for that.

@mangod9 mangod9 modified the milestones: 8.0.0, Future Aug 3, 2023
@ericstj ericstj changed the title BackgroundService StopAsync not triggering when Shutdown in .NET 7 CoreCLR's CNTRL_SHUTDOWN_EVENT handler prevents graceful exit of services during system shutdown Sep 8, 2023
@ericstj
Copy link
Member

ericstj commented Sep 8, 2023

@mangod9 this is a regression from a previous release and I don't think it should be moved to future. This bug prevents graceful service exit due to disabling exception handling and it should be addressed.

@mangod9
Copy link
Member

mangod9 commented Sep 8, 2023

believe there is a workaround in place to help with this issue? Will mark it for 9 to determine the feasibility of the fix.

@ericstj
Copy link
Member

ericstj commented Sep 8, 2023

The workaround is required in the customer's application, and they would need to identify this problem (exceptions not being caught/stop not completing on shutdown) and understand what was going on to know they needed the workaround - which is a pretty big stretch given how hard it is to debug shutdown issues. We would still like to see a fix for this.

@mangod9
Copy link
Member

mangod9 commented Sep 8, 2023

Ok, chatted with @janvorli on this to follow up if there is a good solution on the runtime side.

@jkotas
Copy link
Member

jkotas commented Apr 10, 2024

@janvorli @ericstj What do you expect the runtime fix to be?

If I am interpreting the comments above correctly, this is what's happening in general:

  1. The OS starts sending shutdown callbacks to services (e.g. CONTROL_SHUTDOWN).
  2. The service does not cleanup completely before returning from these callbacks. There is some ongoing async/background activity.
  3. This background activity can encounter unhandled exceptions if it attempts to call into other services that has been shutdown already (e.g. log events using EventLog)

To avoid unhandled exceptions in the last step, I think the service either needs to shutdown completely in the response to the shutdown callback and/or the service needs to start ignoring unhandled exceptions at some point during shutdown. #42275 may help with the later.

@janvorli
Copy link
Member

The primary issue is that a service should not call ExitProcess when receiving the CTRL_SHUTDOWN_EVENT, as described in https://learn.microsoft.com/en-us/windows/console/handlerroutine#remarks:

When a console application is run as a service, it receives a modified default console control handler. This modified handler does not call ExitProcess when processing the CTRL_LOGOFF_EVENT and CTRL_SHUTDOWN_EVENT signals. This allows the service to continue running after the user logs off. If the service installs its own console control handler, this handler is called before the default handler. If the installed handler calls ExitProcess when processing the CTRL_LOGOFF_EVENT signal, the service exits when the user logs off.

The problem got introduced when .NET started to register the control handler in #55333.
To fix that, we should change the behavior of our handler when the .NET app is running as a service. In fact, I've discussed that with Windows developers in the past and they said that the MSDN doc is slightly misleading and this is true for system session, not just services.

@janvorli
Copy link
Member

I haven't looked into the details of our implementation of the code related to the handler, maybe it would be sufficient to return FALSE from the HandlerRoutine if we detect we are running in the system session at this point:

case Interop.Kernel32.CTRL_SHUTDOWN_EVENT:
signal = PosixSignal.SIGTERM;
break;

@ericstj
Copy link
Member

ericstj commented Apr 10, 2024

@janvorli has it right

The problem is the process is getting two different shutdown signals. Services should only be shutdown through SCM - and that's how it works in .NETFramework and earlier .NET versions. It broke with #55333. The Windows default control handler identifies a service process and avoids sending it CTRL_SHUTDOWN_EVENT - the CoreCLR should do the same since it's behaving like the OS and registering a handler for all application types. To behave most like the OS here we should try to identify if the process is a service process - we can't count on it using System.ServiceProcess - some .NET processes opt to PInvoke directly to SCM. If possible we should replicate the OS's detection mechanism.

@jkotas
Copy link
Member

jkotas commented Apr 10, 2024

It broke with #55333

#50527 and #55333 introduced a low-level signal handling API. I do not think that the introduction of these APIs alone broke anything. The break was introduced as side-effect of ConsoleLifeTime using these APIs in #56057.

It does not make sense for this low-level signal handling API to be opinionated and handle app-model specific concerns, like skipping the handler registration for Windows Services. Windows Services are not the only ones with this type of issue. For example, the Win32 GUI apps should be only shutdown through WM_QUIT message.

I think that the problem is caused by ConsoleLifeTime being used by Windows Services. Windows Services are not console apps, so it does not make sense for them to use Console app model. (And similarly, Win32 GUI apps are not console apps and so they should not use ConsoleLifeTime. Dtto for all other appmodels - WinUI, MAUI, Avalonia, ... .)

@janvorli
Copy link
Member

It is true that the #55333 on its own without any SIGTERM registered handler doesn't cause the issue and the problem is that the HandlePosixSignal sets context.Cancel=true which was introduced by the #56057. So, it seems the fix should be rather in the HandlePosixSignal - to not to set context.Cancel for processes running under system session.

@janvorli
Copy link
Member

Or I guess just not using the ConsoleLifetime. That's what you were trying to say, @jkotas, right?

@jkotas
Copy link
Member

jkotas commented Apr 10, 2024

Or I guess just not using the ConsoleLifetime. That's what you were trying to say, @jkotas, right?

Right. ConsoleLifetime implements console appmodel that is not compatible with Windows services.

Note that we have SystemdLifetime that handles same concern for Systemd processes ("Unix services"):

// systemd only sends SIGTERM to the service process, so we only listen for that signal.
// Other signals (ex. SIGINT/SIGQUIT) will be handled by the default .NET runtime signal handler
// and won't cause a graceful shutdown of the systemd service.
_sigTermRegistration = PosixSignalRegistration.Create(PosixSignal.SIGTERM, HandlePosixSignal);
. I assume that ConsoleLifetime would not work well for Systemd processes for similar reasons.

@mangod9 mangod9 modified the milestones: 9.0.0, 10.0.0 Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants