running ps1 script create temp file in everytime

[nAnderYang]

Cento7 and powershell-6.2.3-1.rhel.7.x86_64

#!/bin/bin/pwsh

function xxx{
  echo xxx
}

xxx

create temp file in /tmp at running this script every time ,filename like CoreFxPipe_PSHost.[machine name].[num]+3.None.{scrptname}.ps1

[iSazonov]

@nAnderYang Can you check and confirm with latest 7.0 build?

[nAnderYang]

@iSazonov os: CentOS Linux release 7.7.1908 (Core)

[iSazonov]

@nAnderYang Sorry, I meant we need to repo with PowerShell 7.0 latest build.

[nAnderYang]

@iSazonov update to github powershell-preview-7.0.0_rc.2-1.rhel.7.x86_64.rpm ,no command of pwsh

[root@test tmp]# rpm -qa|grep powershell
powershell-preview-7.0.0_rc.2-1.rhel.7.x86_64
[root@test tmp]# rpm -qa|grep powershell
-bash: pwsh: command not found
[root@test tmp]# which pwsh
/usr/bin/which: no pwsh in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)

[nAnderYang]

i found command in /opt/microsoft/powershell/7-preview/pwsh . change and running the script,that temp files are still being created.

[iSazonov]

no command of pwsh

It is by design to allow Side-By-Side scenario with stable version (pwsh vs pwsh-preview).

@rjmholt Have you any thoughts why the temp file is created?

[SeeminglyScience]

Isn't that the named pipe used for IPC? e.g. for Enter-PSHostProcess?

[iSazonov]

Question is why it is used.

[SeeminglyScience]

Why what's used? Enter-PSHostProcess? If you just mean why is a file created, that's how named pipes work on unix likes.

$pipe = $null
try {
    $pipe = [IO.Pipes.NamedPipeServerStream]::new('mypipe')
    Get-ChildItem /tmp/CoreFxPipe*
    # Should show one with "mypipe" in the file name
} finally {
    ($pipe)?.Dispose()
}

The named pipe is created for PowerShell process to enable Enter-PSHostProcess.

[iSazonov]

@SeeminglyScience My question is about scenario of the issue. Enter-PSHostProcess is not mentioned and it is not clear is it used or not.

[SeeminglyScience]

It's not created on demand, it's part of start up. The named pipe is there whether you use it or not, otherwise there would be no way for Enter-PSHostProcess to signal the process it's trying to enter that it needs to create the pipe.

[iSazonov]

If the named pipe is created every pwsh startup this is answer to the issue question.

[nAnderYang]

i used the scheduled task service, part of which is to control the working baseline of the windows virtual machine under the host through the pwsh script on the host. but over time, i discovered this problem. there are massive files in the / tmp path. the problem is that the files are generated during startup. why not destroy them when exiting?

[iSazonov]

I'd expect the pipe is disposed. @SeeminglyScience Do you know there it is created?

[SeeminglyScience]

Yeah, it's created in the static ctor for RemoteSessionNamedPipeServer:

PowerShell/src/System.Management.Automation/engine/remoting/common/RemoteSessionNamedPipe.cs

Lines 553 to 566 in 70d4a89

	static RemoteSessionNamedPipeServer()
	{
	s_syncObject = new object();
	// All PowerShell instances will start with the named pipe
	// and listener created and running.
	IPCNamedPipeServerEnabled = true;
	CreateIPCNamedPipeServerSingleton();
	#if !CORECLR // There is only one AppDomain per application in CoreCLR, which would be the default
	CreateAppDomainUnloadHandler();
	#endif
	}

(Follow CreateIPCNamedPipeServerSingleton to see it create an instance of RemoteSessionNamedPIpe and save it to a static prop)

It's implemented using NamedPipeServerStream which has a finalizer so it should be doing the clean up on process exit. That's assuming non-Windows reliably runs finalizers, do you know if that's the case @iSazonov?

[iSazonov]

Perhaps it is the case https://stackoverflow.com/questions/12126598/how-and-when-are-c-sharp-static-members-disposed

Stack Overflow

How and when are c# Static members disposed?

I have a class with extensive static members, some of which keep references to managed and unmanaged objects.

For instance, the static constructor is called as soon as the Type is referenced, which

[SeeminglyScience]

Hmm. I know on Windows finalizers are supposed to run when the process is closing, but iirc there is a timeout for the whole finalizer queue. So either that just doesn't happen on non-Windows (or maybe on core at all) or something is blocking the queue.

Either way, even if it does typically work, unless they removed the timeout for processing finalizers; this needs to hook into some process exiting event.

@nAnderYang Does it create (and not remove) a file every time the schedule task runs? And can you provide an example directory listing with file names and sizes? Can you see if it reproduces when running an empty script?

@iSazonov I don't have easy access to a unix machine to test this with, if you do can you verify that this repros by just starting and exiting?

[nAnderYang]

@SeeminglyScience I have currently disabled this feature. However I created a test environment. This problem was reproduced.
Version powershell-6.2.3-1.rhel.7.x86_64 the situation is the same

[root@test tmp]# rpm -qa|grep powershell
powershell-preview-7.0.0_rc.2-1.rhel.7.x86_64
[root@test tmp]# rm -rf * /tmp
[root@test tmp]# echo '#!/opt/microsoft/powershell/7-preview/pwsh

function workthread{
  echo hello
}

workthread'>/tmp/test.ps1
[root@test tmp]# chmod 700 /tmp/test.ps1
[root@test tmp]# for i in $(seq 1 10); do /tmp/test.ps1; done
hello
hello
hello
hello
hello
hello
hello
hello
hello
hello
[root@test tmp]# ls -lh /tmp
total 4.0K
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A72.1591.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A72.1615.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A73.1635.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A73.1656.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A74.1676.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A74.1695.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A74.1715.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A75.1734.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A75.1754.None.test.ps1
srwxr-xr-x. 1 root root  0 Jan 19 09:47 CoreFxPipe_PSHost.D5CE6A76.1774.None.test.ps1
-rwx------. 1 root root 92 Jan 19 09:47 test.ps1
[root@test tmp]# 

[iSazonov]

It seems we need explicitly dispose the named pipe.

/cc @PaulHigin What do you think?

[KUTlime]

I can also confir this issue. I've just deleted about 40 000 files created by a job scheduled in cron.
Now, I have to implement a cron job to cleanup other cron jobs. 😣
I have this issue on v7.0.0 RC1-RTM, Debian Stretch arm7l with the arm32 package.

[nAnderYang]

@KUTlime the bug was not fixed in v7.0.0 release.

[KUTlime]

@nAnderYang Yes, I can see the status Open and I can see thousands of files in my /tmp folders.

I've just wanted to refresh this topic since the last entry was two months and four releases ago. Is there anybody who is actually doing something with this issue?

[nAnderYang]

i don't know if anyone is following this bug. I will not close the isuse without being fixed.

[PaulHigin]

Windows cleans up all resources when a process exits, including named pipes, but it looks like we can't rely on this for Linux systems. For Windows PowerShell we were using the CLR Appdomain unload event to dispose of the server pipe, but that is not implemented in dotNet core. We can use the ProcessExit event to do this for dotNet. That is not always reliable but i think it is our best bet.

Another idea is to make this named pipe listener opt-in. But this diminishes its usefulness somewhat for debugging.

[Zimeon-]

Noticed the same with Powerhell 7.1.4 and Centos7. Use powershell for monitoring and it generates a lot of them, fast.

[PaulHigin]

WG-Remoting
Re-opening issue since it appears the fix isn't working on all platforms. One thing we can consider is making the named pipe listener opt-in. That way most powershell instances will not create the named pipe files.

[iSazonov]

@PaulHigin Makes sense to open issue in .Net Runtime repository?

[yusufozturk]

This is still happening for us, for PowerShell 7.2.3 on Ubuntu 20.04.

Are we really having this issue due to telemetry or diagnostic reasons?

If it's really a telemetry issue, I would like to understand the logic behind making Telemetry and Diagnostic ENABLED by default. I wonder who really uses them. How many developers actually ever needed them? I always think "Telemetry" or "Diagnostic" is something you enable when you think something is wrong.

I understand that some reason you would like to keep telemetry ENABLED for everybody. But then you should be able to fix these issues earlier than 2 years. I disabled them with opt-out, but it's still causing lots of directories. I see that many people have the similar issue, so it means that something is wrong here.

Maybe nobody uses PowerShell + .NET on Linux production machines and nobody cares. But we do. We trusted Microsoft, we migrated to Linux due to Linux support and we have these issues for a long time already.

This might be not related with PowerShell, maybe a pure .NET issue. But we use PowerShell. So if there is a bug with .NET, I assume PowerShell team should open the issue.

I will give it a try with PowerShell 7.3.0-preview.5. I hope newest beta version works. Fingers crossed.

[PaulHigin]

@PowerShell/powershell-committee

We make a best effort to clean up listener named pipes on process exit, but it is not foolproof depending on how the process is exited. Windows cleans up all resources on process exit, including named pipes so we don't have the problem there. However, Linux apparently does not clean up these resources.

The listener allows one local PowerShell process to connect and debug another PowerShell process. This is very convenient but is not needed all of the time, and on Linux machines ends up leaving named pipe files if the process is exited abruptly.

I feel the best option is to make the pipeline listener opt-out through an optional command line switch (-NoListener), and maybe a cmdlet to stop the listener if PowerShell is being hosted.

/cc @SteveL-MSFT who first suggested making this opt-out.

[yusufozturk]

Actually we were having a similar problem on Windows as well. We converted our PowerShell services to Golang apps to avoid these issues, sadly. Here is the issue about that:

#13198

So Windows was also not cleaning the temp files. It was eventually making production server unstable and crashing the apps. Then customer calls us and says our agent crashes their production servers. Even one time, due to thousand of these files on hundreds of servers, we were accused to make huge IO and bring their virtualmachine storage down.

I strongly belive that these files should be created on request. We never need these files but they are keep generated and causes lots of troubles on the large customers. And simply we can not do a cleanup in their /tmp folder because we are afraid to remove a file which is necessary for Linux by mistake.

Thank you for your understanding.

[daxian-dbw]

and maybe a cmdlet to stop the listener if PowerShell is being hosted.

Would it be possible to have it configured by a property in InitialSessionState? So that the -NoListener flag for pwsh can be translated to setting that property, and when hosting PowerShell, one can set the property to not start the listener thread when opening the Runspace.

[PaulHigin]

The listener is per process and not per runspace, so I didn't think it made sense to include it in InitialSessionState. Disabling the listener should probably just be an API for the hosted scenario.

[iSazonov]

The listener allows one local PowerShell process to connect and debug another PowerShell process. This is very convenient but is not needed all of the time, and on Linux machines ends up leaving named pipe files if the process is exited abruptly.

If it is for debug it is not clear why this should be opt-out but not out-in.

[daxian-dbw]

The listener is per process and not per runspace, so I didn't think it made sense to include it in InitialSessionState.

How about naming the property in InitialSessionState to be CreateDebugListenerIfNotYet (or something like that)?
Then it's clear that it will create the listener only if it doesn't yet exist for the process.

[daxian-dbw]

If it is for debug it is not clear why this should be opt-out but not out-in.

@iSazonov That will be a breaking change.

[PaulHigin]

Again, I feel the listener state should not be part of InitialSessionState since it has nothing to do with runspace sessions. A public API is sufficient for opting out of the listener for hosted scenarios. I feel we want opt-out rather than opt-in since opt-in makes it less useful for debugging. It would be difficult or impossible to debug cases where starting the session to be debugged is not controllable.

[iSazonov]

If it is for debug it is not clear why this should be opt-out but not out-in.

@iSazonov That will be a breaking change.

Breaking what?
Debug scenario is rare but today mainstream scenario is broken - why cannot we accept the breaking change?

[daxian-dbw]

It's not broken on Windows as far as I know (files are cleaned up). A user may want to connect to a powershell process running in production and debug some issue, and making it opt-in will break that.

[iSazonov]

If you look at the comments above, not only on Unix, but also on Windows, this feature leads to destruction of the production process. As result I don't think we should think of this feature as code. This feature should be enabled on demand just as we do to enable tracing. And we need an easy and convenient way to do that at any given moment.

[daxian-dbw]

If you look into the issue mentioned in that comment, you will see that was about implicit remoting (remoteIPMoProxy_* files), not named piped. And it was fixed.

[iSazonov]

I'm sorry that I mixed these two problems up. But the problem with remoteIPMoProxy_* is not completely solved. One example.

ipmo dism
exit

It's just that this module (ScheduleTask) has become Core compatible like so many others.
We could say that both problems are of the same nature in terms of PowerShell.

[PaulHigin]

@PowerShell/powershell-committee

The PowerShell committee discussed this today, and regarding whether to make this named pipe connection feature opt-in or opt-out, we agree with @iSazonov that opt-in is the best solution. The feature is for advanced debugging and is rare enough that if it is needed, it is Ok to require a user to set up a special session that has named pipe connections turned on (opt-in) in order to debug an issue.

But before this feature is made opt-in, we feel some time should be spent looking into whether PowerShell could perform clean-up of the orphaned named pipe files on non-Windows machines (this issue is only for non-Windows as Windows OS doesn't use files for named pipes and cleans up process resources on exit). I am personally a bit skeptical about this because PowerShell would likely need root access to clean up named pipe files for other processes.

[iSazonov]

But before this feature is made opt-in, we feel some time should be spent looking into whether PowerShell could perform clean-up of the orphaned named pipe files on non-Windows machines (this issue is only for non-Windows as Windows OS doesn't use files for named pipes and cleans up process resources on exit). I am personally a bit skeptical about this because PowerShell would likely need root access to clean up named pipe files for other processes.

It seems it makes sense to open discussion in .Net Runtime repository - the issue is not PowerShell specific and comes from .Net workaround for named pipes on Unix.

[PaulHigin]

I investigated this some on an Ubuntu system, and PowerShell is not the only thing that creates and leaks named pipe files, when the process is abruptly terminated. The CLR creates three files in the users /tmp directory along with the PowerShell listener named pipe file, which are also leaked after abrupt termination. So disabling the PowerShell listener will not completely solve the problem.

But I believe the dotNET CLR allows you to disable its named pipe files via an environment variable. We can do something similar with PowerShell to allow users to opt-out of creating the listener (and named pipe file) on PowerShell start up. This can be done via command parameter and with policy/configuration setting.

[nighlabs]

Wanted to update the thread on my experience.

Following guidance gleaned from dotnet/runtime#7204 for disabling the debugger FIFO pipes and the resulting https://learn.microsoft.com/en-us/dotnet/core/runtime-config/debugging-profiling page, I set two environment variables: COMPlus_EnableDiagnostics and DOTNET_EnableDiagnostics. Result was none of the three files were created after executing Powershell scripts in an alpine linux container.

Environment variables I set:
COMPlus_EnableDiagnostics='0'
DOTNET_EnableDiagnostics='0'

Debugging profiling config settings - .NET

Learn about run-time settings that configure debugging and profiling for .NET Core apps.

[nAnderYang]

update Thu, 23 Mar 2023 16:19:20 +0800

OS RHEL8.7

$ rpm -qa|grep pow
powershell-7.3.3-1.rh.x86_64

ENV

$ env |grep Diag
COMPlus_EnableDiagnostics=0
DOTNET_EnableDiagnostics=0

normal test

pwsh
PowerShell 7.3.3
PS /tmp> tree -a
.
├── CoreFxPipe_PSHost.D95D5EAF.7285.None.pwsh
├── .dotnet
    └── shm
        └── session3471
            └── PSReadLineHistoryFile_3120419687

3 directories, 2 files
PS /tmp> exit
$ tree -a
.

0 directories, 0 files

unusual exit

$ pwsh
PowerShell 7.3.3
PS /tmp> tree -a
.
├── CoreFxPipe_PSHost.D95D5F61.7430.None.pwsh
└── .dotnet
    └── shm
        └── session3471
            └── PSReadLineHistoryFile_3120419687

3 directories, 2 files
PS /tmp> Killed
$ tree -a
.
├── CoreFxPipe_PSHost.D95D5F61.7430.None.pwsh
└── .dotnet
    └── shm
        └── session3471
            └── PSReadLineHistoryFile_3120419687

3 directories, 2 files

So ... the test results are the same as before.
I recommend using crond to clean up the temp files.