Provide a platform-agnostic way to determine the temp. directory, e.g., via an automatic variable.

[mklement0]

^{Related: #3442 and #4215}

The various supported platforms have different native ways to determine the directory in which to store temporary files and directories:

• Windows:
  • $env:TEMP
• macOS:
  • $env:TMPDIR
• Linux:
  • /tmp

Currently, there is no platform-agnostic way to refer to this directory, which is cumbersome.

An automatic variable, say $TEMP, could provide this abstraction.

Environment data

PowerShell Core v6.0.0-beta.3

[lukeb1961]

what about something like: (New-TemporaryFile).DirectoryName

[mklement0]

@lukeb1961: That indeed yields the platform-specific temporary directory, but it has an unwanted side effect: New-TemporaryFile invariably creates the file whose [System.IO.FileInfo] representation it outputs.

In other words: every time you execute (New-TemporaryFile).DirectoryName, you leave an unused temporary file behind.

Additionally, it's a computationally expensive approach.

[iSazonov]

We've had discussed this in a nutshell. Thank you for opening the Issue.

Internally we already have "TemporaryDirectory", just "GetTemporaryDirectory()" and "CreateTemporaryDirectory()". We could make it public.

[Liturgist]

How about going with $Env:TMP on all platforms? That is not quite like Windows, Mac, or Linux. No turf war.

[mklement0]

@Liturgist: $Env:TMP is not an option, because it is ill-advised to either (a) have PowerShell automatically define environment variables or (b) pretend that non-existent environment variables exist.

An automatic PowerShell variable such as $TMP is an option, however, although the thing to keep in mind is that retroactively introducing automatic variables is technically a breaking change.

[SteveL-MSFT]

If we went the variable route, it would have to be something like $interop:TEMP to avoid name collision. A variable is probably easier to use in scripts than an API although we can certainly support both.

In #3571 I suggested other variables in addition to TEMP

[mklement0]

@SteveL-MSFT:

Good point; an $interop: namespace is indeed a way to avoid collisions.

If we take a step back: If we want PowerShell to become the lingua franca of the shell world, do we really want to frame things as "interop"?

How about trusting PowerShell's automatic variables to Do the Right Thing(TM) generically, which would call for a namespace such as $ps:?

[SteveL-MSFT]

@mklement0 the $ps: prefix/namespace has been proposed before. It would be nice if we had used that from the beginning :)

Personally, I prefer $ps: for exactly your reasoning. Perhaps this can be a discussion point at the next Community Call.

cc @PowerShell/powershell-committee

[mklement0]

I'm glad to hear it, @SteveL-MSFT.

On a related note, I think a separate namespace for all preference variables would be beneficial as well, such as $pspref:

The caveat is that even though collisions are far less likely than with unqualified variable names, someone could have done something like the following:

# Define custom drive 'ps:'
New-PSDrive 'ps' FileSystem '/tmp'; Get-ChildItem ps:

[iSazonov]

Why we cannot use [system.io.path]::GetTempPath()?

https://github.com/dotnet/coreclr/blob/6ba74dc2a7194f8d6c86c3aeab572a074ef645c8/src/mscorlib/shared/System/IO/Path.Unix.cs#L153
https://github.com/dotnet/coreclr/blob/6ba74dc2a7194f8d6c86c3aeab572a074ef645c8/src/mscorlib/shared/System/IO/Path.Windows.cs#L94

[mklement0]

@iSazonov:

That works, but it is both hard to remember and cumbersome to type.
Good to know as a workaround, though.

[iSazonov]

If we have New-TemporaryFile users expect Get-TemporaryPath of Get-TemporaryDirectory.

[mklement0]

@iSazonov:

Since the term path is ambiguous (even though it is used in the .NET API to refer to a directory path in this case), my preference would be Get-TemporaryDirectory, and I certainly wouldn't mind having such a cmdlet.

That said, a cmdlet is a bit heavy-handed for something that returns a single piece of information with no variations in functionality.

Consider the existing Get-Host / $Host duo: Do you ever find yourself using Get-Host instead of $Host?

[SteveL-MSFT]

$ps:Temp would presumably return a static location to put temp stuff. Get-TemporaryDirectory would presumably create a new folder under $ps:temp. Personally, I think having a temp: PS drive may be more useful. Perhaps cleaned up automatically on process exit.

[iSazonov]

temp: - good idea. If we accept it we should have cmdlets for it. And I believe we can enhance *-PSDrive cmdlets.

temp: - initialized to [io.path]::GetTempPath().

We would make temp: optionally per runspaces.

[dragonwolf83]

I would prefer a consistent way of mapping environment variables that are basically the same just expressed differently. Are we saying we will map all path variables to drives, like Home:?

[Liturgist]

@mklement0 - I am not quite sure why Env:TMP is a bad idea. With regard to a) PowerShell creates Env:PSModulePath. With regard to b) I do not know which variables PowerShell would be pretending about. I would like to understand.

[dragonwolf83]

@Liturgist I think the notion is that anything in the $Env: namespace should only be actual environment variables on that platform. It also would be very confusing to have $Env:TEMP and $Env:TMP be defined at the same time. How do I know which one is the one to use to run on Linux, OSX, and Windows just from looking at it?

The proposals for a separate namespace or drive eliminate that issue as it allows for PowerShell to have it's own commonly defined variables that can be used cross-platform and not block access to the real environment variables.

Another idea though for a namespace could be $PSEnv:. It has an implied meaning that anything in that namespace comes from PowerShell and not the system. Then you can cleanly map all the common environment variables and makes it easier to see what is available to use from IntelliSense.

[Liturgist]

@dragonwolf83 - Would Env:PSModulePath be moved to PSEnv:PSModulePath? I fully agree that that there should not be multiple variables such as Env:TEMP and Env:TMP. There should be one good one and let's go with that. Having PSEnv:TEMP or PSEnv:TMP would be fine.

To go with New-TemporaryFile it would be helpful to have New-TemporaryDirectory. To go with this, having a -Path parameter on New-TemporaryFile to specify the directory in which to create the temporary file would be helpful.

[dragonwolf83]

No, I don't think the PowerShell team would move $Env:PSModulePath for three reasons.

1. It would be a breaking change that doesn't add any value
2. It is a real Environment Variable so it would make no sense to move it.
3. There is no need to force a mapping of this environment variable to work cross-platform. It is something PowerShell can add to Environment Variables when installing PowerShell because they own this variable.

In other words, it exists exactly where it belongs.

Take a look at the screenshot below. It shows the Environment Variables defined on my Windows system. It has TEMP and TMP as environment variables for my user Temp path. It also has PSModulePath. So all of those other variables, I would expect to see in $Env: namespace. You can go into cmd and see that all of those variables, including PSModulePath exist there too.

[mklement0]

@Liturgist:

To add to @dragonwolf83's helpful comments and to respond to your comment addressed to me:

I do not know which variables PowerShell would be pretending about. I would like to understand.

My perhaps glaringly-obvious-didn't-need-to-be-said point was that in the absence of PowerShell actually defining such an environment variable (which, as stated, would be ill-advised - also see below), PowerShell shouldn't pretend that it exists.

---

PowerShell respects / defines only two environment variables, according to Get-Help about_Environment_Variables:

• $env:PSExecutionPolicyPreference, when set externally, before invoking powershell, can be used to set the execution policy for the session (currently: Windows PowerShell only).

  • If not set externally, this variable won't be defined in the PowerShell session, and the default / persistently configured value applies.

• $env:PSModulePath, when set externally, before invoking powershell, can be used to define the directories for PowerShell's module search path (the list of directories in which PowerShell looks for modules loaded by name only).

  • If not set externally, PowerShell defines it with a list of standard directories (current-user, all-users, but not system).

    • On my W10 / PSv5.1 system, this env. variable is predefined, as a system-level env. variable, with the system modules directory, $PSHOME/Modules, with the remaining standard directories getting defined at PowerShell startup, prepended to the system-modules dir.

      • It seems that defining a non-standard directory in the system-level variable causes it to be injected after the current-user value.

    • PowerShell Core installations on Unix platforms have no predefined variable.

  • If set externally at the process level (with set PSModulePath=... & powershell ... from cmd.exe or PSModulePath=... powershell ... from POSIX-like shells):

    • Windows PowerShell v5.1 uses just the external value as-is.
    • PowerShell Core injects the external value into the list of standard directories, after the current-user value.

These variables are environment variables for a good reason: they allow you to control PowerShell's startup behavior externally.

By contrast, there is NO good reason to define the directory for temporary files and directories as an environment variable:

• The value is static and can be derived (platform-specifically) from the existing, standardized environment ($env:TMP on Windows, $env:TMPDIR on macOS) / platform conventions (/tmp on Linux).

• There is no good reason to expose this value to child processes via the environment, because (a) if these child processes are not PowerShell processes, they won't know about what environment variables PowerShell defines and (b) any PowerShell session can derive the value independently.

• Conversely, any environment variable that PowerShell does define bears the risk of a name collision with a variable of the same name used differently by other environments. While this problem can be mitigated with a by-convention name prefix - such as PS - the better approach is to avoid defining environment variables altogether, if they're not needed.

^{If a reader is familiar primarily with cmd.exe, it is worth pointing out that cmd.exe -
unlike PowerShell and POSIX-like shells on Unix - knows only environment variables: any variable you define in cmd.exe is implicitly also an environment variable, whereas PowerShell and POSIX-like shells distinguish between session-specific-only shell variables (e.g., $foo in PowerShell) and explicitly-designated-as-such environment variables (e.g., $env:foo in PowerShell).}

---

I definitely like the idea of multiple namespaces for PowerShell's automatic variables (such as the suggested $PSPref: for preference variables), so putting the directory for temporary files in $PSEnv: is a good idea, though we'd have to make clear that - despite the presence of the word 'Env' - these aren't actual environment variables in the established, system sense (accessible to all child processes, irrespective of what executable is being run).

[SteveL-MSFT]

Seems like there's been enough good discussion that perhaps this should be a RFC?

[iSazonov]

I definitely like the idea of multiple namespaces for PowerShell's automatic variables

In [io.path]::GetTempPath() the [io.path] is a namespace - should we introduce a Powershell "native" namespaces like $PSEnv:?

[mklement0]

@SteveL-MSFT:

Before we tackle an RFC, let me try to summarize what I perceive to be the use cases and perhaps get a better understanding of the scope of such an RFC:

• Create a temporary file (in the platform-appropriate location, with a unique, randomly generated name):

  • New-TemporaryFile already provides that.

  • Create a temporary file with a specific filename extension - see Add ability to create a temporary file with a specific extension to New-TemporaryFile #4215

• Create a temporary directory (in the platform-appropriate location, with a unique, randomly generated name):

  • Get path to a temporary directory #3442 requests this ability, proposing a -Directory switch be added to New-TemporaryFile

• Determine the path of the platform-appropriate (root) directory for temporary files and directories.

  • The primary subject of this issue, with the introduction of new, PowerShell-controlled namespace(s) being considered.

• Having the ability to implicitly create a command-scoped temporary and self-deleting file, so as to present the output of a command as a file when calling external utilities (similar to a Bash process substitution) - see Suggestion: support Bash-style process substitutions #4284

• Possibly provide an implicitly defined temp: drive.

  • I think this would only be worthwhile if that drive were script/function-scoped as opposed to session-scoped; i.e., if a script/function could write to this drive without fear of name collisions, similar to the auto-cleaning TestDrive: drive that Pester provides.

  • That said, I fear that doing this would be too expensive and that the scoping is not as clear-cut as in Pester's case.

As for the introduction of PowerShell-controlled namespaces:

Note that the current namespace notation - e.g., $env:PATH is actually based on having an underlying PS drive: all currently supported $<prefix>: "prefixes" refer to drives of the same name; e.g., $env:HOME is the same as (Get-Item Env:/Home).Value.

If we now introduce namespaces such as $PSEnv: without an underlying drive - and an underlying drive would arguably overkill - we'll be departing from that model.
I personally don't think it's a problem, but it should be a conscious, documented decision.

---

So what should an RFC cover?

• Introduction of PowerShell-controlled namespaces?

  • Just for PS-scoped "environment" variables, or, as mentioned, perhaps for preference variables too?

• An implicitly defined temp: drive?

I could tackle the former, but I'm not sure I can come up with a sensible approach to the latter (and it may not be worth doing).

[SteveL-MSFT]

@mklement0 thanks for summarizing all the different related discussions. I think just scoping the Motivation section of the RFC to the first item is sufficient. You can explicitly state that temp: is out of scope for the RFC.

[Liturgist]

@mklement0 - Very nice and helpful summary.

I would much rather have New-TemporaryDirectory over adding a -Directory switch to New-TemporaryFile. New-TemporaryDirectory is more obvious and discoverable. If both are not possible, I vote for New-TemporaryDirectory.

[mklement0]

Thank you, @Liturgist.

I hear you re New-TemporaryDirectory (though there's the proud and confusing Unix tradition of calling any filesystem item a "file"....).

On the opposite end of the spectrum, you could argue (as someone already has, but I forget where) that there should be just one generic New-TemporaryItem cmdlet with -Type File and -Type Directory parameters (conceivably, each drive provider could implement their own temporary locations, though among the ones that ship with PowerShell, it really only makes sense for the FileSystem provider).

Either way, I suggest you make your voice heard at #3442.

---

@SteveL-MSFT: Thank you for the guidance, but despite my initial willingness to tackle this RFC, I'd like to bow out of this [self-]assignment.
I do hope someone else will take it on.

[essentialexch]

Requiring an admin-scripter to use [IO.Path]::GetTempPath() is obscene. It's not discoverable, it's not obvious, and it's not powershell.

I personally believe that promoting $env:tmp is the right answer, but someone could argue it's also not powershell. After some personal consideration, I think a new cmdlet Get-TemporaryDirectory (or a similar name) is the proper answer.

I don't understand "New-TempDirectory". The normal use-case is not creating a directory (which New-* indicates) - it's retrieving the name of an existing directory (which Get-* indicates).

I'm "meh" about a temp: drive because, for it to be discoverable, it also requires one or more cmdlets. Instead of offering more "freedom" to the scripter, a temp: drive feels constraining.

[iSazonov]

Another look to be completely PowerShell native. Can we avoid the explicit use of temporary directories?

# Create the file in temporary directory
$temp= New-Item -Path xyz.txt -Temporary

# Get the file and copy it to temporary directory
$var2 = Get-Item -Path sample.txt; $tempVar2 = Copy-Item $var2 -Temporary

# The temporary directory is automatically created and dropped for the scope 

[DarwinJS]

I feel it should simply abstract away the existing platform with the longest standing powershell platform's references being preferred so that the long standing code can be more easily reused.

Like this:

If (!(test-path env:temp)) 
{
 If (test-path env:tmpdir) {$env:TEMP = $env:tmpdir}
 elseif (test-path '/var/tmp') {$env:TEMP = '/var/tmp'
}

[Liturgist]

Looking back at the subject of this thread, some may want to get what might be considered the traditional "tmp" directory. This could return [System.IO.Path]::GetTempPath(). This would obviate the need for a new namespace or provider (ps:, PSEnv:, etc.).

Get-TempDirectory

If a developer wants to use $Env:TEMP, or $Env:TMPDIR, or /tmp then they are free to do that. They would be making their code platform dependent.

Applications need mechanism to create unique directories and files in which to store things. What if we extended New-Item to be able to create one or more temporary directories and files. It would return a list of the item names. The item names returned will already be created and ready for use.

New-Item -TempDirectory -NamePrefix [string[]]
New-Item -TempFile -NamePrefix [string[]]

[mklement0]

Just to summarize the discussion so far:

We're talking about two complementary use cases here, although one is far more important:

• (a) Most importantly - but note that this is not what this issue was originally about - we need the ability to directly create temporary files and directories with paths that are guaranteed to be unique - without having to worry about the specifics - if a cmdlet creates the file/dir. for you, you needn't worry about where it is stored and what the platform's root dir. for temporary files is.

  • New-TemporaryFile already does that for files.

    • Add ability to create a temporary file with a specific extension to New-TemporaryFile #4215 suggested enhancing it with the ability to specify the filename extension of the file path returned.
    • @Liturgist, can you say more about when -NamePrefix would be useful?

  • Get path to a temporary directory #3442 is about supporting the creation of temp. directories too.

  • Supporting both files and dirs. via New-Item is an option (something you've previously suggested in Get path to a temporary directory #3442 too, @Liturgist, with the option of specifying a target dir. for the temp. item), but given that we have New-TemporaryFile already, perhaps it's easier to simply implement New-TemporaryDirectory for symmetry - alternatively, both could be proxy functions (similar to mkdir on Windows) that defer to an enhanced New-Item.

  • While a temp: drive sounds tempting (no pun intended), I don't think it is feasible, because for that to be useful each scope would have to have its own drive definition based on its own, unique directory in order to be able to freely create items there without risk of name collisions, and managing that sounds like way too much overhead.

  • By contrast, automatic cleanup of explicitly created temporary items on exiting the scope sounds more realistic, along the lines of @iSazonov's suggestion, but there too we should consider the overall performance impact (additional checks on leaving any scope).

• (b) Occasionally - and that is the original subject of this thread - it may be useful to explicitly know a given platform's root directory for creating temporary files and dirs. - e.g., to write a utility that cleans up temporary items on demand, or if you need to check if a given path refers to a temporary item.

  • As discussed in detail before, emulating or actively defining an environment variable on platforms where they have no predefined meaning is ill-advised, so, as familiar as $env:TEMP is, it is not a proper cross-platform solution.

  • Surfacing this root directory via a Get-Temp[orary]Directory cmdlet, as @Liturgist suggests, is definitely an option.

    • That would also allow supporting the distinction between the user and system temporary folders (Windows and macOS), by way of parameters to request one or the other (default should be the user's dir.)
    • Note that [io.path]::GetTempPath() only supports retrieving the user's temp. dir., so for the system one [environment]::GetEnvironmentVariable('TEMP', 'Machine') would have to be used on Windows, and fixed path /tmp on macOS. (Linux doesn't have this distinction, though Unix-like systems do have other types of temporary folders (see https://superuser.com/a/332616/139307), though we probably don't have to worry about supporting them).

  • Another (not necessarily mutually exclusive) option is to surface this directory path via the $sf namespace for special folders proposed in Add support for special folders (known folders) via a separate provider / namespace #6966, along the lines of $sf:Temporary

    • Note that this would build on the way that CoreFx has chosen to surface platform-abstracted locations (although it was originally a Windows-only feature): [System.Environment]::GetFolderPath() - it would be an extension, however, given that the temporary folder(s) are currently not covered there.

[iSazonov]

@mklement0 Thanks for the great summarization!

I think we could continue considering scenarios in which all these options could be useful (like "get a file from web/network, put in temp, process the file (unpack/decoding...), remove the temp"). I think we can do more wonderful things than just return temp directory.

[SteveL-MSFT]

I hit this recently while playing with Azure Functions. I'm so used to having TestDrive: in Pester that I think it would be fine to have an automounted Temp: drive. Of course, unlike TestDrive: PowerShell would not auto-clean up. Thoughts? Let me work on it as an Experimental Feature... #8696

[iSazonov]

Based on above discussion the simple mapping to Temp: drive looks not so useful and not resolve common scenarios. (Users can implement this with one command in profile. In Azure it is possible simply cd to temp.)
I'd prefer that PowerShell Core exposes more magic things for users.

[vexx32]

At that level it's not especially useful, @iSazonov, but the change is helpful to future module authors, who could simply refer to Temp: drive instead of having to code three different ways for it to cover all possible platforms.

[DarwinJS]

@SteveL-MSFT - maybe this issue should be renamed and re-scoped to "implementation-agnostic" to cover things like cloud shells without implying they are a platform in the same way an OS is.

The name change helps reflect the broader value proposition when bringing serverless / non-OS implementions into scope.

[Liturgist]

Thank you for curating the issues, Michael Klement.

It seems that the tradition of a tmp/temp directory is that every program can us it in whatever way it wants. That works until name collision or there are multiple instances of the same program. In the first case, the applications must choose different names. In the second, the application may identify its files by including unique information such as its PID in the filename.

(a) I fully agree that with guaranteed uniqueness, the application should not care about where a temporary file or directory is located or the name.

A constraint should be that it is possible to locate post-mortem the temporary files and directories to enable anomaly remediation (finding and fixing bugs). Does this suggest that automatic temp cleanup should be a preference setting?

The suggestion of -NamePrefix was based on an application being able to identify its own temprrary files in a single pool. Although guaranteed uniqueness obviates this as a requirement, the developer may find comfort in recognizing foo_xxxx.tmp, bar_xxxx.tmp, bosh_xxxx.tmp, and others when mutiple temporary files are in use.

I agree that if we have New-TemporaryFile that we should have New-TemporaryDirectory.

[SteveL-MSFT]

Certainly TEMP: doesn't solve the unique temp folder that New-TemporaryDirectory would solve and the two are orthogonal. Uniqueness is certainly useful particularly if you have multiple processes or even runspaces, but that deviates from what this original issue intended.

[Liturgist]

What was the result of this discussion?

[SteveL-MSFT]

New-TemporaryDirectory is #3442

A Temp: drive resolves this issue.

[Liturgist]

@SteveL-MSFT - Ok. I do not see a Temp: drive from Get-PSDrive in 6.2.1. Am I looking in the wrong place, or when might the Temp: drive become available? #3442 appears to show Waiting - DotNetCore.

Will the Temp: drive be global for the system in the same way a /tmp on *NIX systems is currently?

Will there be a New-TemporaryDirectory to be orthogonal with New-TemporaryFile?

[iSazonov]

@Liturgist It was an experimental feature. Now it is in 7.0. You can download daily build and find it there. It is "global" in PowerShell session.
No immediate plans exist to implement New-TemporaryDirectory - you can discuss in #3442.

[SteveL-MSFT]

@Liturgist if you want to try it on 6.2.1, just do this:

Enable-ExperimentalFeature PSTempDrive

Then restart pwsh.

[ghost]

🎉This issue was addressed in #9872, which has now been successfully released as v7.0.0-preview.2.:tada:

Handy links:

• Release Notes

[LYP951018]

I'm trying TEMP:\ but I found that I could not use TEMP:\ directly in C# methods or external tools, e.g.

[System.IO.Compression.ZipFile]::ExtractToDirectory($path, "TEMP:\foo")

or

unzip foo -d TEMP:\foo

I must use Get-Item TEMP:\ or something to achieve this...

[mklement0]

@LYP951018:

TEMP: is a PowerShell-only drive (you can list them with Get-PSDrive) and such drives aren't visible to the outside world, including .NET

You must translate PS-drive-based paths to their underlying "native" filesystem paths, which is what Convert-Path is for.

Note that there's another pitfall associated with passing filesystem paths to .NET methods (though not to external programs): .NET typically sees a different working directory than PowerShell, so that passing relative paths typically doesn't work as intended - it is best to pass full paths always, which Convert-Path can also help with.

In short: it's good to form a habit of passing filesystem paths to .NET methods via
Convert-Path -LiteralPath.

If you stay in the realm of PowerShell, you won't face that problem; for instance, you could use Expand-Archive.