PSReadLine cannot handle super large history file properly

[ajayguptarrk]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues, especially the pinned issues.

Exception report

System out of memory  and it consume lots of memory when i see in task manager.

Screenshot

Environment data

PS Version: 5.1.14393.5582
PS HostName: ConsoleHost
PSReadLine Version: 1.2
PSReadLine EditMode: Windows
OS: 10.0.14393.0 (rs1_release.160715-1616)
BufferWidth: 139
BufferHeight: 3000

Steps to reproduce

Start powershell in visual code or windows it always crashed

Expected behavior

always crashed

Actual behavior

Start powershell in visual code or windows it always crashed

[StevenBucher98]

It looks like you are on a pretty old version of PSReadLine, can you try upgrading to the latest stable version (2.2.6) and see if you still come across the issue? You can upgrade by running the following:
Install-Module -Name PSReadLine -RequiredVersion 2.2.6
Be sure to restart PowerShell session for changes to take affect

[ajayguptarrk]

Hi, I have updated the same but now again the same exception has come. [image: image.png] *Thanks and RegardsAjay Kumar Gupta *

…

On Mon, 7 Aug 2023 at 21:45, Steven Bucher ***@***.***> wrote: It looks like you are on a pretty old version of PSReadLine, can you try upgrading to the latest stable version (2.2.6) and see if you still come across the issue? You can upgrade by running the following: Install-Module -Name PSReadLine -RequiredVersion 2.2.6 Be sure to restart PowerShell session for changes to take affect — Reply to this email directly, view it on GitHub <#3771 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFMRQWRPGKRBMB3G2EBQTC3XUEIAJANCNFSM6AAAAAA3G46CL4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[StevenBucher98]

@ajayguptarrk I cant seem to see the image you attached to the issue, can you please try to reattach it again? Maybe via GitHub instead of email.

[ajayguptarrk]

Hi, I have updated the same but now again the same exception has come. [image: image.png] Thanks and RegardsAjay Kumar Gupta *
…
On Mon, 7 Aug 2023 at 21:45, Steven Bucher @.> wrote: It looks like you are on a pretty old version of PSReadLine, can you try upgrading to the latest stable version (2.2.6) and see if you still come across the issue? You can upgrade by running the following: Install-Module -Name PSReadLine -RequiredVersion 2.2.6 Be sure to restart PowerShell session for changes to take affect — Reply to this email directly, view it on GitHub <#3771 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMRQWRPGKRBMB3G2EBQTC3XUEIAJANCNFSM6AAAAAA3G46CL4 . You are receiving this because you authored the thread.Message ID: @.*>

[ajayguptarrk]

Please see if you are able to see updated image

[ajayguptarrk]

Sure .please find attached image.

[daxian-dbw]

@ajayguptarrk I still cannot see the updated image. The image above still shows it's a very old version -- 2.0.0-beta2 or prior indicated by the reporting URL:

[StevenBucher98]

Unfortunately it does not seem like we can see the images still @ajayguptarrk

[ajayguptarrk]

Sorry there was some issue in uploading image. Please look updated screen shot and let me know if i can do anything else.

[daxian-dbw]

@ajayguptarrk The size of your history file is too big -- it exceeds the size limit of a generic list in .NET (~2 GB).
We should fix PSReadLine to work with super large history file, but for now, you can easily work around it by deleting some of the early history text from the history file. By default, the history file is saved at c:\Users\<user-name>\AppData\Roaming\Microsoft\Windows\PowerShell\PSReadLine\ConsoleHost_history.txt on Windows.

[kilasuit]

@daxian-dbw this would be another good reason for this feature request as that would move it outside of .NET limitations

[daxian-dbw]

Even after moving to using sqlite for history, the file-based history saving will need to be preserved indefinitely to make it backward compatible. I'm planning to update the code to only read the last 1mb content from the history file, so this issue can be avoided.