[BUG] Restore-PnPRecycleBinItem -RowLimit parameter appears to be broken

[NoahL989]

Hello,

It appears the issue #1598 is not fixed and this is a request to reopen it.

Running into the same issue as #1598 where the -RowLimit parameter appears to not work for Restore-PnPRecycleBinItem.

Even when running the command without the filtering it errors.

Reporting an Issue or Missing Feature

It is an Issue.

Expected behavior

For the cmdlet to work as intended.

Actual behavior

Getting "The attempted operation is prohibited because it exceeds the list view threshold" error when trying to restore something and the -RowLimit parameter does nothing to fix the error.

Steps to reproduce behavior

Connect-PnPOnline -Url https://example.sharepoint.com/sites/sharepointsite -UseWebLogin
Get-PnPRecycleBinItem -RowLimit 15000 | ? -Property DeletedByName -Like "Firstname Lastname" | Restore-PnPRecycleBinItem -Force -RowLimit 15000

OR

Connect-PnPOnline -Url https://example.sharepoint.com/sites/sharepointsite -UseWebLogin
Restore-PnpRecycleBinItem -Identity 22c4d077-2a3d-446e-bf30-e9ece79820a1 -Force -RowLimit 1

What is the version of the Cmdlet module you are running?

(you can retrieve this by executing Get-Module -Name "PnP.PowerShell" -ListAvailable)

Running version 1.11.109

Which operating system/environment are you running PnP PowerShell on?

• Windows
• Linux
• MacOS
• Azure Cloud Shell
• Azure Functions
• Other : please specify

[koskila]

This is fair feedback - I implemented the fix to Get-PnPRecycleBinItem (in #2472), where it should now work, and tested by piping (which should work but will be incredibly slow).

The current implementation of Restore-PnPRecycleBinItem only works with an Identity of some sort, it ignores RowLimit and will never reach the part of code where it would try restoring everything.

Let me take a look at how to improve this...

[koskila]

@gautamdsheth does the simple fix I committed yesterday get rolled up in a nightly build? If it does, @NoahL989 could install the latest nightly build and test if the issue is solved.

[gautamdsheth]

@koskila / @NoahL989 - yes is available in the latest nightly build.

[SamuraiKal]

I'm on nightly build 1.11.128 and I'm running into this issue with a recycle bin that has over 600k items in it, and I have no idea where to begin with fixing it.

The code I've most recently attempted is below.

Even though I have the RowLimit on the Get configured for 150k, it returns all 600k+ items I'm looking for. I tried passing all the items, and a small batch of items (as below), and I've tried without a RowLimit on the Restore, with a 150k RowLimit, and with a 15 RowLimit (as below).

$date1=get-date("10/30/2022")
$date2=get-date("11/2/2022")

$items = Get-PnPRecycleBinItem -RowLimit 150000

$datedItems = $items | ? {($_.DeletedDate -gt $date1 -and $_.DeletedDate -lt $date2)}
$deleters = $datedItems | sort -Unique -Property DeletedByEmail
foreach ($deleter in $deleters) {$deleter.DeletedByEmail; $datedItems | ? {$_.DeletedByEmail -eq $deleter.DeletedByEmail} | measure }

$datedItems | ? {$_.DeletedByEmail -eq 'relevantuser@domain.com'} | measure
$restore = $datedItems | ? {$_.DeletedByEmail -eq 'relevantuser@domain.com'}
$restore | measure

$restore[0..100] | Restore-PnpRecycleBinItem -Force -RowLimit 15

Any help would be greatly appreciated, especially considering the criticality/urgency of probable data loss if this doesn't get sorted soon. Let me know if you need anything else from my end!

[koskila]

@SamuraiKal if you pass Identity to the commandlet (by piping, for example), -RowLimit doesn't do anything. It'll just go through the items one by one. It doesn't really have anything it could use the RowLimit for since it just gets an item to work on.

I debugged the commandlet, and without passed Identity it seems to work. Verified this with my tenant:

RowLimit seems to also work for Get-PnPRecycleBinItem:

Of course, could be that everything changes when you have 600000 items in the recycle bin. I tried achieving that to test this, but all I could do is have my OneDrive be stuck for weeks trying to process the files :)

But perhaps I'm just misunderstanding what's failing for your script... Which row is throwing which error?

[SamuraiKal]

It's the last line that's triggering the error (both below) and the original bug was referenced by this bug #1598. The problem with running the restore without an identity is it would restore thousands of files we don't need in folders that scripts run against, which would cause duplication of scripted work.

PS C:\WINDOWS\system32> $restore[0..100] | Restore-PnpRecycleBinItem -Force -RowLimit 15
Restore-PnpRecycleBinItem : The attempted operation is prohibited because it exceeds the list view threshold.
At line:1 char:20
+ $restore[0..100] | Restore-PnpRecycleBinItem -Force -RowLimit 15
+                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Restore-PnPRecycleBinItem], PSInvalidOperationException
    + FullyQualifiedErrorId : InvalidOperation,PnP.PowerShell.Commands.RecycleBin.RestoreRecycleBinItem

This is how I was hoping to be able to run it and can usually run it this way when we don't have so many items in the bin:

$restore | Restore-PnpRecycleBinItem -Force

[koskila]

Thanks for the details @SamuraiKal.

Microsoft.SharePoint.Client.dll doesn't seem to have a good implementation to restore multiple items (well, short from just restoring everything - in which case RowLimit should help you avoid the threshold issue)...

The way the Identity works is it'll query each piped item by its id from the Site's Recycle Bin - this is not very fast, so iterating through thousands of items will take some time:

I'm guessing one could create a new CommandLet for restoring items by a query since we can get any number of items as long as we use paging, and then restore them (as a collection, even - something like below), but I don't have the bandwidth for this.

@gautamdsheth any thoughts on how this could be done?

A side note / workaround

@SamuraiKal of course, one option is to remove everything that you don't want to restore from the recycle bin (this is going to be painfully slow as well) and then restoreall. Or run the restore for piped items one by one for using foreach-object -parallel - but the CSOM doesn't support this that well, so while it's faster, it'll encounter some issues (and you'll need to retry those items).

Overall, the recycle bin is unfortunately not easy to work with when you have a lot of items to churn through...

[SamuraiKal]

@koskila Thanks for the response! I had scheduled a full restore using -RowLimit to batch it this evening, but I have a new issue now (of course):

PS C:\Windows\system32> Restore-PnpRecycleBinItem -RowLimit 1500 -ErrorAction Continue
Restore-PnpRecycleBinItem : A file with this name "filename.file" already exists in "sites/Sitename/Directory". To restore the file, rename the existing file and try again.

It doesn't continue, it stops and hits the same file over and over. Is there anything that can be done to skip duplicates and keep progressing through? There are going to be hundreds, if not thousands of files that already exist and can't be reasonably renamed. If I can't skip them, can you think of anything else I can do to get these files recovered?

[hsaxena502]

@SamuraiKal : Did you find anything on this? I am stuck with the same issue "A file with this name "filename.file" already exists in"

[SamuraiKal]

@hsaxena502 I did not find anything helpful. The way Microsoft presents the recycle bin currently severely limits the options for interacting with recycle bin objects. MS would need to overhaul their recycle bin interface. We are having to resort to a very messy restore from our backup solution.