[BUG] RemoteEventReceivers are not fired when added via PnP.Powershell

[LeopoldLerch]

When I add an Remote EventReceiver using PnP.Powershell, it simply doesnt get fired.

I used command:

Add-PnPEventReceiver -List Seiten -Name LeoRecTest3 -Url "https://<myOwnSink>.m.pipedream.net/test2.svc" -EventReceiverType ItemUpdated -Synchronization DefaultSynchronization -SequenceNumber 5

This is using pipedram as Sink, but it didn´t work with other sinks as well.

However, uninstalling PnP.Powershell and adding the Eventreceiver (with the same command and same list) and the events are fired. Furthermore, i found out, that i was not able to remove the RER using PnP.Powershell, when it was added via SharepointPnPPowershellOnline. I then get the error that I am not authorized to do so.

Interestingly, the EventReceiver shows up correctly when inspecting the EventReceivers in the List, i was not able to detect any difference between added via PnP.Powershell and SharePointPnPPowerShellOnline. Just one worked while the other didn´t.

Expected behavior

RER is fired, also when added via PnP.Powershell and RER added via SharePointPnPPowerShellOnline can be deleted using PnP.PowerShell

Actual behavior

RER added via PnP.Powershell are not fired. cannot delete RemoteEventReceivers added formerly by using SharePointPnPPowerShellOnline

Steps to reproduce behavior

Add an RemoteEventreceiver via PnP.Powershell and target it to an http-sink like requestbin, hookb.in, pipedream or similar. No request will be received. Added the RemoteEventreceiver via SharePointPnPPowerShellOnline and the request will be received

What is the version of the Cmdlet module you are running?

1.4

Which operating system/environment are you running PnP PowerShell on?

• [ x] Windows
• Linux
• MacOS
• Azure Cloud Shell
• Azure Functions
• Other : please specify

[LeopoldLerch]

Just retried on a complete new Site (modern, as the other were classic). I added 2 Eventreceivers, one with the old Version and one with the New PnP.Powershell, both with the same command, targeting the same Domain (but different endpoints there, one for the old and one for the new). Updated the Item twice, but only the EventReceivers added by the old one were fired, at least only the url for the old one received an request

Results from Rest-Query for EventReceivers:

Name	Value
ReceiverAssembly
ReceiverClass
ReceiverId	260ceaaf-8fb4-4b26-a4a5-b2123912de13
ReceiverName	LeoRecTestOld
SequenceNumber	1000
Synchronization	2
EventType	10002
ReceiverUrl	https://enq408zh8jzqpfm.m.pipedream.net/testOld.svc

Name	Value
ReceiverAssembly
ReceiverClass
ReceiverId	94ec65f3-04ac-4521-beb7-943d5873869b
ReceiverName	LeoRecTestNew
SequenceNumber	1000
Synchronization	2
EventType	10002
ReceiverUrl	https://enq408zh8jzqpfm.m.pipedream.net/testNew.svc

But, only the Url from the "Old"-Event-Receiver gets an Request.

[LeopoldLerch]

@erwinvanhunen Any news about this? This is basically blocking us to use it.

We cannot switch to hooks cause we need to react as soon as possible and RemoteEventReceiver are offering nearly real-time.

[erwinvanhunen]

Hi! We've been looking at this today and we experience no problems removing existing event receivers, added by either version and cross versions of PnP PowerShell. E.g. we could remove a RER added by the legacy PnP PowerShell just fine by using the modern PnP PowerShell and vice versa.

Having said that when we add a remote event receiver, regardless of which PnP PowerShell build we use, none of them calls pipedream. We're still investigating what is happening here.

[erwinvanhunen]

Actually we just found a scenario where deleting failed. If you add a RER using the new PnP PS where you auth with credentials, and then connect again using -UseWebLogin we receive an access denied. Investigating more. The challenge here that is out of the scope of what we can fix, as the error message comes from the server. All other actions seem to work normally, so it's possibly related to how event receivers are registered when using a different auth method.

[LeopoldLerch]

well, no, login worked, i got the unatuhorized error when calling the method to remove the old ER.

Regarding the calls: Pipedream was just one example, i tried different http-sinks, all with the same result. Maybe try to add a path to the domain of pipedream, I think just calling the domain doesn´t usually work as RER-Endpoint, try it with an path like "domainname/fakeendpoint"

[erwinvanhunen]

We tried that actually, with an added endpoint, had no effect on our end.

[LeopoldLerch]

well, i got the messages using the old one. just not with the new one. However, it is just an example, maybe you have another way to verify if Sharepoint sends the Event-messages, i just did not want to deploy a full webservice just for test and demonstration purpose.

[LeopoldLerch]

I used "Connect-PnPOnline" without any additional flags (both times)

[erwinvanhunen]

Right. We got it to work with the new PnP PowerShell. We used an Azure function (HttpTrigger) on the receiving end. What we did different is how we authenticated:

Connect-PnPOnline -Url -UseWebLogin

This uses the 'classic/legacy' way of authenticating and that worked for us. Why it's not working with the new auth we are unsure of right now. The legacy PnP PowerShell uses the same classic auth as -UseWebLogin uses.

[LeopoldLerch]

So using the "-useWebLogin" parameter should work?
However, can i use that parameter in connection with a automated job? like Deployment via Devops? Cause thats what our UseCase in Summary is.

[LeopoldLerch]

"-useWebLogin" just works in our prod-tenant, where also my client is. When targeting our dev or testdenant, i only get an Popup showing a white screen. However, in browser window i am connected to the tenant, so Connection should work.

Legacy Powershell just showing the input Box for username/password and works afterwards. No Browser-Popup

[erwinvanhunen]

The problem with UseWebLogin is two-fold. It's built upon a rather hacky/instable premise: we launch page in a browser component and extract the cookies from that session, which we do if we recognize the correct URL or the cookies in the session, then dress up every request towards SPO with the same cookies. If we do not recognize the correct URL you end up with a blank page. It also limits us from using the Graph APIs and as more and more functionality is introduced to the Graph for new features we need to be able to access it.

We investigated more and we do nothing different between the legacy version and the modern version of PnP PowerShell: result, it's the server acting differently based upon how you authenticate. EventReceivers are a pretty old concept, coming all the way from the on-premises time and have not received much attention unfortunately. The only thing we can advice is to create a support request/bug report at Microsoft so they can look into it.

[LeopoldLerch]

well, its a side-track, but just investigated the urls called by UseWebLogin using fiddler, and all of the urls worked, when copied directly to the browser. However, useWebLogin somehow got a "401" Response from the ADFS-Servers in the mentioned domains. However, using that urls directly in browser worked.

Regarding the ER:
Well, yes, expected that somewhow, as the ERs were shown exactly identicyl when inspecting their Configuration in the list, so I already thought it is more addicted to Sharepoint itself as PNP. But I just expected maybe you have a workaround or idea or at least some better connections to MS g.
When i create a bug report at MS, I do not expect much, as i would have to tell them that when using that version of the tool it works and the other version not. The natural reponse will be, that they will ask me to use the older version of the tool or make a bug at the tool. Cause I can´t really describe them the difference or the issue on their side.

[LeopoldLerch]

Maybe I know why WebLogin is not working for the different domains. I am redirected to the "wia" Endpoint of our adfs-servers, which is for Windows authentication. There, of course, the integrated authentication with my current User of the machine is not working, cause of the domain. However, Browsers at this point show the Authentication-Popup for username-Password, but the Popup of the powershell "useWebLogin" (or the interactive Mode) does not. and thats where the whole story stops.

However, "useWebLogin" seems not usable for automated background (or deployment) tasks anyway.

[LeopoldLerch]

I opened a case at MS. They are currently looking at it, altough they think the best help could come from github (whatever they mean with that). Will come back as soon as I got updates from them

[bhishma]

I had raised similar issue in #241 (comment) . I opened Microsoft ticket.

Microsoft told me that registering RERs using ClientCredential (legacy) method is not technically supported and it's developer's 'party trick'. (Quite surprised! When I asked if they have documentation on that, they said, "we don't have documentation on what not to do")
Registering with the context created with oAuth token
apparently requiress the app Id to be same as that of RER.

Looking forward to hearing your experience. This has been a blocker for us.

[LeopoldLerch]

I have an issue open also. Did not hear anything from that, that is worth reporting, just that they are looking at it.

If they provide an REST-API for EventReceivers, it should work independent of authentication, like the rest of the api. I don´t even get why the way a RER is attached is even important at runtime. If one is added, it should run, or the attachment of it should fail.

The Docs even mention, that it is not necesarry to provide even a digest if you are authenticated using OAuth Token.

There is nowhere in the REST-Api docs to be found, that only cookie-based authentication is supported, nor is this the recommended approach to use it at all nowadays.

[bhishma]

I tried Invoke-PnPSPRestMethod (#241 (comment)) that results in same outcome (RERs registered with pnp.powershell do not fire but same command with old module works).

I was told 'somewhere deep down, the code compares the AppId of the RER and the context, and does not fire if they are not same'!

[LeopoldLerch]

Hi, still in contact with MS.
During investigation for them I found out, that you are still using the goold old SOAP-based API also in new Powershell-Module. Is there a reason why you didn´t switch to REST-API (as SOAP is quite old meanwhile and its endpoints might get deprecated some day)?

However, doesn´t make any difference to the current case, Also when adding the RER via REST-Endpoint, it is not fireing (altough adding itself is successful and it is listed in the existing RERs). At least when using REST via Oauth-Bearer-Token.

[LeopoldLerch]

Update:
Seems that, when attaching an RER using OAuth Sharepoint remembers the AppId (no difference if authenticated as user or per app). RER is added correctly as we know. However, when executing the Sharepoint searches in its AppPrincipals (the ones needed for Sharepoint-Addins using appinv/appregnew). Of course, usually it doesn´t find an appprincipal, because there is no appregistration in sharepoint (with a principal) and fails. However, even if you add an appprincipal for eg the pnp management shell with much permissions, it doesn´t work (but without errors i know of).

Seems to work only using oauth with an token issued from https://accounts.accesscontrol.windows.net, which is the deprecated provider used for the SP addinmodel.

So, for me it seems when the RER is registered using an oauth token, Sharepoint internally is taking this as an SPregistered app, altough there is none.

However, currently MS is saying that the token from the pnp-app is wrong or i should use a token from accesscontrol. In general currently RERs seems to work only in "old" authentication models or attaching it using an SP registered appprincipal.

[LeopoldLerch]

Thing is currently:
When adding an EventReceiver Sharepoint is adding the AppId to the RER-Registration (even when using delegated permissions). When executing it is searching an application-principal (a SP-registered app) and fails, cause there is none

ULS-error according to MS:
The app i:0i.t|ms.sp.ntv|31359c7f-bd7e-475c-86db-fdb8c937548e@2c5e0370-46aa-4fcb-8f09-02e12ebb1f79 does not have required perm Manage from tenant data. TenantPerm=0,SitePerm=0,WebPerm=0

(you see, it is searching for an appprincipal with the Guid of the PNP-App)
Like said, MS is saying that the token used is wrong because of the appid or the app from PNP is wrong and i should use different ways. However, I know that the token is not wrong and that different ways to get the token will be difficult and deliver the same result, just with another appid

Maybe someone from PnP or Vesa Juvonen can help here with some connections? Otherwise MS will block RERs from being used with Modern authentication (as everything else, is more less deprecated or not recommended, when even possible).

[LeopoldLerch]

@erwinvanhunen
It seems to have something to do with the AppId in the token, which makes Sharepoint think it is an SPRegistered App, even when registered via AAD-registered app with delegated permissions. I am quite sure that that is an issue MS should resolve, but they are having troubles seeing that and always pointing to PnP.

However, i also found out, that the Token only contains the "appId" in the v1 version of the token. V2 Tokens might work. However, i did not found out yet, why PnP always getting v1-tokens. As I found out using fiddler, it is using the v2 endpoint, and I also created an custom app for and assingned the "accessTokenAcceptedVersion" to 2, which sould also force to provide an v2 token. However, did not manage to get an v2 token using PnP yet.
Can somebody look at why there is no v2 token issued for PnP-Powershell and if that might help for getting an RER to run?

[LeopoldLerch]

Okay,
Got reply from MS, that RERs are not supported outside of the Provider-hosted-Model (and in context of an SP-registered app /app-principal). The fact, that it worked with legacy-Authentication (cookie-based) has not effect on that. Currently they will not change anything to support RERs using in combination with AAD-registered apps.

Will have a follow-up meeting with Product-group in a while, but that will be only around a way, to prevent RERs being added in the first place. But, even if this will happen, it will take some time.

@erwinvanhunen
In that case RERs will never work with modern authentication. So I would suggest you either remove the Cmdlets completely or only allow it in legacy-mode. So that Users will not run into that troubles.

[Tanddant]

I made a quick webpart sample that let's you add RERs in a user context through rest, feel free to use that as a workaround for this issue

[LeopoldLerch]

Thx, appreciate it.
However, that was not the point. Adding is not the problem. The problem is, that the event-receivers are not fireing (altough added), when they were added using a Oauth-Token received by an AAD-registered app.. Don´t know for the moment, how the pnp-js library is authenticating. But if it is adding the RERs using an oAuth-Token, the RERs will not fire. This "behaviour" is meanwhile confirmed by MS

[Tanddant]

I'm using the webpart context SPHttpClient to add them, and they do fire, while I know this doesn't solve the issue with the PowerShell framework, at least for now it allows you to "quickly" add a RER.

Not as convenient as PowerShell, but it works.

[LeopoldLerch]

Well, at least for me an Webpart is no replacement for my powershell-scripts, as they are run in the Azure-Devops-Pipeline. As that issue is bound to the authentication the only workaround is to stick on the old powershell-version, which uses Cookie-based authentication and support UserCredentials.

Anything else involving Oauth-Authentication with AAD will fail (even Spfx, as soon as they change from cookie to oauth).

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[in8AnkKu]

I am also facing same issue as Remote Event Receiver is not getting triggered if I connect the SharePoint site with below command with PowerShell 7. But if I execute the same with PowerShell 5 then it is working fine.

Connect-PnPOnline -Url https://tenant.sharepoint.com/sites/TestSite -Credentials $cred

Add-PnPEventReceiver -List "Site Pages" -Name "Remove Event Receiver" -Url $ventReceiverUrl -EventReceiverType ItemUpdated -Synchronization Asynchronous

Is there any alternate way to fix it?

[LeopoldLerch]

No, there is no workaround. According to MS RemoteEventReceiver are meant to work within an Sharepoint-Registered App. So, whenever you send an token for authorization with your request, it thinks it is an token for a Sharepoint-Registered-App. However, when that token is a token for an AzureAD-registered-App, this results in a fail cause it will not find an Sharepoint-Registered App. It works only when not using an token at all, which means, you have to be authorized via ClientCredentials/cookie-based Authentication. This is by design and it was even an undocumented and therefore not supported feature that it even worked with cookie-based-Authentication in the past. So, for MS, this is working as designed and "No Issue".

However, when closing that issue, they meant, that I am the first and yet only one having a problem with it. So you are free to make an issue with MS too. Maybe more complaints may force them to do something about it.

The only workaround is to use legacy-Authentication in the Powershell-7 Version or stick with the old Version in Powershell 5.

[bhishma]

@LeopoldLerch , you are not the first :) they say that to everyone.
I opened a similar ticket with Microsoft in February, and they said I was the first.

[sadomovalex]

We have exactly the same issue - RERs are fired only when you authenticated with "UseWebLogin" flag. With other methods RERs are added without error but not fired (wrote details here: Sharepoint Online remote event receivers attached with Add-PnPEventReceiver depend on used authentication method)