Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

exchangelib.errors.ErrorFolderNotFound: No subfolder with name '<custom folder>' when using exchangelib 5.1.0 on Ubuntu 20.04 #1273

Closed
sch0lars opened this issue Feb 11, 2024 · 31 comments
Closed

exchangelib.errors.ErrorFolderNotFound: No subfolder with name '<custom folder>' when using exchangelib 5.1.0 on Ubuntu 20.04 #1273

sch0lars opened this issue Feb 11, 2024 · 31 comments

Comments

@sch0lars
Copy link

Describe the bug
When accessing a created folder via exchangelib 5.1.0 on Ubuntu 20.04, the exception exchangelib.errors.ErrorFolderNotFound is raised. This issue only seems to occur on Ubuntu 20.04 and cannot be replicated on other operating systems, such as Debian or Windows. Interestingly, if calling account.root.tree(), the custom folders do not show up under the Top of Information Store branch.

To Reproduce

  1. Create a custom folder in an Exchange mailbox
  2. Connect to the Exchange mailbox via exchangelib 5.1.0 in an Ubuntu 20.04 environment
  3. Try to access the custom folder

Expected behavior
The folder is accessible from the exchangelib module.

Log output
N/A

Additional context

  • Ubuntu 20.04
  • python 3.x
  • exchangelib 5.1.0
@ecederstrand
Copy link
Owner

This is really weird. exchangelib is written in pure Python and supposedly entirely OS agnostic.

To debug this, I would suggest that you enable debug logging and compare the output from a working and a non-working run of your code that exhibits this error on Ubuntu 20.04.

@ecederstrand
Copy link
Owner

@sch0lars Did you have a chance to revisit this?

@sch0lars
Copy link
Author

sch0lars commented Mar 3, 2024

@ecederstrand, not yet. I downgraded to 4.7.4 to resolve the issue for the time being. I am using the package at work, so I will test it out when I have the time.

@ecederstrand
Copy link
Owner

Ok, so it's not about the Ubuntu version after all?

@sch0lars
Copy link
Author

sch0lars commented Mar 4, 2024

It’s an issue with exchangelib 5.1.0 on Ubuntu 20.04. I confirmed this occurs on multiple machines running the OS. Downgrading to exchangelib 4.7.4 resolves the issue and I could not replicate the issue on Windows or Debian, so it appears to be related changes made in a subsequent version that somehow only impacts Ubuntu 20.04.

@ecederstrand
Copy link
Owner

The best way to debug this is to follow the suggestion in #1273 (comment)

@sch0lars
Copy link
Author

sch0lars commented Mar 7, 2024 •
edited
Loading

I created a script to try and access the folder on WSL Debian and another on an Ubuntu 20.04 VM, and worked from the bottom of the log.

The first difference I noticed was the script on WSL had the following line: DEBUG:exchangelib.version:API version "Exchange2016" worked but server reports version "V2017_07_11". Using "Exchange2016"

The script on the Ubuntu VM logged a different line, despite connecting to the same mailbox: DEBUG:exchangelib.version:API version "Exchange2015" worked but server reports version "V2017_07_11". Using "Exchange2015".

Is there a particular line I should look for through all of the XML output? I cannot just paste the contents in their entirety since I am working on production systems.

@ecederstrand
Copy link
Owner

ecederstrand commented Mar 7, 2024 •
edited
Loading

Those two log lines can be ignored.

You want to look for differences in the XML requests and responses.

@OxyMal
Copy link

OxyMal commented Mar 25, 2024

Hello,

starting from 22.03.24 I am experiencing the same problem. I can only see ‘IPM_SUBTREE’ when I use the following command: account.public_folders_root.tree()
When I try to access a public folder, I get the following error: ‘ErrorFolderNotFound: No subfolder with name ‘SubfolderName’’

@ecederstrand
Copy link
Owner

Did you upgrade exchangelib at that time? If not, then it’s a server issue. If you did, then please follow the advice above.

@Tre-Seibert
Copy link

Posted this in discussion before seeing this issue. I am having the same problem, both on ubuntu and mac os (test env).

Gonna downgrade to see if this resolves my issue as well.

#1286

@Tre-Seibert
Copy link

I downgraded to 4.7.4 on both my production Ubuntu 22.04.3 LTS, and MacOS(test env) and I still get the same issue. @ecederstrand feel free to get in touch with me and I can provide you with more detailed xml logs.

@ecederstrand
Copy link
Owner

@Tre-Seibert Then I don't think it's the same issue. This one is apparently specific to exchangelib 5.1.0 and Ubuntu 20.04.

Also, if you downgraded and still have the issue, then it's probably not caused by exchangelib code, but rather changes on the server?

@OxyMal
Copy link

OxyMal commented Mar 27, 2024

Can’t that be caused by the changes you pushed into the code last week?

@ecederstrand
Copy link
Owner

Releasing a new version of exchangelib would not affect exchangelib 4.7.4 which @Tre-Seibert apparently has the same issue with.

For this issue, we still need to establish what exchangelib 5.1.0 does differently on Ubuntu 20.04 compared to other Ubuntu versions.

@Tre-Seibert
Copy link

I don't think the problem is due to changes on my server, as I tried running the code on my test environment (MacOS Ventura 13.6.4) and Windows 10, with both version 5.1.0 and version 4.7.4. No changes were made to the code or server since running the code in August of 2023, the server has been up 24/7 since august. This error started occurring suddenly about two weeks or so ago.

@Wonkyung-LEE
Copy link

Wonkyung-LEE commented Mar 29, 2024 •
edited by ecederstrand
Loading

@ecederstrand
sorry for indentation. I have same problem.
so, I think it's because the requester and response FolderIds are different.
could you please tell me how to correct this situation?

Additional : my test env is windows11, python 3.9, exchangelib 5.1.0 (and 5.2.0)
I tried : account.public_folders_root.tree()

here is my request XML. (Body Only)

<s:Body>
        <m:GetFolder>
            <m:FolderShape>
                <t:BaseShape>IdOnly</t:BaseShape>
                <t:AdditionalProperties>
                    <t:FieldURI FieldURI="folder:DisplayName"/>
                    <t:FieldURI FieldURI="folder:FolderClass"/>
                    <t:FieldURI FieldURI="folder:PermissionSet"/>
                </t:AdditionalProperties>
            </m:FolderShape>
            <m:FolderIds>
                <t:FolderId Id="AQEuAAADtvn3SgvkmU+d2vZYx4h/7wEAjtiCnOZ2TU64lhu6l2l7IAAAAxYAAAA=" ChangeKey="AQAAABYAAACO2IKc5nZNTriWG7qXaXsgAAAAa4B8"/>
                <t:FolderId Id="AAEuAAAAAAC2+fdKC+SZT53a9ljHiH/vAQCO2IKc5nZNTriWG7qXaXsgAABwj+UjAAA=" ChangeKey="AQAAABYAAACO2IKc5nZNTriWG7qXaXsgAABwrQwt"/>
            </m:FolderIds>
        </m:GetFolder>
    </s:Body>

here is my response XML. (Body Only, masked some attr)

<s:Body>
        <m:GetFolderResponse xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
            <m:ResponseMessages>
                <m:GetFolderResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:Folders>
                        <t:Folder>
                            <t:FolderId Id="AQEuAAADGkRzkKpmEc2byACqAC/EWgMAjtiCnOZ2TU64lhu6l2l7IAAAAxYAAAA=" ChangeKey="AQAAABYAAACO2IKc5nZNTriWG7qXaXsgAAAAa4B8"/>
                            <t:FolderClass>IPF.Note</t:FolderClass>
                            <t:DisplayName>share</t:DisplayName>
                            <t:PermissionSet>
                                <t:Permissions>
                                    <t:Permission>
                                        <t:UserId>
                                            <t:DistinguishedUser>Default</t:DistinguishedUser>
                                        </t:UserId>
                                        <t:CanCreateItems>true</t:CanCreateItems>
                                        <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                                        <t:IsFolderOwner>false</t:IsFolderOwner>
                                        <t:IsFolderVisible>true</t:IsFolderVisible>
                                        <t:IsFolderContact>false</t:IsFolderContact>
                                        <t:EditItems>Owned</t:EditItems>
                                        <t:DeleteItems>Owned</t:DeleteItems>
                                        <t:ReadItems>FullDetails</t:ReadItems>
                                        <t:PermissionLevel>Author</t:PermissionLevel>
                                    </t:Permission>
                                    <t:Permission>
                                        <t:UserId>
                                            <t:DistinguishedUser>Anonymous</t:DistinguishedUser>
                                        </t:UserId>
                                        <t:CanCreateItems>false</t:CanCreateItems>
                                        <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                                        <t:IsFolderOwner>false</t:IsFolderOwner>
                                        <t:IsFolderVisible>false</t:IsFolderVisible>
                                        <t:IsFolderContact>false</t:IsFolderContact>
                                        <t:EditItems>None</t:EditItems>
                                        <t:DeleteItems>None</t:DeleteItems>
                                        <t:ReadItems>None</t:ReadItems>
                                        <t:PermissionLevel>None</t:PermissionLevel>
                                    </t:Permission>
                                    <t:Permission>
                                        <t:UserId>
                                            <t:SID>*****</t:SID>
                                            <t:PrimarySmtpAddress>*****</t:PrimarySmtpAddress>
                                            <t:DisplayName>*****</t:DisplayName>
                                        </t:UserId>
                                        <t:CanCreateItems>true</t:CanCreateItems>
                                        <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                                        <t:IsFolderOwner>true</t:IsFolderOwner>
                                        <t:IsFolderVisible>true</t:IsFolderVisible>
                                        <t:IsFolderContact>true</t:IsFolderContact>
                                        <t:EditItems>All</t:EditItems>
                                        <t:DeleteItems>All</t:DeleteItems>
                                        <t:ReadItems>FullDetails</t:ReadItems>
                                        <t:PermissionLevel>Owner</t:PermissionLevel>
                                    </t:Permission>
                                </t:Permissions>
                            </t:PermissionSet>
                        </t:Folder>
                    </m:Folders>
                </m:GetFolderResponseMessage>
                <m:GetFolderResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:Folders>
                        <t:Folder>
                            <t:FolderId Id="AAEuAAAAAAAaRHOQqmYRzZvIAKoAL8RaAwCO2IKc5nZNTriWG7qXaXsgAABwj+UjAAA=" ChangeKey="AQAAABYAAACO2IKc5nZNTriWG7qXaXsgAABwrQwt"/>
                            <t:DisplayName>*****</t:DisplayName>
                            <t:PermissionSet>
                                <t:Permissions>
                                    <t:Permission>
                                        <t:UserId>
                                            <t:DistinguishedUser>Default</t:DistinguishedUser>
                                        </t:UserId>
                                        <t:CanCreateItems>true</t:CanCreateItems>
                                        <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                                        <t:IsFolderOwner>false</t:IsFolderOwner>
                                        <t:IsFolderVisible>true</t:IsFolderVisible>
                                        <t:IsFolderContact>false</t:IsFolderContact>
                                        <t:EditItems>Owned</t:EditItems>
                                        <t:DeleteItems>Owned</t:DeleteItems>
                                        <t:ReadItems>FullDetails</t:ReadItems>
                                        <t:PermissionLevel>Author</t:PermissionLevel>
                                    </t:Permission>
                                    <t:Permission>
                                        <t:UserId>
                                            <t:DistinguishedUser>Anonymous</t:DistinguishedUser>
                                        </t:UserId>
                                        <t:CanCreateItems>false</t:CanCreateItems>
                                        <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                                        <t:IsFolderOwner>false</t:IsFolderOwner>
                                        <t:IsFolderVisible>false</t:IsFolderVisible>
                                        <t:IsFolderContact>false</t:IsFolderContact>
                                        <t:EditItems>None</t:EditItems>
                                        <t:DeleteItems>None</t:DeleteItems>
                                        <t:ReadItems>None</t:ReadItems>
                                        <t:PermissionLevel>None</t:PermissionLevel>
                                    </t:Permission>
                                    <t:Permission>
                                        <t:UserId>
                                            <t:SID>*****</t:SID>
                                            <t:PrimarySmtpAddress>*****</t:PrimarySmtpAddress>
                                            <t:DisplayName>*****</t:DisplayName>
                                        </t:UserId>
                                        <t:CanCreateItems>true</t:CanCreateItems>
                                        <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                                        <t:IsFolderOwner>true</t:IsFolderOwner>
                                        <t:IsFolderVisible>true</t:IsFolderVisible>
                                        <t:IsFolderContact>true</t:IsFolderContact>
                                        <t:EditItems>All</t:EditItems>
                                        <t:DeleteItems>All</t:DeleteItems>
                                        <t:ReadItems>FullDetails</t:ReadItems>
                                        <t:PermissionLevel>Owner</t:PermissionLevel>
                                    </t:Permission>
                                </t:Permissions>
                            </t:PermissionSet>
                        </t:Folder>
                    </m:Folders>
                </m:GetFolderResponseMessage>
            </m:ResponseMessages>
        </m:GetFolderResponse>
    </s:Body>

@ecederstrand ecederstrand mentioned this issue Apr 2, 2024
Closed
@Tre-Seibert
Copy link

It’s an issue with exchangelib 5.1.0 on Ubuntu 20.04. I confirmed this occurs on multiple machines running the OS. Downgrading to exchangelib 4.7.4 resolves the issue and I could not replicate the issue on Windows or Debian, so it appears to be related changes made in a subsequent version that somehow only impacts Ubuntu 20.04.

Does this still work for you? I have tried every version of exchangelib with no success?

@sch0lars
Copy link
Author

sch0lars commented Apr 3, 2024 •
edited
Loading

It’s an issue with exchangelib 5.1.0 on Ubuntu 20.04. I confirmed this occurs on multiple machines running the OS. Downgrading to exchangelib 4.7.4 resolves the issue and I could not replicate the issue on Windows or Debian, so it appears to be related changes made in a subsequent version that somehow only impacts Ubuntu 20.04.

Does this still work for you? I have tried every version of exchangelib with no success?

@Tre-Seibert, downgrading still works for me. If I call account.root.tree() on 4.7.4, the custom folders are there, but not on 5.1.0. Maybe diffing the changes between the two versions could elucidate what’s going on?

I tried this on a fresh install of Ubuntu 20.04 and confirmed the fix works. The problem is also not present on Windows or WSL, so it seems to be isolated to Ubuntu (or *nix. I haven’t tested on any other OSes).

@shawnlinxl
Copy link

Could this relate to #1288? Maybe Microsoft pushed out updates on one OS first before the others?

@Tre-Seibert
Copy link

Could this relate to #1288? Maybe Microsoft pushed out updates on one OS first before the others?

I think it absolutely does, or these issues would be one huge coincidence. Very curious about the outcome and patches to follow

@ZamElek
Copy link

ZamElek commented Apr 16, 2024

I have the same problem, the script stopped working

@ecederstrand
Copy link
Owner

@ZamElek You mean that you are also getting ErrorFolderNotFound for folders that do exist, on exchangelib 5.1.0 installed on Ubuntu 20.04? And exchangelib 5.1.0 on other Ubuntu versions work?

It's a long shot, but it could be fixed by f65079f.

@ZamElek
Copy link

ZamElek commented Apr 17, 2024 •
edited
Loading

@ZamElek You mean that you are also getting ErrorFolderNotFound for folders that do exist, on exchangelib 5.1.0 installed on Ubuntu 20.04? And exchangelib 5.1.0 on other Ubuntu versions work?

It's a long shot, but it could be fixed by f65079f.

I don't believe the issue lies within the operating system itself. I debugged a script that utilizes the exchangelib to export messages from an Exchange (O365) public folder. This script connects to the EWS API and authenticates via OAuth2 using the delegated credential flow. It functioned correctly until last month. The script runs on Ubuntu 20.04 and uses Python 3. Yesterday, I installed the latest version of exchangelib (v. 5.2.1) and tested it in a Docker image based on Debian GNU/Linux 11 (bullseye) with Python 3.12.2. The same error, "ErrorFolderNotFound," occurs for folders that do indeed exist. I also attempted to export a public folder using the same OAuth2 token but with the .NET library microsoft.exchange.webservices.data.2.2.0 in PowerShell—I copied the OAuth2 token I obtained from exchangelib—and it successfully connected to the public folder and exported messages. It seems likely that Microsoft has altered something on their end.

@ecederstrand
Copy link
Owner

Since you're mentioning public folders, can you please try out f65079f?

@ZamElek
Copy link

ZamElek commented Apr 17, 2024

Since you're mentioning public folders, can you please try out f65079f?

Tried it out and it doesn't work, still get the error: """File "/workspaces/project-closure-main/exchangelib/folders/base.py", line 798, in truediv
raise ErrorFolderNotFound(f"No subfolder with name {other!r}")
exchangelib.errors.ErrorFolderNotFound: No subfolder with name ...""" It just see only the root public folder IPM_SUBTREE

@ZamElek
Copy link

ZamElek commented Apr 17, 2024

I also examined the responses in Fiddler and confirmed that all public folders are in place. I believe that the library is unable to parse the XML responses correctly.

@ecederstrand
Copy link
Owner

I don't have access to a test account with public folders, so you'll have to debug this yourself. Try adding a breakpoint() in __truediv__ and stepping through the code.

@ecederstrand ecederstrand mentioned this issue Apr 22, 2024
Closed
@ecederstrand
Copy link
Owner

@ZamElek It's possible that d9035d0 fixed this issue. Can you try it out?

@ecederstrand
Copy link
Owner

I think the original issue is solved with patches released in v5.3.0. If you still have problems, please open a new issue containing a minimal reproducer, and error message or stack trace, so we're sure which problem we're talking about.

@ZamElek
Copy link

ZamElek commented Apr 29, 2024

@ecederstrand
Thank you, Erik. Apologies for the late response. I had the opportunity to debug the module and discovered that the issue arises because the EWS API returned a different <t:FolderId Id=> for the root IPM_SUBTREE than for the reference <t:ParentFolderId> of the folder itself, from a GetFolder EWS request. The get_children function of RootOfHierarchy tests if f.parent.id == folder.id, and because they differ, it can't retrieve the key from a dict. I suspect that this was modified by Microsoft, so we have opened a support request with them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@ecederstrand @ZamElek @shawnlinxl @Wonkyung-LEE @sch0lars @Tre-Seibert @OxyMal