[Bug]: HTTP REPORT returns 503 on some CalDAV/CardDAV collections #37277

rfc2822 · 2023-03-16T16:28:59Z

⚠️ This issue respects the following points: ⚠️

This is a bug, not a question or a configuration/webserver/proxy issue.
This issue is not already reported on Github (I've searched it).
Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
Nextcloud Server is running on 64bit capable CPU, PHP and OS.
I agree to follow Nextcloud's Code of Conduct.

Bug description

I create this ticket to track the "Nextcloud returns 503 on REPORT for some collections", see bitfireAT/davx5-ose#285, as discussed in Talk today

CC @ChristophWurst @miaulalala @st3iny

Steps to reproduce

Unfortunately not known how

Expected behavior

REPORT should return 207 or at least some error message

Installation method

None

Operating system

Other

PHP engine version

None

Web server

None

Database engine version

None

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

Default user-backend (database)
LDAP/ Active Directory
SSO - SAML
Other

Configuration report

No response

List of activated Apps

not available

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

nickvergessen · 2023-03-20T07:45:27Z

Also my thunderbird is now not showing all the events any more that exist in the web. The report request does show a 207 in this case, but something is still going wrong:

2001:…:7c70 - - [20/Mar/2023:07:41:45 +0000] "REPORT /remote.php/dav/calendars/joas/personal/ HTTP/1.1" 207 1198 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0" 259643

manswiss · 2023-03-20T08:19:24Z

My synchronisation on Android with DavX5 has also stopped working since the update. I also see the 207 and 503 errors in the log.

Der-K-2000 · 2023-03-20T08:29:44Z

I can confirm this issue since RC 2 (I switched from v25 stable to v26 RC 2), with all mobile apps connected to DAVx5 (Android/F-Droid) and with Thunderbird, too.

Also there are no changes after checking and resetting everything or refreshing the password.

ChristophWurst · 2023-03-20T09:11:01Z

What PHP versions are in use? 8.1 or 8.2?

Der-K-2000 · 2023-03-20T09:12:57Z

What PHP versions are in use? 8.1 or 8.2?

PHP 8.1

manswiss · 2023-03-20T09:14:49Z

What PHP versions are in use? 8.1 or 8.2?

I've testet with 8.1 and 8.2. Same result on booth versions

nickvergessen · 2023-03-20T09:20:29Z

What PHP versions are in use?

PHP 8.1.2-1ubuntu2.11

andrelcz · 2023-03-20T09:20:33Z

Same problem here.
DAVx5 - 5.3.1
Nextcloud 26 RC2 (started with RC1), with PHP 8.2.

ChristophWurst · 2023-03-20T09:53:27Z

v26.0.0rc1...v26.0.0rc2

ChristophWurst · 2023-03-20T10:09:06Z

With PHP8.2+nginx+php-fpm and stable26 I can not reproduce the bug. I've tried personal events and ones in shared calendars. They are synced to Thunderbird without any issues.

manswiss · 2023-03-20T10:16:19Z

I use:

Server version: Apache/2.4.55 (Debian)
php8.2-fpm
RC3

Sync in outlook (CalDav Synchronizer) work correct, but not with DavX5 (Android).

ChristophWurst · 2023-03-20T10:18:42Z

https://download.nextcloud.com/server/prereleases/nextcloud-26.0.0beta1.tar.bz2 sync with Thunderbird works too

ChristophWurst · 2023-03-20T10:22:51Z

With PHP8.2+nginx+php-fpm

Same instance with DAVx6 gives lots of

Token is too short for a generated token, should be the password during basic auth

When I log in without an app password (the account doesn't use 2FA)

manswiss · 2023-03-20T10:41:33Z

I'm not sure I understand the last comment correctly. Can you now reproduce the problem with DavX5?
And can I help in any way?

ChristophWurst · 2023-03-20T10:43:48Z

I get logs, but so far all event changes sync to my phone. I can therefore not reproduce.

We are checking if more logging can be added to the dav backend. If we have a logging patch to apply I'll let you know!

manswiss · 2023-03-20T10:48:14Z

If you want I can send you my DavX5 logs. But I want only send you this private, that is not readable for public.

ChristophWurst · 2023-03-20T12:15:01Z

With #37306 applied our production instance syncs with davx5 again but not with Thunderbird: #37301 (comment)

manswiss · 2023-03-20T12:21:27Z

This fix works for me with DavX5. After installing it, I had to restart php8.2-fpm.
Thank you very mutch.

ChristophWurst · 2023-03-20T14:39:37Z

Steps to reproduce

curl -i <domain> -> note the ocxxxxxx cookie based on the instance id
curl -k -i -X REPORT -H 'Cookie: ocxxxxxx=abcdef; cookie_test=test' --user admin:password https://localhost/remote.php/dav/calendars/admin/personal

the values for the cookie, username or password do not matter.

ChristophWurst · 2023-03-20T14:43:16Z

In contrast, curl -k -i -X REPORT -H 'Cookie: ocxxxxxx=abcdef' --user admin:password https://localhost/remote.php/dav/calendars/admin/personal and curl -k -i -X REPORT -H 'Cookie: ocxxxxxx=abcdef; __Host-nc_sameSiteCookielax=true; __Host-nc_sameSiteCookiestrict=true' --user admin:password https://localhost/remote.php/dav/calendars/admin/personal do "work" and produce the expected HTTP500 due to the missing request body.

ChristophWurst · 2023-03-20T14:45:32Z

DAVX5 runs into this scenario. It sends cookie_test but no __Host-nc_sameSiteCookie* cookies

ChristophWurst · 2023-03-20T14:54:46Z

@juliushaertl found that \OC\AppFramework\Http\Request::getProtectedCookieName depends on the session cookie params set at

server/lib/base.php

Line 423 in bbc6eee

ini_set('session.cookie_secure', 'true');

. The latter is skipped for API requests. So the strict cookies miss the secure parameter and the __Host- prefix

…nges Temporary disabled the short cut again to solve issues with CalDAV/CardDAV clients like DAVx5 that use cookies and need a session. See #37277 (comment) and the other comments for further information. Signed-off-by: Joas Schilling <coding@schilljs.com>

…anges Temporary disabled the short cut again to solve issues with CalDAV/CardDAV clients like DAVx5 that use cookies and need a session. See #37277 (comment) and the other comments for further information. Signed-off-by: Joas Schilling <coding@schilljs.com>

nickvergessen · 2023-03-20T15:37:16Z

Had to re-create my PR, because after adjusting the commit message the GitHub PR broken and would not properly transform from draft to PR and open after a close:

fix(session): Fix DAVx5 sync problems by partial reverting session changes #37306

nursoda · 2023-03-20T16:03:56Z

I confirm that applying #37306 leads to DAVx5 syncing my address book against my instance running NC26.0.0RC3 with NGINX and PHP-FPM 8.2 on Arch.

…anges Temporary disabled the short cut again to solve issues with CalDAV/CardDAV clients like DAVx5 that use cookies and need a session. See #37277 (comment) and the other comments for further information. Signed-off-by: Joas Schilling <coding@schilljs.com>

nursoda · 2023-03-21T07:17:12Z

It seems that this workaround does break app update functionality. At least, while lines 414-420 in base.php were commented out, for the calendar app update from 4.3.0-rc2 to 4.3.0 I got this:

$ occ update:check
Update for calendar to version 4.3.0 is available.
1 update available
$ occ app:update --all
calendar new version available: 4.3.0
calendar couldn't be updated

So I uninstalled the App via /settings/apps and tried to re-install. There I get
Could not download app calendar

Same via CLI
$ occ app:install calendar
Error: Could not download app calendar

Manually downloading from https://apps.nextcloud.com/apps/calendar, extracting and enabling calendar did work.

I see this in nextcloud.log

{
  "reqId": "tb5562xZjViM6wTupiPu",
  "level": 2,
  "time": "2023-03-21T07:39:25+01:00",
  "remoteAddr": "",
  "user": "--",
  "app": "appstoreFetcher",
  "method": "",
  "url": "--",
  "message": "Could not connect to appstore: cURL error 7: Failed to connect to apps.nextcloud.com port 443 after 20 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json",
  "userAgent": "--",
  "version": "26.0.0.10",
  "data": {
    "app": "appstoreFetcher"
  }
}
{
  "reqId": "tb5562xZjViM6wTupiPu",
  "level": 3,
  "time": "2023-03-21T07:39:25+01:00",
  "remoteAddr": "",
  "user": "--",
  "app": "no app in context",
  "method": "",
  "url": "--",
  "message": "Could not download app calendar",
  "userAgent": "--",
  "version": "26.0.0.10",
  "exception": {
    "Exception": "Exception",
    "Message": "Could not download app calendar",
    "Code": 0,
    "Trace": [
      {
        "file": "/…/lib/private/Installer.php",
        "line": 193,
        "function": "downloadApp",
        "class": "OC\\Installer",
        "type": "->"
      },
      {
        "file": "/…/core/Command/App/Update.php",
        "line": 107,
        "function": "updateAppstoreApp",
        "class": "OC\\Installer",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/…/3rdparty/symfony/console/Command/Command.php",
        "line": 255,
        "function": "execute",
        "class": "OC\\Core\\Command\\App\\Update",
        "type": "->"
      },
      {
        "file": "/…/3rdparty/symfony/console/Application.php",
        "line": 1009,
        "function": "run",
        "class": "Symfony\\Component\\Console\\Command\\Command",
        "type": "->"
      },
      {
        "file": "/…/3rdparty/symfony/console/Application.php",
        "line": 273,
        "function": "doRunCommand",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/…/3rdparty/symfony/console/Application.php",
        "line": 149,
        "function": "doRun",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/…/lib/private/Console/Application.php",
        "line": 215,
        "function": "run",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/…/console.php",
        "line": 100,
        "function": "run",
        "class": "OC\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/…/occ",
        "line": 11,
        "args": [
          "/…/console.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/…/lib/private/Installer.php",
    "Line": 387,
    "message": "Could not download app calendar",
    "exception": {},
    "CustomMessage": "Could not download app calendar"
  }
}
{
  "reqId": "FBJQ2BKo2DxuZPRd6tMc",
  "level": 3,
  "time": "2023-03-21T07:45:34+01:00",
  "remoteAddr": "192.168.1.2",
  "user": "…",
  "app": "settings",
  "method": "POST",
  "url": "/settings/apps/enable",
  "message": "could not enable apps",
  "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/111.0",
  "version": "26.0.0.10",
  "exception": {
    "Exception": "Exception",
    "Message": "Could not download app calendar",
    "Code": 0,
    "Trace": [
      {
        "file": "/…/apps/settings/lib/Controller/AppSettingsController.php",
        "line": 448,
        "function": "downloadApp",
        "class": "OC\\Installer",
        "type": "->"
      },
      {
        "file": "/…/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 230,
        "function": "enableApps",
        "class": "OCA\\Settings\\Controller\\AppSettingsController",
        "type": "->"
      },
      {
        "file": "/…/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 137,
        "function": "executeController",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->"
      },
      {
        "file": "/…/lib/private/AppFramework/App.php",
        "line": 183,
        "function": "dispatch",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->"
      },
      {
        "file": "/…/lib/private/Route/Router.php",
        "line": 315,
        "function": "main",
        "class": "OC\\AppFramework\\App",
        "type": "::"
      },
      {
        "file": "/…/lib/base.php",
        "line": 1051,
        "function": "match",
        "class": "OC\\Route\\Router",
        "type": "->"
      },
      {
        "file": "/…/index.php",
        "line": 36,
        "function": "handleRequest",
        "class": "OC",
        "type": "::"
      }
    ],
    "File": "/…/lib/private/Installer.php",
    "Line": 387,
    "message": "could not enable apps",
    "exception": {},
    "CustomMessage": "could not enable apps"
  }
}

ChristophWurst · 2023-03-21T07:20:54Z

Failed to connect to apps.nextcloud.com port 443

Network error

ChristophWurst · 2023-03-21T07:21:26Z

~~I can't reach https://apps.nextcloud.com/api/v1/apps.json either~~ I can. It just takes a very long time.

rfc2822 added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Mar 16, 2023

ChristophWurst added 1. to develop Accepted and waiting to be taken care of 26-feedback feature: caldav Related to CalDAV internals feature: carddav Related to CardDAV internals regression and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Mar 16, 2023

This was referenced Mar 17, 2023

[Bug]: Nextcloud 26.0.0 (RC3) CalDav calendar sync problem #37281

Closed

Update to nc26 nextcloud/all-in-one#1846

Closed

ChristophWurst added 2. developing Work in progress and removed 1. to develop Accepted and waiting to be taken care of labels Mar 20, 2023

nickvergessen mentioned this issue Mar 20, 2023

fix(CalDAV): Try to fix DavX5 problems #37301

Closed

4 tasks

ChristophWurst mentioned this issue Mar 20, 2023

fix(security): Log failing strict cookie check #37305

Merged

5 tasks

nickvergessen mentioned this issue Mar 20, 2023

fix(session): Fix DAVx5 sync problems by partial reverting session changes #37306

Merged

4 tasks

ChristophWurst added 4. to release Ready to be released and/or waiting for tests to finish and removed 2. developing Work in progress labels Mar 20, 2023

ChristophWurst mentioned this issue Mar 21, 2023

fix(security)!: Use consistent HTTP status for strict cookie checks #37316

Merged

6 tasks

blizzz closed this as completed in #37306 Mar 21, 2023

solracsf mentioned this issue Mar 28, 2023

[Bug]: Error 503 via DAVx5 in CalDAV/CardDAV, but not every calendar #37450

Closed

9 tasks

szaimen mentioned this issue Apr 14, 2023

tasks for nc26 nextcloud/all-in-one#2336

Closed

45 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: HTTP REPORT returns 503 on some CalDAV/CardDAV collections #37277

[Bug]: HTTP REPORT returns 503 on some CalDAV/CardDAV collections #37277

rfc2822 commented Mar 16, 2023 •

edited

Loading

nickvergessen commented Mar 20, 2023

manswiss commented Mar 20, 2023

Der-K-2000 commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

Der-K-2000 commented Mar 20, 2023

manswiss commented Mar 20, 2023

nickvergessen commented Mar 20, 2023

andrelcz commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

manswiss commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

manswiss commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

manswiss commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023 •

edited by nickvergessen

Loading

manswiss commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023 •

edited

Loading

nickvergessen commented Mar 20, 2023

nursoda commented Mar 20, 2023

nursoda commented Mar 21, 2023 •

edited

Loading

ChristophWurst commented Mar 21, 2023

ChristophWurst commented Mar 21, 2023 •

edited

Loading

[Bug]: HTTP REPORT returns 503 on some CalDAV/CardDAV collections #37277

[Bug]: HTTP REPORT returns 503 on some CalDAV/CardDAV collections #37277

Comments

rfc2822 commented Mar 16, 2023 • edited Loading

⚠️ This issue respects the following points: ⚠️

Bug description

Steps to reproduce

Expected behavior

Installation method

Operating system

PHP engine version

Web server

Database engine version

Is this bug present after an update or on a fresh install?

Are you using the Nextcloud Server Encryption module?

What user-backends are you using?

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

nickvergessen commented Mar 20, 2023

manswiss commented Mar 20, 2023

Der-K-2000 commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

Der-K-2000 commented Mar 20, 2023

manswiss commented Mar 20, 2023

nickvergessen commented Mar 20, 2023

andrelcz commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

manswiss commented Mar 20, 2023 • edited Loading

ChristophWurst commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

manswiss commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023

manswiss commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023 • edited by nickvergessen Loading

manswiss commented Mar 20, 2023 • edited Loading

ChristophWurst commented Mar 20, 2023 • edited Loading

Steps to reproduce

ChristophWurst commented Mar 20, 2023

ChristophWurst commented Mar 20, 2023 • edited Loading

ChristophWurst commented Mar 20, 2023 • edited Loading

nickvergessen commented Mar 20, 2023

nursoda commented Mar 20, 2023

nursoda commented Mar 21, 2023 • edited Loading

ChristophWurst commented Mar 21, 2023

ChristophWurst commented Mar 21, 2023 • edited Loading

rfc2822 commented Mar 16, 2023 •

edited

Loading

manswiss commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023 •

edited by nickvergessen

Loading

manswiss commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023 •

edited

Loading

ChristophWurst commented Mar 20, 2023 •

edited

Loading

nursoda commented Mar 21, 2023 •

edited

Loading

ChristophWurst commented Mar 21, 2023 •

edited

Loading