New calendar subscription throwing 503 Service Unavailable. User agent problems? #4232

starlingfire · 2022-05-19T18:54:25Z

Steps to reproduce

Open the calendar app
Click on "+ New calendar" -> "New subscription from link (read-only)"
Enter a valid URL to an ICS calendar file. I can DM my calendar if needed.

Expected behavior

The calendar is added. When the cronjob runs, it fetches the calendar data from the remote URL. Events show up in the calendar app

Actual behaviour

The calendar is added. When the cronjob runs, the calendar app stays empty.

This issue was already raised and closed in #3546.

The following exception is logged:

{
    "reqId": "Jat0HkX6MM3BdcR2jhwc",
    "level": 2,
    "time": "2022-05-19T19:11:25+02:00",
    "remoteAddr": "192.168.1.1",
    "user": "user",
    "app": "dav",
    "method": "MKCOL",
    "url": "/remote.php/dav/calendars/user/testcalendar",
    "message": "Subscription 25 could not be refreshed due to a network error",
    "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0",
    "version": "23.0.4.1",
    "exception": {
        "Exception": "GuzzleHttp\\Exception\\ServerException",
        "Message": "Server error: `GET https://CALENDAR_REDACTED` resulted in a `503 Service Unavailable` response",
        "Code": 503,
        "Trace": [
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php",
                "line": 69,
                "function": "create",
                "class": "GuzzleHttp\\Exception\\RequestException",
                "type": "::"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 204,
                "function": "GuzzleHttp\\{closure}",
                "class": "GuzzleHttp\\Middleware",
                "type": "::",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 153,
                "function": "callHandler",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "::"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php",
                "line": 48,
                "function": "GuzzleHttp\\Promise\\{closure}",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "::",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 248,
                "function": "run",
                "class": "GuzzleHttp\\Promise\\TaskQueue",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 224,
                "function": "invokeWaitFn",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 269,
                "function": "waitIfPending",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 226,
                "function": "invokeWaitList",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php",
                "line": 62,
                "function": "waitIfPending",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php",
                "line": 187,
                "function": "wait",
                "class": "GuzzleHttp\\Promise\\Promise",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/lib/private/Http/Client/Client.php",
                "line": 223,
                "function": "request",
                "class": "GuzzleHttp\\Client",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/lib/CalDAV/WebcalCaching/RefreshWebcalService.php",
                "line": 240,
                "function": "get",
                "class": "OC\\Http\\Client\\Client",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/lib/CalDAV/WebcalCaching/RefreshWebcalService.php",
                "line": 101,
                "function": "queryWebcalFeed",
                "class": "OCA\\DAV\\CalDAV\\WebcalCaching\\RefreshWebcalService",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/lib/AppInfo/Application.php",
                "line": 287,
                "function": "refreshSubscription",
                "class": "OCA\\DAV\\CalDAV\\WebcalCaching\\RefreshWebcalService",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
                "line": 264,
                "function": "OCA\\DAV\\AppInfo\\{closure}",
                "class": "OCA\\DAV\\AppInfo\\Application",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
                "line": 239,
                "function": "doDispatch",
                "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
                "line": 73,
                "function": "callListeners",
                "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/lib/private/EventDispatcher/SymfonyAdapter.php",
                "line": 123,
                "function": "dispatch",
                "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php",
                "line": 2483,
                "function": "dispatch",
                "class": "OC\\EventDispatcher\\SymfonyAdapter",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/CalendarHome.php",
                "line": 249,
                "function": "createSubscription",
                "class": "OCA\\DAV\\CalDAV\\CalDavBackend",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/lib/CalDAV/CalendarHome.php",
                "line": 88,
                "function": "createExtendedCollection",
                "class": "Sabre\\CalDAV\\CalendarHome",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 1199,
                "function": "createExtendedCollection",
                "class": "OCA\\DAV\\CalDAV\\CalendarHome",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
                "line": 560,
                "function": "createCollection",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
                "line": 89,
                "function": "httpMkcol",
                "class": "Sabre\\DAV\\CorePlugin",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 472,
                "function": "emit",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 253,
                "function": "invokeMethod",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 321,
                "function": "start",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/lib/Server.php",
                "line": 339,
                "function": "exec",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php",
                "line": 35,
                "function": "exec",
                "class": "OCA\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/remote.php",
                "line": 166,
                "args": [
                    "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"
                ],
                "function": "require_once"
            }
        ],
        "File": "/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Exception/RequestException.php",
        "Line": 113,
        "CustomMessage": "Subscription 25 could not be refreshed due to a network error"
    }
}

Calendar app version

3.3.1

CalDAV-clients used

No response

Browser

Firefox 99.0.1 (64-bit)

Client operating system

Solus Linux

Server operating system

Debian Linux

Web server

Nginx

Database engine version

MariaDB

PHP engine version

PHP 8.0

Nextcloud version

23.0.4

Updated from an older installed version or fresh install

Updated from an older version

List of activated apps

No response

Nextcloud configuration

No response

Web server error log

No response

Log file

No response

Browser log

No response

Additional info

I'm experiencing this same issue: I cannot add a calendar due to a 503 error. After some playing around, it looks like the user agent "Nextcloud Webcal Crawler" is causing the problem.

I have confirmed that if I use a user agent switcher and set it to "Nextcloud Webcal Crawler," I cannot download the ICS file. The same is true when using wget:

$ wget --header="Accept-Encoding: text/calendar, application/calendar+json, application/calendar+xml" -U 'Nextcloud Webcal Crawler' https://URL_REDACTED
HTTP request sent, awaiting response... 503 Service Unavailable

If I remove the user agent flag, it works

wget --header="Accept-Encoding: text/calendar, application/calendar+json, application/calendar+xml"  https://URL_REDACTED
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/calendar]
Saving to: file

The user agent is hard-coded in https://github.com/nextcloud/server/blob/0bceaee444c61a613c636e8bd3579c19764f0c66/apps/dav/lib/CalDAV/WebcalCaching/RefreshWebcalService.php#L195

Is there anything I can configure on my end to make this work, or is modifying the php the only way?

The text was updated successfully, but these errors were encountered:

ChristophWurst · 2022-05-20T09:45:56Z

This issue was already raised and closed in #3546.

No. That ticket was about redirects. Your URL points to a service that responds with a HTTP503 error.

ChristophWurst · 2022-05-20T09:47:36Z

Could you check with the solution/service on the other hand and find out why it errors for our user agent? Does it consider it invalid perhaps?

tcitworld · 2022-05-20T10:16:05Z

If you find an acceptable user-agent that still contains Nextcloud Webcal Crawler, we could use it.

starlingfire · 2022-05-20T12:28:16Z

I was testing different strings, and it looks like the server returns a 503 if the word "crawler" appears anywhere in it. Unless it's easy enough to make that value configurable, I might reach out to the site admin to see if they can add an exception for Nextcloud.

> This issue was already raised and closed in #3546. No. That ticket was about redirects. Your URL points to a service that responds with a HTTP503 error.

True.

tcitworld · 2022-05-20T13:04:59Z

We could indeed have a different user agent, like Nextcloud Webcal Service.

We could even expose some other information like some software often do %InstanceName% Webcal Service (Nextcloud/25.0.1; +https://nextcloud-instance.domain), but some of that might be sensitive.

starlingfire · 2022-05-20T13:11:16Z

I just tested Nextcloud Webcal Service and that worked perfectly. It would certainly solve my use case. I don't have a strong opinion either way on including the %InstanceName%.

Some services don't like the "crawler" inside the previous user-agent. Closes nextcloud/calendar#4232 Signed-off-by: Thomas Citharel <tcit@tcit.fr>

starlingfire added 0. to triage Pending approval or rejection bug labels May 19, 2022

tcitworld added a commit to nextcloud/server that referenced this issue May 20, 2022

Rename user-agent used by the Webcal Refresh Service

424fc1b

Some services don't like the "crawler" inside the previous user-agent. Closes nextcloud/calendar#4232 Signed-off-by: Thomas Citharel <tcit@tcit.fr>

tcitworld mentioned this issue May 20, 2022

Rename user-agent used by the Webcal Refresh Service nextcloud/server#32519

Merged

ChristophWurst closed this as completed in nextcloud/server#32519 May 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New calendar subscription throwing 503 Service Unavailable. User agent problems? #4232

New calendar subscription throwing 503 Service Unavailable. User agent problems? #4232

starlingfire commented May 19, 2022

ChristophWurst commented May 20, 2022

ChristophWurst commented May 20, 2022

tcitworld commented May 20, 2022

starlingfire commented May 20, 2022 via email •

edited

Loading

tcitworld commented May 20, 2022

starlingfire commented May 20, 2022

New calendar subscription throwing 503 Service Unavailable. User agent problems? #4232

New calendar subscription throwing 503 Service Unavailable. User agent problems? #4232

Comments

starlingfire commented May 19, 2022

Steps to reproduce

Expected behavior

Actual behaviour

Calendar app version

CalDAV-clients used

Browser

Client operating system

Server operating system

Web server

Database engine version

PHP engine version

Nextcloud version

Updated from an older installed version or fresh install

List of activated apps

Nextcloud configuration

Web server error log

Log file

Browser log

Additional info

ChristophWurst commented May 20, 2022

ChristophWurst commented May 20, 2022

tcitworld commented May 20, 2022

starlingfire commented May 20, 2022 via email • edited Loading

tcitworld commented May 20, 2022

starlingfire commented May 20, 2022

starlingfire commented May 20, 2022 via email •

edited

Loading