Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elaborate on Service Discovery troubleshooting #11113

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Elaborate on Service Discovery troubleshooting #11113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
335aa15
Elaborate on Service Discovery troubleshooting
martin-rueegg Sep 12, 2023
c783cad
Requested changes 1
martin-rueegg Sep 15, 2023
c3919bf
Requested changes 2: typos
martin-rueegg Sep 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
32 changes: 26 additions & 6 deletions admin_manual/installation/nginx.rst
View file
Open in desktop
Expand Up @@ -8,11 +8,11 @@ NGINX configuration
Please note that webservers other than Apache 2.x are not officially supported.

.. note::
This page covers example NGINX configurations to run a Nextcloud server.
This page covers example nginx configurations to run a Nextcloud server.
These configurations examples were originally provided by `@josh4trunks <https://github.com/josh4trunks>`_
and are exclusively community-maintained. (Thank you contributors!)

- You need to insert the following code into **your Nginx configuration file**. Choose the appropriate example based on whether you are deploying :ref:`nginx_webroot_example` (i.e. :code:`https://cloud.example.com/`) or :ref:`nginx_subdir_example` (i.e. :code:`https://cloud.example.com/nextcloud`).
- You need to insert the following code into **your nginx configuration file**. Choose the appropriate example based on whether you are deploying :ref:`nginx_webroot_example` (i.e. :code:`https://cloud.example.com/`) or :ref:`nginx_subdir_example` (i.e. :code:`https://cloud.example.com/nextcloud`).
- Adjust the server directive under :code:`upstream php-handler` to match your PHP installation's configured FPM listener (a misconfiguration here will result in a :code:`502 Bad Gateway` - see :ref:`nginx_php_handler_tips` for details)
- Adjust the existing :code:`server_name` directives found under *both* :code:`server` sections to your real hostname
- Adjust :code:`root` to the webroot of your Nextcloud installation
Expand All @@ -22,7 +22,7 @@ NGINX configuration
- Be careful about line breaks if you copy the examples, as long lines may be
broken for page display and result in an invalid configuration files.
- Some environments might need a ``cgi.fix_pathinfo`` set to ``1`` in their
``php.ini``.
``php.ini``.

.. _nginx_webroot_example:

Expand Down Expand Up @@ -74,9 +74,9 @@ Look for the line that is set to something like:
or
:code:`listen = 127.0.0.1:9000`

If PHP FPM will be running on the same host as NGINX (it's probably a safe assumption it will be if you're unsure), it is recommended you use the UNIX socket (i.e. :code:`/var/run/php/php-fpm.sock`) rather than TCP (:code:`127.0.0.1:9000`) for maximum performance (though either will work as long as your NGINX and PHP FPM configurations match).
If PHP FPM will be running on the same host as nginx (it's probably a safe assumption it will be if you're unsure), it is recommended you use the UNIX socket (i.e. :code:`/var/run/php/php-fpm.sock`) rather than TCP (:code:`127.0.0.1:9000`) for maximum performance (though either will work as long as your nginx and PHP FPM configurations match).

After deciding how you'd prefer to connect NGINX with PHP FPM (and, if necessary, updating your local PHP FPM configuration and restarting FPM), set your NGINX configuration's :code:`upstream php-handler` :code:`server` to match your preference (Note: If using UNIX sockets, prepend :code:`unix:` in the NGINX configuration, but *not* in your PHP FPM :code:`www.conf`).
After deciding how you'd prefer to connect nginx with PHP FPM (and, if necessary, updating your local PHP FPM configuration and restarting FPM), set your nginx configuration's :code:`upstream php-handler` :code:`server` to match your preference (Note: If using UNIX sockets, prepend :code:`unix:` in the nginx configuration, but *not* in your PHP FPM :code:`www.conf`).

Suppressing log messages
^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -123,7 +123,7 @@ Upload of files greater than 10 MiB fails

If you configure nginx (globally) to block all requests to (hidden) dot files,
it may be not possible to upload files greater than 10 MiB using the webpage
due to Nextclouds requirement to upload the file to an url ending with ``/.file``.
due to Nextcloud's requirement to upload the file to an url ending with ``/.file``.

You may require to change:

Expand Down Expand Up @@ -157,3 +157,23 @@ If you just see some correct requests in access log, but no login happens, you c
chown nginx:nginx /var/lib/php/session/
chown root:nginx /var/lib/php/wsdlcache/
chown root:nginx /var/lib/php/opcache/

Troubleshooting
---------------

Service Discovery (aka ``.well-known``)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Command line analysis:
To get an example ``curl`` command, check the :doc:`Troubleshooting Guide <../issues/general_troubleshooting>`
for tips on how to verify the correct redirection of ``/.well-known`` services by consulting the **Tip** boxes
provided in the respective sub-sections of it's :ref:`service-discovery-label` section.

Custom 404 error page:
Please note, that using a custom error page for status code ``404`` (along the lines of ``error_page 404 = @error404;``)
prevents the http header ``x-nextcloud-well-known`` beeing returned from ``index.php``. As a result, the Overview page
in Administration Settings will show a warning, that the endpoints for
:ref:`webfinger and hostinfo <service-discovery-webfinger-and-nodeinfo-label>` are not correctly resolved,
although in fact they may well be. Please use the above mentioned **Command line analysis** and
Comment on lines +176 to +177
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:ref:`webfinger and hostinfo <service-discovery-webfinger-and-nodeinfo-label>` are not correctly resolved,
although in fact they may well be. Please use the above mentioned **Command line analysis** and
:ref:`webfinger and hostinfo <service-discovery-webfinger-and-nodeinfo-label>` are not correctly resolved. Please use the above mentioned **Command line analysis** and

You cannot use a custom error handler with Nextcloud.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your suggestion.

The whole point of the test with curl command line is to find out the real cause of the problem. In my case, I've spent hours in checking redirection, as the warning message suggests there is something wrong with the redirection. Where in fact, the error resulted because a custom error page was configured in the web server for 404 status code. This custom error page removed the required header from the response, which is the actual reason why the test in admin settings overview failed. I.e. it was not the redirection.

So with the sentence above, I try to hint to the possibility that the redirection might be working, yet due to the removal of the header, the test fails. The curl command line analyses does actually test for that header.

As such, it would be counterproductive to remove the part "although in fact they may well be", as it is not enough to just resolve the endpoints correctly.

:ref:`service-discovery-default-well-known-configuration-label` to verify if the header
is forwarded to the client correctly.
234 changes: 214 additions & 20 deletions admin_manual/issues/general_troubleshooting.rst
View file
Open in desktop
Expand Up @@ -202,7 +202,7 @@ these modules:
* mod_xsendfile / X-Sendfile (causing broken downloads if not configured
correctly)

2. NginX
2. NGINX

* ngx_pagespeed
* HttpDavModule
Expand Down Expand Up @@ -250,30 +250,149 @@ Service discovery
Some clients - especially on iOS/macOS - have problems finding the proper
sync URL, even when explicitly configured to use it.

However, there are also several techniques to allow clients to discover services provided by a server. One of these,
which is addressed in this section, is the discovery through *well known* URLs. Further remedies are described extensively
at the `Sabre DAV website <http://sabre.io/dav/service-discovery/>`_

These *well known* URLs are located under the web root driectory of your webserver, whether or not your nextcloud
installation is installed in the root or a subfolder of your web server:

| ``https://example.com/.well-known/``
|

To check if these *well known* URLs are working correctly on your server, you can go to the ``Overview`` page of the
``Administrative Settings`` of your Nextcloud installation, e.g. at ``https://example.com/settings/admin/overview``
and see if there is any warning shown under the ``Security & setup warnings`` section. Additionlly, the specific sections
below show how to verify the correct configuration from a linux command-line (see *Tip* boxes).

.. _service-discovery-caldav-and-carddav-label:

CalDAV and CardDAV
^^^^^^^^^^^^^^^^^^

If you want to use CalDAV or CardDAV clients or other clients that require service discovery
together with Nextcloud it is important to have a correct working setup of the following
URLs:

| ``https://example.com/.well-known/carddav``
| ``https://example.com/.well-known/caldav``
|

Those need to be redirecting your clients to the correct endpoints. If Nextcloud
is running at the document root of your Web server the correct URL is
``https://example.com/remote.php/dav`` for CardDAV and CalDAV and if running in a
subfolder like ``nextcloud``, then ``https://example.com/nextcloud/remote.php/dav``.
Those need to be redirecting your clients to the correct endpoints:

- ``https://example.com/remote.php/dav`` if Nextcloud is running at the document root of your Web server, and
- ``https://example.com/nextcloud/remote.php/dav`` if running in a subfolder like ``nextcloud``.

.. Tip::
To check if your server is set up correctly, use the following command with your correct URL for ``YOUR_SERVER``:

**CalDAV** ::

YOUR_SERVER=https://example.com
SERVICE=caldav
curl -Sv --fail -o /dev/null --GET $YOUR_SERVER/.well-known/$SERVICE 2>&1 \
| grep -iE '^(> get|< (HTTP/|location))'

This should return an output similar to this::

> GET /.well-known/carddav HTTP/2
< HTTP/2 301
< location: https://example.com/remote.php/dav/

**CardDav** ::

YOUR_SERVER=https://example.com
SERVICE=carddav
curl -Sv --fail -o /dev/null --GET $YOUR_SERVER/.well-known/$SERVICE 2>&1 \
| grep -iE '^(> get|< (HTTP/|location))'

This should return an output similar to this::

> GET /.well-known/carddav HTTP/2
< HTTP/2 301
< location: https://example.com/remote.php/dav/

.. _service-discovery-webfinger-and-nodeinfo-label:

WEBFINGER and NODEINFO (and others)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Since version 21 of Nextcloud server, these two endpoints

| ``https://example.com/.well-known/webfinger``
| ``https://example.com/.well-known/nodeinfo``

need to be redirected to the ``index.php`` like so:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As explained above, this is misleading because it's different from our default implementation. webfinger and nodeinfo however are fine with a chain of redirects.


| ``https://example.com/index.php/.well-known/webfinger``
| ``https://example.com/index.php/.well-known/nodeinfo``

or, in case of using Nextcloud in a subfolder like ``nextcloud``:

| ``https://example.com/nextcloud/index.php/.well-known/webfinger``
| ``https://example.com/nextcloud/index.php/.well-known/nodeinfo``

.. note::
For **versions older than v21** of Nextcloud, these URLs needed to be redirected to a different target.
Please consult the Admin Manual of the respective version.

.. Tip::
To check if your server is set up correctly, use the following command with your correct URL for ``YOUR_SERVER``.

**Note:** The return code to the second ``GET`` could also be ``404``, rather than ``200``. Important is that the last line contains ``x-nextcloud-well-known: 1``

**WEBFINGER** ::

YOUR_SERVER=https://example.com
SERVICE=webfinger
curl -SLv --fail -o /dev/null --GET $YOUR_SERVER/.well-known/$SERVICE 2>&1 \
| grep -iE '^(> get|< (HTTP/|location|x-nextcloud-well-known))'

This should return an output similar to this::

> GET /.well-known/webfinger HTTP/2
< HTTP/2 302
< location: https://example.com/index.php/.well-known/webfinger
> GET /index.php/.well-known/webfinger HTTP/2
< HTTP/2 200
< x-nextcloud-well-known: 1

For the first case the :file:`.htaccess` file shipped with Nextcloud should do
this work for you when you're running Apache. You need to make sure that your
Web server is using this file. Additionally, you need the mod_rewrite Apache
module installed and ``AllowOverride All`` set in your :file:`apache2.conf`
or vHost-file to process these redirects. When running Nginx please refer to
:doc:`../installation/nginx`.
**NODEINFO** ::

YOUR_SERVER=https://example.com
SERVICE=nodeinfo
curl -SLv --fail -o /dev/null --GET $YOUR_SERVER/.well-known/$SERVICE 2>&1 \
| grep -iE '^(> get|< (HTTP/|location|x-nextcloud-well-known))'

If your Nextcloud instance is installed in a subfolder called ``nextcloud`` and
you're running Apache, create or edit the :file:`.htaccess` file within the
document root of your Web server and add the following lines::
This should return an output similar to this::

> GET /.well-known/nodeinfo HTTP/2
< HTTP/2 302
< location: https://example.com/index.php/.well-known/nodeinfo
> GET /index.php/.well-known/nodeinfo HTTP/2
< HTTP/2 200
< x-nextcloud-well-known: 1


.. _service-discovery-apache-configuration-label:

Apache Web Server Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The configuration differs depending on whether your Nextcloud instance is running in your Web server's root or in a subfolder.

Running Nextcloud in the webroot folder
"""""""""""""""""""""""""""""""""""""""
- The :file:`.htaccess` file shipped with Nextcloud should do this work for you.
- You need to make sure that your Web server is using this file.
- Additionally, you need the mod_rewrite Apache
module installed and ``AllowOverride All`` set in your :file:`apache2.conf`
or vHost-file to process these redirects.


Running Nextcloud in a subfolder called ``nextcloud``
"""""""""""""""""""""""""""""""""""""""""""""""""""""

- Create or edit the :file:`.htaccess` file within the document root of your Web server and add the following lines::

<IfModule mod_rewrite.c>
RewriteEngine on
Expand All @@ -283,7 +402,8 @@ document root of your Web server and add the following lines::
RewriteRule ^\.well-known/nodeinfo /nextcloud/index.php/.well-known/nodeinfo [R=301,L]
</IfModule>

Make sure to change /nextcloud to the actual subfolder your Nextcloud instance is running in.
- Make sure to change ``/nextcloud`` to the actual subfolder your Nextcloud instance is running in.


.. note:: If you put the above directives directly into an Apache
configuration file (usually within ``/etc/apache2/``)
Expand All @@ -294,7 +414,84 @@ Make sure to change /nextcloud to the actual subfolder your Nextcloud instance i
files by dropping any number of leading slashes, but it does not
do so for the use in its main configuration files.

If you are running NGINX, make sure ``location = /.well-known/carddav {`` and ``location = /.well-known/caldav {`` are properly configured as described in :doc:`../installation/nginx`, adapt to use a subfolder if necessary.
.. _service-discovery-default-well-known-configuration-label:

Default handling of ``/.well-known/`` URLs by Nextcloud
"""""""""""""""""""""""""""""""""""""""""""""""""""""""

Since Nextcloud allows apps to register a service within the ``.well-known`` directory, you may want to redirect
any request to this folder to Nextcloud.

.. warning::
**Make sure you exclude** *well known* **services that are handled by other
applications,** e.g. you should *not* redirect ``/.well-known/acme-challenge`` or ``/.well-known/pki-validation/`` as they
are used for automated SSL certificate deployment.

Also, make sure to add your ``/nextcloud/subfolder``, if your Nextcloud instance is not running in the webroot.

::

<IfModule mod_rewrite.c>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This configuration is done by our .htaccess file.
We should not duplicate it here without a good reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why I mentioned # the standard redirection rules for Nextcloud.

How about my suggested alternative:

      RewriteEngine on
      # The standard redirection rules for Nextcloud; they should already in your .htaccess file and
      # are here just for reference:
      #
      # RewriteRule ^/\.well-known/carddav /remote.php/dav [R=301,L]
      # RewriteRule ^/\.well-known/caldav /remote.php/dav [R=301,L]
      # RewriteRule ^/\.well-known/webfinger /index.php/.well-known/webfinger [R=301,L]
      # RewriteRule ^/\.well-known/nodeinfo /index.php/.well-known/nodeinfo [R=301,L]
      ...

Also note, that I've left the two redirects for webfinger and nodeinfo as they are configured with a 301 permanent redirect, while the general rule below uses a 302 redirect for the reason indicated in the example's comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan. At worst, it's outdated in a few weeks.
Please remove the actual rewrite rules.
Feel free to add a link to our htaccess file.

RewriteEngine on
# The standard redirection rules for Nextcloud; they should already in your .htaccess file and
# are here just for reference:
#
# RewriteRule ^\.well-known/carddav /remote.php/dav [R=301,L]
# RewriteRule ^\.well-known/caldav /remote.php/dav [R=301,L]
# RewriteRule ^\.well-known/webfinger /index.php/.well-known/webfinger [R=301,L]
# RewriteRule ^\.well-known/nodeinfo /index.php/.well-known/nodeinfo [R=301,L]

# Redirect all other requests to /.well-known/ to index.php
# exclude /.well-known/acme-challenge/
RewriteCond "%{REQUEST_URI}" "!^/.well-known/acme-challenge/" [OR]
# exclude /.well-known/pki-validation/
RewriteCond "%{REQUEST_URI}" "!^/.well-known/pki-validation/" [OR]
# add additional excludes, as required by your server setup
# Use a temporary redirect here (302) in case an future service is handled outside of Nextcloud
RewriteRule ^\.well-known/ /index.php/ [R=302,L]
</IfModule>


.. Tip::
To check if your server is set up correctly, use the following command with your correct URL for ``YOUR_SERVER``:

**Any Service** ::

YOUR_SERVER=https://example.com
SERVICE=some-non-existing-service
curl -SLv --fail -o /dev/null --GET $YOUR_SERVER/.well-known/$SERVICE 2>&1 \
| grep -iE '^(> get|< (HTTP/|location|x-nextcloud-well-known))'

This should return an output similar to this::

> GET /.well-known/some-non-existing-service HTTP/2
< HTTP/2 302
< location: https://example.com/index.php/.well-known/some-non-existing-service
> GET /index.php/.well-known/some-non-existing-service HTTP/2
< HTTP/2 404
< x-nextcloud-well-known: 1

**Excluded Service** ::

YOUR_SERVER=https://example.com
SERVICE=acme-challenge/some-file
curl -SLv --fail -o /dev/null --GET $YOUR_SERVER/.well-known/$SERVICE 2>&1 \
| grep -iE '^(> get|< (HTTP/|location|x-nextcloud-well-known))'

This should return an output similar to this (note: no Location header is returned)::

> GET /.well-known/acme-challenge/some-file HTTP/2
< HTTP/2 404


NGINX Web Server Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Make sure ``location = /.well-known/carddav {`` and ``location = /.well-known/caldav {`` are properly configured as
described in :ref:`NGINX configuration <nginx_webroot_example>`, adapt to use a :ref:`subfolder <nginx_subdir_example>` if necessary.

Client Configuration
^^^^^^^^^^^^^^^^^^^^

Now change the URL in the client settings to just use:

Expand All @@ -304,9 +501,6 @@ instead of e.g.

``https://example.com/nextcloud/remote.php/dav/principals/username``.

There are also several techniques to remedy this, which are described extensively at
the `Sabre DAV website <http://sabre.io/dav/service-discovery/>`_.

Troubleshooting sharing
-----------------------------------

Expand Down