[Bug]: DnsPinMiddleware needs a way to skip "addDnsPinning()" in disconnected setups #42901

rseabra · 2024-01-17T19:16:58Z

⚠️ This issue respects the following points: ⚠️

This is a bug, not a question or a configuration/webserver/proxy issue.
This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
I agree to follow Nextcloud's Code of Conduct.

Bug description

When one has a Nextcloud setup in a disconnected environment, that only reaches the Internet via http proxy, Nextcloud should skip any measures requiring protocols like ping or DNS to public addresses.

The only way I could upgrade Nextcloud to 28.0.1 was by adding a "return $handler($request, $options);" just after line 119 of lib/private/Http/Client/DnsPinMiddleware.php.

My "hack" is extremely ugly and results in a code integrity check, of course, but only after doing that did I get to have a Nextcloud setup properly working.

I tried to add an if based on $this->config looking for the proxy option but I don't understand PHP well enough to get it to work.

Steps to reproduce

install or upgrade of Nextcloud in a disconnected environment behind an HTTP proxy
suffer nightmare due to DNS resolutions made in the aforementioned php class file.

Expected behavior

Nextcloud should work without major issues in a disconnected environment.

Installation method

Community Manual installation with Archive

Nextcloud Server version

28

Operating system

RHEL/CentOS

PHP engine version

PHP 8.0

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

Default user-backend (database)
LDAP/ Active Directory
SSO - SAML
Other

Configuration report

{
    "system": {
        "has_internet_connection": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "version": "28.0.1.1",
        "dbtype": "mysql",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpsecure": "tls",
        "mail_smtpauth": true,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "appstoreenabled": true,
        "appstoreurl": "https:\/\/apps.nextcloud.com\/api\/v1",
        "updatechecker": false,
        "log_type": "syslog",
        "log_authfailip": true,
        "forcessl": true,
        "proxy": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "Europe\/Lisbon",
        "remember_login_cookie_lifetime": 3600,
        "session_lifetime": 900,
        "session_keepalive": true,
        "loglevel": 2,
        "maintenance": false,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***"
        ],
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "ldapIgnoreNamingRules": false,
        "login_form_autocomplete": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "mysql.utf8mb4": true,
        "csrf.optout": [
            "\/^***REMOVED SENSITIVE VALUE***\/"
        ],
        "default_phone_region": "PT",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "app_install_overwrite": [
            "files_mindmap",
            "announcementcenter",
            "files_trackdownloads",
            "quicknotes"
        ],
        "trashbin_retention_obligation": "7, auto",
        "versions_retention_obligation": "7, auto",
        "tempdirectory": "\/var\/lib\/nextcloud\/data\/tmp",
        "allow_local_address": true
    }
}

List of activated Apps

Prefer not to list, happens with or without app store apps.

Nextcloud Signing status

No response

Nextcloud Logs

The log is regarding 27.1.5 because that's when I decided to deepdive and try to fix it myself, I don't have a log entry in 28.0.1

Jan 17 17:32:00 myhostname Nextcloud[214427]: {"reqId":"ZagPEGTGlk998L48XXjy1AAAAEc","level":3,"time":"2024-01-17T17:32:00+00:00","remoteAddr":"SENSITIVE VALUE","user":"rui","app":"settings","method":"POST","url":"/index.php/setti
ngs/apps/enable","message":"{\"Exception\":\"OCP\\\\Http\\\\Client\\\\LocalServerException\",\"Message\":\"No DNS record found for github.com\",\"Code\":0,\"Trace\":[{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/3rdparty/guzzlehttp
/guzzle/src/PrepareBodyMiddleware.php\",\"line\":35,\"function\":\"OC\\\\Http\\\\Client\\\\{closure}\",\"class\":\"OC\\\\Http\\\\Client\\\\DnsPinMiddleware\",\"type\":\"->\",\"args\":[\"*** sensitive parameters replaced ***\"]},{\"file\":
\"/var/www/html/cloud/nextcloud-27.1.5/3rdparty/guzzlehttp/guzzle/src/Middleware.php\",\"line\":31,\"function\":\"__invoke\",\"class\":\"GuzzleHttp\\\\PrepareBodyMiddleware\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcl
oud-27.1.5/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php\",\"line\":71,\"function\":\"GuzzleHttp\\\\{closure}\",\"class\":\"GuzzleHttp\\\\Middleware\",\"type\":\"::\",\"args\":[\"*** sensitive parameters replaced ***\"]},{\"file\"
:\"/var/www/html/cloud/nextcloud-27.1.5/3rdparty/guzzlehttp/guzzle/src/Middleware.php\",\"line\":63,\"function\":\"__invoke\",\"class\":\"GuzzleHttp\\\\RedirectMiddleware\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextclou
d-27.1.5/3rdparty/guzzlehttp/guzzle/src/HandlerStack.php\",\"line\":75,\"function\":\"GuzzleHttp\\\\{closure}\",\"class\":\"GuzzleHttp\\\\Middleware\",\"type\":\"::\",\"args\":[\"*** sensitive parameters replaced ***\"]},{\"file\":\"/var/
www/html/cloud/nextcloud-27.1.5/3rdparty/guzzlehttp/guzzle/src/Client.php\",\"line\":331,\"function\":\"__invoke\",\"class\":\"GuzzleHttp\\\\HandlerStack\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/3rdparty
/guzzlehttp/guzzle/src/Client.php\",\"line\":168,\"function\":\"transfer\",\"class\":\"GuzzleHttp\\\\Client\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/3rdparty/guzzlehttp/guzzle/src/Client.php\",\"line\":187,\
"function\":\"requestAsync\",\"class\":\"GuzzleHttp\\\\Client\",\"type\":\"->\",\"args\":[\"*** sensitive parameters replaced ***\"]},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/lib/private/Http/Client/Client.php\",\"line\":230,\
"function\":\"request\",\"class\":\"GuzzleHttp\\\\Client\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/lib/private/Installer.php\",\"line\":296,\"function\":\"get\",\"class\":\"OC\\\\Http\\\\Client\\\\Client\",\"
type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/apps/settings/lib/Controller/AppSettingsController.php\",\"line\":448,\"function\":\"downloadApp\",\"class\":\"OC\\\\Installer\",\"type\":\"->\"},{\"file\":\"/var/www/htm
l/cloud/nextcloud-27.1.5/lib/private/AppFramework/Http/Dispatcher.php\",\"line\":230,\"function\":\"enableApps\",\"class\":\"OCA\\\\Settings\\\\Controller\\\\AppSettingsController\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud
/nextcloud-27.1.5/lib/private/AppFramework/Http/Dispatcher.php\",\"line\":137,\"function\":\"executeController\",\"class\":\"OC\\\\AppFramework\\\\Http\\\\Dispatcher\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/
lib/private/AppFramework/App.php\",\"line\":183,\"function\":\"dispatch\",\"class\":\"OC\\\\AppFramework\\\\Http\\\\Dispatcher\",\"type\":\"->\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/lib/private/Route/Router.php\",\"line\"
:315,\"function\":\"main\",\"class\":\"OC\\\\AppFramework\\\\App\",\"type\":\"::\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/lib/base.php\",\"line\":1068,\"function\":\"match\",\"class\":\"OC\\\\Route\\\\Router\",\"type\":\"->
\"},{\"file\":\"/var/www/html/cloud/nextcloud-27.1.5/index.php\",\"line\":38,\"function\":\"handleRequest\",\"class\":\"OC\",\"type\":\"::\"}],\"File\":\"/var/www/html/cloud/nextcloud-27.1.5/lib/private/Http/Client/DnsPinMiddlewar
e.php\",\"Line\":133,\"message\":\"could not enable apps\",\"exception\":{},\"CustomMessage\":\"could not enable apps\"}","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0","version":"27.1.5.1"}

Additional info

No response

The text was updated successfully, but these errors were encountered:

solracsf · 2024-01-17T20:26:20Z

What about:

server/config/config.sample.php

Lines 855 to 860 in 487de9a

    
           /** 
        
            * Is Nextcloud connected to the Internet or running in a closed network? 
        
            * 
        
            * Defaults to ``true`` 
        
            */ 
        
           'has_internet_connection' => true,

rseabra · 2024-01-17T21:37:28Z

If you look at the file I mentioned, and in particular lines

server/lib/private/Http/Client/DnsPinMiddleware.php

Lines 119 to 120 in 487de9a

    
           ) use ($handler) { 
        
           	if ($options['nextcloud']['allow_local_address'] === true) {

you'll see that it doesn't matter what value you have in has_internet_connection :)

I tried defining allow_local_address => true, but it didn't appear to work either... :(

rseabra · 2024-01-17T21:56:54Z

@szaimen I believe this problem is present since at least somewhere on 25, it's not just a 28 thing, as I definitely got it when I upgraded to 25.0.13

kesselb · 2024-01-18T14:20:38Z

you'll see that it doesn't matter what value you have in has_internet_connection :)

has_internet_connection = true will prevent outgoing requests and the mentioned code is not executed.

allow_local_address => true

server/config/config.sample.php

Lines 708 to 713 in 1334055

    
           /** 
        
            * Allow remote servers with local addresses e.g. in federated shares, webcal services and more 
        
            * 
        
            * Defaults to false 
        
            */ 
        
           'allow_local_remote_servers' => true,

Yet, I don't see how that should help in your case.

When one has a Nextcloud setup in a disconnected environment, that only reaches the Internet via http proxy, Nextcloud should skip any measures requiring protocols like ping or DNS to public addresses.

#40108
#41981

rseabra · 2024-01-18T20:59:57Z

Sigh ... documentation of fix for less experienced people who find this bug...

The configuration one needs to add is...

'dns_pinning' => false,

This should be documented, it's perfectly normal to have a private servers behind http proxies that can't do DNS resolutions of public addresses.

rseabra added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Jan 17, 2024

szaimen added the 28-feedback label Jan 17, 2024

kesselb closed this as completed Jan 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: DnsPinMiddleware needs a way to skip "addDnsPinning()" in disconnected setups #42901

[Bug]: DnsPinMiddleware needs a way to skip "addDnsPinning()" in disconnected setups #42901

rseabra commented Jan 17, 2024 •

edited

solracsf commented Jan 17, 2024

rseabra commented Jan 17, 2024 •

edited

rseabra commented Jan 17, 2024

kesselb commented Jan 18, 2024

rseabra commented Jan 18, 2024 •

edited

[Bug]: DnsPinMiddleware needs a way to skip "addDnsPinning()" in disconnected setups #42901

[Bug]: DnsPinMiddleware needs a way to skip "addDnsPinning()" in disconnected setups #42901

Comments

rseabra commented Jan 17, 2024 • edited

⚠️ This issue respects the following points: ⚠️

Bug description

Steps to reproduce

Expected behavior

Installation method

Nextcloud Server version

Operating system

PHP engine version

Web server

Database engine version

Is this bug present after an update or on a fresh install?

Are you using the Nextcloud Server Encryption module?

What user-backends are you using?

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

solracsf commented Jan 17, 2024

rseabra commented Jan 17, 2024 • edited

rseabra commented Jan 17, 2024

kesselb commented Jan 18, 2024

rseabra commented Jan 18, 2024 • edited

rseabra commented Jan 17, 2024 •

edited

rseabra commented Jan 17, 2024 •

edited

rseabra commented Jan 18, 2024 •

edited