Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest nightly doesn't work with TLS well? #133

Open
treese opened this issue Oct 4, 2017 · 61 comments

Comments

Projects
None yet
@treese
Copy link
Collaborator

commented Oct 4, 2017

I recently updated to the latest development version. If I try to open an https URL with the latest nightly, I get a set of messages like this (in this case, from running list-packages:

Opening TLS connection to ‘melpa.org’...
Opening TLS connection with ‘gnutls-cli --x509cafile nil -p 443 melpa.org’...failed
Opening TLS connection with ‘gnutls-cli --x509cafile nil -p 443 melpa.org --protocols ssl3’...failed
Opening TLS connection to ‘melpa.org’...failed
Failed to download ‘melpa’ archive.

Nothing else has changed about the system, although I suppose there could be some odd version skew with gnutls-cli. I currently have gnutls 3.5.15, which is the latest release on the stable branch.

Has anyone else seen this problem?

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 17, 2017

I figured out the problem. Perhaps no one else has it, but I'll record the solution here in case someone does:

  1. Current stable Aquamacs has an older Emacs setup for making TLS connections, which can use openssl as an external program.

  2. Emacs was asked to stop using openssl in that way because the intent for the s_client subcommand was just for testing.

  3. Emacs 25.3 removed openssl as an external stream connection for TLS.

  4. On my Mac, at least, openssl was being used because gnutls-cli was failing.

  5. Running gnutls-cli from the command line revealed that it did not recognize the issuer for the Let's Encrypt certificate used by MELPA.

  6. The solution (described in How can I retrieve an HTTPS URL on Mac OS X without warnings about an untrusted authority?) on StackExchange is as follows:

    $ brew install libressl
    $ echo $(brew --prefix)/etc/libressl/cert.pem
    /usr/local/etc/libressl/cert.pem

Take the result and add to emacs config something like:

(push "/usr/local/etc/libressl/cert.pem" gnutls-trustfiles)
  1. Enjoy.

Note: you might want to do something like:

(require 'tls)

or

(with-eval-after-load 'tls
    (push "/usr/local/etc/libressl/cert.pem" gnutls-trustfiles))

so that gnutls-trustfiles is defined when you update it.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Oct 17, 2017

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 19, 2017

That's a great question. Here are some quick thoughts for brainstorming; I'd be happy to run more down about them depending on what looks most plausible to start:

  1. Build Aquamacs to compile in the GNU TLS library. The library would have to ship with Aquamacs. We would have to figure out a way of setting the certificate store properly. [If this one is possible, I think it's the best way forward that stays close to the rest of Emacs.]

  2. Build and package gnutls-cli with the Aquamacs distribution.

  3. Advise users about installing gnutls if they want to use secure web connections from Aquamacs. The problem with this is that the package configuration for repositories uses HTTPS for MELPA, so that configuration would be probably need to be changed. As far as I can tell, MELPA requires HTTPS, so it makes for an immediate roadblock for someone unless MELPA is not part of the default Aquamacs configuration.

  4. Write a library wrapper for OpenSSL for Emacs like the GNU TLS one, and ship Aquamacs with it. It is very unlikely that upstream Emacs would want such a thing, but the Mac ships with the core OpenSSL libraries.

  5. Write a sufficiently good replacement command line program for the "openssl s_client" code, and ship a binary for that with Aquamacs.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Oct 20, 2017

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 20, 2017

I'll see if I can build Aquamacs on my system and see if that works.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Jul 23, 2018

Any update on this?

@GeorgeCo

This comment has been minimized.

Copy link

commented Aug 11, 2018

I'm running into this also, using Aquamacs 3.4. I was able to use the workarounds above, i.e.

$ brew install libressl
$ echo $(brew --prefix)/etc/libressl/cert.pem
/usr/local/etc/libressl/cert.pem

then add the following to my .emacs file

(require 'tls)
(push "/usr/local/etc/libressl/cert.pem" gnutls-trustfiles)

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 11, 2018

@GeorgeCo

This comment has been minimized.

Copy link

commented Aug 11, 2018

agreed, we need a fix

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 12, 2018

It still seems that none of the options are easy to do without asking something of Aquamacs users.

One way to approach it might be to figure out how to address this with Aquamacs based on Emacs 26, and triangulate from there. As I understand it, gnutls is required for building Emacs 26 unless you explicitly disable it. Having built-in support would make this problem go away, except possibly for a configuration step which could be added in Aquamacs. The problem then, I think, would be that the gnutls shared library would need to be shipped with Aquamacs. Is that a plausible thing to do?

If we took that approach, we could probably do approximately the same thing with the current Aquamacs.

When I looked into whether or not we could build Aquamacs with a statically-linked gnutls library, it did not seem possible. But I could be wrong.

@rpgoldman

This comment has been minimized.

Copy link
Collaborator

commented Aug 12, 2018

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 12, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 12, 2018

Is there anything similar in the Aquamacs build as an example? Otherwise, what I would imagine is tracking gnutls released versions from Gitlab as a git submodule, building a private copy to install into the application bundle, and add the relevant default configuration and any needed certificates.

This would probably work pretty well both as an update to 3.4 if you wanted, as well as with the Emacs 26 merge.

Does that sound like the right approach?

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 12, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 13, 2018

So after messing around with build configuration for a while, I discovered that the autoconf config.ac for gnutls says:

*** GnuTLS will be build as a static library. That means that library
*** constructors for gnutls_global_init will not be made available to
*** linking applications. If you are building that library for arbitrary
*** applications to link, do not enable static linking.

Also, it turns out that gnutls depends on two other libraries (nettle and gmp), which is adding to the complexity.

Thoughts?

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 13, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 13, 2018

Sorry, I wasn't clear. The gnutls project says "don't do it" for using a static version of the library. We could try going ahead, but it seems dicey. I've been working with homebrew to build it, and we have to hack the recipe to get the static library. I can try to finish the build to see how well it works.

Fortunately it looks like nettle and gmp have the static library built by default in homebrew.

I was puzzling over the right way to add a static library to the Aquamacs build: is it easy to point me in the right direction for that?

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 13, 2018

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 14, 2018

I have installed the libraries on the build server, under /usr/local/Cellar/gnutls. I'm mildly hopeful it's possible. However, we are actively preventing dynamic binding as it seems (see build.sh), so I'm not sure it sees the libraries.

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 14, 2018

David, since I'm fussing around with subtle points of the libraries: it appears to me on my system that the gcc invoked is the one from Xcode, which looks like it is actually clang. Is that how you build it? Or do you have real gcc installed?

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 14, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 14, 2018

So while things were building, I happened to take a look at how Emacs for Mac OSX does it. It controls the build of the libraries with homebrew, and then copies the shared libraries into the application bundle with appropriate path fixups.

This seems like a rather cleaner solution, and would track gnutls well via homebrew if the updates are timely. I don't think it would be hard to implement in the Aquamacs build process.

The relevant build script is at build-emacs-from-tar if you want to take a look.

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 14, 2018

The main thing standing in the way now, I think, is building gnutls and the related homebrew libraries with -mmacosx-version-min=10.9, which would seem to be needed to match the rest of Aquamacs. So far, I haven't been able to figure out how to slip the compiler flag in there without editing the homebrew recipes.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 14, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 17, 2018

OK, I've got a script that pulls in any libraries from Homebrew into the application bundle for distribution. It hooks in near the end of build-aquamacs.

It's not specific to gnutls, so if there are other libraries you wanted to use this way, it should work automatically.

Which branch do you want the pull request against?

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 17, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 17, 2018

I'll take a look at adding it via make install.

Right now, the script reads the compiled Aquamacs binary for any shared libraries in /usr/local, then copies them and any of their dependencies in /usr/local into the bundle. The Homebrew libraries are recompiled/reinstalled as needed to set the minimum Mac OS version for the compiler. It's entirely done as postprocessing on the Aquamacs binary.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Aug 17, 2018

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 18, 2018

Just submitted the pull request for the gnutls libraries.

@seh

This comment has been minimized.

Copy link

commented Aug 23, 2018

See #140.

@paraita

This comment has been minimized.

Copy link

commented Dec 2, 2018

installing gnutls with brew fixed the problem on my machine (Mojave, fresh install of 3.4)

@viking2917

This comment has been minimized.

Copy link

commented Dec 2, 2018

thanks @paraita I will give that a try

@tripleee

This comment has been minimized.

Copy link

commented Jan 16, 2019

If you are looking for something to test with, trying to read Gmail over IMAP in Gnus is one thing which is failing in an unattractive and hard-to-diagnose manner.

Here is a test function I cribbed from http://sange.fi/~atehwa/cgi-bin/piki.cgi/how%20to%20read%20e-mail%20via%20imap%2Bssl%20with%20gnus:

(defun gnus-browse-imaps-server (server)
        "Browse a mail server in Gnus via IMAP-SSL."
        (interactive "sServer name: ")
        (gnus-group-browse-foreign-server
          (list 'nnimap server
             (list 'nnimap-address server)
             '(nnimap-stream ssl)
             '(nnimap-list-pattern ("INBOX" "mail/*" "Mail/*" "INBOX.*"))
             '(nnimap-expunge-on-close ask))))

If you are unfamiliar with Gnus, figuring out how to use that when it works is going to be a challenge, too; but for the time being, just observe how it dies with a nondescript error whilst attempting to connect to the IMAP server.

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 24, 2019

So a new version of gnutls dropped in December, and it uses "TCP fast connect", which requires the connectx system call in Mac OS, which has only been around since El Capitan (10.11). This means that we can't bundle the library and keep backwards compatibility to 10.9.

So far, I haven't been able to get gnutls to build for 10.11 with the rebuild-in-homebrew approach, so we still have a big problem.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Jan 24, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 24, 2019

OK, I found the build problem for using 10.11. There's a bug in homebrew. I submitted a pull request for it. In the meantime, it's a really minor patch to make in the homebrew code on the build system, if you want.

I'm running a couple of times through the full build process I have now to make sure it works correctly after all the messing around trying to debug this.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Jan 24, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 24, 2019

At the moment, there is still a target for install-brewed-libraries in the Makefile to record the proper script invocation, but it is not called by anything. The expectation is that it would be run only for release versions (and maybe nightlies?). It could also be called from build-aquamacs in those cases. Or we could just move it out from there entirely. Let me know what you prefer on that. Everything besides that is wrapped up in the build-homebrew-libraries.sh script.

I did not track down all the places that check for 10.9 as the minimum version, but I don't think they matter that much at this point. The build correctly compiles everything for 10.11.

Do you want me to write changelog entries, and, if so, where?

Then I'll get everything bundled up in a pull request.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Jan 24, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Jan 26, 2019

Just added pull request #146 with what I think is a complete set of changes for the release machinery to build in the gnutls libraries. It does require the latest version of homebrew, which has a bug fix in it (man, tracking that one down was obscure!).

The developer instructions should be updated to say that you'll need to have gnutls installed, via homebrew or otherwise, to build an Aquamacs with usable TLS/SSL code. You can compile without it, but you won't be able to get to, say, MELPA for packages.

Also, I did the copyright assignment with the FSF.

rxa254 added a commit to rxa254/emacs.d that referenced this issue Jan 28, 2019

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Feb 7, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 10, 2019

The nightly that was available on Saturday, 9 Feb, did not have the gnutls libraries in the bundle. The app bundle had no Content/MacOs/lib directory, which is where they would be. Did you run build-homebrew-libraries.sh in the final build stage?

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Feb 10, 2019

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Mar 3, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2019

The nightly build doesn't quite have it yet, because there are a some subtle problems in the script that don't quite work in the nightly build context. It's taken me a bit of time to work through the process of (mostly) replicating the right nightly build on my computer, and then fixing my own bugs.

I think it's possible that the calls to brew may update the package database, and therefore trigger updates on any directly affected packages, which would be gnutls and its dependencies. That would happen only in the case where the they are being recompiled. Other than dependencies, it shouldn't install any other packages.

One effect of this is that configure now searches for libraries in /usr/local, so it may find things like librsvg, libjpeg, and libxml2 if those happen to be installed for some reason. In debugging the script, I disabled those in the build script call to configure. The possible problematic one is libxml2, because Aquamacs normally uses the MacOS built-in version. If those aren't installed on the build machine, it won't be a problem, because configure won't find them. (On the other hand, if you wanted to start using some of those in Aquamacs, that's possible now.)

I'll have an updated pull request with the changes soon; I think we're getting there. The pull request will not change the call to configure; I'm assuming for the moment that the problem libraries are just an issue on my system, not on your build system. Let me know if that's wrong.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Mar 5, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 5, 2019

In the normal case on a nightly build machine, the script would recompile the relevant libraries once, and then it won't do it again unless they are changed so that they don't meet the minimum MacOS version requirement. So the normal course would be that you install/upgrade the libraries, they get rebuilt once, and every night the same binary libraries are used.

It would be possible, if you prefer, to split the script into two parts:

  1. The part that copies the libraries into the Aquamacs application bundle. This would signal an error if the library minimum versions did not match.

  2. The part that recompiles things, which you could run by hand after upgrading libraries with brew.

Happy to do it if that works better.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Mar 5, 2019

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 12, 2019

In case anyone is following along and cares about this, I gave David a new pull request (#148) with a script that should make it possible to distribute Aquamacs with gnutls built in.

@aashudwivedi

This comment has been minimized.

Copy link

commented Mar 15, 2019

@treese thanks a lot for your work.
I am facing a similar issue where package-refresh-contents results in the following error :

Importing package-keyring.gpg...done
Failed to download ‘MELPA’ archive.
Failed to download ‘melpa-stable’ archive.
Failed to download ‘melpa’ archive.
Failed to download ‘gnu’ archive.

I've followed your solution and installed libressl and gnutls (strangely gnutls-cli) is not available from brew.

But the error persists. Would you be able to suggest how to go about fixing it?
Also I've installed GNU emacs26.1 which works fine and is able to install packages but I really like aquamacs.

Also I tried list-packages which results in the following error :

error in process sentinel: Error retrieving: https://stable.melpa.org/packages/archive-contents (error connection-failed "failed with code 22
" :host "" :service 80)
Package refresh done
error in process sentinel: Error retrieving: http://melpa.milkbox.net/packages/archive-contents (error connection-failed "failed with code 22
" :host "" :service 80)
error in process sentinel: Error retrieving: http://melpa.milkbox.net/packages/archive-contents (error connection-failed "failed with code 22
" :host "" :service 80)
@treese

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 18, 2019

@aashudwivedi Were you unable to install gnutls from homebrew? I haven't verified that libressl works myself. What are you setting package-archives to? Are you using https in the links? Also, I have the following configuration lines, which may or may not still be relevant:

(with-eval-after-load 'tls
     (push "/private/etc/ssl/cert.pem" gnutls-trustfiles))
(setq tls-checktrust 'ask)

If you do have gnutls installed, what is the value of the Lisp variable tls-program?

@cl4yton

This comment has been minimized.

Copy link

commented May 28, 2019

Thank you @treese and @davidswelt for all of your work, this has been a very helpful thread.
I think I have followed every step for the recommended manual fix, but loading melpa continues to fail.

Here is my state:
(0) running macOS Mojave (10.14.5)
(1) Installed Aquamacs

Aquamacs 3.4  GNU Emacs 25.3.50.1 (x86_64-apple-darwin14.5.0, NS appkit-1348.17 Version 10.10.5 (Build 14F27))
dated 2018-07-26 rev. 53e2a47a8fbbfce9586fd76891478ac5851ac3b5

(2) brew installed gnutls (v3.6.7.1)
(3) My .emacs.d/init.el consists of the following:

(message ">> init.el file load START!")

(require 'tls)
(with-eval-after-load 'tls
     (push "/private/etc/ssl/cert.pem" gnutls-trustfiles))
(setq tls-checktrust 'ask)

(message ">>>> tls-program: '%s'" tls-program)

(require 'package)
(let* ((no-ssl (and (memq system-type '(windows-nt ms-dos))
                    (not (gnutls-available-p))))
       (proto (if no-ssl "http" "https")))
  (when no-ssl
    (warn "\
Your version of Emacs does not support SSL connections,
which is unsafe because it allows man-in-the-middle attacks.
There are two things you can do about this warning:
1. Install an Emacs version that does support SSL and be safe.
2. Remove this warning from your init file so you won't see it again."))
  ;; Comment/uncomment these two lines to enable/disable MELPA and MELPA Stable as desired
  (add-to-list 'package-archives (cons "melpa" (concat proto "://melpa.org/packages/")) t)
  ;;(add-to-list 'package-archives (cons "melpa-stable" (concat proto "://stable.melpa.org/packages/")) t)
  (when (< emacs-major-version 24)
    ;; For important compatibility libraries like cl-lib
    (add-to-list 'package-archives (cons "gnu" (concat proto "://elpa.gnu.org/packages/")))))
(package-initialize)

(message ">> init.el file load DONE!")

(4) When I launch Aquamacs, the following prints to the *Messages* buffer:

Loading prestart plugin files ...
... done.
Wrote /Users/claytonm/Library/Preferences/Aquamacs Emacs/Packages/.nosearch
Shell: /bin/bash
Warning: Key <remap> <scroll-up> already bound to `cua-scroll-up' by minor modes (cua-mode).  Use ‘define-key’ instead.
Warning: Key <remap> <scroll-down> already bound to `cua-scroll-down' by minor modes (cua-mode).  Use ‘define-key’ instead.
Loading /Users/claytonm/Library/Preferences/Aquamacs Emacs/Recent Files.el (source)...done
Cleaning up the recentf list...done (0 removed)
one-buffer-one-frame-mode disabled.
Warning: Login shell did not return environment.
0 environment variables imported from login shell (/bin/bash).
Loading /Applications/Aquamacs.app/Contents/Resources/lisp/aquamacs/edit-modes/tex-site.el (source)...done
Loading /Applications/Aquamacs.app/Contents/Resources/lisp/aquamacs/edit-modes/auctex.el (source)...done
Loading /Applications/Aquamacs.app/Contents/Resources/lisp/aquamacs/edit-modes/auctex/preview-latex.el (source)...done
Loading /Applications/Aquamacs.app/Contents/Resources/lisp/aquamacs/edit-modes/auctex/preview-latex.el (source)...done
Loading plugins ...
Loading /Applications/Aquamacs.app/Contents/Resources/lisp/aquamacs/site-start.el (source)...done
... done.
Loading ‘custom-file’ failed.
Loading /Users/claytonm/Library/Preferences/Aquamacs Emacs/Preferences.el (source)...done
>> init.el file load START!
>>>> tls-program: ’(gnutls-cli --x509cafile %t -p %p %h gnutls-cli --x509cafile %t -p %p %h --protocols ssl3)’
>> init.el file load DONE!
one-buffer-one-frame-mode disabled.
Mark set [26 times]
Loading /Users/claytonm/Library/Preferences/Aquamacs Emacs/frame-positions.el (source)...done
Mark set [5 times]
Aquamacs is based on GNU Emacs, a part of the GNU/Linux system. It is Free Software: you can improve and redistribute it under the GNU General Public License, version 3 or later. (C) 2018
Free Software Foundation, and D. Reitter. No Warranty.

(5) As shown in the output, tls-program is bound to:
(gnutls-cli --x509cafile %t -p %p %h gnutls-cli --x509cafile %t -p %p %h --protocols ssl3)
(6) When I then M-x package-refresh-contents, I get the following in the *Messages* buffer:

Opening TLS connection to ‘stable.melpa.org’...
Opening TLS connection with ‘gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 stable.melpa.org’...failed
Opening TLS connection with ‘gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 stable.melpa.org --protocols ssl3’...failed
Opening TLS connection to ‘stable.melpa.org’...failed
Failed to download ‘melpa-stable’ archive.
Opening TLS connection to ‘melpa.org’...
Opening TLS connection with ‘gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 melpa.org’...failed
Opening TLS connection with ‘gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 melpa.org --protocols ssl3’...failed
Opening TLS connection to ‘melpa.org’...failed
Failed to download ‘melpa’ archive.
Package refresh done
You can run the command ‘package-refresh-contents’ with M-x pa-r- RET
Package refresh done

(7) I also tried the above but adding the following after (tls-checktrust 'ask) in init.el

(setq tls-program
      '("gnutls-cli -p %p --dh-bits=2048 --ocsp --x509cafile=%t --insecure --priority='SECURE192:+SECURE128:-VERS-ALL:+VERS-TLS1.2:%%PROFILE_MEDIUM' %h"))

as well as trying the complete listing linked by @paolodedios ...

... but get the same failure to download melpa.

Any suggestions about fixes to my above setup or next steps for debugging are much appreciated!

@treese

This comment has been minimized.

Copy link
Collaborator Author

commented May 29, 2019

@cl4yton Can you run gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 stable.melpa.org in a terminal to see what it says? Feel free to email me directly (treese@acm.org) while we sort it out.

@bestlem

This comment has been minimized.

Copy link

commented Jun 12, 2019

Sorry to add to this.

Does the latest nightly build of Aquamacs (11 March) now include gnutls library. I can't see it as a .dylib in the Aquamacs bundle

I ask - I have the working workaround using gnutls-cli for normal use for use with the release Aquamacs.

However I now want to use a elisp library that builds by using cask and Makefiles . It does not compile just using the byte compile on Aquamacs startup - it actually totally fails that as one of its .el files kicks off the compilation. cask of course needs a fixed emacs as their bug cask/cask#418 ended with its OK we have fixed Homebrew build

For bonus points I use Macports not homebrew and am still using OpenSSL for libressl. So if Aquamacs relies on an outside dynamic lib I have rather a lot to dig into.

@davidswelt

This comment has been minimized.

Copy link
Owner

commented Jun 12, 2019

@cl4yton

This comment has been minimized.

Copy link

commented Jun 12, 2019

Many thanks to @treese for helping solve my problem: Aquamacs not able to connect to melpa.org.

Quick summary: Running M-x package-refresh-content was failing because my Aquamacs did not have gnutls-cli in its PATH (for me, gnutls-cli executable is in /usr/local/bin after installing using homebrew)

Solution: Add the following line to ~/Library/Preferences/Aquamacs Emacs/Preferences.el

(setenv "PATH" (concat "~/bin.sh:/usr/local/bin:" (getenv "PATH")))`

NOTE: change the path /usr/local/bin if your gnutls-cli executable lives elsewhere.

More details:

The following reviews the debugging steps we followed, which provided the evidence that the problem was with the PATH Aquamacs has. I'm reproducing these details here in case it helps others in a similar situation (and to distinguish from other potential causes).

First, we confirmed that gnutls-cli could be executed from a (non-emacs) terminal:

$ gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 stable.melpa.org

The (partial) output:

Processed 77 CA certificate(s).
Resolving 'stable.melpa.org:443'...
Connecting to '178.128.185.1:443'...
- Certificate type: X.509
- Got a certificate list of 2 certificates.

<snip>

- Handshake was completed

- Simple Client Mode:

- Peer has closed the GnuTLS connection

The last line (Peer has closed``) only appears after about 60 seconds of inactivity. The above confirms that from the terminal, gnutls-cli` executes successfully.

From within Aquamacs, opened the *Messages* buffer and then executed M-x package-refresh-content and observed that the connection to 'melpa.org' failed.

We then used interactive Emacs Lisp (IELM), executing M-x elm (still within Aquamacs), and evaluated the following form in the REPL, with resulting error:

ELISP> (url-retrieve-synchronously "https://melpa.org/packages/")
*** Eval error ***  Could not create connection to melpa.org:443

However, if I did the same thing within my Mac terminal emacs (GNU Emacs 26.2), I instead got success (i.e., within terminal emacs, execute M-x elm and then evaluated the same form as above):

ELISP> (url-retrieve-synchronously "https://melpa.org/packages/")
#<buffer  *http melpa.org:443*>

You can access the *http melpa.org:433* buffer (NOTE: there is a space at the start of the buffer name!):

HTTP/1.1 200 OK
Server: nginx
Date: Fri, 31 May 2019 04:00:53 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 306
Last-Modified: Fri, 14 Dec 2018 04:45:09 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5c133555-132"
Strict-Transport-Security: max-age=15768000
Accept-Ranges: bytes

<html>
  <head>
    <title>Package Listing</title>
    <meta http-equiv="refresh" content="5; url=/" />
  </head>
  <body>
    <h1>Package Listing</h1>
    <p>
      Please see the <a href="/">main page</a> for a full package
      listing. You will be redirected automatically.
    </p>
  </body>
</html>

This shows in more detail that terminal emacs can access melpa.org, but Aquamacs could not.

We then tried a lower-level IELM command, again back in Aquamacs (within IELM):

ELISP> (setq xx (start-process "xtls" "XTLS" "/bin/bash" "-c" "gnutls-cli --x509cafile /private/etc/ssl/cert.pem -p 443 stable.melpa.org" ))
#<process xtls>

Now the buffer shows:

/bin/bash: gnutls-cli: command not found

Process xtls exited abnormally with code 127

Finally, within a shell launched within Aquamacs (M-x shell), we confirmed that Aquamacs could not find gnutls-cli, and /usr/local/bin is not in the environment PATH.

bash-3.2$ which gnutls-cli
bash-3.2$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/Library/TeX/texbin:/usr/local/texlive/2019/bin:/usr/local/texlive/2018/bin

This led to the above solution of adding the path manually to Aquamacs Preferences.el

@bestlem

This comment has been minimized.

Copy link

commented Jun 12, 2019

Yes Macports is still developed it has emacs develop as of emacs-app-devel @20190413 and the Mos port latest version and openssl as of 28-May-2019 Someone has started work for macOS 10.15.

From the logs I see it is the copy of the dylibs that fails.

I locally built changing the build to use Macports' gnutls and that works albeit with the gnutls library etc not in the bundle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.