infinite stack recursion in upgradable_dependents #5431

ploxiln · 2018-12-21T07:19:44Z

are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh
ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
ran brew update and can still reproduce the problem?
ran brew doctor, fixed all issues and can still reproduce the problem?
ran brew config and brew doctor and included their output with your issue?

I only reproduced with libpng, but this seems to me like an issue with the core framework, or something that should be detected in the core framework.

What you were trying to do (and why)

upgrade libpng ... from source ... I know you guys seem to not like that anymore, but I started with gentoo about 15 years ago ...

What happened (include command output)

libpng upgrade worked, then homebrew hit an apparently infinite stack recursion while checking dependents

==> make install
...
/usr/local/Cellar/libpng/1.6.36: 27 files, 1.2MB, built in 1 minute 33 seconds
Error: stack level too deep
/usr/local/Homebrew/Library/Homebrew/options.rb:26:in `hash'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:313:in `add'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:388:in `block in merge'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:96:in `each'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:96:in `each_entry'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:96:in `do_with_enum'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:388:in `merge'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:90:in `initialize'
/usr/local/Homebrew/Library/Homebrew/options.rb:64:in `new'
/usr/local/Homebrew/Library/Homebrew/options.rb:64:in `initialize'
/usr/local/Homebrew/Library/Homebrew/options.rb:60:in `new'
/usr/local/Homebrew/Library/Homebrew/options.rb:60:in `create'
/usr/local/Homebrew/Library/Homebrew/tab.rb:256:in `used_options'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:355:in `from_keg'
/usr/local/Homebrew/Library/Homebrew/keg.rb:416:in `to_formula'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:200:in `map'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:200:in `block in upgradable_dependents'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:190:in `upgradable_dependents'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:217:in `block in upgradable_dependents'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:190:in `upgradable_dependents'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:217:in `block in upgradable_dependents'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:190:in `upgradable_dependents'
... repeats ...

What you expected to happen

Maybe nothing after the successful install, I expect libpng-1.6.36 to be fully ABI compatible with libpng-1.6.35, or maybe homebrew automatically re-builds all the formula which depend on libpng:

$ brew uses --installed --include-optional libpng | cat
cairo
freetype
gd
graphviz
gst-plugins-good
qrencode
webp

$ brew uses --installed libpng | cat
cairo
fontconfig
freetype
gd
gobject-introspection
graphviz
gst-libav
gst-plugins-base
gst-plugins-good
gst-plugins-ugly
gst-rtsp-server
harfbuzz
libass
libbluray
libsoup
mpv
pango
qrencode
vala
webp

interesting how --include-optional just avoids including transitive dependents in this case

Step-by-step reproduction instructions (by running `brew` commands)

I did this a couple times to check it wasn't just a fluke. I still have libpng-1.6.35 installed, so first:

$ brew uninstall --ignore-dependencies libpng
Uninstalling /usr/local/Cellar/libpng/1.6.36... (28 files, 1.2MB)
libpng 1.6.35 is still installed.
Remove all versions with `brew uninstall --force libpng`.

Then:

$ brew upgrade libpng
==> Upgrading 1 outdated package:
libpng 1.6.36
==> Upgrading libpng 
...

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have HOMEBREW_BUILD_FROM_SOURCE set. This environment variable is
intended for use by Homebrew developers. If you are encountering errors,
please try unsetting this. Please do not file issues if you encounter
errors when using this environment variable.

Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
  brew install p11-kit

Run `brew missing` for more details.

$ brew config
HOMEBREW_VERSION: 1.8.6-14-ga2b4d4a
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: a2b4d4aac294ec0872cd84d44f65bd5bd507e166
Last commit: 3 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: a4af237fa6836920fdec9d9c8a3c14260123ac11
Core tap last commit: 70 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_BUILD_FROM_SOURCE: 1
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_NO_ANALYTICS: 1
HOMEBREW_NO_AUTO_UPDATE: 1
HOMEBREW_NO_EMOJI: 1
CPU: quad-core 64-bit haswell
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1000
Git: 2.20.1 => /usr/local/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 1.8.0_192
macOS: 10.13.6-x86_64
CLT: 10.1.0.0.1.1539992718
Xcode: N/A
XQuartz: 2.7.11 => /opt/X11

I was in the process of upgrading stuff:

$ brew outdated
awscli (1.16.60) < 1.16.70
bison (3.2.2) < 3.2.3
cmake (3.13.1) < 3.13.2
fdk-aac (0.1.6) < 2.0.0
ffmpeg (4.1) < 4.1_1
glib (2.58.1) < 2.58.2
gnu-sed (4.5) < 4.6
gobject-introspection (1.58.1) < 1.58.2
graphite2 (1.3.12) < 1.3.13
gst-plugins-ugly (1.14.4_1) < 1.14.4_2
harfbuzz (2.1.3) < 2.2.0
icu4c (62.1) < 63.1
media-info (18.08.1) < 18.12
meson (0.48.2) < 0.49.0
node@8 (8.12.0) < 8.14.1
opam (2.0.1) < 2.0.2
pixman (0.34.0_1) < 0.36.0
x264 (r2854) < r2917

The text was updated successfully, but these errors were encountered:

ploxiln · 2018-12-21T07:45:29Z

I hacked in a print to get some clue about the formulae causing this loop:

--- a/Library/Homebrew/cmd/upgrade.rb
+++ b/Library/Homebrew/cmd/upgrade.rb
@@ -186,12 +186,13 @@ module Homebrew
   def upgradable_dependents(kegs, formulae)
     formulae_to_upgrade = Set.new
     formulae_pinned = Set.new
 
     formulae.each do |formula|
       descendants = Set.new
+      puts "considering formula: #{formula.full_name}"
 
       dependents = kegs.select do |keg|
         keg.runtime_dependencies

the output is confusing because this is a recursion inside a loop, but it eventually did come to a consistent repeating pattern:

...
considering formula: gst-rtsp-server
considering formula: gst-rtsp-server
considering formula: gst-plugins-bad
considering formula: gst-plugins-base
considering formula: gst-libav
considering formula: gst-plugins-bad
considering formula: gst-plugins-good
considering formula: gst-rtsp-server
considering formula: gst-plugins-good
considering formula: gst-rtsp-server
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
considering formula: libsoup
considering formula: gst-plugins-good
considering formula: ghostscript
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
considering formula: libsoup
considering formula: gst-plugins-good
considering formula: ghostscript
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
considering formula: libsoup
considering formula: gst-plugins-good
considering formula: ghostscript
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
... repeats ...

ploxiln · 2018-12-21T07:56:33Z

I think an interesting question is: how did libtiff get into that output ...

$ brew info libtiff
libtiff: stable 4.0.10 (bottled)
TIFF library and utilities
http://libtiff.maptools.org/
/usr/local/Cellar/libtiff/4.0.10 (246 files, 3.5MB) *
  Built from source on 2018-11-28 at 21:06:13
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libtiff.rb
==> Dependencies
Required: jpeg (installed)
Optional: xz (installed)

MikeMcQuaid · 2018-12-21T10:27:26Z

Sorry, given this affects just you we'll accept a pull request for a fix or would need to see this reproducible with no brew doctor warnings. Will reopen if others experience this.

ploxiln · 2018-12-21T10:47:23Z

eh well it's just HOMEBREW_BUILD_FROM_SOURCE, the "p11-kit" one is erroneous:

$ brew info gnutls
gnutls: stable 3.5.19 (bottled)
GNU Transport Layer Security (TLS) Library
https://gnutls.org/
/usr/local/Cellar/gnutls/3.5.19 (1,104 files, 7.3MB) *
  Built from source on 2018-07-19 at 00:39:41 with: --without-p11-kit
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/gnutls.rb
==> Dependencies
Build: pkg-config (installed)
Required: gmp (installed), libtasn1 (installed), libunistring (installed), nettle (installed)
Recommended: p11-kit (uninstalled)
Optional: guile (uninstalled), unbound (uninstalled)
==> Options
--with-guile
	Build with guile support
--with-unbound
	Build with unbound support
--without-p11-kit
	Build without p11-kit support

Frizlab · 2018-12-23T10:15:49Z

I have the same issue with a simple brew upgrade. The upgrade command declared upgrading ffmpeg 4.1 -> 4.1_1, rust 1.31.0 -> 1.31.1, gnu-sed 4.5 -> 4.7, libssh 0.8.4 -> 0.8.5, gnutls 3.5.19 -> 3.6.5.

I have the following installed:

adns			ffmpeg			graphite2		libgcrypt		libvo-aacenc		opencore-amr		readline		theora
asciidoc		ffmpeg@2.8		gti			libgpg-error		libvorbis		openh264		rlwrap			tree
autoconf		flac			gtk-doc			libgsm			libvpx			openjpeg		rsync			two-lame
autoconf-archive	fontconfig		harfbuzz		libidn			libxml2			openssl			rtmpdump		unrar
automake		freetype		htop			libidn2			libyaml			openvpn			rubberband		util-linux
bat			frei0r			httpie			libksba			little-cms2		opus			ruby			vapor
boost			fribidi			hugo			libmodplug		lz4			p11-kit			rust			watch
cabextract		game-music-emu		icu4c			libogg			lzo			p7zip			sdl			wavpack
cairo			gdbm			intltool		libpng			makedepend		pango			sdl2			webp
carthage		gdk-pixbuf		itstool			libressl		meson-internal		pass			sl			wget
chromaprint		gettext			jpeg			librsvg			mpg123			pcre			snappy			wine
cimg			git-lfs			jq			libsamplerate		nasm			phash			source-highlight	winetricks
cloc			glib			lame			libsndfile		ncdu			pigz			speedtest-cli		x264
cmake			gmp			leptonica		libsoxr			nettle			pinentry		speex			x265
coreutils		gnu-getopt		lftp			libssh			ninja			pixman			sphinx-doc		xmlto
ctls			gnu-sed			libass			libssh2			nmap			pkg-config		sqlite			xvid
cython			gnupg			libassuan		libtasn1		node			postgresql		swiftlint		xz
dep			gnutls			libbs2b			libtiff			npth			pstree			swig			yasm
docbook			go			libcaca			libtool			ocaml			pv			tcl-tk			youtube-dl
docbook-xsl		gobject-introspection	libcroco		libunistring		ocaml-findlib		python			tesseract		zeromq
doxygen			gpac			libdvdcss		libusb			ocamlbuild		python@2		texi2html		zimg
fdk-aac			gpgme			libffi			libvidstab		oniguruma		qrencode		the_silver_searcher

I also have the HOMEBREW_BUILD_FROM_SOURCE var set to 1.

MikeMcQuaid · 2018-12-23T10:49:38Z

I also have the HOMEBREW_BUILD_FROM_SOURCE var set to 1.

As the docs note: that’s an unsupported configuration.

Frizlab · 2018-12-23T11:29:41Z

I got the issue on another computer with the var unset, just when compiling from source.

MikeMcQuaid · 2018-12-23T11:54:59Z

@Frizlab If someone can provide a simple reproduction without HOMEBREW_BUILD_FROM_SOURCE set we've got something we can look at fixing.

MikeMcQuaid · 2018-12-23T11:55:57Z

Additionally we need a fuller backtrace in a Gist, thanks.

Frizlab · 2018-12-23T11:56:17Z

Will try and do that when/if I find the time.

MikeMcQuaid closed this as completed Dec 21, 2018

lock bot added the outdated PR was locked due to age label Jan 22, 2019

lock bot locked as resolved and limited conversation to collaborators Jan 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

infinite stack recursion in upgradable_dependents #5431

infinite stack recursion in upgradable_dependents #5431

ploxiln commented Dec 21, 2018 •

edited

ploxiln commented Dec 21, 2018

ploxiln commented Dec 21, 2018

MikeMcQuaid commented Dec 21, 2018

ploxiln commented Dec 21, 2018

Frizlab commented Dec 23, 2018

MikeMcQuaid commented Dec 23, 2018

Frizlab commented Dec 23, 2018

MikeMcQuaid commented Dec 23, 2018

MikeMcQuaid commented Dec 23, 2018

Frizlab commented Dec 23, 2018 via email

infinite stack recursion in upgradable_dependents #5431

infinite stack recursion in upgradable_dependents #5431

Comments

ploxiln commented Dec 21, 2018 • edited

What you were trying to do (and why)

What happened (include command output)

What you expected to happen

Step-by-step reproduction instructions (by running brew commands)

ploxiln commented Dec 21, 2018

ploxiln commented Dec 21, 2018

MikeMcQuaid commented Dec 21, 2018

ploxiln commented Dec 21, 2018

Frizlab commented Dec 23, 2018

MikeMcQuaid commented Dec 23, 2018

Frizlab commented Dec 23, 2018

MikeMcQuaid commented Dec 23, 2018

MikeMcQuaid commented Dec 23, 2018

Frizlab commented Dec 23, 2018 via email

ploxiln commented Dec 21, 2018 •

edited

Step-by-step reproduction instructions (by running `brew` commands)