Octave 6.4.0 - err, 8.1.0 - build (older ticket)

[apjanke]

Octave 6.4.0 is out already. Let's roll an Octave.app installer for it.

I'd like to get a 6.3.0 installer (#229) done first, so we don't have a gap in our history.

Let's see if we can get Apple Silicon support (#205) in here, too.

Here's the ticket for homebrew-octave-app formula changes needed to support this: https://github.com/octave-app/homebrew-octave-app/issues/7.

References

• https://github.com/octave-app/homebrew-octave-app/issues/7
• Octave 8.x and macOS 11+ support (main ticket) #243 (main ticket)
  • This is the new main ticket for all this Octave 8.x / macOS 11+ / 2023 / new Homebrew / Apple Silicon stuff. Most updates will be happening there.

[stevemaser]

(Hey, we are fine if you skip 6.3.0 -- since I'm the one that asked about it.) :-)

[apjanke]

Darnit. Now we've got another problem: Some of the Homebrewed build tools we use are now broken on macOS 10.13 and 10.14: #234

[vinilet]

I have a Apple M1 with Big Sur, if you want I can install the octave-app for ARM and see if it works.

[apjanke]

Thanks! I uh... gotta build an octave-app for ARM first. That's a few items down on the TODO list right now. We'll get there!

Also, we have an M1 Mac build box for Octave.app! Shout out to MacStadium for providing this hardware for zero dollars; boo on me for still not getting this build working so we can actually make use of it.

[thrynae]

I suspect it would be fine to skip 6.4 as well in favor of the brand spanking new 7.1?

I applaud every effort you're putting into this niche use case. Thank you for your trouble.

[apjanke]

Probably so: I'm not aware of any use cases or user populations that would actually prefer 6.3 or 6.4 over 7.x. (Though I kinda like to have them for completeness.)

Anyway things are looking up a bit on this front: I have a new M1 MacBook Pro coming this week that will be suitable as a build box and let me run VMs for a variety of macOS versions.

[thrynae]

That sounds very nice.

This side note is probably better suited to another thread, but is there a reason why installing on removable media won't work? I can borrow a MacBook, but re-installing Octave every time I want to run my test scripts is a pain.

[mpiqueca]

As far as I know, an M1 Mac can run macOS 12 (Monterey) in a VM but not Mac OS 11 Big Sur (or of course any earlier release). I routinely run several Monterey versions on my M1 Mac mini using “Parallels”, and occasionally run an ARM Linux that way. Michael Pique ***@***.*** +1 858-354-4391 On Apr 10, 2022, at 21:16, Andrew Janke ***@***.***> wrote:  Probably so: I'm not aware of any use cases or user populations that would actually prefer 6.3 or 6.4 over 7.x. (Though I kinda like to have them for completeness.) Anyway things are looking up a bit on this front: I have a new M1 MacBook Pro coming this week that will be suitable as a build box and let me run VMs for a variety of macOS versions. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[thrynae]

Is there still work happening on this project? It would be a tremendous shame if this dies a silent death. I'm not asking for a release date for v8 (life tends to get in the way of things and all that), but I would like to know if it is still an active project. I don't know whether I can help, but if there are things others can help with, that may be a way to get this neat idea moving again.

[apjanke]

Is there still work happening on this project?

Definitely. But it's slow.

I'm still interested in maintaining Octave.app as a viable project. But we're at a bit of an impasse here: I would like to maintain support for Octave.app on older versions of macOS, because that's what I use and most of my Macs run, and also because I think that's a substantial portion of Octave.app's user base, which I think is mostly academic - students and researchers.

Octave.app is built on Mac Homebrew, which only supports the last 3 major releases of macOS, and at least for some of the stuff needed for Octave.app, is not just unsupported but broken on earlier macOS versions. I've been looking at doing a full migration to MacPorts, which supports older macOS (and is more of a "pro" tool anyway) - #139 - but that will be substantial work. And that's been holding stuff up.

So in the mean time, I could just roll a new release of Octave.app built on and requiring a newer version of macOS, and post that release with some caveats. Probably the best way to get things rolling here.

[cbm755]

Just an update from GitLab perspective: the CI over there runs only MacOS 12 now---we used to be able to build for MacOS 10.13.6, 11.x, etc, but cannot any more.

So while I agree with this in principle:

I would like to maintain support for Octave.app on older versions of macOS

It might not be do-able in an automated low-effort way... The approach I'm planning in a different (non-Octave) project, is to provide macOS 12 arm64 and macOS 12 x86_64 (and hopefully eventually single universal2 binaries) in an automated fashion (via CI/CD). Then I'll do macOS 11 on a "best effort" (manual) basis.

[apjanke]

So while I agree with this in principle:

I would like to maintain support for Octave.app on older versions of macOS

It might not be do-able in an automated low-effort way...

On this topic, I've actually crossed over myself recently, @cbm755. So many apps and services have dropped support for older macOS versions that, at least for me, it's just untenable without being a paid job. I'm currently in the process of abandoning 10.14 Mojave and upgrading even on my personal machines.

New plan: How about I just roll out a new Octave.app release for macOS 11+ using the current Homebrew-based build system, and for users still on older macOS versions, sorry folks. :/

[cbm755]

Sounds good to me (I'm not a macOS user but I'm well-versed in them not upgrading their OSes).

Does this stuff build automatically on Github Actions? If not, we could consider trying to build it on GitLab CI with their macOS cloud thingy: that would further push you to macOS 12 however. See here for example: https://gitlab.com/gnu-octave/octave-pythonic/-/merge_requests/84

[apjanke]

Does this stuff build automatically on Github Actions?

Haha, no. This stuff doesn't build automatically at all. The way an Octave.app build & release happens is I pop open one of my laptops, fire up a "clean" VM (which can build Octave.app without interacting with all the other junk I have on my machine) which I manually created in VMware Fusion and set up with Xcode and MacTeX and stuff, fire it off, and if it succeeds, upload it to GitHub Releases via the browser.

[...] well-versed in them not upgrading their OSes

In this case, it's not Apple and macOS that's the issue, it's Homebrew, the community-supported third-party package manager for installing Linux/Unix stuff on macOS, that is the issue. Homebrew only supports the last three versions of macOS. And I wanted to support more macOS versions.

(That is not a diss on Homebrew! I wouldn't want to support more OS versions myself in their situation, and how I'm using Homebrew to build Octave.app as a native Mac app bundle here is explicitly not a supported use case for Homebrew.)

But now I'm okay with supporting less macOS versions, so that's less of an issue.

I would love to have Octave.app build automatically through GitHub Actions or some other CI system. The problem is, Octave.app is a three or four hour build on a beefy box. (Because of all the dependencies. An Ocatve.app build is also a build of all its deps, which is a lot. Including Qt. Because of how its packaging works, an Octave.app build isn't just Octave linked to regular libs; it's a build of the whole darn Homebrew numerics ecosystem under a special prefix.) I don't know any free/FLOSS CI providers who are handing out that sort of CI server time for free. But I'm definitely open to having my mind changed here! I didn't even know that GitLab supported macOS until you mentioned it here.

[cbm755]

Because of how its packaging works,

I assume this is b/c of binary signing. I'll admit I do not have that (fastlane?) working on Gitlab CI/CD yet :( I would like to in a different project.

who are handing out that sort of CI server time

I'm not sure what the SaaS runner limit is per job. 60 mins by default but I just changed Pythonic to 2 hours... However, I dimly recall it was a lower upper bound for mac during the beta period (https://docs.gitlab.com/ee/ci/runners/saas/macos_saas_runner.html).

[apjanke]

Because of how its packaging works,
I assume this is b/c of binary signing.

No – I don't have binary signing working at all. (Though I bet that issue is related.) I'm afraid this is much more basic: Octave is not designed as a "native Mac app" or macOS style program that's intended to be distributed as a Mac "app bundle" (the things you see in /Applications that are special "blessed" directories). It's a traditional Unix/Linux style program where it has dependencies on a bunch of other Unix packages or libraries, and it builds and links against them, and then installs into a "prefix" like /usr/local, with its dependencies probably in a mix of /usr and /usr/local. And when Octave builds, the installation location of many of those dependencies is baked in to the binary.

There's about 120 of those package dependencies that are not supplied by macOS itself...

[Cellar] $ pwd
/Applications/Octave-6.2.0.app/Contents/Resources/usr/Cellar
[Cellar] $ ls
aom                 gdbm                   ilmbase           libtiff      netpbm-octave-app        qrupdate
arpack              gdk-pixbuf             imagemagick       libtool      ninja                    qscintilla2-octave-app
autoconf            gettext                isl               libvorbis    octave-octave-app@6.2.0  qt-octave-app
automake            ghostscript            jasper            libx11       open-mpi                 readline
bison               gl2ps                  jpeg              libxau       openblas                 shared-mime-info
cairo               glib                   libde265          libxcb       openexr                  sqlite
cfitsio             glpk                   libevent          libxdmcp     openjdk                  suite-sparse
cmake               gmp                    libffi            libxext      openjpeg                 sundials
docbook             gnu-getopt             libheif           libxrender   openssl@1.1              szip
docbook-xsl         gnu-sed                liblqr            little-cms2  opus                     tbb
eigen               gnu-tar                libmpc            lua          pango                    texinfo
epstool             gnuplot-octave-app     libogg            lzip         pcre                     util-macros
fftw                gobject-introspection  libomp            lzo          pixman                   webp
fig2dev-octave-app  graphicsmagick         libpng            m4           pkg-config               x265
flac                graphite2              libpthread-stubs  meson        plotutils                xmlto
fontconfig          gsl                    librsb            metis        portaudio                xorgproto
freetype            harfbuzz               librsvg           mpdecimal    proj@5                   xtrans
fribidi             hdf5                   libsndfile        mpfr         pstoedit                 xz
gcc                 hwloc                  libsodium         nasm         python@3.9               yasm
gd                  icu4c                  libssh2           netcdf       qhull                    zeromq
[Cellar] $ ls | wc -l
     120
[Cellar] $

Plus Qt and MacTeX, which need to be installed differently, and are big installs themselves. The idea of Octave.app is that it ships Octave as a standalone app bundle that a user can just drag and drop in to /Applications and not have to worry about installing any of those hundred dependencies and getting them in the right place etc. So the Octave.app build system creates an /Applications/Octave-<version>.app directory, installs a full Homebrew installation under there, uses brew to build & install all of Octave's dependencies and Octave itself under there, and then packages the whole shebang as an app bundle. (That's why it's like 3 GB.)

The next step would be to code-sign that resulting app bundle, but I've never gotten that to work. I suspect it's because of the nontraditional structure of this app, but don't really know; I've never gotten a useful error message or diagnostic out of it. But currently you don't need code-signing at all to build Octave.app, and even if we did, for CI/CD purposes (at least testing, but probably the whole basic build up to the signing stage), you could just build an unsigned app. And then have a maintainer code-sign that CI-produced build when you want to "bless" it for an actual release.

... I just changed Pythonic to 2 hours...

Oh... 2 hours is probably enough time to get a useful CI/CD run for Octave.app, especially if I could pre-build some of the rarely-changing dependencies, like OpenBLAS, gcc, and Qt. (Yes, I'm afraid Octave.app builds a whole GCC because you need it for the build and and at run time so Octave users can build oct-files and MEX files. And it needs a Qt build because it uses a special hacked Qt to avoid some dumb "bad filesystem change id" warnings from Qt's directory-watcher tool.) Pre-build those somehow, plus one other long-build dependency I can't remember right now, and the build time drops by hours and maybe gets under that CI/CD time limit. Depending on how many cores the build runner has, I guess; I'm usually building on a 4- or 8-core box.

The ridiculous, but performant, thing to do would be to set up a whole custom binary repo for Homebrew bottles or MacPorts binaries, do custom Octave.app-compliant builds of all of Octave's dependences (targeting the special installation location), and use those in the build. Then you could do it by building each package individually in a single CI worker run, and I think that would get all except the very longest-running individual package builds under an hour. And then interactive builds of Octave.app (for when you're developing it) would go a lot faster, too. I had always dismissed this in the past as Too Much, but if it enabled using a cloud CI/CD/build service, maybe it's time to reconsider that...? The current build time for Octave.app is really a pain, and has slowed down my development of it. Homebrew doesn't have good support for versioning I think, so this might entail switching to MacPorts.

[apjanke]

Okay, I finally found some time to work on this. (Sorry, I caught Covid a few weeks ago and haven't gotten much done lately.)

Here's the new master ticket to track all the work that goes in to this: #243. I made a separate ticket so I don't spam y'all with comments about internal development work here; feel free to follow that ticket if you want detailed progress info.

Making progress. I've now got an Octave 8.3 building on macOS 12 and kinda running, but with some annoying GUI bugs that need to be fixed before making a beta.

Might be able to get an Apple Silicon release built, too; the build seems to be working there, but I'm not sure if I'm accidentally relying on some Rosetta stuff.

[apjanke]

8.4 builds are working. Prerelease alpha builds of Octave.app 8.4.0 are up for testing (e.g. alpha3 here), they seem to be working on macOS 12-14, and there's a native Apple Silicon build.

Closing this ticket as a duplicate of #243, since that other ticket has grown to take over this whole "new release please" thing. Please join us over there if you'd like further updates.