[Bug]: Fails to compile with PyParsing >= 3.0.4

[TheSonicMaster]

Checklist

• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for a bug that matches the one I want to file, without success.

Flatpak version

1.12.2-47-g4da815f3 (git describe --tags)

What Linux distribution are you using?

Other (specify below)

Linux distribution version

All distributions using PyParsing >= 3.0.4 are affected

What architecture are you using?

x86_64

How to reproduce

1. git clone https://github.com/flatpak/flatpak.git
2. cd flatpak
3. NOCONFIGURE=1 ./autogen.sh
4. ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --disable-static --with-system-bubblewrap --with-system-dbus-proxy --with-dbus-config-dir=/usr/share/dbus-1/system.d
5. make

Expected Behavior

Expected it to build successfully.

Actual Behavior

Fails to build near the start of make:

mkdir -p ./common
mkdir -p ./common
mkdir -p ./common
  GEN      common/flatpak-dbus-generated.c
  GEN      common/flatpak-document-dbus-generated.c
  GEN      common/flatpak-enum-types.h
  GEN      common/flatpak-systemd-dbus-generated.c
  GEN      common/flatpak-enum-types.c
  GEN      common/flatpak-variant-private.h
mkdir -p ./app
  GEN      app/flatpak-permission-dbus-generated.c
mkdir -p ./portal
  GEN      portal/flatpak-permission-dbus.c
mkdir -p ./portal
  GEN      portal/flatpak-portal-dbus.c
Parse error: Expected end of text, found 'type'  (at char 218), (line:12, col:1)
make: *** [Makefile:7681: common/flatpak-variant-private.h] Error 1

Additional Information

Tested on both stable version 1.12.2 and the latest git checkout, 1.12.2-47-g4da815f3.

My only workaround was to downgrade PyParsing to 2.4.7, which works, however this is only a temporary workaround, not a permanent fix.

[mwleeds]

I assume the build is tripping on the variant-schema-compiler subproject since that uses pyparsing. So it might be worth opening an issue at https://gitlab.gnome.org/alexl/variant-schema-compiler/-/issues but we can leave this open too

[imrebuild]

By removing .leaveWhitespace() in subprojects/variant-schema-compiler/variant-schema-compiler, I can build flatpak with pyparsing 3.0.6.

It generates same result as pyparsing 2.4.7 for ./subprojects/variant-schema-compiler/variant-schema-compiler --outfile-header common/flatpak-variant-private.h --outfile common/flatpak-variant-impl-private.h --prefix var ./data/flatpak-variants.gv.

I'm not sure if flatpak has a convenient way to patch submodule. I also can't confirm it won't impact other parts of flatpak. At least I can build flatpak and run applications.

[mwleeds]

We don't want to patch the submodule. We should just apply the patch to the variant-schema-compiler repo linked above, if it's correct, and then update the submodule so we get the patched version.

[imrebuild]

The problem is I don't know. I just read some source code of variant-schema-compiler and pyparsing, guessing that one change in pyparsing 3.0.2 may be the reason. And it fixed my problem on my machine.

But there are far more questions. Does this the only change need to be done? Will it impact other projects using this compiler? Will it break the function for people who use pyparsing prior 3.0.2? Only the author can answer these questions and decide the best solution.

I'll leave it as a temporary workaround, for people who want to build flakpak with higher version pyparsing.

[mwleeds]

Looks like we're not the only project to have whitespace-related issues with recent versions of PyParsing: pydot/pydot#277 (comment)

(that comment is by the author/maintainer of pyparsing)

[mwleeds]

I used git bisect to find that this error is caused by pyparsing/pyparsing@4ab17bb

[imrebuild]

More specifically, pyparsing/pyparsing@4ab17bb#diff-651b49e05968bd7f42614f05b7e9d94b30f52db27eb692ce20ef87e9799c624fR3392 is why I removed .leaveWhitespace().

[mwleeds]

Removing leaveWhitespace() from line 1743 in variant-schema-compiler makes a functional difference to the parser that we don't want:

• with leaveWhitespace(), using pyparsing before commit 4ab17bb, variant-schema-compiler errors out if line 12 of flatpak-variants.gv is modified to have a space after the apostrophe: type RefMap [] ' RefMapEntry {
• without leaveWhitespace(), using pyparsing after commit 4ab17bb, variant-schema-compiler completes successfully, despite the whitespace between ' and RefMapEntry

[mwleeds]

Fix available here: https://gitlab.gnome.org/alexl/variant-schema-compiler/-/merge_requests/11