Problems with GHC 9.2

[qrilka]

I was trying to check my package xlsx with GHC 9.2 and found it failing at https://github.com/qrilka/xlsx/blob/master/test/Main.hs#L57 which uses xeno-based fast parsing. Quick look shows some odd results from xeno.
I tried xeno with GHC 9.2.1 and with the following stack.yaml

packages:
- .
extra-deps:
- git: https://github.com/chrisdone/hexml.git
  commit: 22923b55ca7390f1973ddfd0d0e517f13cf8a8a4
- bytestring-mmap-0.2.2@sha256:d04e6bc5a158dd292757d3b9b032beb8a2c43e768777d64a8289abf89f612f67
- hexpat-0.20.13
- List-0.6.2
- libxml-0.1.1
- mutable-containers-0.3.4@sha256:aad13ec7e9686725fbbae6af1852055cc2d73f00109b5a313bb007ac74f5ecc2,2303
- vector-0.12.3.1@sha256:040210919e5ce454dcee3320f77803da3dbda579c8428dc25ff0155732234808,7946
- mono-traversable-1.0.15.3@sha256:059bf3c05cdbef2d06b765333fe41c2168ced2503a23de674e2a59ceb2548c48,2060
- primitive-0.7.3.0@sha256:6b28a1c0572f5ca50597ba8388aeade21515842969ae192cdc6bfca81367bf56,2951
- hashable-1.4.0.1@sha256:0251ad5228be07909a385152a3ff634fac1f892611f89904aea5c5e7af411e5d,4790
- split-0.2.3.4@sha256:a6df9c3e806ee7cb50bc980a183fc1156f35022a39430dabac0bf9456fe18a4b,2647
- unordered-containers-0.2.15.0@sha256:7e84950317c31e9c33f11e6338c1dcc9be0141ff696cf5249985c4625b9e144a,5217
- vector-algorithms-0.8.0.4@sha256:bf4760b23a0fee09abb8c9e3c952c870f5dc9780876e9d7e38ab2bdd98c8f283,3752
- extra-1.7.10@sha256:e384751317577554f873812358fab022da02aa9a286c9341308fac83f4d766c5,2691
- hspec-2.9.1@sha256:5daa1483240c194fdcf3c6ba446b56462dbf1f52d525339acea24e24f242065a,1709
- QuickCheck-2.14.2@sha256:4ce29211223d5e6620ebceba34a3ca9ccf1c10c0cf387d48aea45599222ee5aa,7736
- clock-0.8.2@sha256:473ffd59765cc67634bdc55b63c699a85addf3a024089073ec2a862881e83e2a,4313
- hspec-core-2.9.1@sha256:8241004297d18ceb44ee8d3a3c33222931bec926f55690ee52cca7a7da331e8b,5575
- hspec-discover-2.9.1@sha256:c16e37a84166bacf853f14e2acef098535cac655693b76523278381ad313f5a7,2157
- hspec-expectations-0.8.2@sha256:e2db24881baadc2d9d23b03cb629e80dcbda89a6b04ace9adb5f4d02ef8b31aa,1594
- HUnit-1.6.2.0@sha256:1a79174e8af616117ad39464cac9de205ca923da6582825e97c10786fda933a4,1588
- ansi-terminal-0.11@sha256:97470250c92aae14c4c810d7f664c532995ba8910e2ad797b29f22ad0d2d0194,3307
- call-stack-0.4.0@sha256:ac44d2c00931dc20b01750da8c92ec443eb63a7231e8550188cb2ac2385f7feb,1200
- quickcheck-io-0.2.0@sha256:7bf0b68fb90873825eb2e5e958c1b76126dcf984debb998e81673e6d837e0b2d,1133
- random-1.2.1@sha256:8bee24dc0c985a90ee78d94c61f8aed21c49633686f0f1c14c5078d818ee43a2,6598
- setenv-0.1.1.3@sha256:c5916ac0d2a828473cd171261328a290afe0abd799db1ac8c310682fe778c45b,1053
- splitmix-0.1.0.4@sha256:714a55fd28d3e2533bd5b49e74f604ef8e5d7b06f249c8816f6c54aed431dcf1,6483
- tf-random-0.5@sha256:14012837d0f0e18fdbbe3d56e67da8622ee5e20b180abce952dd50bd9f36b326,3983
- colour-2.3.6@sha256:ebdcbf15023958838a527e381ab3c3b1e99ed12d1b25efeb7feaa4ad8c37664a,2378
resolver: ghc-9.2.1

I get a failure

Xeno.DOM tests
  test 1 [✘]
Xeno.DOM tests
  DOM from bytestring substring [✘]
  Leading whitespace characters are accepted by parse [✔]
  children test [✘]
  attributes [✘]
  xml prologue test [✘]
hexml tests
  "<test id='bob'>here<extra/>there</test>" [✔]
  "<test /><close />" [✔]
  "<test /><!-- comment > --><close />" [✔]
  "<test id=\"bob value\" another-attr=\"test with <\">here </test> more text at the end<close />" [✔]
  "<test></more>" [✔]
  "<test" [✔]
  "<?xml version=\"1.1\"?>\n<greeting>Hello, world!</greeting>" [✔]
  "<some-example/>" [✔]
  "<a numeric1=\"attribute\"/>" [✔]
  "<also.a.dot></also.a.dot>" [✔]
  "<test><![CDATA[Oneliner CDATA.]]></test>" [✘]
  "<test><![CDATA[<strong>This is strong but not XML tags.</strong>]]></test>" [✘]
  "<test><![CDATA[A lonely ], sad isn't it?]]></test>" [✘]
  namespaces [✔]
robust XML tests
  DOM from bytestring substring [✘]
  Leading whitespace characters are accepted by parse [✔]
  children test [✘]
  attributes [✘]
  xml prologue test [✘]
  html doctype test [✘]
  hexml tests
    "<test id='bob'>here<extra/>there</test>" [✔]
    "<test /><close />" [✔]
    "<test /><!-- comment > --><close />" [✔]
    "<test id=\"bob value\" another-attr=\"test with <\">here </test> more text at the end<close />" [✔]
    "<test></more>" [✔]
    "<test" [✔]
    "<?xml version=\"1.1\"?>\n<greeting>Hello, world!</greeting>" [✔]
    "<some-example/>" [✔]
    "<a numeric1=\"attribute\"/>" [✔]
    "<also.a.dot></also.a.dot>" [✔]
    "<test><![CDATA[Oneliner CDATA.]]></test>" [✘]
    "<test><![CDATA[<strong>This is strong but not XML tags.</strong>]]></test>" [✘]
    "<test><![CDATA[A lonely ], sad isn't it?]]></test>" [✘]
    namespaces [✔]
  recovers unclosed tag [ ]Right (Node "<" [("<a a","<")] [Element (Node "<a " [] [])])
  recovers unclosed tag [✘]
  ignores too many closing tags [✔]
skipDoctype
  strips initial doctype declaration [✔]
  strips doctype after spaces [✔]
  does not strip anything after or inside element [✔]

Failures:

  test/Main.hs:29:18: 
  1) Xeno.DOM tests test 1
       expected: "catalog"
        but got: "<?xml v"

  To rerun use: --match "/Xeno.DOM tests/test 1/"

  test/Main.hs:45:23: 
  2) Xeno.DOM tests DOM from bytestring substring
       expected: "valid"
        but got: "<vali"

  To rerun use: --match "/Xeno.DOM tests/DOM from bytestring substring/"

  test/Main.hs:55:44: 
  3) Xeno.DOM tests children test
       expected: ["test","test","b","test","test"]
        but got: ["<roo","<roo","<","<roo","<roo"]

  To rerun use: --match "/Xeno.DOM tests/children test/"

  test/Main.hs:58:53: 
  4) Xeno.DOM tests attributes
       expected: [("id","1"),("extra","2")]
        but got: [("<r","<"),("<root","<")]

  To rerun use: --match "/Xeno.DOM tests/attributes/"

  test/Main.hs:63:23: 
  5) Xeno.DOM tests xml prologue test
       expected: "greeting"
        but got: "<?xml ve"

  To rerun use: --match "/Xeno.DOM tests/xml prologue test/"

  test/Main.hs:76:36: 
  6) hexml tests "<test><![CDATA[Oneliner CDATA.]]></test>"
       expected: Right [CData "Oneliner CDATA."]
        but got: Right [CData "<test><![CDATA["]

  To rerun use: --match "/hexml tests/\"<test><![CDATA[Oneliner CDATA.]]></test>\"/"

  test/Main.hs:76:36: 
  7) hexml tests "<test><![CDATA[<strong>This is strong but not XML tags.</strong>]]></test>"
       expected: Right [CData "<strong>This is strong but not XML tags.</strong>"]
        but got: Right [CData "<test><![CDATA[<strong>This is strong but not XML"]

  To rerun use: --match "/hexml tests/\"<test><![CDATA[<strong>This is strong but not XML tags.</strong>]]></test>\"/"

  test/Main.hs:76:36: 
  8) hexml tests "<test><![CDATA[A lonely ], sad isn't it?]]></test>"
       expected: Right [CData "A lonely ], sad isn't it?"]
        but got: Right [CData "<test><![CDATA[A lonely ]"]

  To rerun use: --match "/hexml tests/\"<test><![CDATA[A lonely ], sad isn't it?]]></test>\"/"

  test/Main.hs:89:25: 
  9) robust XML tests DOM from bytestring substring
       expected: "valid"
        but got: "<vali"

  To rerun use: --match "/robust XML tests/DOM from bytestring substring/"

  test/Main.hs:99:44: 
  10) robust XML tests children test
       expected: ["test","test","b","test","test"]
        but got: ["<roo","<roo","<","<roo","<roo"]

  To rerun use: --match "/robust XML tests/children test/"

  test/Main.hs:102:53: 
  11) robust XML tests attributes
       expected: [("id","1"),("extra","2")]
        but got: [("<r","<"),("<root","<")]

  To rerun use: --match "/robust XML tests/attributes/"

  test/Main.hs:107:23: 
  12) robust XML tests xml prologue test
       expected: "greeting"
        but got: "<?xml ve"

  To rerun use: --match "/robust XML tests/xml prologue test/"

  test/Main.hs:111:23: 
  13) robust XML tests html doctype test
       expected: "greeting"
        but got: "<greetin"

  To rerun use: --match "/robust XML tests/html doctype test/"

  test/Main.hs:119:38: 
  14) robust XML tests, hexml tests, "<test><![CDATA[Oneliner CDATA.]]></test>"
       expected: Right [CData "Oneliner CDATA."]
        but got: Right [CData "<test><![CDATA["]

  To rerun use: --match "/robust XML tests/hexml tests/\"<test><![CDATA[Oneliner CDATA.]]></test>\"/"

  test/Main.hs:119:38: 
  15) robust XML tests, hexml tests, "<test><![CDATA[<strong>This is strong but not XML tags.</strong>]]></test>"
       expected: Right [CData "<strong>This is strong but not XML tags.</strong>"]
        but got: Right [CData "<test><![CDATA[<strong>This is strong but not XML"]

  To rerun use: --match "/robust XML tests/hexml tests/\"<test><![CDATA[<strong>This is strong but not XML tags.</strong>]]></test>\"/"

  test/Main.hs:119:38: 
  16) robust XML tests, hexml tests, "<test><![CDATA[A lonely ], sad isn't it?]]></test>"
       expected: Right [CData "A lonely ], sad isn't it?"]
        but got: Right [CData "<test><![CDATA[A lonely ]"]

  To rerun use: --match "/robust XML tests/hexml tests/\"<test><![CDATA[A lonely ], sad isn't it?]]></test>\"/"

  test/Main.hs:131:34: 
  17) robust XML tests recovers unclosed tag
       expected: "a"
        but got: "<"

  To rerun use: --match "/robust XML tests/recovers unclosed tag/"

Randomized with seed 1893189390

Finished in 0.0063 seconds
45 examples, 17 failures

xeno                > Test suite xeno-test failed

[jappeace]

I suspect this has to do with the change of internal representation of bytestring:

• https://hackage.haskell.org/package/bytestring/changelog
  (I mean specifically mean the "Change internal representation of ByteString, removing offset")
• Remove the offset parameter completely haskell/bytestring#175

It appears impossible to put a version bound on xeno to be lower then 0.11.0.0 due to the base restriction.
Not that it'd be a great solution, but it'd confirm the theory at least, I end up with this issue:

In the dependencies for bytestring-0.10.12.1:
    base-4.16.0.0 from stack configuration does not match >=4.2 && <4.16  (latest matching version is 4.15.0.0)
needed due to xeno-0.4.2 -> bytestring-0.10.12.1

But I think going forward we should fix the offset like it is for the new bytestring anyway,
I'll check with pragma's which bytestring is being used.

[avieth]

Here's a potential non-backwards-compatible fix. I'd appreciate a review. Tests pass but I didn't think too hard about what I was doing.

[jappeace]

@avieth if you want, open a PR.

you can use this trick to make it backwards compatible:

#if MIN_VERSION_bytestring(0,11,0)
    NEW_CODE
#else
    OLD_CODE
#endif

It's best to factor that out in a dedicated function per branch, but inline also works.
If you don't want to or can't, let me know and I'll open a PR based on your solution (keeping the commit log intact).

[ocramz]

First of all, thank you everyone for looking at this and @avieth for getting the ball rolling on a PR. I have a funny feeling about that unsafeDupablePerformIO when subtracting pointer offsets. It could create problems if xeno is used in a multithreaded application, I think.

[avieth]

Pull request here featuring backwards compatibility.

I have a funny feeling about that unsafeDupablePerformIO when subtracting pointer offsets. It could create problems if xeno is used in a multithreaded application, I think.

Should be fine I think. The IO is only needed to get a hold of the Ptrs, and the function called against those pointers is pure. Whenever those IOs are run, the foreign pointers are guaranteed not to be finalized, since the bytestrings are retained by the call to parse.

[ocramz]

Closed via #56 . Thanks all!

[ocramz]

Forgot to say, I've published xeno 0.5 with the fix in, try it out !