FPyLLL Strategies inside SageMath binaries

[malb]

On Debian 8, after downloading Sage 8.8 as follows:

$ curl -O https://www.mirrorservice.org/sites/www.sagemath.org/linux/64bit/sage-8.8-Debian_GNU_Linux_8-x86_64.tar.bz2
$ tar xf sage-8.8-Debian_GNU_Linux_8-x86_64.tar.bz2

and launching Sage as follows:

$ ./SageMath/sage

running the following commands:

sage: from fpylll import *
sage: BKZ.DEFAULT_STRATEGY

gives an incorrect result:

'/.../local/share/fplll/strategieparse error - unpreprocessing_blpruning_parameteError: gptr == nullpoints/default.json'

The output should be a filename but it's not.
A build from source doesn't have that problem.

Reported upstream:

• Default strategy inside SageMath 8.8 Binaries fplll/fpylll#147

Upstream: Fixed upstream, in a later stable release.

CC: @isuruf @malb @slel @vbraun @dimpase

Component: packages: standard

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/28664

[malb]

comment:1

Seems a precompiled Sage binary also cannot load strategy files stored elsewhere, so something is really broken there:

┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 8.9, Release Date: 2019-09-29                     │
│ Using Python 2.7.15. Type "help()" for help.                       │
└────────────────────────────────────────────────────────────────────┘
sage: from fpylll import *
sage: BKZ.DEFAULT_STRATEGY_PATH
'/bulk/home/malb/software/SageMath/local/share/fplll/strategieparse error - unpreprocessing_blpruning_parameteError: gptr == nulls'
sage: BKZ.DEFAULT_STRATEGY
'/bulk/home/malb/software/SageMath/local/share/fplll/strategieparse error - unpreprocessing_blpruning_parameteError: gptr == nulls/default.json'
sage: print load_strategies_json("/home/malb/projects/lattices/fplll/strategies/default.json")[60]
Strategy< 60, (), 1.00-1.00>

The above should print Strategy< 60, (40), 0.29-0.50>

See fplll/fpylll#159

Any ideas?

[isuruf]

comment:2

This looks like a bug in binary-pkg which looks for \0 character at the end of the path, but the path being stored in a C++ string, the NULL character does not exist at the end.

[isuruf]

comment:3

Adding -std=c++11 to CXXFLAGS forces the C++ compiler to enforce C++11 which enforces that std::string terminate with \0 in the internal buffer.

From my limited testing, this fixes the issue.

[malb]

comment:4

Oh, thanks!

I'm not clear though on where -std=c++11 is missing, FPLLL is currently being compiled with:

AX_CXX_COMPILE_STDCXX([11],[noext],[mandatory])

which should set -std=c++11. But from my local testing it doesn't seem to (?)

[isuruf]

comment:5

File m4/ax_cxx_compile_stdcxx.m4 in fplll repo is old. Updating to the latest version fixes the issue.

[isuruf]

comment:6

@malb, the updated macro at https://github.com/fplll/fplll/pull/394/files forces the -std=c++11. New versions of gcc support some C++11 features without the flag, but does not enforce some C++11 features like \0 at the end of the string buffer. This default checking is removed in https://github.com/fplll/fplll/pull/394/files#diff-38b7f4802a0916ba0fab7e320746bfdbL62-L69

[malb]

Changed upstream from Reported upstream. Developers deny it's a bug. to Reported upstream. Developers acknowledge bug.

[malb]

comment:7

Thank you! I'll wait for the fallout of the 5.3.0 release for a bit and then will cut a new release 5.3.1 with that fix in.

[malb]

Changed upstream from Reported upstream. Developers acknowledge bug. to Fixed upstream, but not in a stable release.

[malb]

comment:9

Fixed in FPLLL 5.3.1

[malb]

Changed upstream from Fixed upstream, but not in a stable release. to Fixed upstream, in a later stable release.

[malb]

comment:10

See #28886 for updates of FP(y)LLL which should fix this issue.

[embray]

comment:11

Ticket retargeted after milestone closed

[mkoeppe]

comment:12

Moving tickets to milestone sage-9.2 based on a review of last modification date, branch status, and severity.

[slel]

comment:13

• Update FP(Y)LLL #28886, merged in Sage 9.1.beta1, upgraded to FPLLL 5.3.2 and FPyLLL 0.5.1.dev.
• FPLLL 5.3.3 #30021, merged in Sage 9.2.beta4, upgraded to FPLLL 5.3.3.

On Debian 10 buster, after downloading Sage 9.1 as follows:

$ SAGE_VERSION=9.1
$ DEBIAN_VERSION=`source /etc/os-release ; echo $VERSION_ID`
$ MIRROR=https://www.mirrorservice.org/sites/www.sagemath.org/linux/64bit
$ TARBALL=sage-$SAGE_VERSION-Debian_GNU_Linux_$DEBIAN_VERSION-x86_64.tar.bz2
$ curl -O $MIRROR/$TARBALL
$ tar xf $TARBALL

and launching Sage as follows:

$ ./SageMath/sage -q

running the following two commands:

sage: from fpylll import *
sage: BKZ.DEFAULT_STRATEGY

still gives the same incorrect result:

'/.../local/share/fplll/strategieparse error - unpreprocessing_blpruning_parameteError: gptr == nullpointer.'

while with Sage built from source on the same machine:

sage: from fpylll import *
sage: BKZ.DEFAULT_STRATEGY
b'/.../local/share/fplll/strategies/default.json'

So even after #28886 Sage built from source gets this right
while Sage binaries don't.

[slel]

comment:14

What causes this difference between the buildbot-produced binaries
and Sage built from source?

[isuruf]

comment:15

Is fplll built with C++11 in the Sage 9.1 binaries?

[mkoeppe]

comment:17

Moving to 9.4, as 9.3 has been released.

[dimpase]

Reviewer: Dima Pasechnik