Build error with g++ 9 snapshot

[eddelbuettel]

While gcc 9 is of course still unreleased, someone looking after the CRAN archive for R already ran some tests, and RcppAnnoy fails to install. I can replicate that under Docker with Debian's gcc-snapshot (currently at 20180908). We see

/usr/lib/gcc-snapshot/bin/g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -I"/usr/local/lib/R/site-library/Rcpp/include"    -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-udh3fS/r-base-3.5.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c annoy.cpp -o annoy.o
In file included from annoy.cpp:39:
../inst/include/annoylib.h: In instantiation of 'void AnnoyIndex<S, T, Distance, Random>::_get_all_nns(const T*, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = float; Distance = Angular; Random = Kiss64Random; size_t = long unsigned int]':
../inst/include/annoylib.h:602:5:   required from 'void AnnoyIndex<S, T, Distance, Random>::get_nns_by_item(S, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = float; Distance = Angular; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:69:9:   required from 'std::vector<T> Annoy<S, T, Distance, Random>::getNNsByItem(S, size_t) [with S = int; T = float; Distance = Angular; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:142:50:   required from here
../inst/include/annoylib.h:737:68: error: cannot bind packed field 'nd->Angular::Node<int, float>::children[1]' to 'int&'
737 |         q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h:738:68: error: cannot bind packed field 'nd->Angular::Node<int, float>::children[0]' to 'int&'
738 |         q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h: In instantiation of 'void AnnoyIndex<S, T, Distance, Random>::_get_all_nns(const T*, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = float; Distance = Euclidean; Random = Kiss64Random; size_t = long unsigned int]':
../inst/include/annoylib.h:602:5:   required from 'void AnnoyIndex<S, T, Distance, Random>::get_nns_by_item(S, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = float; Distance = Euclidean; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:69:9:   required from 'std::vector<T> Annoy<S, T, Distance, Random>::getNNsByItem(S, size_t) [with S = int; T = float; Distance = Euclidean; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:168:52:   required from here
../inst/include/annoylib.h:737:68: error: cannot bind packed field 'nd->Minkowski::Node<int, float>::children[1]' to 'int&'
737 |         q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h:738:68: error: cannot bind packed field 'nd->Minkowski::Node<int, float>::children[0]' to 'int&'
738 |         q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h: In instantiation of 'void AnnoyIndex<S, T, Distance, Random>::_get_all_nns(const T*, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = float; Distance = Manhattan; Random = Kiss64Random; size_t = long unsigned int]':
../inst/include/annoylib.h:602:5:   required from 'void AnnoyIndex<S, T, Distance, Random>::get_nns_by_item(S, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = float; Distance = Manhattan; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:69:9:   required from 'std::vector<T> Annoy<S, T, Distance, Random>::getNNsByItem(S, size_t) [with S = int; T = float; Distance = Manhattan; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:194:52:   required from here
../inst/include/annoylib.h:737:68: error: cannot bind packed field 'nd->Minkowski::Node<int, float>::children[1]' to 'int&'
737 |         q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h:738:68: error: cannot bind packed field 'nd->Minkowski::Node<int, float>::children[0]' to 'int&'
738 |         q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h: In instantiation of 'void AnnoyIndex<S, T, Distance, Random>::_get_all_nns(const T*, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = long unsigned int; Distance = Hamming; Random = Kiss64Random; size_t = long unsigned int]':
../inst/include/annoylib.h:602:5:   required from 'void AnnoyIndex<S, T, Distance, Random>::get_nns_by_item(S, size_t, size_t, std::vector<T>*, std::vector<T>*) [with S = int; T = long unsigned int; Distance = Hamming; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:69:9:   required from 'std::vector<T> Annoy<S, T, Distance, Random>::getNNsByItem(S, size_t) [with S = int; T = long unsigned int; Distance = Hamming; Random = Kiss64Random; size_t = long unsigned int]'
annoy.cpp:220:50:   required from here
../inst/include/annoylib.h:737:68: error: cannot bind packed field 'nd->Hamming::Node<int, long unsigned int>::children[1]' to 'int&'
737 |         q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
    |                                                        ~~~~~~~~~~~~^
../inst/include/annoylib.h:738:68: error: cannot bind packed field 'nd->Hamming::Node<int, long unsigned int>::children[0]' to 'int&'
738 |         q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));
    |                                                        ~~~~~~~~~~~~^
make: *** [/usr/lib/R/etc/Makeconf:168: annoy.o] Error 1
ERROR: compilation failed for package ‘RcppAnnoy’
* removing ‘/usr/local/lib/R/site-library/RcppAnnoy’

and that code seems unchanged between what I have in RcppAnnoy and Annoy:

      } else {
        T margin = D::margin(nd, v, _f);
        q.push(make_pair(D::pq_distance(d, margin, 1), nd->children[1]));
        q.push(make_pair(D::pq_distance(d, margin, 0), nd->children[0]));
      }

This is of course pretty much at the core of Annoy. Thoughts about how to avoid this while not giving up performance?

[erikbern]

my guess is we just need to cast pointer types explicitly

[eddelbuettel]

I can probably test something. My Docker container with this is still up.

[erikbern]

actually looking at it again i'm confused. seems like make_pair is acting up

maybe we need to be explicit about what template to use? i think you could do something like make_pair<x, y>(...) right? my c++ is getting more and more rusty

[eddelbuettel]

Good call on the cast. A simple static_cast<S>(...) was all it took: eddelbuettel@dbe5bfe

I send a PR if it passes Travis.

[erikbern]

nice!

[dkobak]

I'm suddenly getting lots of warnings when compiling with g++ included in Ubuntu 19.10 (which I think is g++ 9). Everything still compiles but all the warnings are a bit frightening :) Is it related to the errors here?

[eddelbuettel]

What we had above were errors which stop us in our tracks. As you mention 19.10, I just installed my RcppAnnoy (wrapper around Annoy) on it, and yes -- with g++-9 I also see warnings but those are just that: warnings. And g++ tells us how to suppress them: add -Wno-address-of-packed-member. And when I do that all is quiet apart from the one #pragma added to make noise:

edd@rob:~/git/rcppannoy(master)$ install.r 
* installing *source* package found in current working directory ...
* installing *source* package ‘RcppAnnoy’ ...
** using staged installation
** libs
ccache g++  -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -I"/usr/local/lib/R/site-library/Rcpp/include"   -fpic  -g -O3 -Wall -pipe -pedantic -Wno-misleading-indentation -Wno-unused -Wno-ignored-attributes -Wno-parentheses -Wno-address-of-packed-member -c annoy.cpp -o annoy.o
In file included from annoy.cpp:39:
../inst/include/annoylib.h:84:17: note: #pragma message: Using no AVX instructions
   84 | #pragma message "Using no AVX instructions"
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
ccache gcc -I"/usr/share/R/include" -DNDEBUG -I../inst/include/ -I"/usr/local/lib/R/site-library/Rcpp/include"   -fpic  -g -O3 -Wall -pipe -pedantic -Wno-misleading-indentation -Wno-unused -Wno-ignored-attributes -Wno-parentheses -Wno-address-of-packed-member -std=gnu99 -c init.c -o init.o
ccache g++ -std=gnu++11 -Wl,-S -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o RcppAnnoy.so annoy.o init.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-rcppannoy/00new/RcppAnnoy/libs
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppAnnoy)
edd@rob:~/git/rcppannoy(master)$ 

That said, CRAN is always reminding us that this does get a permanent warning from the SAN/UBSAN analyser ... to which I usually respond that it is "by design". See
https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-UBSAN/RcppAnnoy/RcppAnnoy-Ex.Rout
https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-UBSAN/RcppAnnoy/RcppAnnoy-Ex.Rout

Edit: Typos edited on 05 Jan 2020.

[dkobak]

Thanks! Adding -Wno-address-of-packed-member worked great to remove scary warnings.