Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix build with PDFium #1400

Closed
revathi-murali opened this issue Aug 20, 2019 · 33 comments
Closed

fix build with PDFium #1400

revathi-murali opened this issue Aug 20, 2019 · 33 comments

Comments

@revathi-murali
Copy link

revathi-murali commented Aug 20, 2019

We are upgrading to libvips 8.8.1 and looks like there are some changes made in 8.8.1 with pdfium linking and I get the following error when building vips:

`/usr/lib/gcc/x86_64-alpine-linux-musl/8.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lc++
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:750: libvips.la] Error 1
make[3]: *** [Makefile:877: install-recursive] Error 1
make[2]: *** [Makefile:632: install-recursive] Error 1
make[1]: *** [Makefile:943: install-strip] Error 2
make: *** [Makefile:9: build] Error 2

Command used to build libvips

aclocal && automake && \
	./configure --prefix=/usr \
		--with-pdfium-includes=/usr/include/pdfium \
		--with-pdfium-libraries=/usr/lib \
		--without-python \
		--without-gsf \
		--enable-debug=no \
		--disable-dependency-tracking \
		--disable-static \
		--enable-silent-rules && \
	make -s install-strip

Since alpine uses musl libc, I am not sure if glibc/libc++ works fine on musl. Any pointers on this?

Note: We are using the following patches already

  1. https://github.com/jcupitt/docker-builds/blob/31cc11fcd160f2f722eb66841891e6d00c79da10/pdfium/data/build-pdfium.sh#L29-L33
@jcupitt
Copy link
Member

jcupitt commented Aug 21, 2019

Something about the way you've built pdfium, I guess. I know libvips itself builds fine on alpine.

C++ link problems are awful :( The two usual solutions are not to use -lc++, or to link with g++. You could check the .pc file that pdfium build makes.

@revathi-murali
Copy link
Author

revathi-murali commented Aug 21, 2019

Sure I will check the .pc file and update you.

@vposham
Copy link

vposham commented Aug 24, 2019

I had issues (exactly this one - jcupitt/docker-builds#2) linking pdfium with vips 8.7.4.

So, Im currently using separate microservices - one to do image processing by using vips and bimg without pdfium for all image formats and another one for all PDF operations without using vips by calling pdfium (by using pdfium binaries) and libjpeg header files using cgo.

I always was tried to build vips on debain based systems and ran into jcupitt/docker-builds#2 this issue.

@jcupitt, Do you suggest me to use alpine based base image for linking pdfium with vips even for 8.7.4to avoid this issue (jcupitt/docker-builds#2)?

@revathi-murali , If you've managed to make vips 8.7.4 to work with pdfium, could you please share that docker file?

Thanks

@jcupitt
Copy link
Member

jcupitt commented Dec 7, 2019

Build with pdfium needs fixing :( Let's leave this issue as a reminder.

@jcupitt jcupitt changed the title Compiling vips with pdfium - cannot find lc++ in musl fix build with PDFium Dec 7, 2019
@tomrucki
Copy link

Taking binaries from https://github.com/bblanchon/pdfium-binaries and then installing libicu-dev libc++-dev libc++abi-dev it seems to work. But I'm not sure how valid that approach is?

@jcupitt
Copy link
Member

jcupitt commented Feb 15, 2020

If it works, that's great, thank you for the update.

We need to sit down and fix this properly :(

@vposham
Copy link

vposham commented Feb 17, 2020

If anyone needs pdfium working with binaries (without libvips) - refer this - https://stackoverflow.com/a/60254335/7438923

@asterikx
Copy link

@jcupitt do you know if there is any progress on this?

I'm facing the same issue as OP. I need to build libvips with PDFium support for AWS Lambda (Amazon Linux 1).

I fetch the latest pre-built PDFium binary from https://github.com/bblanchon/pdfium-binaries and build libvips with the --with-pdfium-includes and --with-pdfium-libraries options pointing to the corresponding paths:

curl -L https://github.com/bblanchon/pdfium-binaries/releases/latest/download/pdfium-linux.tgz > pdfium-linux.tgz && \
  mkdir -p /opt/pdfium-linux && \
  tar -xf pdfium-linux.tgz -C /opt
./configure \
  --prefix=/opt \
  --disable-gtk-doc \
  --without-magick \
  --with-expat=/opt && \
  --with-pdfium-includes=/opt/include \
  --with-pdfium-libraries=/opt/lib
make && \
  make install

Error message:

make[3]: Entering directory `/build/vips-8.9.2/libvips'
/bin/sh ../libtool  --tag=CXX   --mode=link g++  -g -O2 -no-undefined -version-info 54:2:12   -o libvips.la -rpath /opt/lib  resample/libresample.la arithmetic/libarithmetic.la colour/libcolour.la conversion/libconversion.la convolution/libconvolution.la deprecated/libdeprecated.la foreign/libforeign.la freqfilt/libfreqfilt.la histogram/libhistogram.la draw/libdraw.la iofuncs/libiofuncs.la morphology/libmorphology.la mosaicing/libmosaicing.la create/libcreate.la -lz     -lpng12    -ltiff   -ljpeg   -pthread -lgthread-2.0 -lglib-2.0   -Wl,--export-dynamic -pthread -lgmodule-2.0 -lgobject-2.0 -lglib-2.0   -L/opt/lib -lexpat     -llcms2      -L/opt/lib -lpdfium -lc++ -licuuc         -lm 
libtool: link: g++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-amazon-linux/4.8.5/../../../../lib64/crti.o /usr/lib/gcc/x86_64-amazon-linux/4.8.5/crtbeginS.o  -Wl,--whole-archive resample/.libs/libresample.a arithmetic/.libs/libarithmetic.a colour/.libs/libcolour.a conversion/.libs/libconversion.a convolution/.libs/libconvolution.a deprecated/.libs/libdeprecated.a foreign/.libs/libforeign.a freqfilt/.libs/libfreqfilt.a histogram/.libs/libhistogram.a draw/.libs/libdraw.a iofuncs/.libs/libiofuncs.a morphology/.libs/libmorphology.a mosaicing/.libs/libmosaicing.a create/.libs/libcreate.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/usr/lib64 -lz -lpng12 -ltiff -ljpeg -lgthread-2.0 -lgmodule-2.0 -lgobject-2.0 -lglib-2.0 -L/opt/lib /usr/lib64/libexpat.so -llcms2 -lpdfium -lc++ -licuuc -L/usr/lib/gcc/x86_64-amazon-linux/4.8.5 -L/usr/lib/gcc/x86_64-amazon-linux/4.8.5/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-amazon-linux/4.8.5/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-amazon-linux/4.8.5/crtendS.o /usr/lib/gcc/x86_64-amazon-linux/4.8.5/../../../../lib64/crtn.o  -g -O2 -pthread -Wl,--export-dynamic -pthread   -pthread -Wl,-soname -Wl,libvips.so.42 -o .libs/libvips.so.42.12.2
/usr/bin/ld: cannot find -lc++
collect2: error: ld returned 1 exit status

I also tried to build from source (see PDFium repo), but the ./build/install-build-deps.sh script does not support Amazon Linux 1:

ERROR: The only supported distros are
 	Ubuntu 14.04 LTS (trusty with EoL April 2022)
 	Ubuntu 16.04 LTS (xenial with EoL April 2024)
 	Ubuntu 18.04 LTS (bionic with EoL April 2028)
 	Ubuntu 20.04 LTS (focal with Eol April 2030)
 	Ubuntu 19.04 (disco)
 	Ubuntu 19.10 (eoan)
 	Debian 8 (jessie) or later

I appreciate any help!

@jcupitt
Copy link
Member

jcupitt commented Jul 25, 2020

Sorry, we've been getting ready for the 8.10 release. Perhaps in the 8.11 cycle.

@asterikx
Copy link

@jcupitt you don't have to apologize for anything ;) I'll try to use Poppler in the meantime. I hope improved PDFium support can make it into 8.11.

@Projkt-James
Copy link

Projkt-James commented Sep 28, 2020

Just to continue on from #1837, I am able to successfully build libvips 8.9.1 (haven't test 8.10+ yet) referencing the latest linux build from https://github.com/bblanchon/pdfium-binaries and have tested pdf to jpg renders via both CLI and Sharp with no issue. Fortunately the warning doesn't appear to be fatal. I am happy to provide a dockerfile or guidance for anyone struggling with the libc++ libicu linkage step.

@asterikx
Copy link

I am happy to provide a dockerfile or guidance for anyone struggling with the libc++ libicu linkage step.

@Projkt-James I would gladly see the Dockerfile 👍

@vposham
Copy link

vposham commented Oct 15, 2020

I am happy to provide a dockerfile or guidance for anyone struggling with the libc++ libicu linkage step.

@Projkt-James I would gladly see the Dockerfile 👍

#1400 (comment)

https://stackoverflow.com/a/60254335/7438923

@jcupitt
Copy link
Member

jcupitt commented Oct 15, 2020

Nice! Dockerfile copied here for reference:

FROM ubuntu:16.04
# Specify pdfium version
ARG PdfiumVersion=4026

# Install pkg-config, etc.
RUN apt-get -yqq update \
    && apt-get clean \
    && apt-get install -yqq apt-utils pkg-config tzdata \
    && dpkg-reconfigure -f noninteractive tzdata

# Create .pc file for pkg-config
RUN echo "\n" \
     "prefix=/home\n" \
     "Name: pdfium\n" \
     "Description: pdfium\n" \
     "Version: $PdfiumVersion\n" \
     "Requires:\n" \
     "Libs: -L/home/lib -lpdfium\n" \
     "Cflags: -I/home/include\n" >  /home/pdfium.pc

# Download and extract pdfium binary
RUN cd /home \
    && wget --quiet https://github.com/bblanchon/pdfium-binaries/releases/download/chromium%2F$PdfiumVersion/pdfium-linux.tgz \
    && tar -xf pdfium-linux.tgz \
    && rm pdfium-linux.tgz

# Setting up paths for pkg-config
ENV LD_LIBRARY_PATH=/home/lib
ENV PKG_CONFIG_PATH=/home/

## COPY YOUR APP TO /app/src/yourApp
# BUILD YOUR APP
WORKDIR /app/src/yourApp

# RUN your app which is linked to pdfium
ENTRYPOINT [“./yourApp"]

jcupitt added a commit that referenced this issue Oct 16, 2020
It had bitrotted a bit. Thanks @Projkt-James.

See #1400
@jcupitt
Copy link
Member

jcupitt commented Oct 16, 2020

I've fixed build with pdfium -- it builds cleanly against the current binary from https://github.com/bblanchon/pdfium-binaries

There's a comment in pdfiumload.c with build notes:

/* How to build against PDFium:
 *
 * Download the prebuilt binary from: 
 *
 *      https://github.com/bblanchon/pdfium-binaries
 *
 * Untar to the libvips install prefix, for example:
 *
 *      cd ~/vips
 *      tar xf ~/pdfium-linux.tgz
 *
 * Create a pdfium.pc like this (update the version number):
 *

VIPSHOME=/home/john/vips
cat > $VIPSHOME/lib/pkgconfig/pdfium.pc << EOF
     prefix=$VIPSHOME
     exec_prefix=\${prefix}
     libdir=\${exec_prefix}/lib
     includedir=\${prefix}/include
     Name: pdfium
     Description: pdfium
     Version: 4290
     Requires:
     Libs: -L\${libdir} -lpdfium
     Cflags: -I\${includedir}
EOF

 * 
 */

Then just ./configure etc and it should build cleanly and work.

I'll try to add support for VipsSource as well.

@jcupitt
Copy link
Member

jcupitt commented Oct 16, 2020

OK, all done! I'll close. This will be in 8.11. Thanks for explaining how to fix the linking.

@jcupitt jcupitt closed this as completed Oct 16, 2020
@asterikx
Copy link

asterikx commented Nov 5, 2020

I decided to give it a try again. My setup differs slightly since I need to build for lambda (everything must end up in /opt)

Unfortunately, libvips is no detecting my pdfium installation. I created pdfium.pc in /opt/lib/pkgconfig (which is in my PKG_CONFIG_PATH) with the following content:

prefix=/opt
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
Name: pdfium
Description: pdfium
Version: 4313
Requires:
Libs: -L${libdir} -lpdfium
Cflags: -I${includedir}

and extracted the pre-built binaries into /opt.

libvips' output includes the following lines:

checking for PDFIUM... libraries (none), headers -I/opt/include
PDF import with PDFium:                 no

I can't really see what is wrong with my setup.

I hope someone can help me here.

Dockerfile
FROM lambci/lambda:build-go1.x AS compile-image

ENV BUILD_DIR="/tmp/build"
ENV INSTALL_DIR="/opt"


# Create all necessary directories
#
RUN set -Eeuxo pipefail \
  && mkdir -p ${BUILD_DIR} \
  ${INSTALL_DIR}/{bin,doc,include,lib,lib64,libexec,sbin,share} \
  ${INSTALL_DIR}/{lib,lib64}/pkgconfig


# Install Development Tools
#
WORKDIR /tmp

RUN set -Eeuxo pipefail \
  && yum -y update \
  && yum -y install \
  gcc72-c++ \
  nasm \
  ninja-build \
  && yum -y clean all

RUN pip3 install meson


# Set GCC to 7.2.1-2 (default is 4.8.5-28 on lambci/lambda:build-go1.x image)
RUN update-alternatives --set gcc /usr/bin/gcc72


# Configure Default Compiler Variables
#
ENV PKG_CONFIG_PATH="${INSTALL_DIR}/lib64/pkgconfig:${INSTALL_DIR}/lib/pkgconfig" \
  PKG_CONFIG="/usr/bin/pkg-config" \
  PATH="${INSTALL_DIR}/bin:${PATH}"

ENV LD_LIBRARY_PATH="${INSTALL_DIR}/lib64:${INSTALL_DIR}/lib"


# Install Libpng (http://www.linuxfromscratch.org/blfs/view/svn/general/libpng.html)
#
# Pre-installed on Amazon Linux: 1.2.49
# Required by (optional): libvips
#
ENV LIBPNG_VERSION=1.6.37
ENV LIBPNG_BUILD_DIR=${BUILD_DIR}/libpng

RUN set -Eeuxo pipefail \
  && mkdir -p ${LIBPNG_BUILD_DIR} \
  && curl -L https://downloads.sourceforge.net/libpng/libpng-${LIBPNG_VERSION}.tar.xz \
  | tar xJC ${LIBPNG_BUILD_DIR} --strip-components=1

WORKDIR ${LIBPNG_BUILD_DIR}/

RUN set -Eeuxo pipefail \
  && ./configure \
  --prefix=${INSTALL_DIR} \
  --with-sysroot=${INSTALL_DIR} \
  --disable-static \
  && make V=0 \
  && make install


# Install Libjpeg-Turbo (http://www.linuxfromscratch.org/blfs/view/svn/general/libjpeg.html)
#
# Pre-installed on Amazon Linux: no
# Required by (optional): libvips
#
ENV LIBJPEG_TURBO_VERSION=2.0.5
ENV LIBJPEG_TURBO_BUILD_DIR=${BUILD_DIR}/libjpeg

RUN set -Eeuxo pipefail \
  && mkdir -p ${LIBJPEG_TURBO_BUILD_DIR}/build \
  && curl -L https://downloads.sourceforge.net/libjpeg-turbo/libjpeg-turbo-${LIBJPEG_TURBO_VERSION}.tar.gz \
  | tar xzC ${LIBJPEG_TURBO_BUILD_DIR} --strip-components=1

WORKDIR ${LIBJPEG_TURBO_BUILD_DIR}/build/

RUN set -Eeuxo pipefail \
  && cmake .. \
  -DCMAKE_PREFIX_PATH=${INSTALL_DIR} \
  -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} \
  -DCMAKE_BUILD_TYPE=RELEASE \
  -DENABLE_STATIC=FALSE \
  -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \
  && make V=0 \
  && make install


# Install libexif (http://www.linuxfromscratch.org/blfs/view/svn/general/libexif.html)
#
# Pre-installed on Amazon Linux: no
# Required by (optional): libvips # https://github.com/libvips/libvips/issues/1749#issuecomment-666312296
#
ENV LIBEXIF_VERSION=0.6.22
ENV LIBEXIF_VERSION_SNAKECASE=0_6_22
ENV LIBEXIF_BUILD_DIR=${BUILD_DIR}/libexif

RUN set -Eeuxo pipefail \
  && mkdir -p ${LIBEXIF_BUILD_DIR} \
  && curl -L https://github.com/libexif/libexif/releases/download/libexif-${LIBEXIF_VERSION_SNAKECASE}-release/libexif-${LIBEXIF_VERSION}.tar.xz \
  | tar xJC ${LIBEXIF_BUILD_DIR} --strip-components=1

WORKDIR ${LIBEXIF_BUILD_DIR}

RUN set -Eeuxo pipefail \
  && ./configure --prefix=${INSTALL_DIR} \
  --disable-static \
  && make V=0 \
  && make install


# Install Little CMS (http://www.linuxfromscratch.org/blfs/view/svn/general/lcms2.html)
#
# Pre-installed on Amazon Linux: 2.6
# Required by (recommended): libvips # https://github.com/libvips/libvips/issues/1749#issuecomment-666606745
#
ENV LCMS2_VERSION=2.11
ENV LCMS2_BUILD_DIR=${BUILD_DIR}/lcms

RUN set -Eeuxo pipefail \
  && mkdir -p ${LCMS2_BUILD_DIR} \
  && curl -L https://downloads.sourceforge.net/lcms/lcms2-${LCMS2_VERSION}.tar.gz \
  | tar xzC ${LCMS2_BUILD_DIR} --strip-components=1

WORKDIR ${LCMS2_BUILD_DIR}/

RUN set -Eeuxo pipefail \
  && ./configure  \
  --prefix=${INSTALL_DIR} \
  --disable-static \
  && make V=0 \
  && make install


# Install orc (https://gitlab.freedesktop.org/gstreamer/orc/-/blob/master/.gitlab-ci.yml#L15)
#
# Pre-installed on Amazon Linux: no
# Required by (recommended): libvips # https://github.com/libvips/libvips/issues/1749#issuecomment-666312296
#
ENV ORC_VERSION=0.4.32
ENV ORC_BUILD_DIR=${BUILD_DIR}/orc

RUN set -Eeuxo pipefail \
  && mkdir -p ${ORC_BUILD_DIR} \
  && curl -L https://gstreamer.freedesktop.org/src/orc/orc-${ORC_VERSION}.tar.xz \
  | tar xJC ${ORC_BUILD_DIR} --strip-components=1

WORKDIR ${ORC_BUILD_DIR}

RUN set -Eeuxo pipefail \
  && mkdir build \
  && cd build \
  && meson --prefix=${INSTALL_DIR} --werror \
  && ninja \
  && ninja install


# Install PDFium (http://www.linuxfromscratch.org/blfs/view/svn/general/cmake.html)
#
# Pre-installed on Amazon Linux: no
#
ENV PDFIUM_VERSION=4313

RUN set -Eeuxo pipefail \
  && mkdir -p ${INSTALL_DIR} \
  && curl -L https://github.com/bblanchon/pdfium-binaries/releases/download/chromium%2F${PDFIUM_VERSION}/pdfium-linux.tgz \
  | tar xC ${INSTALL_DIR}

RUN echo -e "\
  prefix=${INSTALL_DIR}\n\
  exec_prefix=\${prefix}\n\
  libdir=\${exec_prefix}/lib\n\
  includedir=\${prefix}/include\n\
  Name: pdfium\n\
  Description: pdfium\n\
  Version: ${PDFIUM_VERSION}\n\
  Requires:\n\
  Libs: -L\${libdir} -lpdfium\n\
  Cflags: -I\${includedir}\n" > ${INSTALL_DIR}/lib/pkgconfig/pdfium.pc

RUN cat ${INSTALL_DIR}/lib/pkgconfig/pdfium.pc


# Install libvips (https://libvips.github.io/libvips/install.html)
#
# Pre-installed on Amazon Linux: no
#
ENV LIBVIPS_VERSION=8.10.2
ENV LIBVIPS_BUILD_DIR=${BUILD_DIR}/libvips

RUN set -Eeuxo pipefail \
  && mkdir -p ${LIBVIPS_BUILD_DIR} \
  && curl -L https://github.com/libvips/libvips/releases/download/v${LIBVIPS_VERSION}/vips-${LIBVIPS_VERSION}.tar.gz \
  | tar xzC ${LIBVIPS_BUILD_DIR} --strip-components=1

WORKDIR ${LIBVIPS_BUILD_DIR}

RUN set -Eeuxo pipefail \
  && ./configure --prefix=${INSTALL_DIR} \
  --without-gsf \
  --without-fftw \
  --without-magick \
  --without-OpenEXR \
  --without-nifti \
  --without-heif \
  # --without-pdfium \
  --with-pdfium-includes=${INSTALL_DIR}/include \
  --with-pdfium-libraries=${INSTALL_DIR}/lib \
  --without-rsvg \
  --without-openslide \
  --without-matio \
  --without-cfitsio \
  --without-libwebp \
  --without-pangoft2 \
  --without-tiff \
  --without-giflib \
  --without-imagequant \
  --disable-gtk-doc \
  && make V=0 \
  && make install \
  # not sure what this does and how to copy it (lambda layers are unpacked to /opt)
  && echo /opt/lib > /etc/ld.so.conf.d/libvips.conf \
  && ldconfig


FROM lambci/lambda:build-go1.x AS runtime-image

ENV SOURCE_DIR="/opt"
ENV INSTALL_DIR="/opt"

ENV PATH="/opt/bin:${PATH}" \
  LD_LIBRARY_PATH="${INSTALL_DIR}/lib64:${INSTALL_DIR}/lib"

# Install zip
#
RUN set -Eeuxo pipefail \
  && yum update -y \
  && yum -y install zip \
  && yum -y clean all

# Copy all binaries/libaries
#
RUN set -Eeuxo pipefail \
  && mkdir -p ${INSTALL_DIR}/{etc,bin,var,share,lib}

COPY --from=compile-image /lib64/libuuid.so.* ${INSTALL_DIR}/lib/
COPY --from=compile-image ${SOURCE_DIR}/share/ /tmp/share
COPY --from=compile-image ${SOURCE_DIR}/bin/ ${INSTALL_DIR}/bin/
COPY --from=compile-image ${SOURCE_DIR}/var/ ${INSTALL_DIR}/var/
COPY --from=compile-image ${SOURCE_DIR}/lib/ ${INSTALL_DIR}/lib/
COPY --from=compile-image ${SOURCE_DIR}/lib64/ ${INSTALL_DIR}/lib/

@asterikx
Copy link

asterikx commented Nov 5, 2020

When I add the options --with-pdfium-includes=${INSTALL_DIR}/include and --with-pdfium-libraries=${INSTALL_DIR}/lib, livips recognizes pdfium.

However, the built later fails with:

make[3]: Entering directory `/tmp/build/libvips/libvips'
  CXXLD    libvips.la
/usr/bin/ld: cannot find -lc++
collect2: error: ld returned 1 exit status
make[3]: *** [libvips.la] Error 1
make[3]: Leaving directory `/tmp/build/libvips/libvips'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/tmp/build/libvips/libvips'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/tmp/build/libvips'
make: *** [all] Error 2

which it doesn't when built without pdfium support.

Shouldn't --with-pdfium-includes and --with-pdfium-libraries options normally be unnecessary since libvips will auto-detect those using pkgconfig?

@jcupitt
Copy link
Member

jcupitt commented Nov 5, 2020

Hi @asterikx, thanks for testing this.

libvips' output includes the following lines:
checking for PDFIUM... libraries (none), headers -I/opt/include

That looks odd. I see:

checking for PDFIUM... yes
configure: WARNING: PDFium found, disabling poppler

I wonder if you don't have git master libvips, or if you've not run autogen.sh since updating it?

For reference, my config.log looks like this:

configure:20958: checking for PDFIUM
configure:20965: $PKG_CONFIG --exists --print-errors "pdfium >= 4200"
configure:20968: $? = 0
configure:20982: $PKG_CONFIG --exists --print-errors "pdfium >= 4200"
configure:20985: $? = 0
configure:21027: result: yes
configure:21034: WARNING: PDFium found, disabling poppler

My pkg-config shows:

john@yingna ~/GIT/libvips (master) $ pkg-config pdfium --libs
-L/home/john/vips/lib -lpdfium
john@yingna ~/GIT/libvips (master) $ pkg-config pdfium --cflags
-I/home/john/vips/include

@jcupitt
Copy link
Member

jcupitt commented Nov 5, 2020

Oh heh I just saw your dockerfile, sorry.

Yes, this fix is git master libvips only, and you will need to use autogen.sh, not configure.

@asterikx
Copy link

asterikx commented Nov 5, 2020

@jcupitt thank you! It finally works 🙂

@nator
Copy link

nator commented Dec 14, 2020

Sorry for noise on this ticket, but I've gotten this built locally and I'm not sure how to tell if it's actually using PDFium? I see this in config.log:

3035-configure:21056: checking for RSVG
3036-configure:21063: $PKG_CONFIG --exists --print-errors "librsvg-2.0 >= 2.40.3 cairo >= 1.2"
3037-configure:21066: $? = 0
3038-configure:21080: $PKG_CONFIG --exists --print-errors "librsvg-2.0 >= 2.40.3 cairo >= 1.2"
3039-configure:21083: $? = 0
3040-configure:21129: result: yes

but vips -l still shows poppler?

          VipsForeignLoadPdfFile (pdfload), load PDF with libpoppler (.pdf), priority=0, is_a, get_flags, get_flags_filename, header, load
          VipsForeignLoadPdfBuffer (pdfload_buffer), load PDF with libpoppler, priority=0, is_a_buffer, get_flags, get_flags_filename, header, load```

Is there any way to tell if pdfium is actually being used by libvips? Should I remove poppler from the system so there's no chance it's used?

@jcupitt
Copy link
Member

jcupitt commented Dec 15, 2020

Hi @nator,

You should see:

$ vips -l | grep -i pdf
        VipsForeignLoadPdf (pdfload_base), load PDF with PDFium, priority=0
...

If you have both pdfium and poppler installed, libvips will pick pdfium, so I would guess your pdfium install probably failed somehow. Did you make the .pc file?

@nator
Copy link

nator commented Dec 15, 2020

Oops I pasted the entirely wrong line from config.log :(

configure:20848: gcc -c -g -O2 -I/usr/local/include  conftest.c >&5
configure:20848: $? = 0
configure:20882: gcc -o conftest -g -O2 -I/usr/local/include   conftest.c -L/usr/local/lib -lpdfium -lc++ -licuuc  >&5
configure:20882: $? = 0
configure:20912: result: libraries /usr/local/lib, headers -I/usr/local/include
configure:20930: WARNING: PDFium found, disabling poppler

Yes, I made the .pc file, but let me double check everything one last time now that I know to expect "load PDF with PDFium" in vips -l. Thanks for answering!

@jcupitt
Copy link
Member

jcupitt commented Dec 15, 2020

The safest check is the summary at the end of configure. During configure you should see:

...
checking for HEIF_AVIF... no
checking for struct heif_decoding_options.convert_hdr_to_8bit... no
checking for PDFIUM... yes
configure: WARNING: PDFium found, disabling poppler
checking for RSVG... yes
checking for ZLIB... yes
...

Then in the summary at the end:

...
file import with matio: 		yes
PDF import with PDFium: 		yes
PDF import with poppler-glib: 		no
  (requires poppler-glib 0.16.0 or later)
SVG import with librsvg-2.0: 		yes
...

@nator
Copy link

nator commented Dec 15, 2020

I think I mixed up my docker scripts and I've been building 8.10.2 instead of master. :( Trying that...

@jcupitt
Copy link
Member

jcupitt commented Dec 15, 2020

There's 8.10.4 now and it includes a fixed PDFium build. I'd try that.

@nator
Copy link

nator commented Dec 15, 2020

Hmm. I'm stumped:

root@28d2d1efdea9:/vips-8.10.4# grep PDF config.log 
configure:20825: checking for PDFIUM
configure:20930: WARNING: PDFium found, disabling poppler
| #define HAVE_PDFIUM 1
PDF import with PDFium: 		yes
PDF import with poppler-glib: 		no

root@28d2d1efdea9:/# vips -v
vips-8.10.4-Mon Dec 14 09:20:17 UTC 2020

root@28d2d1efdea9:/# vips -l | grep -i pdf                                                                                                                   
        VipsForeignLoadPdf (pdfload_base), load PDF with libpoppler, priority=0
          VipsForeignLoadPdfFile (pdfload), load PDF with libpoppler (.pdf), priority=0, is_a, get_flags, get_flags_filename, header, load
          VipsForeignLoadPdfBuffer (pdfload_buffer), load PDF with libpoppler, priority=0, is_a_buffer, get_flags, get_flags_filename, header, load

With 8.10.4 I did not use autogen.sh -- is that still required maybe? I'm building on Ubuntu 20.04 due to upstream requirements, if that helps debug this at all...

@nator
Copy link

nator commented Dec 15, 2020

ah, built by hand and I see lots of build errors I was missing in docker:

In file included from ../../libvips/include/vips/vips.h:130,
                 from pdfiumload.c:62:
pdfiumload.c: In function ‘vips_foreign_load_pdf_close’:
pdfiumload.c:148:33: warning: passing argument 1 of ‘FPDF_ClosePage’ from incompatible pointer type [-Wincompatible-pointer-types]
  148 |  VIPS_FREEF( FPDF_ClosePage, pdf->page );
../../libvips/include/vips/memory.h:42:28: note: in definition of macro ‘VIPS_FREEF’
   42 |                 (void) F( (S) ); \
      |                            ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:905:57: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  905 | FPDF_EXPORT void FPDF_CALLCONV FPDF_ClosePage(FPDF_PAGE page);
      |                                               ~~~~~~~~~~^~~~
In file included from ../../libvips/include/vips/vips.h:130,
                 from pdfiumload.c:62:
pdfiumload.c:149:37: warning: passing argument 1 of ‘FPDF_CloseDocument’ from incompatible pointer type [-Wincompatible-pointer-types]
  149 |  VIPS_FREEF( FPDF_CloseDocument, pdf->doc );
../../libvips/include/vips/memory.h:42:28: note: in definition of macro ‘VIPS_FREEF’
   42 |                 (void) F( (S) ); \
      |                            ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:913:65: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  913 | FPDF_EXPORT void FPDF_CALLCONV FPDF_CloseDocument(FPDF_DOCUMENT document);
      |                                                   ~~~~~~~~~~~~~~^~~~~~~~
In file included from ../../libvips/include/vips/vips.h:130,
                 from pdfiumload.c:62:
pdfiumload.c: In function ‘vips_foreign_load_pdf_get_page’:
pdfiumload.c:222:34: warning: passing argument 1 of ‘FPDF_ClosePage’ from incompatible pointer type [-Wincompatible-pointer-types]
  222 |   VIPS_FREEF( FPDF_ClosePage, pdf->page );
../../libvips/include/vips/memory.h:42:28: note: in definition of macro ‘VIPS_FREEF’
   42 |                 (void) F( (S) ); \
      |                            ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:905:57: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  905 | FPDF_EXPORT void FPDF_CALLCONV FPDF_ClosePage(FPDF_PAGE page);
      |                                               ~~~~~~~~~~^~~~
pdfiumload.c:229:39: warning: passing argument 1 of ‘FPDF_LoadPage’ from incompatible pointer type [-Wincompatible-pointer-types]
  229 |   if( !(pdf->page = FPDF_LoadPage( pdf->doc, page_no )) ) {
      |                                    ~~~^~~~~
      |                                       |
      |                                       struct fpdf_document_t__ **
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:662:65: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  662 | FPDF_EXPORT FPDF_PAGE FPDF_CALLCONV FPDF_LoadPage(FPDF_DOCUMENT document,
      |                                                   ~~~~~~~~~~~~~~^~~~~~~~
pdfiumload.c:229:19: warning: assignment to ‘struct fpdf_page_t__ **’ from incompatible pointer type ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} [-Wincompatible-pointer-types]
  229 |   if( !(pdf->page = FPDF_LoadPage( pdf->doc, page_no )) ) {
      |                   ^
pdfiumload.c: In function ‘vips_foreign_load_pdf_set_image’:
pdfiumload.c:289:30: warning: passing argument 1 of ‘FPDF_GetMetaText’ from incompatible pointer type [-Wincompatible-pointer-types]
  289 |   len = FPDF_GetMetaText( pdf->doc, metadata->tag, text, 1024 );
      |                           ~~~^~~~~
      |                              |
      |                              struct fpdf_document_t__ **
In file included from pdfiumload.c:71:
/usr/include/fpdf_doc.h:388:72: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  388 | FPDF_EXPORT unsigned long FPDF_CALLCONV FPDF_GetMetaText(FPDF_DOCUMENT document,
      |                                                          ~~~~~~~~~~~~~~^~~~~~~~
pdfiumload.c: In function ‘vips_foreign_load_pdf_header’:
pdfiumload.c:331:39: warning: passing argument 1 of ‘FPDF_GetPageCount’ from incompatible pointer type [-Wincompatible-pointer-types]
  331 |  pdf->n_pages = FPDF_GetPageCount( pdf->doc );
      |                                    ~~~^~~~~
      |                                       |
      |                                       struct fpdf_document_t__ **
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:650:63: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  650 | FPDF_EXPORT int FPDF_CALLCONV FPDF_GetPageCount(FPDF_DOCUMENT document);
      |                                                 ~~~~~~~~~~~~~~^~~~~~~~
In file included from ../../libvips/include/vips/vips.h:116,
                 from pdfiumload.c:62:
pdfiumload.c:366:26: warning: passing argument 1 of ‘FPDF_GetPageWidth’ from incompatible pointer type [-Wincompatible-pointer-types]
  366 |    FPDF_GetPageWidth( pdf->page ) * pdf->scale );
      |                       ~~~^~~~~~
      |                          |
      |                          struct fpdf_page_t__ **
../../libvips/include/vips/util.h:71:40: note: in definition of macro ‘VIPS_RINT’
   71 | #define VIPS_RINT( V ) __builtin_rint( V )
      |                                        ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:685:62: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  685 | FPDF_EXPORT double FPDF_CALLCONV FPDF_GetPageWidth(FPDF_PAGE page);
      |                                                    ~~~~~~~~~~^~~~
In file included from ../../libvips/include/vips/vips.h:116,
                 from pdfiumload.c:62:
pdfiumload.c:368:27: warning: passing argument 1 of ‘FPDF_GetPageHeight’ from incompatible pointer type [-Wincompatible-pointer-types]
  368 |    FPDF_GetPageHeight( pdf->page ) * pdf->scale );
      |                        ~~~^~~~~~
      |                           |
      |                           struct fpdf_page_t__ **
../../libvips/include/vips/util.h:71:40: note: in definition of macro ‘VIPS_RINT’
   71 | #define VIPS_RINT( V ) __builtin_rint( V )
      |                                        ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:707:63: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  707 | FPDF_EXPORT double FPDF_CALLCONV FPDF_GetPageHeight(FPDF_PAGE page);
      |                                                     ~~~~~~~~~~^~~~
pdfiumload.c: In function ‘vips_foreign_load_pdf_generate’:
pdfiumload.c:457:37: warning: passing argument 2 of ‘FPDF_RenderPageBitmap’ from incompatible pointer type [-Wincompatible-pointer-types]
  457 |   FPDF_RenderPageBitmap( bitmap, pdf->page,
      |                                  ~~~^~~~~~
      |                                     |
      |                                     struct fpdf_page_t__ **
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:860:64: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  860 |                                                      FPDF_PAGE page,
      |                                                      ~~~~~~~~~~^~~~
pdfiumload.c: In function ‘vips_foreign_load_pdf_file_header’:
pdfiumload.c:614:17: warning: assignment to ‘struct fpdf_document_t__ **’ from incompatible pointer type ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} -Wincompatible-pointer-types]
  614 |  if( !(pdf->doc = FPDF_LoadDocument( file->filename, NULL )) ) {
      |                 ^
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -I../../libvips/include -DG_DISABLE_ASSERT -DG_DISABLE_CHECKS -pthread -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pango-1.0 -I/usr/include/fribidi -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/libgsf-1 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/libxml2 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/include/libpng16 -I/usr/include/libexif -I/usr/lib/x86_64-linux-gnu/hdf5/serial/include -pthread -I/usr/include/librsvg-2.0 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/local/include -I/usr/include/OpenEXR -I/usr/include/openslide -I/usr/include/x86_64-linux-gnu -g -O2 -MT pdfload.lo -MD -MP -MF .deps/pdfload.Tpo -c pdfload.c -o pdfload.o >/dev/null 2>&1
pdfiumload.c: In function ‘vips_foreign_load_pdf_buffer_header’:
pdfiumload.c:690:17: warning: assignment to ‘struct fpdf_document_t__ **’ from incompatible pointer type ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} -Wincompatible-pointer-types]
  690 |  if( !(pdf->doc = FPDF_LoadMemDocument( buffer->buf->data,
      |                 ^

I'm using the 4355 PDFium binary, perhaps that's too new...

@nator
Copy link

nator commented Dec 15, 2020

Build with master and I'm sorted. Phew. Sorry for the endless thread :)

@jcupitt
Copy link
Member

jcupitt commented Dec 15, 2020

You're right, looking at the code you need git master libvips (8.11) for pdfium, I'd forgotten. Anyway, I'm glad it's working.

I made a demo Dockerfile that builds git master libvips against pdfium on 20.04, it might help someone:

https://github.com/jcupitt/docker-builds/blob/master/libvips-pdfium-ubuntu20.04/Dockerfile

@nator
Copy link

nator commented Dec 15, 2020 via email

@Projkt-James
Copy link

Projkt-James commented Dec 17, 2020

ah, built by hand and I see lots of build errors I was missing in docker:

In file included from ../../libvips/include/vips/vips.h:130,
                 from pdfiumload.c:62:
pdfiumload.c: In function ‘vips_foreign_load_pdf_close’:
pdfiumload.c:148:33: warning: passing argument 1 of ‘FPDF_ClosePage’ from incompatible pointer type [-Wincompatible-pointer-types]
  148 |  VIPS_FREEF( FPDF_ClosePage, pdf->page );
../../libvips/include/vips/memory.h:42:28: note: in definition of macro ‘VIPS_FREEF’
   42 |                 (void) F( (S) ); \
      |                            ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:905:57: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  905 | FPDF_EXPORT void FPDF_CALLCONV FPDF_ClosePage(FPDF_PAGE page);
      |                                               ~~~~~~~~~~^~~~
In file included from ../../libvips/include/vips/vips.h:130,
                 from pdfiumload.c:62:
pdfiumload.c:149:37: warning: passing argument 1 of ‘FPDF_CloseDocument’ from incompatible pointer type [-Wincompatible-pointer-types]
  149 |  VIPS_FREEF( FPDF_CloseDocument, pdf->doc );
../../libvips/include/vips/memory.h:42:28: note: in definition of macro ‘VIPS_FREEF’
   42 |                 (void) F( (S) ); \
      |                            ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:913:65: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  913 | FPDF_EXPORT void FPDF_CALLCONV FPDF_CloseDocument(FPDF_DOCUMENT document);
      |                                                   ~~~~~~~~~~~~~~^~~~~~~~
In file included from ../../libvips/include/vips/vips.h:130,
                 from pdfiumload.c:62:
pdfiumload.c: In function ‘vips_foreign_load_pdf_get_page’:
pdfiumload.c:222:34: warning: passing argument 1 of ‘FPDF_ClosePage’ from incompatible pointer type [-Wincompatible-pointer-types]
  222 |   VIPS_FREEF( FPDF_ClosePage, pdf->page );
../../libvips/include/vips/memory.h:42:28: note: in definition of macro ‘VIPS_FREEF’
   42 |                 (void) F( (S) ); \
      |                            ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:905:57: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  905 | FPDF_EXPORT void FPDF_CALLCONV FPDF_ClosePage(FPDF_PAGE page);
      |                                               ~~~~~~~~~~^~~~
pdfiumload.c:229:39: warning: passing argument 1 of ‘FPDF_LoadPage’ from incompatible pointer type [-Wincompatible-pointer-types]
  229 |   if( !(pdf->page = FPDF_LoadPage( pdf->doc, page_no )) ) {
      |                                    ~~~^~~~~
      |                                       |
      |                                       struct fpdf_document_t__ **
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:662:65: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  662 | FPDF_EXPORT FPDF_PAGE FPDF_CALLCONV FPDF_LoadPage(FPDF_DOCUMENT document,
      |                                                   ~~~~~~~~~~~~~~^~~~~~~~
pdfiumload.c:229:19: warning: assignment to ‘struct fpdf_page_t__ **’ from incompatible pointer type ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} [-Wincompatible-pointer-types]
  229 |   if( !(pdf->page = FPDF_LoadPage( pdf->doc, page_no )) ) {
      |                   ^
pdfiumload.c: In function ‘vips_foreign_load_pdf_set_image’:
pdfiumload.c:289:30: warning: passing argument 1 of ‘FPDF_GetMetaText’ from incompatible pointer type [-Wincompatible-pointer-types]
  289 |   len = FPDF_GetMetaText( pdf->doc, metadata->tag, text, 1024 );
      |                           ~~~^~~~~
      |                              |
      |                              struct fpdf_document_t__ **
In file included from pdfiumload.c:71:
/usr/include/fpdf_doc.h:388:72: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  388 | FPDF_EXPORT unsigned long FPDF_CALLCONV FPDF_GetMetaText(FPDF_DOCUMENT document,
      |                                                          ~~~~~~~~~~~~~~^~~~~~~~
pdfiumload.c: In function ‘vips_foreign_load_pdf_header’:
pdfiumload.c:331:39: warning: passing argument 1 of ‘FPDF_GetPageCount’ from incompatible pointer type [-Wincompatible-pointer-types]
  331 |  pdf->n_pages = FPDF_GetPageCount( pdf->doc );
      |                                    ~~~^~~~~
      |                                       |
      |                                       struct fpdf_document_t__ **
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:650:63: note: expected ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} but argument is of type ‘struct fpdf_document_t__ **’
  650 | FPDF_EXPORT int FPDF_CALLCONV FPDF_GetPageCount(FPDF_DOCUMENT document);
      |                                                 ~~~~~~~~~~~~~~^~~~~~~~
In file included from ../../libvips/include/vips/vips.h:116,
                 from pdfiumload.c:62:
pdfiumload.c:366:26: warning: passing argument 1 of ‘FPDF_GetPageWidth’ from incompatible pointer type [-Wincompatible-pointer-types]
  366 |    FPDF_GetPageWidth( pdf->page ) * pdf->scale );
      |                       ~~~^~~~~~
      |                          |
      |                          struct fpdf_page_t__ **
../../libvips/include/vips/util.h:71:40: note: in definition of macro ‘VIPS_RINT’
   71 | #define VIPS_RINT( V ) __builtin_rint( V )
      |                                        ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:685:62: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  685 | FPDF_EXPORT double FPDF_CALLCONV FPDF_GetPageWidth(FPDF_PAGE page);
      |                                                    ~~~~~~~~~~^~~~
In file included from ../../libvips/include/vips/vips.h:116,
                 from pdfiumload.c:62:
pdfiumload.c:368:27: warning: passing argument 1 of ‘FPDF_GetPageHeight’ from incompatible pointer type [-Wincompatible-pointer-types]
  368 |    FPDF_GetPageHeight( pdf->page ) * pdf->scale );
      |                        ~~~^~~~~~
      |                           |
      |                           struct fpdf_page_t__ **
../../libvips/include/vips/util.h:71:40: note: in definition of macro ‘VIPS_RINT’
   71 | #define VIPS_RINT( V ) __builtin_rint( V )
      |                                        ^
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:707:63: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  707 | FPDF_EXPORT double FPDF_CALLCONV FPDF_GetPageHeight(FPDF_PAGE page);
      |                                                     ~~~~~~~~~~^~~~
pdfiumload.c: In function ‘vips_foreign_load_pdf_generate’:
pdfiumload.c:457:37: warning: passing argument 2 of ‘FPDF_RenderPageBitmap’ from incompatible pointer type [-Wincompatible-pointer-types]
  457 |   FPDF_RenderPageBitmap( bitmap, pdf->page,
      |                                  ~~~^~~~~~
      |                                     |
      |                                     struct fpdf_page_t__ **
In file included from pdfiumload.c:70:
/usr/include/fpdfview.h:860:64: note: expected ‘FPDF_PAGE’ {aka ‘struct fpdf_page_t__ *’} but argument is of type ‘struct fpdf_page_t__ **’
  860 |                                                      FPDF_PAGE page,
      |                                                      ~~~~~~~~~~^~~~
pdfiumload.c: In function ‘vips_foreign_load_pdf_file_header’:
pdfiumload.c:614:17: warning: assignment to ‘struct fpdf_document_t__ **’ from incompatible pointer type ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} -Wincompatible-pointer-types]
  614 |  if( !(pdf->doc = FPDF_LoadDocument( file->filename, NULL )) ) {
      |                 ^
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../.. -I../../libvips/include -DG_DISABLE_ASSERT -DG_DISABLE_CHECKS -pthread -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pango-1.0 -I/usr/include/fribidi -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/libgsf-1 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/libxml2 -fopenmp -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/local/include/ImageMagick-7 -I/usr/include/libpng16 -I/usr/include/libexif -I/usr/lib/x86_64-linux-gnu/hdf5/serial/include -pthread -I/usr/include/librsvg-2.0 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/local/include -I/usr/include/OpenEXR -I/usr/include/openslide -I/usr/include/x86_64-linux-gnu -g -O2 -MT pdfload.lo -MD -MP -MF .deps/pdfload.Tpo -c pdfload.c -o pdfload.o >/dev/null 2>&1
pdfiumload.c: In function ‘vips_foreign_load_pdf_buffer_header’:
pdfiumload.c:690:17: warning: assignment to ‘struct fpdf_document_t__ **’ from incompatible pointer type ‘FPDF_DOCUMENT’ {aka ‘struct fpdf_document_t__ *’} -Wincompatible-pointer-types]
  690 |  if( !(pdf->doc = FPDF_LoadMemDocument( buffer->buf->data,
      |                 ^

I'm using the 4355 PDFium binary, perhaps that's too new...

I was building with 8.9.1 and majority of the warnings are non fatal from what I can tell. A lot of the issues I was having inside docker for older libvips versions was linking of libcxx and libicu.

alon-ne added a commit to wix-playground/libvips that referenced this issue Dec 21, 2020
* better webp load sanity checking

see libvips@d93d9bb#r40846309

* block fuzz data over 100kb

Many codecs can take a huge amount of time attempting to read large
random objects. jpeg_read_header(), for example, can take ~10s on a 1mb
of random data.

Ignore fuzz objects over 100kb.

See https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24383

* deprecate vips_popen()

it didn't work well on win, and we no longer use it anyway

* detect RLE overflow in radiance

old-style radiance RLE could overflow harmlessly

see https://oss-fuzz.com/testcase-detail/4918518961930240

* better dbg msg in spngload

* try to work around some broken heic images

see libvips#1574

* revise heic thumbnail workaround

* Add C++ bindings for new_from_memory_steal()

new_from_memory_steal() will create a new image with the input
buffer and will "move" the data into the image. The buffer is then
managed by the image, and will be freed when it goes out of scope.

* Don't check g_signal_connect()'s return

* Remove cast in free() call

* Remove redundant part of comment

* Add parameter name for unused image

* fix typo

see libvips/pyvips#198 (comment)

thanks Tremeschin

* fix write ICC profile to webp

ooops, a typo broke ICC profile write to webp 18 days ago

thanks augustocdias

see libvips#1767

* put minimise support back into pdfload

* docs fix

pandoc changed the name of their top-level section node

* start 8.10.1

following the doc generation fix

* remove redefinition of typedefs

We had this in a couple of places:

	typedef struct _A A;
	typedef struct _A A;

Some old gccs (eg. centos6) throw errors for this.

See libvips#1774

* add missing funcs to public C API

The C API was missing vips_jpegload_source and vips_svgload_source.
Thanks to augustocdias.

See libvips#1780

* update wrapper script

fixes "make check".

* experiment with doxygen for C++ docs

* fix regression in thumbnail of pyr tiff

The new subifd pyramid thumbnail code broke the old page-based pyramid
detector.

Thanks tand826

See libvips#1784

* move vips-operators.h into the header

doxy hates #include inside classes

* Ensure magick buffer+file checks use consistent min length guard

Prevents a zero-length buffer from crashing GetImageMagick

It looks like the fix for magick7 in libvips#1642 is also now required
for magick6 as the assertion appears to have been backported.

* add optional params to doc strings

* Ensure magick buffer+file checks use consistent min length guard

Prevents a zero-length buffer from crashing GetImageMagick

It looks like the fix for magick7 in libvips#1642 is also now required
for magick6 as the assertion appears to have been backported.

* prevent 0-length buffers reaching imagemagick

im6 seems to have added an assert for this

see libvips#1785

* add doxygen to the build system

configure tests for it, make runs it, make install copies the generated
html

* revise cpp codegen again

use f'' strings, polish formatting

* add doc comments for VError and VInterpolate

* a few more doc comments

* convert no-profile CMYK to RGB on save

Use the fallback cmyk profile to convert to RGB on save if the image has
no embedded profile.

Thanks augustocdias.

See libvips#1767

* fix some unknown types

We were missing VipsInterpolate and guint64. Add guint64 set() as well.

see libvips@636e265#commitcomment-41589463

* improve C++ API

Make VObject inheritance public, so we can have a single set() for all
VObject-derived types.

* note how to list interpolators

* Ensure SVG loader skips input with chars outside x09-x7F range

Add test with example valid WebP image that happens to contain
the string '<svg' within its compressed image data.

* Ensure SVG loader skips input with chars outside x09-x7F range

Add test with example valid WebP image that happens to contain
the string '<svg' within its compressed image data.

* note svg fix

* improve docs for arrayjoin

see libvips/pyvips#202

* Fix a small memory leak in sinkscreen

* start sinkscreen thread on first use

we were starting the sinkscreen background thread during vips_init() --
instead, start it on first use

see libvips#1792

* fix typo

* better mask sizing for gaussmat

We were calculating the mask size incorrectly for small masks.

Thanks johntrunc

see libvips#1793

* fix handling of "squash" param in tiffsave

the deprecated param was not being detected correctly, breaking vips7
compat in some cases

see libvips#1801

* fix jpegload autorotate

thanks chregu

see libvips/php-vips#105

* note render thread change in changelog

* update magick metadata naming

IM seem to have changed their rules for naming metadata chunks. They are
now lowercase and ICM is renamed to ICC. Add a simple test too.

See libvips/ruby-vips#246

* revise doxy flags to configure

* add a README.md for cpp

* add some more C++ docs

* more C++ docs

* better dint rules

We had some special cases coded for dhint inheritance, but they could
fail in some edge cases. Revert to something simpler and more
predictable.

see libvips#1810

* don't add generated latex to repo

* finish C++ doc comments

* integrate new C++ docs in main docs

* more small doc tweaks

* Ensure VImage::set uses class to determine type

Prevents null GType and associated segfault

* don't set JFIF res if we will set EXIF res

Some JPEG loaders give priority to JFIF resolution over EXIF resolution
tags. This patch makes libvips not write the JFIF res tags if it will be
writing the EXIF res tags.

See libvips/ruby-vips#247

* typo in recent cpp API improvements

We had G_VALUE_TYPE instead of G_OBJECT_TYPE, oops. Thanks @lovell.

see libvips#1812

* fix TIFF thumbnail of buffer and source

We had dropped a couple of patches.

see libvips#1815

* fix tiff thumbnail from buffer and source

We were missing the new tiff thumbnail logic on the source and buffer
paths.

see libvips#1815

* add a .gitignore for the new cpp api

to stop accidentally adding it to 8.10

* raise minimum libheif version to 1.3

We didn't compile with anything less than 1.3 anyway.

see libvips#1817

* block doxy latex output too

* libheif: expose speed parameter (currently AV1 compression only)

Supports both aom and rav1e encoders by limiting to a 0-8 range.

(The rav1e encoder accepts speed values of 9 and 10 but these
use 64x64 blocks more suited to video than images.)

* fix dzsave iiif dimensions

dzsave in iiif mode could set info.json dimensions off by one

thanks Linden6

see libvips#1818

* allow both dpi and scale to be set for pdfload

pdfload didn't allow both dpi and scale to be set. This patch makes the
two settings combine if both are given.

thanks le0daniel

see libvips#1824

* tiny thumbnail speedup

thumbnail can skip premultiply/unpre if there's no residual resize

* allow gaussblur sigma 0

meaning no blur (obviosuly)

* oop typo

* Verify ISO/3GPP2 signature in heifload is_a check

* Verify ISO/3GPP2 signature in heifload is_a check

* better heif signature detection

* heifload: simplify is_a check of first 4 bytes

Allow multiples of 4, up to 32, as chunk length

* heifload: simplify is_a check of first 4 bytes

Allow multiples of 4, up to 32, as chunk length

* revise heif sniffing again

* note VImage::new_from_memory_steal() in ChangeLog

plus doxy commnets etc., see libvips#1758

* Fix test failure on ARM-based Windows

The optional parameters of vips_gaussnoise were incorrectly
passed within vips_fractsurf. This was discovered when running
the libvips testsuite on Windows 10 IoT (ARM32).

* Fix test failure on ARM-based Windows

The optional parameters of vips_gaussnoise were incorrectly
passed within vips_fractsurf. This was discovered when running
the libvips testsuite on Windows 10 IoT (ARM32).

* note fractsurf fix in changelog

* heifload: prevent reading beyond end of source buffer

* heifload: prevent reading beyond end of source buffer

* heifload: prevent reading beyond end of source buffer

* revise heifload EOF detection

VipsSource used a unix-style model where read() returns 0 to mean EOF.
libheif uses a model where a separate call to wait_for_file_size()
beforehand is used to check thaht the read will be OK, and then the
read() is expected to never fail.

We were trying to put EOF detection into libheif read(), but that's not
the right way to do it. Instead, test for EOF in wait_for_file_size().

see libvips#1833

* get docs building with goi 1.66+

It builds now, but some doc sections are missing. Fix this properly in
8.11.

See libvips#1836

* note improvements to iprofile

The docs had fallen behind a bit ... iprofile is no longer usually necessary.

see libvips#1843

* update vipsthumbnail docs for --export-profile

and --input-profile

* add stdin, stdout to vipsthumbnail

eg.

	vipsthumbnail stdin[page=2] -o .jpg[Q=90]

mirror of syntax in new_from_file etc.

* improve seek on pipes

There were a few issues in VipsSource around seeking on pipes. With this
patch, EOF detection is better, and pipe sources automatically turn into memory
sources when EOF is hit.

see libvips#1829

* revise pipe sources (again)

Simplify and cleanup.

* pdfload was missing a rewind on source

* add a test for vipsthumbnail of stdin/stdout

* libheif: expose speed parameter (currently AV1 compression only)

Supports both aom and rav1e encoders by limiting to a 0-8 range.

(The rav1e encoder accepts speed values of 9 and 10 but these
use 64x64 blocks more suited to video than images.)

* note new "speed" param in heifsave

To help support the rapid move to AVIF.

see libvips#1819 (comment)

* fix a regression in the C path for dilate/erode

A ++ had been dropped in the recent refactoring. Credit to kleisauke.

See libvips#1846

* fix build with libheif save buit not load

We had some definitions inside the #ifdef HEIFLOAD.

Thanks estepnv

libvips#1844

* heifload: expose heif-compression metadata

* Speed up VIPS_ARGUMENT_COLLECT_SET

By using G_VALUE_COLLECT_INIT, see:
https://bugzilla.gnome.org/show_bug.cgi?id=603590

* fix heifload with libheif 1.6

heif_avif wasn't added until libheif 1.7

* Separate lock for PDFium

* move the pdfium lock init

move it inside the existing ONCE

see libvips#1857

* better GraphicsMagick image write

We were not setting matte or depth correctly, thanks bfriesen.

* get pdium load working again

It had bitrotted a bit. Thanks @Projkt-James.

See libvips#1400

* fix pdfium mutex init

We need to make the mutex in _class_init, not _build, since we can lock
even if _build is not called.

* add pdfium load from source

* improve BGRA -> RGBA conversion

* revise BGRA->RGBA

* relax is_a heic test rules

32 was a little too small, see libvips#1861

* fix vips7 webp load

webp load using the vips7 interface was crashing, thanks barryspearce

see libvips#1860

* fix out of bounds exif read in heifload

We were subtracting 4 from the length of the exif data block without
checking that there were 4 or more bytes there.

* only remove main image (ifd0) orientation tag

we were stripping all orientation tags on autorot

see libvips#1865

* fix two small bugs in test_connections.c

We were passing NULL rather than argv0 to VIPS_INIT(), and we were not
freeing the loaded file.

thanks zodf0055980

see libvips#1867

* add mssing g_option_context_free() to vipsedit

We were not freeing the argument parse context in vipsedit.c.

Thanks zodf0055980

see libvips#1868

* Revert "Remove round-to-nearest behaviour"

This reverts commit ac30bad

* Round sum values to the nearest integer in *_notab

* Fix centre convention

* fix out of bounds read in tiffload

libtiff can change the value of some fields while scanning a corrupt
TIFF file and this could trigger an out of bounds read.

This patch makes tiffload more cautious about rescanning a TIFF
directory before reading out header fields.

* fix tiff pyramid save region-shrink

we'd forgotton to connect it up

thanks imgifty

see libvips#1875

* add tests for tiff pyr save region-shrink flag

we were testing the flag before, but not that the result was correct

see libvips#1875

* flush target at end of write

we were missing end-of-write flushes on four save operations

thanks harukizaemon

see libvips/ruby-vips#256

* missing include

* Update Examples.md

Just some issues I found while testing the examples:
- Reference on header for the file try255.py
- Typo on parameter from bigtif to bigtiff
- Use explicit call to python interpreter from command line

* update examples to py3

* fix compiler warning

* block annoying INFO messages on some older glibs

Some old glibs can display INFO messages by default. Block these
ourselves.

See libvips#1876

* fix icc-profiles and dzsave --no-strip

We were not copying metadata down pyramid layers in dzsave, so
--no-strip didn't allow icc profiles on tiles.

Thanks altert

See libvips#1879

* Ensure that streams are properly read in spngload

* better GraphicsMagick image write

We were not setting matte or depth correctly, thanks bfriesen.

* add read loops to heif and ppm as well

We were not looping on vips_source_read() in these loaders, so they
could fail when reading from very slow pipes.

See kleisauke/net-vips#101

* fix changelog after GM backport

* note read fixes in changelog

* forgot to advance the buffer pointers

thanks kleis

see kleisauke/net-vips#101 (comment)

* add read loops to gifload

and check for error in ppnmload.

* add "seed" param to perlin, worley and gaussnoise

see libvips#1884

* block 0 width or height images from imagemagick

IM could return 0 width and/or height for some crafted images. Block
these.

Thanks @Koen1999.

See libvips#1890

* oops typo in magick7 load

* fix a possible read loop for truncated gifs

* gifload: ensure total height of all pages is sanitised

* reformat

* backport gifheight check

ensure gifheight can't oevrflow

see libvips#1892

* oops typo in magick7 load

* make ppm load default to msb first

We has lsb first as the default, breaking 16-bit PPM load. Thanks ewelot.

see libvips#1894

* byteswap on ppm save, if necessary

this was missing, thanks ewelot

see libvips#1894

* Ensure vipsload only byte swaps if necessary

Prior to this commit, MSB-ordered vips images were always byte swapped
on both little- and big endian systems. And LSB-ordered vips images
were loaded without a byte swap. This works correctly on little endian
systems, but will not work on big endian systems where the byte swap
must be done vice versa.

This commit ensures that the byte swap only takes place when needed.

See libvips#1847.

* Simplify MSB-ordered image check

* Determine endianness at compile time

* Port Ruby test case to Python

* force binary mode for connections on win

stdin / stdout (for example) are created in text mode by default on
win. We need to flip them to binary mode for connection read and write.

See https://stackoverflow.com/questions/65014352/pipe-libvips-cli-output-to-stdout-in-windows

* better test for output to target

We used to enable write to stdout if the first character of an output filename
was ".", eg.:

	vips copy x.jpg .png

But this will also enable write to stdout for things like:

	vips copy x.jpg ./y.png

This patch also tests that the rightmost "." in a filename is also the
first character.

Thanks barryspearce

See libvips#1906

* block deprecation warnings from libgsf

with an uglu gcc progma

* docs clarification

libvips#1912

* don't add date in ppmsave if @strip is set

see libvips#1913

* add is_a_source to ppmload

ppmload_source was missing an ia_a test

see libvips#1915

* note ppmload fix

* fix ppmsave regression

ppm strip dropped magic number

* n comment

* webpload: ensure first frame is not blended

* make webp frame blend do doround to nearest

see libvips#1918

* fix heif load fails with 0 length metadata

fixes libvips#1901

* fix range clips for casts to and from int

Fix two bugs:

- clip in casts from int32 and uint32 could overflow -- do these as gint64 now

- clip in casts from float to int could overflow since float32 can't
  represent the full range of int32 without losing precision -- do these
  as double

And add some more tests.

Thanks ewelot.

see libvips#1922

* note HEIC fix in changelog

see libvips#1921

* fix heif load fails with 0 length metadata

fixes libvips#1901

* note HEIC fix in changelog

see libvips#1921

* forgot changelog update

* improve website link in docs

it was being rewritten by the export script

see libvips#1928

* fix spng detection

This patch was dropped from 8.10.3 release 1, annoyingly.

* start 8.10.4

with a dropped patch from 8.10.3

* allow spng.pc and libspng.ps for libspng discovery

* webpload: prevent divide-by-zero when blending pixels

Adds a test case to prevent regression - see commit 6eaf1ed

* bump version for animated webp load fix

Co-authored-by: John Cupitt <jcupitt@gmail.com>
Co-authored-by: Kyle Schwarz <zeranoe@gmail.com>
Co-authored-by: Lovell Fuller <github@lovell.info>
Co-authored-by: Kleis Auke Wolthuizen <github@kleisauke.nl>
Co-authored-by: DarthSim <darthsim@gmail.com>
Co-authored-by: Antonio Martínez <amalbala@gmail.com>
Co-authored-by: Daniel Dennedy <ddennedy@gopro.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants