Cannot be build on Julia v1.0

[MaWo2]

Hello.
I tried to install ImageMagick.jl under Julia v1.0 on Windows 8.1 64 bit. It fails during building, because one of the files (Zlib.v1.2.11.x86_64-w64-mingw32.tar.gz) cannot be unpacked.

┌ Error: Error building `ImageMagick`:
│ ERROR: LoadError: LoadError: Could not unpack C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\usr\downloads\Zlib.v1.2.11.x86_64-w64-mingw32.tar.gz into C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\usr
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #unpack#98(::Bool, ::Function, ::String, ::String) at C:\Julia\Julia-packages\v1.0\packages\BinaryProvider\pWVqZ\src\PlatformEngines.jl:600
│  [3] #unpack at .\none:0 [inlined]
│  [4] macro expansion at .\logging.jl:322 [inlined]
│  [5] #install#115(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at C:\Julia\Julia-packages\v1.0\packages\BinaryProvider\pWVqZ\src\Prefix.jl:358
│  [6] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at .\none:0
│  [7] top-level scope at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\build_Zlib.v1.2.11.jl:36
│  [8] include at .\boot.jl:317 [inlined]
│  [9] top-level scope at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\build.jl:15 [inlined]
│  [10] top-level scope at .\none:0
│  [11] include at .\boot.jl:317 [inlined]
│  [12] include_relative(::Module, ::String) at .\loading.jl:1038
│  [13] include(::Module, ::String) at .\sysimg.jl:29
│  [14] include(::String) at .\client.jl:388
│  [15] top-level scope at none:0
│ in expression starting at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\build_Zlib.v1.2.11.jl:32
│ in expression starting at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\build.jl:11
│ ERROR: Can not create symbolic link : Zugriff verweigert : C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\usr\bin\libz.dll
│ ERROR: Can not create symbolic link : Zugriff verweigert : C:\Julia\Julia-packages\v1.0\packages\ImageMagick\W81WA\deps\usr\bin\libz-1.dll
│
│ 7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
│
│
│ Extracting archive:
│ --
│ Path =
│ Type = tar
│ Code Page = UTF-8
│
│
│ Sub items Errors: 2
│
│ Archives with Errors: 1
│
│ Sub items Errors: 2
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Operations.jl:1068

I tried to manually unpack the file into the assigned folder, but it still throws the same error.

[SimonDanisch]

I really hope you're on an old version, since i just tagged a fix for this ;)
https://github.com/JuliaIO/ImageMagick.jl/releases/tag/v0.7.1

[MaWo2]

You were right, I was still on v0.7.0, which I had downloaded this morning.
However, the problem remains. The new Error message looks similar, but is slightly different at the end. Could the problem be connected to 7-zip? 16.04 is an old version. My computer already has 18.05 installed.

(v1.0) pkg> build ImageMagick
  Building Conda ──────→ `C:\Julia\Julia-packages\v1.0\packages\Conda\m7vem\deps\build.log`
  Building FFTW ───────→ `C:\Julia\Julia-packages\v1.0\packages\FFTW\p7sLQ\deps\build.log`
  Building ImageMagick → `C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\build.log`
┌ Error: Error building `ImageMagick`:
│ ERROR: LoadError: LoadError: Could not unpack C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\usr\downloads\libpng.v1.6.31.x86_64-w64-mingw32.tar.gz into C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\usr
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #unpack#98(::Bool, ::Function, ::String, ::String) at C:\Julia\Julia-packages\v1.0\packages\BinaryProvider\UTYxu\src\PlatformEngines.jl:600
│  [3] macro expansion at .\none:0 [inlined]
│  [4] #install#115(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at C:\Julia\Julia-packages\v1.0\packages\BinaryProvider\UTYxu\src\Prefix.jl:358
│  [5] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at .\none:0
│  [6] top-level scope at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\build_libpng.v1.0.0.jl:36
│  [7] include at .\boot.jl:317 [inlined]
│  [8] top-level scope at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\build.jl:15 [inlined]
│  [9] top-level scope at .\none:0
│  [10] include at .\boot.jl:317 [inlined]
│  [11] include_relative(::Module, ::String) at .\loading.jl:1038
│  [12] include(::Module, ::String) at .\sysimg.jl:29
│  [13] include(::String) at .\client.jl:388
│  [14] top-level scope at none:0
│ in expression starting at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\build_libpng.v1.0.0.jl:32
│ in expression starting at C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\build.jl:11
│
│ 7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
│
│
│ Extracting archive:
│ ERROR: Can not create symbolic link : Zugriff verweigert : C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\usr\bin\libpng-config
│ --
│ Path =
│ Type = tar
│ Code Page = UTF-8
│
│ ERROR: Can not create symbolic link : Zugriff verweigert : C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\usr\lib\libpng.dll.a
│ ERROR: Can not create symbolic link : Zugriff verweigert : C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\usr\lib\pkgconfig\libpng.pc
│ ERROR: Can not create symbolic link : Zugriff verweigert : C:\Julia\Julia-packages\v1.0\packages\ImageMagick\d5KBL\deps\usr\lib\libpng.a
│
│ Sub items Errors: 4
│
│ Archives with Errors: 1
│
│ Sub items Errors: 4
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Operations.jl:1068

[ejohlin]

Having the same issues, libpng.v1.6.31.x86_64-w64-mingw32.tar.gz can't be unpacked, giving the same kind of
ERROR: Can not create symbolic link : A required privilege is not held by the client.
errors.

I'm running Windows 10 with Julia 7.0. This unfortunately seems to make the whole Images package unusable as I can't load anything... or is there a way around this?

[musm]

The work around is to run Julia from an elevated command prompt then build ImageMagick. It's not ideal but it works for now.

[MaWo2]

The work around works, but only as long as the packages are not installed into Julia's default location (user folder). If the packages reside in the user folder, the admin account has no access to them and cannot build them.

[timholy]

Getting something like this on Travis too, see https://travis-ci.org/JuliaImages/ImageView.jl/builds/423974711?utm_source=github_status&utm_medium=notification

[tlnagy]

I'm running into this too where ImageMagick fails to install Travis CI during testing for OMETIFF.jl: https://travis-ci.org/tlnagy/OMETIFF.jl/jobs/424544699

InitError: error compiling __init__: could not load library "/home/travis/.julia/packages/ImageMagick/d5KBL/deps/usr/lib/libMagickWand-6.Q16.so"
  /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/travis/.julia/packages/ImageMagick/d5KBL/deps/usr/lib/libpng16.so.16)
  Stacktrace:
   [1] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:627
   [2] _require_from_serialized(::String) at ./loading.jl:678
   [3] macro expansion at ./logging.jl:317 [inlined]
   [4] _require(::Base.PkgId) at ./loading.jl:941
   [5] require(::Base.PkgId) at ./loading.jl:852
   [6] require(::Module, ::Symbol) at ./logging.jl:311
   [7] eval at ./boot.jl:319 [inlined]
   [8] topimport(::Symbol) at /home/travis/.julia/packages/FileIO/kflOT/src/loadsave.jl:13
   [9] checked_import(::Symbol) at /home/travis/.julia/packages/FileIO/kflOT/src/loadsave.jl:30
   [10] #load#27(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Stream{DataFormat{:TIFF},IOStream}) at /home/travis/.julia/packages/FileIO/kflOT/src/loadsave.jl:180
   [11] load at /home/travis/.julia/packages/FileIO/kflOT/src/loadsave.jl:172 [inlined]
   [12] (::getfield(Main, Symbol("##20#22")))(::IOStream) at /home/travis/build/tlnagy/OMETIFF.jl/test/runtests.jl:96
   [13] #open#294(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(Main, Symbol("##20#22")), ::String) at ./iostream.jl:369
   [14] open(::Function, ::String) at ./iostream.jl:367
   [15] macro expansion at /home/travis/build/tlnagy/OMETIFF.jl/test/runtests.jl:95 [inlined]
   [16] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [17] top-level scope at /home/travis/build/tlnagy/OMETIFF.jl/test/runtests.jl:84
   [18] include at ./boot.jl:317 [inlined]
   [19] include_relative(::Module, ::String) at ./loading.jl:1038
   [20] include(::Module, ::String) at ./sysimg.jl:29
   [21] include(::String) at ./client.jl:388
   [22] top-level scope at none:0
   [23] eval(::Module, ::Any) at ./boot.jl:319
   [24] macro expansion at ./logging.jl:317 [inlined]
   [25] exec_options(::Base.JLOptions) at ./client.jl:219
   [26] _start() at ./client.jl:421
  during initialization of module ImageMagick

[timholy]

A workaround is to have your testscript say import ImageMagick before it loads anything else that might bring in a copy of zlib (workaround credit due to @RalphAS, see JuliaImages/ImageView.jl#156 (comment)).

[tlnagy]

Is there a way to do that so that it only does that on Linux (since QuartzImageIO works fine)? Maybe something like:

if Sys.islinux()
    import ImageMagick
end

?

[sjkelly]

On Windows 10 symlinks can be created without admin level by enabling Developer mode: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

Enabling developer mode seems to have fixed things for me.

[Cruor]

Turns out some of the tarballs have symlinks, which causes extraction to fail on Windows.

[tragu]

Same issue here I am using juno/atom showing the error while building Imagemagick package

[tragu]

On Windows 10 symlinks can be created without admin level by enabling Developer mode: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

Enabling developer mode seems to have fixed things for me.

It worked

[hearnsj]

I just came across this issue with Julia 1.0.0 and Windows 10.
Cannot install ImageMagick due to administrative privilege being needed.

I guess the README needs update for Windows 10 to say that Developer Mode should be enabled.
I further guess I should be kicked into submitting a PR!

[simonbyrne]

Upstream issue: JuliaIO/LibpngBuilder#2

[SimonDanisch]

and JuliaIO/ImageMagickBuilder#6

[simonbyrne]

@SimonDanisch What exactly is the problem, and what needs to be fixed? This is a major pain point on Windows.

[SimonDanisch]

JuliaIO/ImageMagickBuilder#6 needs to get into shape....

[SimonDanisch]

or https://github.com/JuliaIO/ImageMagickBuilder/pull/5^^

[bjarthur]

what's the latest with the libz problem? the hack to import ImageMagick before using Images works, but is painful because the latter has to re-compile itself everytime this way. is anyone working on a more permanent solution? i could lend a hand if not.

[bjarthur]

actually, the precompilation issue is only for julia 0.7. perhaps it's time for me to start using 1.0 in production

[mcarpe]

So, is there any solution yet for users without admin rights on Windows 7?

[jordancluts]

I just wan't to comment to remind people of this very vexing issue. I wish I had the skills to help.

[hhaensel]

If I am right, one possibility is that the maintainers of this package provide tar files without symlinks. Another is that you manually extract the files from the archive and replace the symlinks by renamed copies of the original files. But this is tedious to do with any update of the package...

As a workaround I have written a windows batch script which extracts the contents of the tar-file, tracks errors and copies the original files instead of the symlinks. This script is then called instead of 7z.exe - it's a dirty hack though.

This could also be realised by a Julia script, but my Julia knowledge with respect to error handling is not good enough yet. :-(

So, here's my dirty hack
Modify lines 197/198 of PlatformEngines.jl

#            pipeline(`$exe7z x $(tarball_path) -y -so`,
#                     `$exe7z x -si -y -ttar -o$(out_path)`)
            pipeline(`C:\\Users\\hh\\Desktop\\Misc\\hh\\hh.bat $(tarball_path) $(out_path)`)

the code of the script hh.bat is

@title Unpack_with_symlinks
@echo off

::Extracting files while replacing symlinked files with the originals
::To be called instead of 7z, e.g. to avoid errors when using BinaryProvider from Julia
::replpace the existing pipeline command in unpack_7z by
::pipeline(`C:\\Users\\hh\\Desktop\\Misc\\hh\\hh.bat $(tarball_path) $(out_path)`)
::HH, 2019-04-29

setlocal EnableExtensions EnableDelayedExpansion

set "tarfile=%1"
echo Extracting "%tarfile%"

if "%tarfile:~-3%" == ".gz" (
	7z x %tarfile% * -r -y -so > temp.tar
	set "tarfile=temp.tar"
)

set "dest=%2"
if "%dest%" == "" set "dest=."

call :strLen dest destLen

for /F "tokens=4* delims=:" %%a in ('7z x %tarfile% * -r -y -o%dest% 2^>^&1^| find "symbolic"') do (

	set "symlinkPath=%%~a"
	set "symlinkPath=!symlinkPath:~1!"

	set "origPath=.!symlinkPath:~%destLen%!"
	
	for /f "delims=" %%B in ('7z x %tarfile% !origPath! -r -y -so') do set "target=%%B"

	set "symlinkName=%%~nxa"
	call :strLen symLinkName linkLen
	echo; 
	echo Symlink warning:
	echo    !origPath! --^> !target!
	rem echo symlinkName: !symlinkName!
	rem echo symlinkPath: !symlinkPath!
	rem echo linkLen: !linkLen!
	call set "targetPath=%%symlinkPath:~0,-!linkLen!%%!!target!"
	call echo     copying "!targetPath!" to "!symlinkPath!" instead!
	copy !targetPath! !symlinkPath! > NUL
)

if "%tarfile%" == "temp.tar" del /F temp.tar

endlocal
exit /B

:strLen
setlocal enabledelayedexpansion

:strLen_Loop
   if not "!%1:~%len%!"=="" set /A len+=1 & goto :strLen_Loop
(endlocal & set %2=%len%)
goto :eof

you can also manually call the script and track what happens :-)
If called without destination argument, the file will be extracted into the current folder.

[hhaensel]

I have submitted a PR for a Julian workaround.

[timholy]

Should work on Julia 1.3+

[AntumDeluge]

Not directly related to Julia, just want to comment on what I was told about 7-zip & symbolic links on Windows platform: 7-zip uses the mklink command for converting the symbolic links. And apparently, mklink is what actually requires elevated privileges.