Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports for Julia v0.6.3 #26915

Merged
merged 42 commits into from May 27, 2018
Merged

Backports for Julia v0.6.3 #26915

merged 42 commits into from May 27, 2018

Conversation

ararslan
Copy link
Member

@ararslan ararslan commented Apr 26, 2018

Still to do:

Next steps:

  • Nanosoldier
  • PackageEvaluator

Omitted:

  • CircleCI configuration, also omitted from 0.6.1 and 0.6.2.
  • Changes to allow building on FreeBSD, also omitted from 0.6.1 and 0.6.2. I tried backporting them this time but wasn't able to get things working, so unless someone has a vested interest in first-class FreeBSD support for 0.6.3, I'll leave them out of this release as well. 0.6.3 will have FreeBSD support.

jaakkor2 and others added 8 commits April 26, 2018 12:22
Should allow files >2 GB to be truncated on Windows.

Fix #24466

Checking Windows 32bit build https://julialang-s3.julialang.org/bin/winnt/x86/0.6/julia-0.6.1-win32.exe, libgit2.dll already uses `_chsize_s`, so both 32bit and 64bit should be able to use `_chsize_s`.

Ref #24616
(cherry picked from commit ff045af)
…4902)

* Make setindex!(v::RowVector, ...) return v rather than v's parent.

Ref #24902
(cherry picked from commit e28b507)
Computes the dependency graph subset induced by the requirements
after filter_versions has propagated the implicit requirements,
rather than before, which ensures that only the parts reachable
from some requirement are kept. In practice, it removes many more
useless parts of the graph.

(Plus other tiny improvements.)

Ref #24904
(cherry picked from commit 9b00059)
Set the packages as fixed when checking instead of just
requiring them; propagate the resolution info; improve
the dependency computation efficiency; some refactoring.

Ref #24904
(cherry picked from commit 114e82e)
Fixes the rare condition where two packages were
being decimated at once in mutually inconsistent
versions.

Ref #24904
(cherry picked from commit 450c8c4)
Ref #24904
(cherry picked from commit 208f633)
@ararslan ararslan added this to the 0.6.x milestone Apr 26, 2018
@ararslan
Copy link
Member Author

@nanosoldier runbenchmarks(ALL, vs=":release-0.6")

@ararslan
Copy link
Member Author

ararslan commented Apr 27, 2018

Tests are looking good on Linux and Windows, but there seems to be a problem with the macOS build:

In file included from /Users/travis/build/JuliaLang/julia/src/APInt-C.cpp:3:
./llvm-version.h:3:10: fatal error: 'llvm/Config/llvm-config.h' file not found
#include <llvm/Config/llvm-config.h>
         ^
make[2]: /usr/local/Cellar/llvm39-julia/3.9.1_1/bin/llvm-config: Command not found

Any ideas, @staticfloat? Maybe an issue with the tap?

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

@ViralBShah
Copy link
Member

Do we have anything to test whether the GPU infrastructure will keep working on 0.6.3? @nishanth @aviks @maleadt @MikeInnes

@maleadt
Copy link
Member

maleadt commented Apr 27, 2018

CUDAdrv/LLVM/CUDAnative/CuArrays test fine on this branch right now.
I'll test again when the other patches land.

@ararslan
Copy link
Member Author

ararslan commented May 2, 2018

@nanosoldier runbenchmarks(ALL, vs=":release-0.6")

@iblislin
Copy link
Member

iblislin commented May 3, 2018

Does this need to be backported? #25537

@ararslan
Copy link
Member Author

ararslan commented May 3, 2018

Julia 0.6 doesn't carry other patches for LLVM versions after 3.9.1; I don't think we supported later versions in 0.6, in which case backporting that would be unnecessary. I could be wrong though. @vchuravy?

@vchuravy
Copy link
Sponsor Member

vchuravy commented May 3, 2018

Indeed, the support LLVM version is 3.9.1 and I think we shouldn't backport patches for versions that we didn't even have developmental support.

@iblis17 to have a particular reason for wanting to backport that patch? Are you using Julia 0.6 with LLVM 4.0.0?

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

@iblislin
Copy link
Member

iblislin commented May 3, 2018

@vchuravy oh okay, I'm fine with not to backporting it.
My situation is quite special. I have a Ryzen box. It's a buggy CPU. I built 0.6 with LLVM 4.0.1 + newer openblas. I manually applied that patch to get CuArrays work.

@iblislin
Copy link
Member

iblislin commented May 3, 2018

FreeBSD CI build is here: https://freebsdci.julialang.org/#/builders/1/builds/9129

@iblislin
Copy link
Member

iblislin commented May 3, 2018

FreeBSD CI got segfault 🤔

@ararslan
Copy link
Member Author

ararslan commented May 3, 2018

Looks like the hashing tests caused a segfault on FreeBSD. I can't reproduce that locally. @iblis17, can you?

@iblislin
Copy link
Member

iblislin commented May 3, 2018

I can access the core file, it points to test/bitarray.jl.
https://gist.github.com/iblis17/8fb77e32dcb94d58fa5463c76aa0f3c1

I'm building this brach on local...

@ararslan
Copy link
Member Author

ararslan commented May 3, 2018

@iblis17 Any update on your end? Building this branch on FreeBSD 11.1 after git clean -fdx produces no segfaults for me in any of the tests. Perhaps the build cache on the CI worker needs to be cleared for this to work there?

@iblislin
Copy link
Member

iblislin commented May 4, 2018

I don't get segfault locally also.
Rerun CI with git clean -fdx: https://freebsdci.julialang.org/#/builders/1/builds/9150

@iblislin
Copy link
Member

iblislin commented May 4, 2018

FreeBSD CI passed now.
As your guess, seems the cache clutters things up.

@ararslan
Copy link
Member Author

ararslan commented May 4, 2018

Great, thanks for confirming!

Once PackageEvaluator finishes this should be good to go (assuming all's well there).

ararslan and others added 22 commits May 26, 2018 21:48
FreeBSD's system libgcc_s declares its GCC version as 4.6, which is too
old to build Julia. Since gfortran doesn't come installed by default,
when it's installed it's done so through a GCC port, which includes its
own libgcc_s. Linking to libgfortran from the port and to the system's
libgcc_s causes versioning issues which wreck the build.

This commit works around this issue by determining the version of GCC
used for gfortran, then linking in the libgcc_s corresponding to that
version everywhere. It's a bit of a kludge, but it gets the job done.
With this change, all libraries which need to link to libgcc_s link to
the proper version.

Ref #21788
(cherry picked from commit 7d83d99)
Ref #22352
(cherry picked from commit 8b8fa11)
Tweak the order of libgcc_s in DT_NEEDED on FreeBSD so that building on FreeBSD does not require `BUILD_CUSTOM_LIBCXX`.

See also: #21788, #22352

Ref #22656
(cherry picked from commit d90c215)
The value from rusage.ru_maxrss is in kilobytes.

Ref #22997
(cherry picked from commit f60636f)
Should help with crashes when loading multiple libLLVM versions in
the same process, as happens with mesa/llvmpipe when mesa is
dynamically linked against libLLVM. See

  #19606

patch: https://reviews.llvm.org/D31524
commit: https://reviews.llvm.org/rL300496

Ref #25597
(cherry picked from commit edabf42)
* Avoid out of bounds read in mul! for SymTridiagonal

Fixes #26994

* Also handle 0x0 case

Ref #27008
(cherry picked from commit f90c599)
…pect inference to be non-terminating

this same issue applied to several other similar functions
similarly, need to avoid nesting the use of functional code (like map)
too deeply inside the array code to avoid the appearance of indeterminate recursion in inference

(minimally cherry-picked just for base/array.jl from 4f1b479
* Bump Documenter and deps.
* Canonical URLs and build cleanup. (#25623)
* Hide "The Julia Language" in navbar. (#27146)
@ararslan
Copy link
Member Author

I've removed the commit from #26897 to see whether that fixes the permissions error on Windows. That PR was the first time it was observed, even though the change looks entirely unrelated.

@ararslan
Copy link
Member Author

Well, I can't explain why, but removing that commit seems to have fixed it.

@ararslan
Copy link
Member Author

Looks like there's a network issue or something with the i686 Linux build on Travis; it keeps getting checksum mismatches then rebuilding everything. I'm not sure why it would only happen for i686 though, as the x86-64 build works fine...

@ararslan
Copy link
Member Author

The branch cache probably got polluted with master somehow. That happened when I did 0.6.2 as well. Since Travis is trying to rebuild everything I'm just going to stop it since it will just eat into our CI time just to eventually time out. So this PR should be good to merge I think.

@ararslan ararslan changed the title WIP: Backports for Julia v0.6.3 Backports for Julia v0.6.3 May 27, 2018
@ararslan ararslan merged commit 261e488 into release-0.6 May 27, 2018
@ararslan ararslan deleted the aa/backports-0.6.3 branch May 27, 2018 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet