NEWS.md

Julia v0.5.0 Release Notes

New language features

• Generator expressions: f(i) for i in 1:n (#4470). This returns an iterator that computes the specified values on demand. This is useful for computing, e.g. sum(f(i) for i in 1:n) without creating an intermediate array of values.

• Generators and comprehensions support filtering using if (#550) and nested iteration using multiple for keywords (#4867).

• Fused broadcasting syntax: f.(args...) is equivalent to broadcast(f, args...) (#15032), and nested f.(g.(args...)) calls are fused into a single broadcast loop (#17300). Similarly, the syntax x .= ... is equivalent to a broadcast!(identity, x, ...) call and fuses with nested "dot" calls; also, x .+= y and similar is now equivalent to x .= x .+ y, rather than x = x .+ y (#17510).

• Macro expander functions are now generic, so macros can have multiple definitions (e.g. for different numbers of arguments, or optional arguments) (#8846, #9627). However note that the argument types refer to the syntax tree representation, and not to the types of run time values.

• Varargs functions like foo{T}(x::T...) may now restrict the number of such arguments using foo{T,N}(x::Vararg{T,N}) (#11242).

• x ∈ X is now a synonym for x in X in for loops and comprehensions, as it already was in comparisons (#13824).

• The PROGRAM_FILE global is now available for determining the name of the running script (#14114).

• The syntax x.:sym (e.g. Base.:+) is now supported, while using x.(:sym) or x.(i) for field access are deprecated in favor of getfield (#15032).

• Function return type syntax function f()::T has been added (#1090). Values returned from a function with such a declaration will be converted to the specified type T.

• Many more operators now support . prefixes (e.g. .≤) (#17393). However, users are discouraged from overloading these, since they are mainly parsed in order to implement backwards compatibility with planned automatic broadcasting of dot operators in Julia 0.6 (#16285). Explicitly qualified operator names like Base.≤ should now use Base.:≤ (prefixed by @compat if you need 0.4 compatibility via the Compat package).

• User-extensible bounds check elimination is now possible with the new @boundscheck macro (#14474). This macro marks bounds checking code blocks, which the compiler may remove when encountered inside an @inbounds call.

Experimental language features

• Support for multi-threading. Loops with independent iterations can be easily parallelized with the Threads.@threads macro.

• Support for arrays with indexing starting at values different from 1. The array types are expected to be defined in packages, but now Julia provides an API for writing generic algorithms for arbitrary indexing schemes (#16260).

Language changes

• Each function and closure now has its own type. The captured variables of a closure are fields of its type. Function is now an abstract type, and is the default supertype of functions and closures. All functions, including anonymous functions, are generic and support all features (e.g. keyword arguments). Instead of adding methods to call, methods are added by type using the syntax (::ftype)(...) = .... call is deprecated (#13412). A significant result of this language change is that higher order functions can be specialized on their function arguments, leading to much faster functional programming, typically as fast as if function arguments were manually inlined. See below for details.

• Square brackets and commas (e.g. [x, y]) no longer concatenate arrays, and always simply construct a vector of the provided values. If x and y are arrays, [x, y] will be an array of arrays (#3737, #2488, #8599).

• using and import are now case-sensitive even on case-insensitive filesystems (common on Mac and Windows) (#13542).

• Relational algebra symbols are now allowed as infix operators (#8036): ⨝, ⟕, ⟖, ⟗ for joins and ▷ for anti-join.

• A warning is always given when a method is overwritten; previously, this was done only when the new and old definitions were in separate modules (#14759).

• The if keyword cannot be followed immediately by a line break (#15763).

• Juxtaposition of numeric literals ending in . (e.g. 1.x) is no longer allowed (#15731).

• The built-in NTuple type has been removed; NTuple{N,T} is now implemented internally as Tuple{Vararg{T,N}} (#11242).

• Use of the syntax x::T to declare the type of a local variable is deprecated. In the future this will always mean type assertion, and declarations should use local x::T instead (#16071). When x is global, x::T = ... and global x::T used to mean type assertion, but this syntax is now reserved for type declaration (#964).

• Dictionary comprehension syntax [ a=>b for x in y ] is deprecated. Use Dict(a=>b for x in y) instead (#16510).

• Parentheses are no longer allowed around iteration specifications, e.g. for (i = 1:n) (#17668).

Breaking changes

This section lists changes that do not have deprecation warnings.

• All dimensions indexed by scalars are now dropped, whereas previously only trailing scalar dimensions would be omitted from the result (#13612). This is a very major behavioral change, but should cause obvious failures. To retain a dimension sliced with a scalar i slice with i:i instead.

• The assignment operations .+=, .*= and so on now generate calls to broadcast! on the left-hand side (or call to view(a, ...) on the left-hand side if the latter is an indexing expression, e.g. a[...]). This means that they will fail if the left-hand side is immutable (or does not support view), and will otherwise change the left-hand side in-place (#17510, #17546).

• Method ambiguities no longer generate warnings when files are loaded, nor do they dispatch to an arbitrarily-chosen method; instead, a call that cannot be resolved to a single method results in a MethodError at run time, rather than the previous definition-time warning (#6190).

• Array comprehensions preserve the dimensions of the input ranges. For example, [2x for x in A] will have the same dimensions as A (#16622).

• The result type of an array comprehension depends only on the types of elements computed, instead of using type inference (#7258). If the result is empty, then type inference is still used to determine the element type.

• reshape is now defined to always share data with the original array. If a reshaped copy is needed, use copy(reshape(a)) or copy! to a new array of the desired shape (#4211).

• mapslices now re-uses temporary storage. Recipient functions that expect input slices to be persistent should copy data to other storage (#17266). All usages of mapslices should be carefully audited since this change can cause silent, incorrect behavior, rather than failing noisily.

• Local variables and arguments are represented in lowered code as numbered Slot objects instead of as symbols (#15609).

• The information that used to be in the ast field of the LambdaStaticData type is now divided among the fields code, slotnames, slottypes, slotflags, gensymtypes, rettype, nargs, and isva in the LambdaInfo type (#15609).

• A <: B is parsed as Expr(:(<:), :A, :B) in all cases (#9503). This also applies to the >: operator.

• Simple 2-argument comparisons like A < B are parsed as calls instead of using the :comparison expression type (#15524). The :comparison expression type is still produced in ASTs when comparisons are chained (e.g. A < B ≤ C).

• map on a dictionary now expects a function that expects and returns a Pair. The result is now another dictionary instead of an array (#16622).

• Bit shift operations (i.e. <<, >>, and >>>) now handle negative shift counts differently: Negative counts are interpreted as shifts in the opposite direction. For example, 4 >> -1 == 4 << +1 == 8. Previously, negative counts would implicitly overflow to large positive counts, always yielding either 0 or -1.

Library improvements

• Strings (#16107):

  • The UTF8String and ASCIIString types have been merged into a single String type (#16058). Use isascii(s) to check whether a string contains only ASCII characters. The ascii(s) function now converts s to String, raising an ArgumentError exception if s is not pure ASCII.

  • The UTF16String and UTF32String types and corresponding utf16 and utf32 converter functions have been removed from the standard library. If you need these types, they have been moved to the LegacyStrings.jl package. In the future, more robust Unicode string support will be provided by the StringEncodings.jl package. If you only need these types to call wide string APIs (UTF-16 on Windows, UTF-32 on UNIX), consider using the new transcode function (see below) or the Cwstring type as a ccall argument type, which also ensures correct NUL termination of string data.

  • A transcode(T, src) function is now exported for converting data between UTF-xx Unicode encodings (#17323).

  • The basic string construction routines are now string(args...), String(s), unsafe_string(ptr) (formerly bytestring(ptr)), and unsafe_wrap(String, ptr) (formerly pointer_to_string) (#16731).

  • Comparisons between Chars and Integers are now deprecated (#16024): 'x' == 120 now produces a warning but still evaluates to true. In the future it may evaluate to false or the comparison may be an error. To compare characters with integers you should either convert the integer to a character value or convert the character to the corresponding code point first: e.g. 'x' == Char(120) or Int('x') == 120. The former is usually preferable.

  • Support for Unicode 9 (#17402).

• Arrays and linear algebra:

  • Dimensions indexed by multidimensional arrays add dimensions. More generally, the dimensionality of the result is the sum of the dimensionalities of the indices (#15431).

  • New normalize and normalize! convenience functions for normalizing vectors (#13681).

  • QR matrix factorization:

    • New method for generic QR with column pivoting (#13480).

    • New method for polar decompositions of AbstractVectors (#13681).

  • A new SparseVector type allows for one-dimensional sparse arrays. Slicing and reshaping sparse matrices now return vectors when appropriate. The sparsevec function returns a one-dimensional sparse vector instead of a one-column sparse matrix. The SparseMatrix module has been renamed to SparseArrays (#13440).

  • Rank one update and downdate functions, lowrankupdate, lowrankupdate!, lowrankdowndate, and lowrankdowndate!, have been introduced for dense Cholesky factorizations (#14243, #14424).

  • All sparse methods now retain provided numerical zeros as structural nonzeros; to drop numerical zeros, use dropzeros! (#14798, #15242).

  • setindex! methods for sparse matrices and vectors no longer purge allocated entries on zero assignment. To drop stored entries from sparse matrices and vectors, use Base.SparseArrays.dropstored! (#17404).

  • Concatenating dense and sparse matrices now returns a sparse matrix (#15172).

• Files and I/O:

  • The open function now respects umask on UNIX when creating files (#16466, #16502).

  • A new function walkdir() returns an iterator that walks the tree of a directory (#8814, #13707).

    for (root, dirs, files) in walkdir(expanduser("~/.julia/v0.5/Plots/src"))
        println("$(length(files)) \t files in $root")
    end
    19    files in /Users/me/.julia/v0.5/Plots/src
    15    files in /Users/me/.julia/v0.5/Plots/src/backends
    4     files in /Users/me/.julia/v0.5/Plots/src/deprecated
    

  • A new function chown() changes the ownership of files (#15007).

  • Display properties can now be passed among output functions (e.g. show) using an IOContext object (#13825).

  • Cmd(cmd; ...) now accepts new Windows-specific options windows_verbatim (to alter Windows command-line generation) and windows_hide (to suppress creation of new console windows) (#13780).

  • There is now a default no-op flush(io) function for all IO types (#16403).

• Parallel computing:

  • pmap keyword arguments err_retry=true and err_stop=false are deprecated. Action to be taken on errors can be specified via the on_error keyword argument. Retry is specified via retry_n, retry_on and retry_max_delay (#15409, #15975, #16663).

  • The functions remotecall, remotecall_fetch, and remotecall_wait now have the function argument as the first argument to allow for do-block syntax (#13338).

• Statistics:

  • Improve performance of quantile (#14413).

  • extrema can now operate over a region (#15550).

  • cov and cor don't use keyword arguments anymore and are therefore now type stable (#13465).

  • Histogram functionality has been deprecated in Base. Use the StatsBase.jl package instead (#6842, #16450).

• Testing:

  • The Base.Test module now has a @testset feature to bundle tests together and delay throwing an error until the end (#13062).

  • The new features are mirrored in the BaseTestNext.jl package for users who would like to use the new functionality on Julia v0.4.

  • The BaseTestDeprecated.jl package provides the old-style handler functionality, for compatibility with code that needs to support both Julia v0.4 and v0.5.

• Package management:

  • The package system (Pkg) is now based on the libgit2 library, rather than running the git program, increasing performance (especially on Windows) (#11196).

  • Package-development functions like Pkg.tag and Pkg.publish have been moved to an external PkgDev package (#13387).

  • Updating only a subset of the packages is now supported, e.g. Pkg.update("Example") (#17132).

• Miscellanous:

  • Prime number related functions have been moved from Base to the Primes.jl package (#16481).

  • Most of the combinatorics functions have been moved from Base to the Combinatorics.jl package (#13897).

  • New foreach function for calling a function on every element of a collection when the results are not needed (#13774). Compared to map(f, v), which allocates and returns a result array, foreach(f, v) calls f on each element of v, returning nothing.

  • The new Base.StackTraces module makes stack traces easier to use programmatically (#14469).

  • The libjulia library is now properly versioned and installed to the public <prefix>/lib directory, instead of the private <prefix>/lib/julia directory (#16362).

  • System reflection is now more consistently exposed from Sys and not Base (e.g. constants such as WORD_SIZE and CPU_CORES). OS_NAME has been replaced by Sys.KERNEL and always reports the name of the kernel (as reported by uname). The @windows_only and @osx family of macros have been replaced with functions such as is_windows() and is_apple(). There is now also a @static macro that will evaluate the condition of an if-statement at compile time, for when a static branch is required (#16219).

  • Date and DateTime values can now be rounded to a specified resolution (e.g., 1 month or 15 minutes) with floor, ceil, and round (#17037).

Compiler/Runtime improvements

• Machine SIMD types can be represented in Julia as a homogeneous tuple of VecElement (#15244).

• The performance of higher-order and anonymous functions has been greatly improved. For example, map(x->2x, A) performs as well as 2.*A(#13412).

• On windows, a DLL of standard library code is now precompiled and used by default, improving startup time (#16953).

• LLVM has been upgraded to version 3.7.1, improving the quality of generated code and debug info. However compile times may be slightly longer (#14623).

New architectures

This release greatly improves support for ARM, and introduces support for Power.

• ARM: #14194, #14519, #16645, #16621

• Power: #16455, #16404

Deprecated or removed

• The following function names have been simplified and unified (#13232):

  • get_bigfloat_precision -> precision(BigFloat)

  • set_bigfloat_precision -> setprecision

  • with_bigfloat_precision -> setprecision

  • get_rounding -> rounding

  • set_rounding -> setrounding

  • with_rounding -> setrounding

• The method A_ldiv_B!(SparseMatrixCSC, StridedVecOrMat) has been deprecated in favor of versions that require the matrix to be in factored form (#13496).

• chol(A,Val{:U/:L}) has been deprecated in favor of chol(A) (#13680).

• rem1(x,y) is discontinued due to inconsistency for x==0. Use mod1 instead (#14140).

• The FS module has been renamed to Filesystem. Calling the functions isreadable, iswritable, and isexecutable on filesystem paths has been deprecated (#12819).

• RemoteRef has been deprecated in favor of RemoteChannel (#14458).

• super has been renamed to supertype (#14335).

• parseip(str) has been deprecated in favor of parse(IPAddr, str) (#14676).

• readall has been renamed to readstring, and readbytes has been renamed to read (#14608, #14660).

• fieldoffsets(x) has been deprecated in favor of calling fieldoffset(x, i) on each field (#14777).

• issym is deprecated in favor of issymmetric to match similar functions (ishermitian, ...) (#15192).

• scale is deprecated in favor of either α*A, Diagonal(x)*A, or A*Diagonal(x) (#15258).

• istext has been renamed to istextmime (#12872, #15708).

• "Functor" types are no longer necessary and have been deprecated (#15804). To maintain performance on older versions of Julia the Compat.jl package provides a @functorize macro.

• bitunpack(B) and bitpack(A) have been deprecated in favor of Array(B) and BitArray(A), respectively (#16010).

• xdump is removed, and dump now simply shows the full representation of a value. dump should not be overloaded, since it is for examining concrete structure (#4163).

• sprandbool has been deprecated in favor of sprand(Bool, ...) or sprand(rng, Bool, ...) (#11688, #16098).

• The lowercase symbol function has been deprecated in favor of the Symbol constructor (#16154).

• writemime is deprecated, and output methods specifying a MIME type are now methods of show (#14052).

• BLAS utility functions blas_set_num_threads, blas_vendor, and check_blas have been moved to the BLAS module as BLAS.set_num_threads, BLAS.vendor, and BLAS.check (#10548, #16600).

• print_escaped has been renamed to escape_string, print_unescaped has been renamed to unescape_string, and print_joined has been renamed to join (#16603).

• pointer_to_string has been renamed to unsafe_wrap(String, ...), and pointer_to_array has been renamed to unsafe_wrap(Array, ...) (#16731).

• sub and slice have been deprecated in favor of view (#16972).

• Sparse matrix functions etree, ereach, csc_permute, and symperm have been moved to the SuiteSparse.jl package (#12231, #17033).

• The no-op transpose fallback for non-numeric arrays has been deprecated. Consider introducing suitable transpose methods or calling permutedims(x, (2, 1)) for matrices and reshape(x, 1, length(x)) for vectors. (#13171, #17075, #17374).

• The following macros have been deprecated (#16219):

  • @windows is deprecated in favor of is_windows()
  • @unix is deprecated in favor of is_unix()
  • @osx is deprecated in favor of is_apple()
  • @linux is deprecated in favor of is_linux()
  • @windows_only is deprecated in favor of if is_windows()
  • @unix_only is deprecated in favor of if is_unix()
  • @osx_only is deprecated in favor of if is_apple()
  • @linux_only is deprecated in favor of if is_linux()
  • NOTE: Using @static could be useful/necessary when used in a function's local scope. See details at the section entitled Handling Operating System Variation in the manual.

Command-line option changes

• The -F flag to load ~/.juliarc has been deprecated in favor of --startup-file=yes (#9482).

• The -f and --no-startup flags to disable loading of ~/.juliarc have been deprecated in favor of --startup-file=no (#9482).

• The -P and --post-boot flags for evaluating an expression in "interactive mode" have been deprecated in favor of -i -e (#16854).

• The --no-history-file flag to disable loading of ~/.julia_history has been deprecated in favor of --history-file=no (#9482).

Language tooling improvements

• The Julia debugger makes its debut with this release. Install it with Pkg.add("Gallium"), and the documentation should get you going. The JuliaCon talk on Gallium shows off various features of the debugger.

• The Juno IDE has matured significantly, and now also includes support for plotting and debugging.

• Cxx.jl provides a convenient FFI for calling C++ code from Julia.