In [Arrays, Numbers, and Colors](@ref page_arrays_colors) we discussed how one can
convert the element type of an array a = [1,2,3,4] using a syntax
like Float64.(a). You might be curious what effect, if any,
Int.(a) has:
julia> a = [1,2,3,4]
4-element Vector{Int64}:
1
2
3
4
julia> b = Int.(a)
4-element Vector{Int64}:
1
2
3
4
There's certainly no obvious change, and as you'd expect b == a
returns true. Beyond having equal size and elements, there's a more
extensive notion of "sameness": do a and b refer to the same
storage area in memory? We can test that in the following ways:
julia> a === b # note: 3 equal signs!
false
or more generally by setting a value and seeing whether the change is reflected in the other:
julia> b[1] = 5
5
julia> b
4-element Vector{Int64}:
5
2
3
4
julia> a
4-element Vector{Int64}:
1
2
3
4
Since the types of a and b are identical, both tests tell us that
a and b are independent objects, even if they (initially) had the
same values.
This occurs because f.(a) (which calls the function broadcast(f, a)) always allocates a new array to return its values. However, not
all functions operate this way. One good example is view:
julia> v = view(a, :)
4-element view(::Vector{Int64}, :) with eltype Int64:
1
2
3
4
v and a have the same values, but again they are distinct objects:
julia> v == a
true
julia> v === a
false
However, they share the same memory:
julia> v[1] = 10
10
julia> v
4-element view(::Vector{Int64}, :) with eltype Int64:
10
2
3
4
julia> a
4-element Vector{Int64}:
10
2
3
4
Consequently, v is a "view" of the values stored in a. While this
usage of view is trivial, more generally it can be used to select a
rectangular region of interest, which is a common operation in image
processing; this region is selected without copying any data, and any
manipulations of the values within this region are reflected in the
original (parent) array. See the documentation on view, by typing
?view, for more information.
view is not the only function with this property: another good
example is reshape, which can be used to change the dimensions of an
array:
julia> r = reshape(a, 2, 2)
2×2 Matrix{Int64}:
10 3
2 4
julia> r[1,2] = 7
7
julia> r
2×2 Matrix{Int64}:
10 7
2 4
julia> a
4-element Vector{Int64}:
10
2
7
4
Notice that the return type of reshape is just an Array, one which
happens to be serving as a view of a. However, some inputs cannot be
represented as a view with an Array. For example:
julia> r = reshape(1:15, 3, 5)
3×5 reshape(::UnitRange{Int64}, 3, 5) with eltype Int64:
1 4 7 10 13
2 5 8 11 14
3 6 9 12 15
A UnitRange is represented compactly-storing only the starting and
stopping values-so there is no memory location that can be referenced to
access all values. In such cases, reshape returns a ReshapedArray,
which is a generic "view type" that handles reshaping of any kind of
AbstractArray.
The output of both view and reshape are always views: make a
change in either the parent or the view, and the change is reflected
in the other.
[Arrays, Numbers, and Colors](@ref page_arrays_colors) also introduced the fixed-point
numbers used in some representations of color (or grayscale)
information. If you want to switch representation, you can use the
reinterpret function:
julia> using FixedPointNumbers
julia> x = 0.5N0f8
0.502N0f8
julia> y = reinterpret(x) # alternatively, use: reinterpret(UInt8, x)
0x80
julia> reinterpret(N0f8, y)
0.502N0f8
You can apply this to arrays:
julia> a = [0.2N0f8, 0.8N0f8]
2-element Array{N0f8,1} with eltype N0f8:
0.2N0f8
0.8N0f8
julia> b = reinterpret.(a)
2-element Vector{UInt8}:
0x33
0xcc
Because of the f.(a) call, b does not share memory with a:
julia> b[2] = 0xff
0xff
julia> a
2-element Array{N0f8,1} with eltype N0f8:
0.2N0f8
0.8N0f8
Often this might not be a problem, but sometimes you might wish that these referenced the same underlying object. For such situations, JuliaImages, through the ImageCore package (which is bundled with Images), implements views that can perform this reinterpretation:
julia> using Images
julia> v = rawview(a)
2-element reinterpret(UInt8, ::Array{N0f8,1}):
0x33
0xcc
julia> v[2] = 0xff
0xff
julia> a
2-element Array{N0f8,1} with eltype N0f8:
0.2N0f8
1.0N0f8
We can understand clearly from below code that v is an immutable object, or is just
a reference of a. It doesn't has a separate memory allocated to it:
julia> a = [0.2N0f8,0.8N0f8]
2-element Array{N0f8,1} with eltype N0f8:
0.2N0f8
0.8N0f8
julia> v = rawview(a)
2-element reinterpret(UInt8, ::Array{N0f8,1}):
0x33
0xcc
julia> pointer_from_objref(a) #function used to find address of an object
Ptr{Nothing} @0x000000011cbb19f0
julia> pointer_from_objref(v) #v is just a immutable reference to a, no separate memory allocated to it.
ERROR: pointer_from_objref cannot be used on immutable objects
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] pointer_from_objref(x::Any)
@ Base ./pointer.jl:146
[3] top-level scope
@ none:1The opposite transformation is normedview:
julia> c = [0x11, 0x22]
2-element Vector{UInt8}:
0x11
0x22
julia> normedview(c)
2-element reinterpret(N0f8, ::Vector{UInt8}):
0.067N0f8
0.133N0f8
normedview allows you to pass the interpreted type as the first
argument, i.e., normedview(N0f8, A), and indeed it's required to do
so unless A has element type UInt8, in which case normedview
assumes you want N0f8.
Like reshape, both rawview and normedview might return an
Array or a more complicated type (a ReinterpretArray, or a
MappedArray from the
MappedArrays package),
depending on the types of the inputs.
In [Arrays, Numbers, and Colors](@ref page_arrays_colors), we pointed out that one can
convert a numeric array to a grayscale array with Gray.(a); the
opposite transformation can be performed with real.(b). Handling RGB
colors is a little more complicated, because the dimensionality of the
array changes. One approach is to use Julia's comprehensions:
julia> a = reshape(collect(0.1:0.1:0.6), 3, 2)
3×2 Matrix{Float64}:
0.1 0.4
0.2 0.5
0.3 0.6
julia> c = [RGB(a[1,j], a[2,j], a[3,j]) for j = 1:2]
2-element Array{RGB{Float64},1} with eltype RGB{Float64}:
RGB{Float64}(0.1,0.2,0.3)
RGB{Float64}(0.4,0.5,0.6)
julia> x = [getfield(c[j], i) for i = 1:3, j = 1:2]
3×2 Matrix{Float64}:
0.1 0.4
0.2 0.5
0.3 0.6
While this approach works, it's not without flaws:
- this implementation relies on the two-dimensionality of
a; a 3d array (producing a 2d color image) would need a different implementation - the use of
getfieldassumes that elements ofchave fields and that they are in the orderr,g,b. Given the large number of different representations of RGB supported by ColorTypes, neither of these assumptions is entirely safe. - it always makes a copy of the data
To address these weaknesses, JuliaImages provides two complementary view function, colorview and channelview:
julia> colv = colorview(RGB, a)
2-element reinterpret(reshape, RGB{Float64}, ::Matrix{Float64}) with eltype RGB{Float64}:
RGB{Float64}(0.1,0.2,0.3)
RGB{Float64}(0.4,0.5,0.6)
julia> chanv = channelview(c)
3×2 reinterpret(reshape, Float64, ::Array{RGB{Float64},1}) with eltype Float64:
0.1 0.4
0.2 0.5
0.3 0.6
colorview and channelview always return a view of the original
array.
Another use for colorview is to combine multiple grayscale images into a single color image. For example:
using Colors, Images
r = range(0,stop=1,length=11)
b = range(1,stop=0,length=11)
img1d = colorview(RGB, r, zeroarray, b)
# output
11-element mappedarray(RGB{Float64}, ImageCore.extractchannels, ::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}}, ::ImageCore.ZeroArray{Float64, 1, Base.OneTo{Int64}}, ::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}}) with eltype RGB{Float64}:
RGB{Float64}(0.0,0.0,1.0)
RGB{Float64}(0.1,0.0,0.9)
RGB{Float64}(0.2,0.0,0.8)
RGB{Float64}(0.3,0.0,0.7)
RGB{Float64}(0.4,0.0,0.6)
RGB{Float64}(0.5,0.0,0.5)
RGB{Float64}(0.6,0.0,0.4)
RGB{Float64}(0.7,0.0,0.3)
RGB{Float64}(0.8,0.0,0.2)
RGB{Float64}(0.9,0.0,0.1)
RGB{Float64}(1.0,0.0,0.0)
results (in IJulia) in
zeroarray is a special constant that "expands" to return the
equivalent of an all-zeros array with axes matching the other
inputs to colorview.
When you've separated colors into a separate color dimension, some
code might assume that color is the last (slowest) dimension. You can
convert directly using Julia's permutedims function:
julia> pc = permutedims(a, (2,1))
2×3 Matrix{Float64}:
0.1 0.2 0.3
0.4 0.5 0.6
permutedims explicitly creates a new array with the data rearranged in memory. It's also possible to perform something similar
as a view:
julia> pv = PermutedDimsArray(a, (2,1))
2×3 PermutedDimsArray(::Matrix{Float64}, (2, 1)) with eltype Float64:
0.1 0.2 0.3
0.4 0.5 0.6
While this looks the same, pv (unlike pc) shares memory with a; this is
an apparent permutation, achieved by having the indexing of a
PermutedDimsArray swap the input indexes whenever individual
elements are accessed.
One thing to be aware of is that the performance of these two might
differ, for reasons that have to do with how CPUs and memory work
rather than any limitation of Julia. If a is large and you want to
access all three elements corresponding to the color channels of a
single pixel, pv will likely be more efficient because values are
adjacent in memory and thus likely share a cache line. Conversely, if
you want to access different pixels from a single color channel
sequentially, pc may be more efficient (for the same reason).
Sometimes when you want to compare two images, one might be of a
different size than another. You can create array views that have
common indices with paddedviews:
julia> a1 = reshape([1,2], 2, 1)
2×1 Matrix{Int64}:
1
2
julia> a2 = [1.0,2.0]'
1×2 adjoint(::Vector{Float64}) with eltype Float64:
1.0 2.0
julia> a1p, a2p = paddedviews(0, a1, a2); # 0 is the fill value
julia> a1p
2×2 PaddedView(0, ::Matrix{Int64}, (Base.OneTo(2), Base.OneTo(2))) with eltype Int64:
1 0
2 0
julia> a2p
2×2 PaddedView(0.0, adjoint(::Vector{Float64}), (Base.OneTo(2), Base.OneTo(2))) with eltype Float64:
1.0 2.0
0.0 0.0
This can be especially useful in conjunction with colorview to
compare two (or more) grayscale images. See
Keeping track of location with unconventional indices for more
information.
Sometimes it's helpful to combine several images into a single view for further array-like manipulation.
julia> img1 = reshape(1:8, (2,4))
2×4 reshape(::UnitRange{Int64}, 2, 4) with eltype Int64:
1 3 5 7
2 4 6 8
julia> img2 = reshape(11:18, (2,4))
2×4 reshape(::UnitRange{Int64}, 2, 4) with eltype Int64:
11 13 15 17
12 14 16 18
julia> sv = StackedView(img1, img2)
2×2×4 StackedView{Int64, 3, Tuple{Base.ReshapedArray{Int64, 2, UnitRange{Int64}, Tuple{}}, Base.ReshapedArray{Int64, 2, UnitRange{Int64}, Tuple{}}}}:
[:, :, 1] =
1 2
11 12
[:, :, 2] =
3 4
13 14
[:, :, 3] =
5 6
15 16
[:, :, 4] =
7 8
17 18
julia> imgMatrix = reshape(sv, (2, 8))
2×8 reshape(::StackedView{Int64, 3, Tuple{Base.ReshapedArray{Int64, 2, UnitRange{Int64}, Tuple{}}, Base.ReshapedArray{Int64, 2, UnitRange{Int64}, Tuple{}}}}, 2, 8) with eltype Int64:
1 2 3 4 5 6 7 8
11 12 13 14 15 16 17 18
If you want to use some of these views but have an application where
the sharing of memory is actually problematic, keep in mind that you
can always call Julia's copy function to create a copy of the
array. The type of the resulting copy might not be identical to the
original, but the values will be the same.
When Julia displays an array as text, there is usually a 1-line
summary at the top showing the array type. You may have already
noticed that JuliaImages uses an unconventional syntax for summarizing
information about certain kinds of arrays. For example, the type of
pv above is
PermutedDimsArray{Float64,2,(2,1),(2,1),Matrix{Float64}}but when you display such an object, in the summary line it prints as
2×3 PermutedDimsArray(::Matrix{Float64}, (2, 1)) with eltype Float64This is intended to result in more easily-readable information about types.
The main motivation for this is that different view types can be
combined freely, and when you do so sometimes the type gets quite
long. For example, suppose you have a disk file storing a m×n×3×t UInt8 array representing an RGB movie (t being the time axis). To
have it display as an RGB movie, you might create the following view of
the array A:
using Images
using Random
Random.seed!(1234)
A = rand(UInt8, 5, 6, 3, 10);
mov = colorview(RGB, normedview(PermutedDimsArray(A, (3,1,2,4))));
summary(mov)
typeof(mov)
While there is little or no performance cost to making use of JuliaImage's convenient views, sometimes the types can get complicated!