-
Notifications
You must be signed in to change notification settings - Fork 55
/
index.html
77 lines (67 loc) · 23.2 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
<!DOCTYPE html>
<html lang="en"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><title>Arrays, Numbers, and Colors · JuliaImages</title><script data-outdated-warner src="../../assets/warner.js"></script><link href="https://cdnjs.cloudflare.com/ajax/libs/lato-font/3.0.0/css/lato-font.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/juliamono/0.039/juliamono-regular.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/fontawesome.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/solid.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/brands.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.13.11/katex.min.css" rel="stylesheet" type="text/css"/><script>documenterBaseURL="../.."</script><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js" data-main="../../assets/documenter.js"></script><script src="../../siteinfo.js"></script><script src="../../../versions.js"></script><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../../assets/themes/documenter-dark.css" data-theme-name="documenter-dark" data-theme-primary-dark/><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../../assets/themes/documenter-light.css" data-theme-name="documenter-light" data-theme-primary/><script src="../../assets/themeswap.js"></script><link href="../../democards/gridtheme.css" rel="stylesheet" type="text/css"/></head><body><div id="documenter"><nav class="docs-sidebar"><a class="docs-logo" href="../../"><img src="../../assets/logo.png" alt="JuliaImages logo"/></a><div class="docs-package-name"><span class="docs-autofit"><a href="../../">JuliaImages</a></span></div><form class="docs-search" action="../../search/"><input class="docs-search-query" id="documenter-search-query" name="q" type="text" placeholder="Search docs"/></form><ul class="docs-menu"><li><a class="tocitem" href="../../">Home</a></li><li><a class="tocitem" href="../../install/">Getting started</a></li><li><span class="tocitem">Tutorials</span><ul><li><a class="tocitem" href="../quickstart/">Quickstart</a></li><li class="is-active"><a class="tocitem" href>Arrays, Numbers, and Colors</a><ul class="internal"><li><a class="tocitem" href="#Numbers-versus-colors"><span>Numbers versus colors</span></a></li><li><a class="tocitem" href="#Colors-beyond-the-pale"><span>Colors beyond the pale</span></a></li><li><a class="tocitem" href="#fixedpoint"><span>A consistent scale for floating-point and "integer" colors: fixed-point numbers</span></a></li><li><a class="tocitem" href="#More-fixed-point-numbers"><span>More fixed-point numbers</span></a></li><li><a class="tocitem" href="#A-note-on-arithmetic-overflow"><span>A note on arithmetic overflow</span></a></li></ul></li><li><a class="tocitem" href="../conversions_views/">Conversions vs. views</a></li><li><a class="tocitem" href="../indexing/">Arrays: more advanced indexing</a></li></ul></li><li><span class="tocitem">Packages</span><ul><li><a class="tocitem" href="../../pkgs/">Introduction</a></li><li><a class="tocitem" href="../../pkgs/axes/">ImageAxes.jl</a></li><li><a class="tocitem" href="../../pkgs/metadata/">ImageMetaData.jl</a></li><li><a class="tocitem" href="../../pkgs/segmentation/">ImageSegmentation.jl</a></li><li><a class="tocitem" href="../../pkgs/transformations/">ImageTransformations.jl</a></li><li><a class="tocitem" href="../../pkgs/features/">ImageFeatures.jl</a></li></ul></li><li><a class="tocitem" href="../../examples/">Demos</a></li><li><a class="tocitem" href="../../function_reference/">References</a></li><li><a class="tocitem" href="../../api_comparison/">Comparison with other image processing frameworks</a></li></ul><div class="docs-version-selector field has-addons"><div class="control"><span class="docs-label button is-static is-size-7">Version</span></div><div class="docs-selector control is-expanded"><div class="select is-fullwidth is-size-7"><select id="documenter-version-selector"></select></div></div></div></nav><div class="docs-main"><header class="docs-navbar"><nav class="breadcrumb"><ul class="is-hidden-mobile"><li><a class="is-disabled">Tutorials</a></li><li class="is-active"><a href>Arrays, Numbers, and Colors</a></li></ul><ul class="is-hidden-tablet"><li class="is-active"><a href>Arrays, Numbers, and Colors</a></li></ul></nav><div class="docs-right"><a class="docs-edit-link" href="https://github.com/JuliaImages/juliaimages.github.io/blob/source/docs/src/tutorials/arrays_colors.md" title="Edit on GitHub"><span class="docs-icon fab"></span><span class="docs-label is-hidden-touch">Edit on GitHub</span></a><a class="docs-settings-button fas fa-cog" id="documenter-settings-button" href="#" title="Settings"></a><a class="docs-sidebar-button fa fa-bars is-hidden-desktop" id="documenter-sidebar-button" href="#"></a></div></header><article class="content" id="documenter-page"><h1 id="page_arrays_colors"><a class="docs-heading-anchor" href="#page_arrays_colors">Arrays, Numbers, and Colors</a><a id="page_arrays_colors-1"></a><a class="docs-heading-anchor-permalink" href="#page_arrays_colors" title="Permalink"></a></h1><p>In Julia, an image is just an array, and many of the ways you manipulate images come from the general methods to work with <a href="http://docs.julialang.org/en/stable/manual/arrays/">multidimensional arrays</a>. For example,</p><pre><code class="language-julia-repl hljs">julia> img = rand(2,2)
2×2 Matrix{Float64}:
0.366796 0.210256
0.523879 0.819338</code></pre><p>defines an "image" <code>img</code> of 64-bit floating-point numbers. You should be able to use this as an image in most or all functions in JuliaImages.</p><p>We'll be talking quite a bit about handling arrays. This page will focus on the "element type" (<code>eltype</code>) stored in the array. In case you're new to Julia, if <code>a</code> is an array of integers:</p><pre><code class="language-julia-repl hljs">julia> a = [1,2,3,4]
4-element Vector{Int64}:
1
2
3
4</code></pre><p>then either of the following creates a new array where the element type is <code>Float64</code>:</p><pre><code class="language-julia hljs">map(Float64, a)
Float64.(a) # short for broadcast(Float64, a)</code></pre><p>For example,</p><pre><code class="language-julia hljs">julia> Float64.(a)
4-element Vector{Float64}:
1.0
2.0
3.0
4.0</code></pre><p>Arrays are indexed with square brackets (<code>a[1]</code>), with indexing starting at 1 by default. A two-dimensional array like <code>img</code> can be indexed as <code>img[2,1]</code>, which would be the second row, first column. Julia also supports "linear indexing," using a single integer to address elements of an arbitrary multidimensional array in a manner that (in simple cases) reflects the memory offset of the particular element. For example, <code>img[3]</code> corresponds to <code>img[1,2]</code> (numbering goes down columns, and then wraps around at the top of the next column, because Julia arrays are stored in "column major" order where the fastest dimension is the first dimension).</p><h2 id="Numbers-versus-colors"><a class="docs-heading-anchor" href="#Numbers-versus-colors">Numbers versus colors</a><a id="Numbers-versus-colors-1"></a><a class="docs-heading-anchor-permalink" href="#Numbers-versus-colors" title="Permalink"></a></h2><p>For the array <code>img</code> we created above, you can display it as a grayscale image using ImageView. But if you happen to be following along in Juno or IJulia, you might notice that <code>img</code> does not <em>display</em> as an image: instead, it prints as an array of numbers as shown above. Arrays of "plain numbers" are not displayed graphically, because they might represent something numerical (e.g., a matrix used for linear algebra) rather than an image. To indicate that this is worthy of graphical display, convert the element type to a color chosen from the <a href="https://github.com/JuliaGraphics/Colors.jl">Colors</a> package:</p><p><img src="../assets/arrays_colors/float_gray.png" alt="float_gray"/></p><p>Here we used <code>Gray</code> to indicate that this array should be interpreted as a grayscale image. (Note that the Images package re-exports Colors, so you can alternatively say <code>using Images</code>.)</p><p>Under the hood, what is <code>Gray</code> doing? It's informative to see the "raw" object, displayed as text:</p><p><img src="../assets/arrays_colors/float_gray_text.png" alt="float_gray_text"/></p><p>(Users of Juno or the Julia command-line REPL interface will see this representation immediately.)</p><p>You can see this is a 2×2 array of <code>Gray{Float64}</code> objects. You might be curious how these <code>Gray</code> objects are represented. In the command-line REPL, it looks like this (the same command works with IJulia):</p><pre><code class="language-julia hljs">julia> dump(imgg[1,1])
ColorTypes.Gray{Float64}
val: Float64 0.36679641243992434</code></pre><p><code>dump</code> shows the "internal" representation of an object. You can see that <code>Gray</code> is a <a href="https://docs.julialang.org/en/v1/manual/types/">type</a> (technically, an immutable <code>struct</code>) with a single field <code>val</code>; for <code>Gray{Float64}</code>, <code>val</code> is a 64-bit floating point number. Using <code>val</code> directly is not recommended: you can extract the <code>Float64</code> value with the accessor functions <code>real</code> or <code>gray</code> (the reason for the latter name will be clearer when we discuss RGB colors).</p><p>What kind of overhead do these objects incur?</p><pre><code class="language-julia hljs">julia> sizeof(img)
32
julia> sizeof(imgg)
32</code></pre><p>The answer is "none": they don't take up any memory of their own, nor do they typically require any additional processing time. The <code>Gray</code> "wrapper" is just an <em>interpretation</em> of the values, one that helps clarify that this should be displayed as a grayscale image. Indeed, <code>img</code> and <code>imgg</code> compare as equal:</p><pre><code class="language-julia hljs">julia> img == imgg
true</code></pre><p>There's more to say on this topic, but we'll wait until we discuss <a href="../conversions_views/#page_conversions_views">Conversions vs. views</a>.</p><h2 id="Colors-beyond-the-pale"><a class="docs-heading-anchor" href="#Colors-beyond-the-pale">Colors beyond the pale</a><a id="Colors-beyond-the-pale-1"></a><a class="docs-heading-anchor-permalink" href="#Colors-beyond-the-pale" title="Permalink"></a></h2><p><code>Gray</code> is not the only color in the universe:</p><p><img src="../assets/arrays_colors/randrgb.png" alt="randrgb"/></p><p>Let's look at <code>imgc</code> as text (shown here in the REPL):</p><pre><code class="language-julia hljs">julia> imgc
2×2 Array{ColorTypes.RGB{Float32},2}:
RGB{Float32}(0.75509,0.965058,0.65486) RGB{Float32}(0.696203,0.142474,0.783316)
RGB{Float32}(0.705195,0.953892,0.0744661) RGB{Float32}(0.571945,0.42736,0.548254)
julia> size(imgc)
(2,2)
julia> dump(imgc[1,1])
ColorTypes.RGB{Float32}
r: Float32 0.7550899
g: Float32 0.9650581
b: Float32 0.65485954</code></pre><p>Here we see one of the primary differences between Julia's approach to images and that of several other popular frameworks: <code>imgc</code> does not have a dimension of the array devoted to the "color channel." Instead, every element of the array corresponds to a complete pixel's worth of information. Often this simplifies the logic of many algorithms, sometimes allowing a single implementation to work for both color and grayscale images.</p><p>You can extract the individual color channels using their field names (<code>r</code>, <code>g</code>, and <code>b</code>), but as you'll see in a moment, a more universal approach is to use accessor functions:</p><pre><code class="language-julia hljs">julia> c = imgc[1,1]; (red(c), green(c), blue(c))
(0.7550899f0,0.9650581f0,0.65485954f0)</code></pre><p>Julia's Colors package allows the same color to be represented in several different ways, and this can facilitate interaction with other tools. For example, certain C libraries permit or prefer the order of the color channels to be different:</p><pre><code class="language-julia hljs">julia> dump(BGR(c))
ColorTypes.BGR{Float32}
b: Float32 0.65485954
g: Float32 0.9650581
r: Float32 0.7550899</code></pre><p>or even to pack the red, green, and blue colors–-together with a dummy "alpha" (transparency) channel–-into a single 32-bit integer:</p><pre><code class="language-julia hljs">julia> c24 = RGB24(c); dump(c24)
ColorTypes.RGB24
color: UInt32 12711591
julia> c24.color
0x00c1f6a7</code></pre><p>From first (the first two hex-digits after the "0x") to last (the final two hex-digits), the order of the channels here is alpha, red, green, blue:</p><pre><code class="language-julia hljs">julia> 0xc1/0xff
0.7568627450980392
julia> 0xf6/0xff
0.9647058823529412
julia> 0xa7/0xff
0.6549019607843137</code></pre><p>These values are close to the channels of <code>c</code>, but have been rounded off–-each channel is encoded with only 8 bits, so some approximation of the exact floating-point value is unavoidable.</p><h2 id="fixedpoint"><a class="docs-heading-anchor" href="#fixedpoint">A consistent scale for floating-point and "integer" colors: fixed-point numbers</a><a id="fixedpoint-1"></a><a class="docs-heading-anchor-permalink" href="#fixedpoint" title="Permalink"></a></h2><p><code>c24</code> does not have an <code>r</code> field, but we can still use <code>red</code> to extract the red channel:</p><pre><code class="language-julia hljs">julia> r = red(c24)
0.757N0f8</code></pre><p>This may look fairly strange at first, so let's unpack this carefully. Notice first that the "floating-point" portion of this number matches (to within the precision of the rounding) the value of <code>red(c)</code>. The <code>N0f8</code> means "<strong>N</strong>ormalized with <strong>8 f</strong>ractional bits, with <strong>0</strong> bits left for representing values higher than 1." This is a <a href="https://en.wikipedia.org/wiki/Fixed-point_arithmetic">fixed-point number</a>–-rather like floating-point numbers, except that the decimal does not "float". Internally, these are represented in terms of the 8-bit unsigned integer <code>UInt8</code></p><pre><code class="language-julia hljs">julia> dump(r)
FixedPointNumbers.N0f8
i: UInt8 193</code></pre><p>(Note that <code>N0f8</code> is an abbreviation; the full typename is <code>Normed{UInt8, 8}</code>.) <code>N0f8</code> <em>interprets</em> this 8-bit integer as a value lying between 0 and 1, with 0 corresponding to <code>0x00</code> and 1 corresponding to <code>0xff</code>. This interpretation affects how the number is used for arithmetic and conversion to and from other values. Stated another way, <code>r</code> behaves as</p><pre><code class="language-julia hljs">julia> r == 193/255
true</code></pre><p>for essentially all purposes (but see <a href="#A-note-on-arithmetic-overflow">A note on arithmetic overflow</a>).</p><p>This has a very important consequence: <strong>in many other image frameworks, the "meaning" of an image depends on how it is stored, but in Julia the meaning can be assigned independently of storage representation.</strong> For example, in a different language/framework, the following sequence:</p><pre><code class="nohighlight hljs">img = uint8(255*rand(10, 10, 3));
figure; image(img)
imgd = double(img); % convert to double-precision, but don't change the values
figure; image(imgd)</code></pre><p>might produce the following images:</p><table><tr><th style="text-align: center">img</th><th style="text-align: center">imgd</th></tr><tr><td style="text-align: center"><img src="../assets/arrays_colors/uint8.png" alt="checker"/></td><td style="text-align: center"><img src="../assets/arrays_colors/uint8_to_double.png" alt="checker2"/></td></tr></table><p>The one on the right looks white because floating-point types are interpreted on a 0-to-1 colorscale (and all of the entries in <code>img</code> happen to be 1 or higher), whereas <code>uint8</code> is interpreted on a 0-to-255 colorscale. Unfortunately, two arrays that are <em>numerically identical</em> have very different meanings as images.</p><p>Many frameworks offer convenience functions for converting images from one representation to another, but this can be a source of bugs if we go to compare images: in most number systems we would agree that <code>255 != 1.0</code>, and this fact means that you sometimes need to be quite careful when converting from one representation to another. Conversely, using these Julia packages <strong>there is no discrepancy in "meaning" between the encoding of images represented as floating point or 8-bit (or 16-bit) fixed-point numbers: 0 always means "black" and 1 or something greater than 1 always means "white" or "saturated."</strong></p><p>Now, this doesn't prevent you from constructing pixels with values out of this range:</p><p><img src="../assets/arrays_colors/saturated_spectrum.png" alt="saturated_spectrum"/></p><p>Notice that the first two yellows look identical, because both the red and green color channels are 1 or higher and consequently are saturated.</p><p>However, you should be aware that for <em>integer</em> inputs, the default is to use the <code>N0f8</code> element type, and this type cannot represent values outside the range from 0 to 1:</p><pre><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia> RGB(8,2,0)</code><code class="nohighlight hljs ansi" style="display:block;">ERROR: ArgumentError: (8, 2, 0) are integers in the range 0-255, but integer inputs are encoded with the N0f8
type, an 8-bit type representing 256 discrete values between 0 and 1.
Consider dividing your input values by 255, for example: RGB{N0f8}(8/255,2/255,0/255)
Or use `reinterpret(N0f8, x)` if `x` is a `UInt8`.
See the READMEs for FixedPointNumbers and ColorTypes for more information.</code></pre><p>The error message here reminds you how to resolve a common mistake, trying to construct red as <code>RGB(255, 0, 0)</code>. In Julia, that should always be <code>RGB(1, 0, 0)</code>.</p><h2 id="More-fixed-point-numbers"><a class="docs-heading-anchor" href="#More-fixed-point-numbers">More fixed-point numbers</a><a id="More-fixed-point-numbers-1"></a><a class="docs-heading-anchor-permalink" href="#More-fixed-point-numbers" title="Permalink"></a></h2><p>16-bit images can be expressed in terms of the <code>N0f16</code> type. Let's compare the maximum values (<code>typemax</code>) and smallest-difference (<code>eps</code>) representable with <code>N0f8</code> and <code>N0f16</code>:</p><pre><code class="language-julia-repl hljs">julia> using FixedPointNumbers
julia> (typemax(N0f8), eps(N0f8))
(1.0N0f8, 0.004N0f8)
julia> (typemax(N0f16), eps(N0f16))
(1.0N0f16, 2.0e-5N0f16)</code></pre><p>You can see that this type also has a maximum value of 1, but is higher precision, with the gap between adjacent numbers being much smaller.</p><p>Many cameras (particularly, scientific cameras) now return 16-bit values. However, some cameras do not provide a full 16 bits worth of information; for example, the camera might be 12-bit and return values between <code>0x0000</code> and <code>0x0fff</code>. As an <code>N0f16</code>, the latter displays as nearly black:</p><p><img src="../assets/arrays_colors/12bit_black.png" alt="12bit_black"/></p><p>Since the camera is saturated, this is quite misleading–-it should instead display as white.</p><p>This again illustrates one of the fundamental problems about assuming that the <em>representation</em> (a 16-bit integer) also describes the <em>meaning</em> of the number. In Julia, we decouple these by providing many different fixed-point number types. In this case, the natural way to interpret these values is by using a fixed-point number with 12 fractional bits; this leaves 4 bits that we can use to represent values bigger than 1, so the number type is called <code>N4f12</code>:</p><pre><code class="language-julia-repl hljs">julia> (typemax(N4f12), eps(N4f12))
(16.0037N4f12, 0.0002N4f12)</code></pre><p>You can see that the maximum value achievable by an <code>N4f12</code> is approximately 16 = 2^4.</p><p>Using this <code>N4f12</code> interpretation of the 16 bits, the color displays correctly as white:</p><p><img src="../assets/arrays_colors/12bit_white.png" alt="12bit_black"/></p><p>and acts like 1 for all arithmetic purposes. Even though the raw representation as <code>0x0fff</code> is the same, we can endow the number with appropriate meaning through its type.</p><h2 id="A-note-on-arithmetic-overflow"><a class="docs-heading-anchor" href="#A-note-on-arithmetic-overflow">A note on arithmetic overflow</a><a id="A-note-on-arithmetic-overflow-1"></a><a class="docs-heading-anchor-permalink" href="#A-note-on-arithmetic-overflow" title="Permalink"></a></h2><p>Sometimes, being able to construct a color values outside 0 to 1 is useful. For example, if you want to compute the average color in an image, the natural approach is to first sum all the pixels and then divide by the total number of pixels. At an intermediate stage, the sum will typically result in a color that is well beyond saturation.</p><p>It's important to note that arithmetic with <code>N0f8</code> numbers, like arithmetic with <code>UInt8</code>, overflows:</p><pre><code class="language-julia-repl hljs">julia> 0xff + 0xff
0xfe
julia> 1N0f8 + 1N0f8
0.996N0f8
julia> 0xfe/0xff # the first result corresponds to the second result
0.996078431372549</code></pre><p>Consequently, if you're accumulating values, it's advisable to accumulate them in an appropriate floating-point type, such as <code>Float32</code>, <code>Gray{Float64}</code>, or <code>RGB{Float32}</code>.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../quickstart/">« Quickstart</a><a class="docs-footer-nextpage" href="../conversions_views/">Conversions vs. views »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.5 on <span class="colophon-date" title="Monday 30 August 2021 13:59">Monday 30 August 2021</span>. Using Julia version 1.6.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>