Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 195 lines (154 sloc) 9.377 kb
48cf0ffa » rrnewton
2011-06-24 Adding a DEVLOG file to keep track of development.
1
2
3 DEVLOG: A collection of notes accumulated during development.
4 =============================================================
5
6
7 [2011.06.24] (transient) Regression in stdGen performance.
8 ----------------------------------------------------------
9
10 I just added a simple benchmark to make sure that whatever fix I
11 introduce for trac ticket #5133 does not regress performance. Yet in
12 doing so I discovered that I'm getting much worse performance out of
13 rev 130e421e912d than I'm seeing in my installed random-1.0.0.3 package.
14
15 Current version:
16 How many random numbers can we generate in a second on one thread?
17 Cost of rdtsc (ffi call): 100
18 Approx getCPUTime calls per second: 234,553
19 Approx clock frequency: 3,335,220,196
20 First, timing with System.Random interface:
21 68,550,189 random ints generated [constant zero gen] ~ 48.65 cycles/int
22 900,889 random ints generated [System.Random stdGen] ~ 3,702 cycles/int
23
24 random-1.0.0.3 version:
25 How many random numbers can we generate in a second on one thread?
26 Cost of rdtsc (ffi call): 75
27 Approx getCPUTime calls per second: 215,332
28 Approx clock frequency: 3,334,964,738
29 First, timing with System.Random interface:
30 71,683,748 random ints generated [constant zero gen] ~ 46.52 cycles/int
31 13,609,559 random ints generated [System.Random stdGen] ~ 245 cycles/int
32
33 A >13X difference!!
34 Both are compiled with the same options. The only difference is which
35 System.Random is used.
36
37 When did the regression occur?
38
39 * e059ed15172585310f9c -- 10/13/2010 perf still good
40 * 6c43f80f48178ac617 -- SplittableGen introduced, still good perf
41 * 130e421e912d394653a4 -- most recent, bad performance
42
43 Ok... this is very odd. It was a heisenbug becuase it's disappeared
44 now! I'll leave this note here to help remember to look for it in the
45 future.
46 -Ryan
47
48
49 [2011.06.24] Timing non-int types
50 ---------------------------------
51
52 The results are highly uneven:
53
54 Cost of rdtsc (ffi call): 84
55 Approx getCPUTime calls per second: 220,674
56 Approx clock frequency: 3,336,127,273
57 First, timing with System.Random interface:
58 112,976,933 randoms generated [constant zero gen] ~ 29.53 cycles/int
59 14,415,176 randoms generated [System.Random stdGen] ~ 231 cycles/int
60 70,751 randoms generated [System.Random Floats] ~ 47,153 cycles/int
61 70,685 randoms generated [System.Random CFloats] ~ 47,197 cycles/int
62 2,511,635 randoms generated [System.Random Doubles] ~ 1,328 cycles/int
63 70,494 randoms generated [System.Random CDoubles] ~ 47,325 cycles/int
64 858,012 randoms generated [System.Random Integers] ~ 3,888 cycles/int
65 4,756,213 randoms generated [System.Random Bools] ~ 701 cycles/int
66
67 As you can see, all the types that use the generic randomIvalFrac /
68 randomFrac definitions perform badly. What's more, the above results
9e494aa4 » rrnewton
2011-09-30 minor: DEVLOG comments
69 INCLUDE an attempt to inline:
48cf0ffa » rrnewton
2011-06-24 Adding a DEVLOG file to keep track of development.
70
71 {-# INLINE randomIvalFrac #-}
72 {-# INLINE randomFrac #-}
73 {-# INLINE randomIvalDouble #-}
74
75 After reimplementing random/Float these are the new results:
76
77 Cost of rdtsc (ffi call): 100
78 Approx getCPUTime calls per second: 200,582
79 Approx clock frequency: 3,334,891,942
80 First, timing with System.Random interface:
81 105,266,949 randoms generated [constant zero gen] ~ 31.68 cycles/int
82 13,593,392 randoms generated [System.Random stdGen] ~ 245 cycles/int
83 10,962,597 randoms generated [System.Random Floats] ~ 304 cycles/int
84 11,926,573 randoms generated [System.Random CFloats] ~ 280 cycles/int
85 2,421,520 randoms generated [System.Random Doubles] ~ 1,377 cycles/int
86 2,535,087 randoms generated [System.Random CDoubles] ~ 1,315 cycles/int
87 856,276 randoms generated [System.Random Integers] ~ 3,895 cycles/int
88 4,976,373 randoms generated [System.Random Bools] ~ 670 cycles/int
89
90 (But I still need to propagate these changes throughout all types / API calls.)
91
92
327401e4 » rrnewton
2011-06-28 Added tests and benchmarks for BIG Integers.
93
94 [2011.06.28] Integer Generation via random and randomR
95 -------------------------------------------------------
96
97 Back on the master branch I notice that while randomIvalInteger does
98 well for small ranges, it's advantage doesn't scale to larger ranges:
99
100 range (-100,100):
101 5,105,290 randoms generated [System.Random Integers] ~ 653 cycles/int
102
103 range (0,2^5000):
104 8,969 randoms generated [System.Random BIG Integers] ~ 371,848 cycles/int
105
9e494aa4 » rrnewton
2011-09-30 minor: DEVLOG comments
106
107
108 [2011.08.25] Validating release version 1.0.1.0 rev 40bbfd2867
109 --------------------------------------------------------------
110
111 This is a bugfix release without SplittableGen. It passed (cd tests;
112 make test) on my Mac Os 10.6 machine.
113
114 I ran GHC validate using the following fingerprint
115
116 .|c5056b932a06b4adce5167a5cb69f1f0768d28ec
117 ghc-tarballs|e7b7b152083f7c3e3559e557a239757d41ac02a6
118 libraries/Cabal|3dcc425495523ab6142027097cb598a4d2ad810a
119 libraries/Win32|085b11285b6adbc6484d9c21f5e0b8105556869c
120 libraries/array|fa295423e7404d3d1d3d82655b2b44d50f045a44
121 libraries/base|a57369f54bd25a1de5d477f3c363b3bafd17d168
122 libraries/binary|9065df2120254601c68c3a28264fd062abde9354
123 libraries/bytestring|caad22630f73e0e9b1b61f4da34f8aefcc6001d8
124 libraries/containers|667591b168c804d3eeae503dff1c848ed6852412
125 libraries/directory|d44f52926278319286804d8333149dd13d4ecc82
126 libraries/dph|b23b45a9e8fce985327b076997d61ab0ddc7b2f7
127 libraries/extensible-exceptions|e77722871a5812d52c467e3a8fd9c7b97cdec521
128 libraries/filepath|fd381017dca45de5c94dac85a6233516a6b6963d
129 libraries/ghc-prim|0a84a755e1248b4d50f6535a0ce75af0bb21b0ad
130 libraries/haskeline|8787a64417500efffc9c48032ee7c37315fb2547
131 libraries/haskell2010|470b34b6c0336339eac9fbcfb6020e46b5154bfe
132 libraries/haskell98|5590c0f042d6d07352e0bf49cedeef5ba0821f23
133 libraries/hoopl|b98db91cd0c53ddb2c275c04823f9c379774104b
134 libraries/hpc|7c726abec939b11af1ecf89740ca8d04e6a1360d
135 libraries/integer-gmp|65c73842bca2f075b65f418a5ff34753b185e0d7
136 libraries/integer-simple|10202111c59f0695ef782d1ec9e6fc992933fc9a
137 libraries/mtl|a41470c1890073e678f0dca2a9ef4c02d55bf7c6
138 libraries/old-locale|104e7e5a7b33424f34e98825a0d4ccb7614ca7c2
139 libraries/old-time|81e0c8a4b98d4b084eefe75bedf91a44edd31888
140 libraries/pretty|036fb8dfbb9d4a787fcd150c2756b4899be4e942
141 libraries/primitive|74479e07b92b8859eae473e5cc86b40decae1d6e
142 libraries/process|68ba490d6691f55eab19a249379144831055e2ac
143 libraries/random|3fb0e9e42b54d7b01b794fc27d4d678d7d74ff0e
144 libraries/template-haskell|02362d12e5ae0af20d637eec97db51f6827a1625
145 libraries/terminfo|baec6aff59d13ba294b370f9563e8068706392ce
146 libraries/unix|f55638fb5c6badd385c51a41de7ff96ef106de42
147 libraries/utf8-string|ec2b85940a256dbc8771e5e2065ca8f76cc802d0
148 libraries/vector|1e72d46bdc8488a84558b64ac63632cef1d8a695
149 libraries/xhtml|cb2cf6c34d815fdf4ed74efeb65e1993e7bda514
150 testsuite|26c608a0c31d56917099e4f48bf58c1d1e92e61c
151 utils/haddock|d54959189f33105ed09a59efee5ba34f53369282
152 utils/hsc2hs|f8cbf37ab28ab4512d932678c08c263aa412e008
153
154
155
156 First validating in the context of a slightly stale GHC head
157 (7.3.20110727) on a mac.
158
159
b2443d55 » rrnewton
2011-09-30 Fixed bug in Ticket 5501. But there appears to be a significant perfo…
160 [2011.09.30] Redoing timings after bugfix in version 1.0.1.1
161 ------------------------------------------------------------
162
163 It looks like there has been serious performance regression (3.33ghz
164 nehalem still).
165
166 How many random numbers can we generate in a second on one thread?
167 Cost of rdtsc (ffi call): 38
168 Approx getCPUTime calls per second: 7,121
169 Approx clock frequency: 96,610,524
170 First, timing System.Random.next:
171 148,133,038 randoms generated [constant zero gen] ~ 0.65 cycles/int
172 12,656,455 randoms generated [System.Random stdGen/next] ~ 7.63 cycles/int
173
174 Second, timing System.Random.random at different types:
175 676,066 randoms generated [System.Random Ints] ~ 143 cycles/int
176 3,917,247 randoms generated [System.Random Word16] ~ 24.66 cycles/int
177 2,231,460 randoms generated [System.Random Floats] ~ 43.29 cycles/int
178 2,269,993 randoms generated [System.Random CFloats] ~ 42.56 cycles/int
179 686,363 randoms generated [System.Random Doubles] ~ 141 cycles/int
180 2,165,679 randoms generated [System.Random CDoubles] ~ 44.61 cycles/int
181 713,702 randoms generated [System.Random Integers] ~ 135 cycles/int
182 3,647,551 randoms generated [System.Random Bools] ~ 26.49 cycles/int
183 4,296,919 randoms generated [System.Random Chars] ~ 22.48 cycles/int
184
185 Next timing range-restricted System.Random.randomR:
186 4,307,214 randoms generated [System.Random Ints] ~ 22.43 cycles/int
187 4,068,982 randoms generated [System.Random Word16s] ~ 23.74 cycles/int
188 2,059,264 randoms generated [System.Random Floats] ~ 46.92 cycles/int
189 1,960,359 randoms generated [System.Random CFloats] ~ 49.28 cycles/int
190 678,978 randoms generated [System.Random Doubles] ~ 142 cycles/int
191 2,009,665 randoms generated [System.Random CDoubles] ~ 48.07 cycles/int
192 4,296,452 randoms generated [System.Random Integers] ~ 22.49 cycles/int
193 3,689,999 randoms generated [System.Random Bools] ~ 26.18 cycles/int
194 4,367,577 randoms generated [System.Random Chars] ~ 22.12 cycles/int
195 6,650 randoms generated [System.Random BIG Integers] ~ 14,528 cycles/int
196
Something went wrong with that request. Please try again.