1
1
perl6-bench: Tools to benchmark and compare Perl family language implementations
2
2
3
+ TABLE OF CONTENTS
4
+ -----------------
5
+ * QUICK START
6
+ * NETWORK TRAFFIC
7
+ * BENCHMARKING TIPS
8
+ * PREREQUISITES
9
+
10
+
3
11
QUICK START
4
12
-----------
5
13
NOTE: This is an EARLY RELEASE, and no attempt has been made to bulletproof
@@ -8,13 +16,45 @@ NOTE: This is an EARLY RELEASE, and no attempt has been made to bulletproof
8
16
A sample sequence for building, benchmarking, and comparing a few
9
17
compilers is as follows:
10
18
19
+ # First step chews network bandwidth to mirror repos; see next section
11
20
./bench setup
12
21
./bench extract nqp niecza rakudo
13
22
./bench build nqp/2012.11.1 rakudo/2012.11 niecza/v24
14
23
./bench time nqp/2012.11.1 rakudo/2012.11 niecza/v24
15
24
./bench analyze --compare results/*/*.json
16
25
17
26
27
+ NETWORK TRAFFIC
28
+ ---------------
29
+ perl6-bench tries to front-load as much of the network traffic as possible,
30
+ so that you can run "./bench setup" once on a fast network, then disconnect
31
+ and do benchmarking to your heart's content without touching the network
32
+ again (unless you run "./bench fetch" to bring in new upstream commits).
33
+
34
+ During the setup process, 'bench' clones a bare mirror of the git repos of
35
+ every component it knows about. After that, the extract command simply
36
+ makes local clones of these bare mirrors as needed, not touching the network
37
+ at all. Also, special care is taken during component builds so that components
38
+ such as NQP and Rakudo that want to automatically clone other components during
39
+ build don't do so over the network, making fast local clones instead.
40
+
41
+ Thus, after setup you can build, benchmark, and analyze without touching the
42
+ network again. Eventually though you may want to grab the latest changes to
43
+ the component repos, e.g. to benchmark a new release when it comes out. You
44
+ can do this with "./bench fetch", which takes care to only update the bare
45
+ mirrors across the network -- not requiring nearly as much bandwidth as the
46
+ original setup since only new commits and tags are pulled -- and then update
47
+ all extracted checkouts locally from those mirrors.
48
+
49
+ NOTE: There is one exception to this magic -- Niecza's build process needs to
50
+ download a ZIP of an older release in order to build a new one. Unfortunately,
51
+ because these are changed quite often, it would waste quite a bit of bandwidth
52
+ to download all the ZIPs at once during setup if the user doesn't plan to
53
+ benchmark *every* Niecza release. For this reason, if you plan to benchmark
54
+ Niecza when you are not on a fast network, you should at least *build* the
55
+ Niecza releases you want to test while still well-connected.
56
+
57
+
18
58
BENCHMARKING TIPS
19
59
-----------------
20
60
Make sure you stop background processes when benchmarking! Mail programs,
0 commit comments