Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
6207 lines (5620 sloc) 266 KB

Bellini’s LabBook

XXXX-XX-XX Proper emacs configuration file

I recommend you use Arnaud’s emacs configuration file, available here:

2015-09-15 Planning

Sep/OctComplete Java Simulator for the Paje file format
Nov/DecDatabase modeling for Paje traces
Jan/FebExperimentation and Performance Evaluation
Mar/JunWrite the thesis

2015-09-15 Structure / Draft of TCC

  1. Introduction (3 pages max)
    • Context (current scenario with pajeng c++)
    • Motivation
  2. Basic Concepts? (5 pages max)
    • JavaCC?
    • Ferramentas de análise de desempenho
    • MySQL ?
  3. PajeNG
    • How it is used by others (pjdump, etc)
    • Description of how it works internally
    • The problems with the current approach
      • Focus on your contribution
  4. Java Simulator
    • JavaCC / Perf. Eval. against others / Why JavaCC?
  5. Plugins
    • Null
    • PjDump
    • MySQL
      • Database modeling
      • arguments (batches, server, etc.)
  6. Performance Evaluation
    • Overview of what is being evaluated and why (my sql is the only NEW feature)
    • Methodology
    • Results and grafics
    • Possible explanations
  7. Conclusion

2016-02-15 Performance issues

Tais report that her solution to DB insertion has severe performance issues. The initial solution was that every pop event become a insert. That didn’t work, so Tais optimize with an alternative implementation which uses StringBuffer and create many insert commands on that object. They only get executed at the end of the parsing. Tais was never capable to evaluate if this solution is faster because execution runs out of memory.

So, by looking into this SO entry:

We propose a new attempt to solve the performance problem. The SO entry proposes four solutions, we will take the last one, which seems the best. To correctly adapt this fourth solution to Tais’s case, we should limit the batch size to something acceptable. If not, we will run out of memory again because the (usual) trace file size is just too big.

So, next plans are like this:

  • Tais will limit the size of the batch considering the number of operations on it.
  • Each batch will contain five StringBuffers, one for each paje type (Container, State, Link, Variable, and Event).
  • Every time something gets added to one of these string buffers, the operation count of the batch gets increased.
  • When the batch size reaches a (configurable) value X, the code executes the batch, zero it, and we start again filling.
  • Tais will do a quick performance analysis by changing the (configurable) value X to see the impact in execution time.

How to know the baseline:

  • Find out how long it take to only parse with the Java version. You can do that by checking out the no-db branch.

2016-02-26 Meeting with Tais

How to get it working.

Install mysql, root password should be root:

sudo apt-get install mysql-server

Run the pajeDB.sql to create the tables.

Fix compile.sh to have the correct CLASSPATH.

Problems found:

  • Program does not terminate

Plugin:

  • Today, SQL commands are created in the middle of the PajeSimulator source code. Moreover, the different strategies (searching for performance) are commented in the same source code. This leads to design that is difficult to manage, and also hard to evaluate the different performance options.
  • So my suggestion is to create a plugin for each type of output from the simulator. You will go back in time so the Paje Simulator source code is as clean as possible, and instrument this code to contain specific calls when something should be registered (such as the pop, the end links, and so on).
  • The plugin will be called in this different instrumentation points and do what it was design to do. A first plugin to be implemented is the one with the SQL insert commands.
  • Alternative versions to look for better performance should be implemented as other plugins, that could be subclasses (derived) from the initial one, if there is some functionality sharing between them.
  • Plugins already proposed by Tais:
    • Multiple batches, but are handle sequentially
    • Multiple batches, but with threads
    • Print the SQL commands to output
  • Class model for this plugin story
    • PajePlugin
      • PajePluginMultipleBatchesSequential
        • PajePluginMultipleBatchesThreads
      • PajePluginPrintSQLCommandsToOutput
      • PajePluginNull
      • PajePluginOutputPJDUMP
  • Clean PajeSimulator
    • Create PajePlugin
  • Exhaustive list of instrumentations points for the plugins
    • All types and values definitions
    • All create containers
    • Pops
    • Events
    • Add, Sub, Set variables
    • Start and end of links

2016-02-28 Starting Aiyra

The new implementation with plugins will be called Aiyra. The code is here. There are only two classes with a print informing which plugin was chosen.

How to use

java br.ufrgs.inf.tlbellini.PajeGrammar -f <path-to-trace-file> \
                                        -m <comment (optional)> \
                                        -p <plugin (default: null)> \
                                        - ... <options for the specific plugin>

Plugins available

  • “pjdump”: it outputs a CSV based on the pj_dump format
  • “mysql” : saves the data in a MySQL database

References

Command line reader: http://www.javaworld.com/article/2074849/core-java/processing-command-line-arguments-in-java–case-closed.html

Memory Usage: http://www.vogella.com/tutorials/JavaPerformance/article.html#runtimeinfo_memory https://developer.apple.com/library/mac/documentation/Performance/Conceptual/ManagingMemory/Articles/VMPages.html

Performance

commit: “create poc of integration of plugins and the Options package”

  • scorep-cg.A.64.paje - 2.1GB - 367918ms
  • scorep-lu.B.64.paje - 776.7MB - 77917ms
  • traces-16Mega.paje - 16MB - 1500ms

Plugins

PajeDumpPlugin: using System.out.println which is bad in performance, but I couldn’t find other solution that would not giva e heap space problem to big files. The current solution implies the user to decide how many lines will be in the dump. The higher the number, better the performance but more memory is used. In my computer, I have JavaHeapSpace problem.

2016-02-29 Adding PajeInsertDBPlugin

Notes

-When destroy container: just need to insert states

  • Event: inserts when it happens
  • Var: inserts when it is created or updated
  • Link: inserts when it is complete

2016-03-15 How to Use

DB Plugin

  1. Install MySQL and MySQL Workbench
  2. Create an instance (local or in a server) and choose a name and password to it. Use port 3306.
  3. execute pajeDB.sql script in the created instance.

Now you have a database called “paje”.

Execute the simulator passing the server name (-s) name (-u) and password (-pwd) of the instance you created in step 2. Defaults are “localhost”, “root”, “root”.

java br.ufrgs.inf.tlbellini.PajeGrammar -f <path-to-trace-file> \
                                        -m <comment (optional)> \
                                        -p mysql \
                                        -s <server-name> \
                                        -u <username> \
                                        -pwd <password> \

PajeDump Plugin

Execute:

java br.ufrgs.inf.tlbellini.PajeGrammar -f <path-to-trace-file> \
                                        -m <comment (optional)> \
                                        -p pjdump \
                                        -l <lines-per-dump (default: 100000)>

“lines-per-dump” is the number of lines that will be printed each time. Higher numbers takes less time but use more memory.

2016-03-15 - How to Create a Plugin

In order to create a new plugin you’ll need to: MODIFY OptionsHandler.java in br.ufrgs.inf.tlbellini; CREATE a NEW class in br.ufrgs.inf.tlbellini.plugins that EXTENDS PajePlugin class. It can also extend any other plugin if they have many similarities.

MODIFY OptionsHandler:

  1. If your plugin DOES NOT NEED an extra argument, SKIP this instruction

In the OptionsHandler constructor, add the new argument as follows:

opt.getSet().addOption(“<alias>”, Options.Separator.<SEPARATOR>, Options.Multiplicity.<MULTIPLICITY>);

<alias> = the alias to identify the argument <SEPARATOR> = COLON, EQUALS, BLANK, NONE <MULTIPLICITY> = ONCE, ONCE_OR_MORE, ZERO_OR_ONE, ZERO_OR_MORE

  1. Explain your new entry (if any) and plugin in printOptionsHelper() method.
  2. In pluginHandler() method, add a “case” to your plugin. Make all the assignments you need to do and initialize it. to get the value given for the argument: opt.getSet().getOption(“<alias”).getResultValue(0) to check if the value was set: opt.getSet().isSet(“<alias>”) to initialize: PajeGrammar.plugin = new YourPluginClass(…);

CREATE NEW CLASS:

  1. Create a class that extends PajePlugin or any Plugin available.

Instrumentation Points:

2016-03-29 Meeting with Tais

Performance Evalution

  • DB Insertion Plugin
    • Multiple batches, but are handle sequentially
  • What is the experimental configuration?
    • Software and hardware
    • Software: everything in the same machine

What we shall measure to evaluate?

  • Batch size
    • Input size
      • Lucas will provide a trace file generator 128K, 128M, 1G
    • Batch size in operations
      • 128
      • 256
      • 512
      • 1024
      • 2048
    • Four platform configurations
      • luiza Tais’ personal notebook (Local/Mac) - Tais
      • guarani Lucas’ guarani (Linux/Local) - Lucas
      • orion Orion (Linux/Local) - Tais
      • hptais Tais’ work notebook is client (Windows), guarani is server (Linux) (Remote) - Tais
        • Find out the interconnection topology
        • The latency and the bandwidth of the network (limits)
          • Find out a metric to see if you reach the network limit
  • What to measure
    • Global execution time
      • Only simulation
      • Only insertion
    • Global memory occupation
    Only Simulation TimeOnly Insertion TimeGlobal Memory Occupation
    • How batches have been generated
      • When a batch has started, when it has ended
      • Format suggestion for dumping this kind of information
        Batch numberStart timeEnd timeSize in bytesSize in number of operations
  • Comparison against C++
    • Measure only the simulation time
      • Null plugin against
        • pj_dump with -q
        • pj_dump with -q and -f (for flex)
    • Input size
      • 128K, 128M, 1G
    • Platforms
      • Tais’ personal notebook (Local) - Tais
      • Lucas’ guarani (Local) - Lucas
      • Orion (Local) - Tais
    Only simulation time

Experimental Design

  • How many replications? 30.
  • Statistical framework: gaussian distribution, with mean, sd, se
    • se: 3*sd/sqrt(n), CI of 99.7% assuming gaussian
    • no statistical tests (t-test), because we’ll use plots to analyze with mean+se and mean-se
  • See Jain 1991, Part III, Chapter 16.
  • Factors
    • Input size
      • small, big
    • Batch size
      • small, big
    • Platform
      • two platforms wildly different
  • Real Factors and levels (see chapter 23 on page 381)
    • input size: 3
    • batch size: 6
    • platforms: 4
  • 3*6*4
    input_size = 3;
    batch_levels = 6;
    platforms = 4;
    r=30;
    total_number_of_experiments = input_size*batch_levels*platforms*r
    minutes_each = 15;
    total_number_of_experiments * minutes_each / 60 / 24
        
    [1] 22.5
        

C++ Comparison Analysis

  • Version
    • Ayira
    • pajeng with flex
    • pajeng without flex
  • Input size
  • 30 replications
Input SizeVersion

Changes to the *mysql* plugin

  • add a new parameter to tell the batch size (default is size of input)
  • add a flag (disable by default) to tell the plugin to measure each batch execution
    • a vector will be used to keep this in memory during execution; only at the end that this is dumped to CSV
    • generates a new file that has the CSV info in the beggining of each line and then the following information you have measured
    Batch numberStart timeEnd timeSize in bytesSize in number of operations

How to measure memory utilization

  • MemoryUsage class in Java, check the used() method
  • Compare against top (the column that indicates the virtual memory allocated)

2016-03-29 DoE for C++ Comparison

The C++ Comparison has two factors: version and input, each one with three levels. For version, the levels are ayira, pj, pjflex. For input, the levels are small (128K), medium (131072K, or 128M), and big (1048576K or 1G). We shall use 30 replications randomizing experiments to reduce external interference. Two things are measured: the execution time represents the simulation time, the platform is the identification of the platform where measurements took place. Platform can be one of luiza, guarani, orion, hptais (TODO: rename this last one).

To understand what functions below do, check:

You’ll need the DoE.base R package.

Check the file Cpp_Comparison.csv in the designs directory. It contains your experimental design for running the Cpp Comparison experimental session. Each line is a experiment. You should run each experiment following the factor configuration, providing the execution time in the last column.

You can read that CSV with:

csv <- read.csv("designs/Cpp_Comparison.csv");
csv;
    name run.no.in.std.order run.no run.no.std.rp version  input time platform
1      1                   4      1          4.10   ayira medium   NA       NA
2      2                   9      2          9.10  pjflex    big   NA       NA
3      3                   2      3          2.10      pj  small   NA       NA
4      4                   5      4          5.10      pj medium   NA       NA
5      5                   7      5          7.10   ayira    big   NA       NA
6      6                   6      6          6.10  pjflex medium   NA       NA
7      7                   3      7          3.10  pjflex  small   NA       NA
8      8                   1      8          1.10   ayira  small   NA       NA
9      9                   8      9          8.10      pj    big   NA       NA
10    10                   6     10          6.20  pjflex medium   NA       NA
11    11                   7     11          7.20   ayira    big   NA       NA
12    12                   8     12          8.20      pj    big   NA       NA
13    13                   4     13          4.20   ayira medium   NA       NA
14    14                   3     14          3.20  pjflex  small   NA       NA
15    15                   5     15          5.20      pj medium   NA       NA
16    16                   2     16          2.20      pj  small   NA       NA
17    17                   9     17          9.20  pjflex    big   NA       NA
18    18                   1     18          1.20   ayira  small   NA       NA
19    19                   2     19          2.30      pj  small   NA       NA
20    20                   9     20          9.30  pjflex    big   NA       NA
21    21                   6     21          6.30  pjflex medium   NA       NA
22    22                   1     22          1.30   ayira  small   NA       NA
23    23                   3     23          3.30  pjflex  small   NA       NA
24    24                   4     24          4.30   ayira medium   NA       NA
25    25                   5     25          5.30      pj medium   NA       NA
26    26                   7     26          7.30   ayira    big   NA       NA
27    27                   8     27          8.30      pj    big   NA       NA
28    28                   6     28          6.40  pjflex medium   NA       NA
29    29                   7     29          7.40   ayira    big   NA       NA
30    30                   5     30          5.40      pj medium   NA       NA
31    31                   8     31          8.40      pj    big   NA       NA
32    32                   1     32          1.40   ayira  small   NA       NA
33    33                   4     33          4.40   ayira medium   NA       NA
34    34                   9     34          9.40  pjflex    big   NA       NA
35    35                   3     35          3.40  pjflex  small   NA       NA
36    36                   2     36          2.40      pj  small   NA       NA
37    37                   2     37          2.50      pj  small   NA       NA
38    38                   6     38          6.50  pjflex medium   NA       NA
39    39                   9     39          9.50  pjflex    big   NA       NA
40    40                   3     40          3.50  pjflex  small   NA       NA
41    41                   8     41          8.50      pj    big   NA       NA
42    42                   4     42          4.50   ayira medium   NA       NA
43    43                   5     43          5.50      pj medium   NA       NA
44    44                   7     44          7.50   ayira    big   NA       NA
45    45                   1     45          1.50   ayira  small   NA       NA
46    46                   4     46          4.60   ayira medium   NA       NA
47    47                   5     47          5.60      pj medium   NA       NA
48    48                   2     48          2.60      pj  small   NA       NA
49    49                   9     49          9.60  pjflex    big   NA       NA
50    50                   3     50          3.60  pjflex  small   NA       NA
51    51                   7     51          7.60   ayira    big   NA       NA
52    52                   8     52          8.60      pj    big   NA       NA
53    53                   6     53          6.60  pjflex medium   NA       NA
54    54                   1     54          1.60   ayira  small   NA       NA
55    55                   7     55          7.70   ayira    big   NA       NA
56    56                   1     56          1.70   ayira  small   NA       NA
57    57                   2     57          2.70      pj  small   NA       NA
58    58                   5     58          5.70      pj medium   NA       NA
59    59                   4     59          4.70   ayira medium   NA       NA
60    60                   8     60          8.70      pj    big   NA       NA
61    61                   9     61          9.70  pjflex    big   NA       NA
62    62                   6     62          6.70  pjflex medium   NA       NA
63    63                   3     63          3.70  pjflex  small   NA       NA
64    64                   8     64          8.80      pj    big   NA       NA
65    65                   7     65          7.80   ayira    big   NA       NA
66    66                   5     66          5.80      pj medium   NA       NA
67    67                   6     67          6.80  pjflex medium   NA       NA
68    68                   9     68          9.80  pjflex    big   NA       NA
69    69                   1     69          1.80   ayira  small   NA       NA
70    70                   3     70          3.80  pjflex  small   NA       NA
71    71                   4     71          4.80   ayira medium   NA       NA
72    72                   2     72          2.80      pj  small   NA       NA
73    73                   3     73          3.90  pjflex  small   NA       NA
74    74                   4     74          4.90   ayira medium   NA       NA
75    75                   5     75          5.90      pj medium   NA       NA
76    76                   7     76          7.90   ayira    big   NA       NA
77    77                   9     77          9.90  pjflex    big   NA       NA
78    78                   1     78          1.90   ayira  small   NA       NA
79    79                   6     79          6.90  pjflex medium   NA       NA
80    80                   2     80          2.90      pj  small   NA       NA
81    81                   8     81          8.90      pj    big   NA       NA
82    82                   3     82          3.10  pjflex  small   NA       NA
83    83                   4     83          4.10   ayira medium   NA       NA
84    84                   8     84          8.10      pj    big   NA       NA
85    85                   1     85          1.10   ayira  small   NA       NA
86    86                   2     86          2.10      pj  small   NA       NA
87    87                   9     87          9.10  pjflex    big   NA       NA
88    88                   5     88          5.10      pj medium   NA       NA
89    89                   7     89          7.10   ayira    big   NA       NA
90    90                   6     90          6.10  pjflex medium   NA       NA
91    91                   6     91          6.11  pjflex medium   NA       NA
92    92                   4     92          4.11   ayira medium   NA       NA
93    93                   5     93          5.11      pj medium   NA       NA
94    94                   7     94          7.11   ayira    big   NA       NA
95    95                   9     95          9.11  pjflex    big   NA       NA
96    96                   3     96          3.11  pjflex  small   NA       NA
97    97                   8     97          8.11      pj    big   NA       NA
98    98                   1     98          1.11   ayira  small   NA       NA
99    99                   2     99          2.11      pj  small   NA       NA
100  100                   3    100          3.12  pjflex  small   NA       NA
101  101                   4    101          4.12   ayira medium   NA       NA
102  102                   7    102          7.12   ayira    big   NA       NA
103  103                   2    103          2.12      pj  small   NA       NA
104  104                   5    104          5.12      pj medium   NA       NA
105  105                   6    105          6.12  pjflex medium   NA       NA
106  106                   1    106          1.12   ayira  small   NA       NA
107  107                   8    107          8.12      pj    big   NA       NA
108  108                   9    108          9.12  pjflex    big   NA       NA
109  109                   3    109          3.13  pjflex  small   NA       NA
110  110                   2    110          2.13      pj  small   NA       NA
111  111                   1    111          1.13   ayira  small   NA       NA
112  112                   7    112          7.13   ayira    big   NA       NA
113  113                   9    113          9.13  pjflex    big   NA       NA
114  114                   8    114          8.13      pj    big   NA       NA
115  115                   5    115          5.13      pj medium   NA       NA
116  116                   6    116          6.13  pjflex medium   NA       NA
117  117                   4    117          4.13   ayira medium   NA       NA
118  118                   4    118          4.14   ayira medium   NA       NA
119  119                   2    119          2.14      pj  small   NA       NA
120  120                   1    120          1.14   ayira  small   NA       NA
121  121                   5    121          5.14      pj medium   NA       NA
122  122                   6    122          6.14  pjflex medium   NA       NA
123  123                   8    123          8.14      pj    big   NA       NA
124  124                   7    124          7.14   ayira    big   NA       NA
125  125                   3    125          3.14  pjflex  small   NA       NA
126  126                   9    126          9.14  pjflex    big   NA       NA
127  127                   5    127          5.15      pj medium   NA       NA
128  128                   6    128          6.15  pjflex medium   NA       NA
129  129                   2    129          2.15      pj  small   NA       NA
130  130                   1    130          1.15   ayira  small   NA       NA
131  131                   8    131          8.15      pj    big   NA       NA
132  132                   3    132          3.15  pjflex  small   NA       NA
133  133                   7    133          7.15   ayira    big   NA       NA
134  134                   9    134          9.15  pjflex    big   NA       NA
135  135                   4    135          4.15   ayira medium   NA       NA
136  136                   7    136          7.16   ayira    big   NA       NA
137  137                   4    137          4.16   ayira medium   NA       NA
138  138                   8    138          8.16      pj    big   NA       NA
139  139                   2    139          2.16      pj  small   NA       NA
140  140                   3    140          3.16  pjflex  small   NA       NA
141  141                   1    141          1.16   ayira  small   NA       NA
142  142                   5    142          5.16      pj medium   NA       NA
143  143                   9    143          9.16  pjflex    big   NA       NA
144  144                   6    144          6.16  pjflex medium   NA       NA
145  145                   2    145          2.17      pj  small   NA       NA
146  146                   4    146          4.17   ayira medium   NA       NA
147  147                   9    147          9.17  pjflex    big   NA       NA
148  148                   1    148          1.17   ayira  small   NA       NA
149  149                   8    149          8.17      pj    big   NA       NA
150  150                   5    150          5.17      pj medium   NA       NA
151  151                   6    151          6.17  pjflex medium   NA       NA
152  152                   7    152          7.17   ayira    big   NA       NA
153  153                   3    153          3.17  pjflex  small   NA       NA
154  154                   5    154          5.18      pj medium   NA       NA
155  155                   8    155          8.18      pj    big   NA       NA
156  156                   9    156          9.18  pjflex    big   NA       NA
157  157                   2    157          2.18      pj  small   NA       NA
158  158                   3    158          3.18  pjflex  small   NA       NA
159  159                   6    159          6.18  pjflex medium   NA       NA
160  160                   7    160          7.18   ayira    big   NA       NA
161  161                   4    161          4.18   ayira medium   NA       NA
162  162                   1    162          1.18   ayira  small   NA       NA
163  163                   7    163          7.19   ayira    big   NA       NA
164  164                   3    164          3.19  pjflex  small   NA       NA
165  165                   2    165          2.19      pj  small   NA       NA
166  166                   6    166          6.19  pjflex medium   NA       NA
167  167                   5    167          5.19      pj medium   NA       NA
168  168                   9    168          9.19  pjflex    big   NA       NA
169  169                   4    169          4.19   ayira medium   NA       NA
170  170                   8    170          8.19      pj    big   NA       NA
171  171                   1    171          1.19   ayira  small   NA       NA
172  172                   3    172          3.20  pjflex  small   NA       NA
173  173                   7    173          7.20   ayira    big   NA       NA
174  174                   2    174          2.20      pj  small   NA       NA
175  175                   1    175          1.20   ayira  small   NA       NA
176  176                   6    176          6.20  pjflex medium   NA       NA
177  177                   9    177          9.20  pjflex    big   NA       NA
178  178                   5    178          5.20      pj medium   NA       NA
179  179                   8    179          8.20      pj    big   NA       NA
180  180                   4    180          4.20   ayira medium   NA       NA
181  181                   2    181          2.21      pj  small   NA       NA
182  182                   4    182          4.21   ayira medium   NA       NA
183  183                   9    183          9.21  pjflex    big   NA       NA
184  184                   8    184          8.21      pj    big   NA       NA
185  185                   7    185          7.21   ayira    big   NA       NA
186  186                   1    186          1.21   ayira  small   NA       NA
187  187                   5    187          5.21      pj medium   NA       NA
188  188                   3    188          3.21  pjflex  small   NA       NA
189  189                   6    189          6.21  pjflex medium   NA       NA
190  190                   6    190          6.22  pjflex medium   NA       NA
191  191                   7    191          7.22   ayira    big   NA       NA
192  192                   2    192          2.22      pj  small   NA       NA
193  193                   4    193          4.22   ayira medium   NA       NA
194  194                   9    194          9.22  pjflex    big   NA       NA
195  195                   1    195          1.22   ayira  small   NA       NA
196  196                   8    196          8.22      pj    big   NA       NA
197  197                   3    197          3.22  pjflex  small   NA       NA
198  198                   5    198          5.22      pj medium   NA       NA
199  199                   5    199          5.23      pj medium   NA       NA
200  200                   3    200          3.23  pjflex  small   NA       NA
201  201                   6    201          6.23  pjflex medium   NA       NA
202  202                   8    202          8.23      pj    big   NA       NA
203  203                   2    203          2.23      pj  small   NA       NA
204  204                   7    204          7.23   ayira    big   NA       NA
205  205                   1    205          1.23   ayira  small   NA       NA
206  206                   9    206          9.23  pjflex    big   NA       NA
207  207                   4    207          4.23   ayira medium   NA       NA
208  208                   3    208          3.24  pjflex  small   NA       NA
209  209                   8    209          8.24      pj    big   NA       NA
210  210                   6    210          6.24  pjflex medium   NA       NA
211  211                   2    211          2.24      pj  small   NA       NA
212  212                   9    212          9.24  pjflex    big   NA       NA
213  213                   7    213          7.24   ayira    big   NA       NA
214  214                   5    214          5.24      pj medium   NA       NA
215  215                   4    215          4.24   ayira medium   NA       NA
216  216                   1    216          1.24   ayira  small   NA       NA
217  217                   7    217          7.25   ayira    big   NA       NA
218  218                   9    218          9.25  pjflex    big   NA       NA
219  219                   1    219          1.25   ayira  small   NA       NA
220  220                   4    220          4.25   ayira medium   NA       NA
221  221                   8    221          8.25      pj    big   NA       NA
222  222                   5    222          5.25      pj medium   NA       NA
223  223                   6    223          6.25  pjflex medium   NA       NA
224  224                   3    224          3.25  pjflex  small   NA       NA
225  225                   2    225          2.25      pj  small   NA       NA
226  226                   6    226          6.26  pjflex medium   NA       NA
227  227                   5    227          5.26      pj medium   NA       NA
228  228                   2    228          2.26      pj  small   NA       NA
229  229                   1    229          1.26   ayira  small   NA       NA
230  230                   7    230          7.26   ayira    big   NA       NA
231  231                   4    231          4.26   ayira medium   NA       NA
232  232                   9    232          9.26  pjflex    big   NA       NA
233  233                   8    233          8.26      pj    big   NA       NA
234  234                   3    234          3.26  pjflex  small   NA       NA
235  235                   8    235          8.27      pj    big   NA       NA
236  236                   9    236          9.27  pjflex    big   NA       NA
237  237                   1    237          1.27   ayira  small   NA       NA
238  238                   5    238          5.27      pj medium   NA       NA
239  239                   7    239          7.27   ayira    big   NA       NA
240  240                   6    240          6.27  pjflex medium   NA       NA
241  241                   2    241          2.27      pj  small   NA       NA
242  242                   4    242          4.27   ayira medium   NA       NA
243  243                   3    243          3.27  pjflex  small   NA       NA
244  244                   3    244          3.28  pjflex  small   NA       NA
245  245                   8    245          8.28      pj    big   NA       NA
246  246                   2    246          2.28      pj  small   NA       NA
247  247                   1    247          1.28   ayira  small   NA       NA
248  248                   7    248          7.28   ayira    big   NA       NA
249  249                   5    249          5.28      pj medium   NA       NA
250  250                   9    250          9.28  pjflex    big   NA       NA
251  251                   6    251          6.28  pjflex medium   NA       NA
252  252                   4    252          4.28   ayira medium   NA       NA
253  253                   1    253          1.29   ayira  small   NA       NA
254  254                   3    254          3.29  pjflex  small   NA       NA
255  255                   7    255          7.29   ayira    big   NA       NA
256  256                   4    256          4.29   ayira medium   NA       NA
257  257                   6    257          6.29  pjflex medium   NA       NA
258  258                   5    258          5.29      pj medium   NA       NA
259  259                   2    259          2.29      pj  small   NA       NA
260  260                   8    260          8.29      pj    big   NA       NA
261  261                   9    261          9.29  pjflex    big   NA       NA
262  262                   1    262          1.30   ayira  small   NA       NA
263  263                   7    263          7.30   ayira    big   NA       NA
264  264                   3    264          3.30  pjflex  small   NA       NA
265  265                   6    265          6.30  pjflex medium   NA       NA
266  266                   4    266          4.30   ayira medium   NA       NA
267  267                   5    267          5.30      pj medium   NA       NA
268  268                   2    268          2.30      pj  small   NA       NA
269  269                   8    269          8.30      pj    big   NA       NA
270  270                   9    270          9.30  pjflex    big   NA       NA

NA means not available. Last two columns should have simulation time for that factor combination and the platform where experiments took place.

2016-03-29 DoE for Batch Size Investigation

The batch size investigation will have two factors: input, and batch. The first one with three levels, the second one with six levels. The levels for input are small (128K), medium (131072K, or 128M), and big (1048576K or 1G). The levels for batch are A, B, C, D, E and F. Each letter is a integer number which represents the number of SQL operations in each batch. Based on preliminary runs, Tais will decide which values to use instead of letters A to F. I’ll use them so it’s easy to map them to values once they are defined. We shall again use 30 replications randomizing experiments to reduce external interference. Four things are measured: the global simulation time (all time except DB insertions), the global insertion time (the sum of the time spent executing all DB insertions - the batches), the maximum memory occupation during the experiment, and the platform. Platform can be one of luiza, guarani, orion, hptais (TODO: rename this last one). The memory occupation should be measure with exception care, since the maximum may not appear at the end of the execution, specially if batches are small.

csv <- read.csv("designs/Batch_Size_Investigation.csv");
csv;
    name run.no.in.std.order run.no run.no.std.rp  input batch sim_time
1      1                   8      1          8.10 medium     C       NA
2      2                  18      2         18.10    big     F       NA
3      3                   5      3          5.10 medium     B       NA
4      4                  12      4         12.10    big     D       NA
5      5                  16      5         16.10  small     F       NA
6      6                   6      6          6.10    big     B       NA
7      7                  15      7         15.10    big     E       NA
8      8                   3      8          3.10    big     A       NA
9      9                  13      9         13.10  small     E       NA
10    10                  10     10         10.10  small     D       NA
11    11                   7     11          7.10  small     C       NA
12    12                  17     12         17.10 medium     F       NA
13    13                   4     13          4.10  small     B       NA
14    14                  11     14         11.10 medium     D       NA
15    15                  14     15         14.10 medium     E       NA
16    16                   2     16          2.10 medium     A       NA
17    17                   9     17          9.10    big     C       NA
18    18                   1     18          1.10  small     A       NA
19    19                   3     19          3.20    big     A       NA
20    20                   5     20          5.20 medium     B       NA
21    21                  14     21         14.20 medium     E       NA
22    22                   1     22          1.20  small     A       NA
23    23                   6     23          6.20    big     B       NA
24    24                  10     24         10.20  small     D       NA
25    25                  13     25         13.20  small     E       NA
26    26                  17     26         17.20 medium     F       NA
27    27                  12     27         12.20    big     D       NA
28    28                  16     28         16.20  small     F       NA
29    29                   7     29          7.20  small     C       NA
30    30                  11     30         11.20 medium     D       NA
31    31                   8     31          8.20 medium     C       NA
32    32                  15     32         15.20    big     E       NA
33    33                   4     33          4.20  small     B       NA
34    34                   9     34          9.20    big     C       NA
35    35                  18     35         18.20    big     F       NA
36    36                   2     36          2.20 medium     A       NA
37    37                   3     37          3.30    big     A       NA
38    38                  11     38         11.30 medium     D       NA
39    39                   5     39          5.30 medium     B       NA
40    40                   6     40          6.30    big     B       NA
41    41                   7     41          7.30  small     C       NA
42    42                  17     42         17.30 medium     F       NA
43    43                  12     43         12.30    big     D       NA
44    44                  10     44         10.30  small     D       NA
45    45                  14     45         14.30 medium     E       NA
46    46                   4     46          4.30  small     B       NA
47    47                  16     47         16.30  small     F       NA
48    48                   2     48          2.30 medium     A       NA
49    49                   9     49          9.30    big     C       NA
50    50                  18     50         18.30    big     F       NA
51    51                  13     51         13.30  small     E       NA
52    52                   8     52          8.30 medium     C       NA
53    53                  15     53         15.30    big     E       NA
54    54                   1     54          1.30  small     A       NA
55    55                  14     55         14.40 medium     E       NA
56    56                   1     56          1.40  small     A       NA
57    57                   5     57          5.40 medium     B       NA
58    58                  12     58         12.40    big     D       NA
59    59                  15     59         15.40    big     E       NA
60    60                   2     60          2.40 medium     A       NA
61    61                  16     61         16.40  small     F       NA
62    62                  13     62         13.40  small     E       NA
63    63                   9     63          9.40    big     C       NA
64    64                   8     64          8.40 medium     C       NA
65    65                   7     65          7.40  small     C       NA
66    66                  18     66         18.40    big     F       NA
67    67                   6     67          6.40    big     B       NA
68    68                  10     68         10.40  small     D       NA
69    69                  17     69         17.40 medium     F       NA
70    70                   3     70          3.40    big     A       NA
71    71                   4     71          4.40  small     B       NA
72    72                  11     72         11.40 medium     D       NA
73    73                   6     73          6.50    big     B       NA
74    74                   9     74          9.50    big     C       NA
75    75                  10     75         10.50  small     D       NA
76    76                  13     76         13.50  small     E       NA
77    77                   8     77          8.50 medium     C       NA
78    78                   2     78          2.50 medium     A       NA
79    79                  17     79         17.50 medium     F       NA
80    80                  18     80         18.50    big     F       NA
81    81                   7     81          7.50  small     C       NA
82    82                   3     82          3.50    big     A       NA
83    83                   4     83          4.50  small     B       NA
84    84                  14     84         14.50 medium     E       NA
85    85                   1     85          1.50  small     A       NA
86    86                  15     86         15.50    big     E       NA
87    87                  12     87         12.50    big     D       NA
88    88                   5     88          5.50 medium     B       NA
89    89                  16     89         16.50  small     F       NA
90    90                  11     90         11.50 medium     D       NA
91    91                  12     91         12.60    big     D       NA
92    92                   8     92          8.60 medium     C       NA
93    93                  18     93         18.60    big     F       NA
94    94                  11     94         11.60 medium     D       NA
95    95                  16     95         16.60  small     F       NA
96    96                  17     96         17.60 medium     F       NA
97    97                   9     97          9.60    big     C       NA
98    98                   5     98          5.60 medium     B       NA
99    99                  13     99         13.60  small     E       NA
100  100                   3    100          3.60    big     A       NA
101  101                   4    101          4.60  small     B       NA
102  102                   7    102          7.60  small     C       NA
103  103                   2    103          2.60 medium     A       NA
104  104                  15    104         15.60    big     E       NA
105  105                   6    105          6.60    big     B       NA
106  106                   1    106          1.60  small     A       NA
107  107                  10    107         10.60  small     D       NA
108  108                  14    108         14.60 medium     E       NA
109  109                   6    109          6.70    big     B       NA
110  110                   4    110          4.70  small     B       NA
111  111                   1    111          1.70  small     A       NA
112  112                   2    112          2.70 medium     A       NA
113  113                   7    113          7.70  small     C       NA
114  114                  18    114         18.70    big     F       NA
115  115                  10    115         10.70  small     D       NA
116  116                   5    116          5.70 medium     B       NA
117  117                  16    117         16.70  small     F       NA
118  118                  17    118         17.70 medium     F       NA
119  119                  15    119         15.70    big     E       NA
120  120                  12    120         12.70    big     D       NA
121  121                  11    121         11.70 medium     D       NA
122  122                  13    122         13.70  small     E       NA
123  123                   8    123          8.70 medium     C       NA
124  124                  14    124         14.70 medium     E       NA
125  125                   3    125          3.70    big     A       NA
126  126                   9    126          9.70    big     C       NA
127  127                  10    127         10.80  small     D       NA
128  128                  11    128         11.80 medium     D       NA
129  129                   5    129          5.80 medium     B       NA
130  130                   1    130          1.80  small     A       NA
131  131                   2    131          2.80 medium     A       NA
132  132                   9    132          9.80    big     C       NA
133  133                   7    133          7.80  small     C       NA
134  134                  14    134         14.80 medium     E       NA
135  135                  13    135         13.80  small     E       NA
136  136                  12    136         12.80    big     D       NA
137  137                   4    137          4.80  small     B       NA
138  138                   8    138          8.80 medium     C       NA
139  139                  17    139         17.80 medium     F       NA
140  140                   3    140          3.80    big     A       NA
141  141                  15    141         15.80    big     E       NA
142  142                  16    142         16.80  small     F       NA
143  143                  18    143         18.80    big     F       NA
144  144                   6    144          6.80    big     B       NA
145  145                   4    145          4.90  small     B       NA
146  146                   8    146          8.90 medium     C       NA
147  147                  18    147         18.90    big     F       NA
148  148                   2    148          2.90 medium     A       NA
149  149                  11    149         11.90 medium     D       NA
150  150                  12    150         12.90    big     D       NA
151  151                   3    151          3.90    big     A       NA
152  152                  10    152         10.90  small     D       NA
153  153                  14    153         14.90 medium     E       NA
154  154                   5    154          5.90 medium     B       NA
155  155                  17    155         17.90 medium     F       NA
156  156                   9    156          9.90    big     C       NA
157  157                  15    157         15.90    big     E       NA
158  158                  13    158         13.90  small     E       NA
159  159                   6    159          6.90    big     B       NA
160  160                   7    160          7.90  small     C       NA
161  161                  16    161         16.90  small     F       NA
162  162                   1    162          1.90  small     A       NA
163  163                  13    163         13.10  small     E       NA
164  164                   5    164          5.10 medium     B       NA
165  165                   3    165          3.10    big     A       NA
166  166                  15    166         15.10    big     E       NA
167  167                  14    167         14.10 medium     E       NA
168  168                  17    168         17.10 medium     F       NA
169  169                   6    169          6.10    big     B       NA
170  170                   7    170          7.10  small     C       NA
171  171                  12    171         12.10    big     D       NA
172  172                  16    172         16.10  small     F       NA
173  173                  11    173         11.10 medium     D       NA
174  174                   2    174          2.10 medium     A       NA
175  175                   1    175          1.10  small     A       NA
176  176                  10    176         10.10  small     D       NA
177  177                   9    177          9.10    big     C       NA
178  178                  18    178         18.10    big     F       NA
179  179                   8    179          8.10 medium     C       NA
180  180                   4    180          4.10  small     B       NA
181  181                   3    181          3.11    big     A       NA
182  182                   7    182          7.11  small     C       NA
183  183                   4    183          4.11  small     B       NA
184  184                   8    184          8.11 medium     C       NA
185  185                   5    185          5.11 medium     B       NA
186  186                   1    186          1.11  small     A       NA
187  187                  17    187         17.11 medium     F       NA
188  188                   9    188          9.11    big     C       NA
189  189                   2    189          2.11 medium     A       NA
190  190                   6    190          6.11    big     B       NA
191  191                  12    191         12.11    big     D       NA
192  192                  10    192         10.11  small     D       NA
193  193                  16    193         16.11  small     F       NA
194  194                  11    194         11.11 medium     D       NA
195  195                  13    195         13.11  small     E       NA
196  196                  15    196         15.11    big     E       NA
197  197                  18    197         18.11    big     F       NA
198  198                  14    198         14.11 medium     E       NA
199  199                  10    199         10.12  small     D       NA
200  200                   5    200          5.12 medium     B       NA
201  201                  13    201         13.12  small     E       NA
202  202                   7    202          7.12  small     C       NA
203  203                   4    203          4.12  small     B       NA
204  204                   8    204          8.12 medium     C       NA
205  205                   3    205          3.12    big     A       NA
206  206                  11    206         11.12 medium     D       NA
207  207                  14    207         14.12 medium     E       NA
208  208                  12    208         12.12    big     D       NA
209  209                  16    209         16.12  small     F       NA
210  210                   6    210          6.12    big     B       NA
211  211                   2    211          2.12 medium     A       NA
212  212                   9    212          9.12    big     C       NA
213  213                  15    213         15.12    big     E       NA
214  214                  17    214         17.12 medium     F       NA
215  215                  18    215         18.12    big     F       NA
216  216                   1    216          1.12  small     A       NA
217  217                  13    217         13.13  small     E       NA
218  218                  14    218         14.13 medium     E       NA
219  219                   1    219          1.13  small     A       NA
220  220                  10    220         10.13  small     D       NA
221  221                   2    221          2.13 medium     A       NA
222  222                  17    222         17.13 medium     F       NA
223  223                  16    223         16.13  small     F       NA
224  224                   4    224          4.13  small     B       NA
225  225                   7    225          7.13  small     C       NA
226  226                   6    226          6.13    big     B       NA
227  227                   5    227          5.13 medium     B       NA
228  228                  18    228         18.13    big     F       NA
229  229                  12    229         12.13    big     D       NA
230  230                  15    230         15.13    big     E       NA
231  231                  11    231         11.13 medium     D       NA
232  232                   9    232          9.13    big     C       NA
233  233                   8    233          8.13 medium     C       NA
234  234                   3    234          3.13    big     A       NA
235  235                  15    235         15.14    big     E       NA
236  236                  18    236         18.14    big     F       NA
237  237                   1    237          1.14  small     A       NA
238  238                  12    238         12.14    big     D       NA
239  239                   3    239          3.14    big     A       NA
240  240                  14    240         14.14 medium     E       NA
241  241                   7    241          7.14  small     C       NA
242  242                  16    242         16.14  small     F       NA
243  243                  17    243         17.14 medium     F       NA
244  244                  13    244         13.14  small     E       NA
245  245                   8    245          8.14 medium     C       NA
246  246                   2    246          2.14 medium     A       NA
247  247                  11    247         11.14 medium     D       NA
248  248                  10    248         10.14  small     D       NA
249  249                   5    249          5.14 medium     B       NA
250  250                   9    250          9.14    big     C       NA
251  251                   6    251          6.14    big     B       NA
252  252                   4    252          4.14  small     B       NA
253  253                   2    253          2.15 medium     A       NA
254  254                   6    254          6.15    big     B       NA
255  255                  16    255         16.15  small     F       NA
256  256                   9    256          9.15    big     C       NA
257  257                  11    257         11.15 medium     D       NA
258  258                  13    258         13.15  small     E       NA
259  259                   7    259          7.15  small     C       NA
260  260                  15    260         15.15    big     E       NA
261  261                   1    261          1.15  small     A       NA
262  262                  10    262         10.15  small     D       NA
263  263                  12    263         12.15    big     D       NA
264  264                   3    264          3.15    big     A       NA
265  265                  17    265         17.15 medium     F       NA
266  266                   4    266          4.15  small     B       NA
267  267                   5    267          5.15 medium     B       NA
268  268                  18    268         18.15    big     F       NA
269  269                   8    269          8.15 medium     C       NA
270  270                  14    270         14.15 medium     E       NA
271  271                   3    271          3.16    big     A       NA
272  272                  14    272         14.16 medium     E       NA
273  273                   1    273          1.16  small     A       NA
274  274                  11    274         11.16 medium     D       NA
275  275                   7    275          7.16  small     C       NA
276  276                   6    276          6.16    big     B       NA
277  277                   5    277          5.16 medium     B       NA
278  278                  15    278         15.16    big     E       NA
279  279                  18    279         18.16    big     F       NA
280  280                  13    280         13.16  small     E       NA
281  281                  12    281         12.16    big     D       NA
282  282                   2    282          2.16 medium     A       NA
283  283                  10    283         10.16  small     D       NA
284  284                  17    284         17.16 medium     F       NA
285  285                   4    285          4.16  small     B       NA
286  286                  16    286         16.16  small     F       NA
287  287                   8    287          8.16 medium     C       NA
288  288                   9    288          9.16    big     C       NA
289  289                  12    289         12.17    big     D       NA
290  290                  15    290         15.17    big     E       NA
291  291                  16    291         16.17  small     F       NA
292  292                  17    292         17.17 medium     F       NA
293  293                   4    293          4.17  small     B       NA
294  294                   3    294          3.17    big     A       NA
295  295                  10    295         10.17  small     D       NA
296  296                   8    296          8.17 medium     C       NA
297  297                   6    297          6.17    big     B       NA
298  298                   9    298          9.17    big     C       NA
299  299                   5    299          5.17 medium     B       NA
300  300                   1    300          1.17  small     A       NA
301  301                  13    301         13.17  small     E       NA
302  302                  14    302         14.17 medium     E       NA
303  303                  18    303         18.17    big     F       NA
304  304                  11    304         11.17 medium     D       NA
305  305                   2    305          2.17 medium     A       NA
306  306                   7    306          7.17  small     C       NA
307  307                   2    307          2.18 medium     A       NA
308  308                  10    308         10.18  small     D       NA
309  309                  11    309         11.18 medium     D       NA
310  310                  15    310         15.18    big     E       NA
311  311                   8    311          8.18 medium     C       NA
312  312                   3    312          3.18    big     A       NA
313  313                   7    313          7.18  small     C       NA
314  314                  18    314         18.18    big     F       NA
315  315                  14    315         14.18 medium     E       NA
316  316                   9    316          9.18    big     C       NA
317  317                   1    317          1.18  small     A       NA
318  318                  13    318         13.18  small     E       NA
319  319                   6    319          6.18    big     B       NA
320  320                  16    320         16.18  small     F       NA
321  321                   4    321          4.18  small     B       NA
322  322                  17    322         17.18 medium     F       NA
323  323                   5    323          5.18 medium     B       NA
324  324                  12    324         12.18    big     D       NA
325  325                   8    325          8.19 medium     C       NA
326  326                  18    326         18.19    big     F       NA
327  327                   4    327          4.19  small     B       NA
328  328                   1    328          1.19  small     A       NA
329  329                   2    329          2.19 medium     A       NA
330  330                  11    330         11.19 medium     D       NA
331  331                  13    331         13.19  small     E       NA
332  332                  15    332         15.19    big     E       NA
333  333                  12    333         12.19    big     D       NA
334  334                  10    334         10.19  small     D       NA
335  335                   3    335          3.19    big     A       NA
336  336                  17    336         17.19 medium     F       NA
337  337                   7    337          7.19  small     C       NA
338  338                   6    338          6.19    big     B       NA
339  339                   9    339          9.19    big     C       NA
340  340                  16    340         16.19  small     F       NA
341  341                  14    341         14.19 medium     E       NA
342  342                   5    342          5.19 medium     B       NA
343  343                   5    343          5.20 medium     B       NA
344  344                  15    344         15.20    big     E       NA
345  345                   4    345          4.20  small     B       NA
346  346                  13    346         13.20  small     E       NA
347  347                  10    347         10.20  small     D       NA
348  348                   9    348          9.20    big     C       NA
349  349                   6    349          6.20    big     B       NA
350  350                   3    350          3.20    big     A       NA
351  351                  12    351         12.20    big     D       NA
352  352                   8    352          8.20 medium     C       NA
353  353                  16    353         16.20  small     F       NA
354  354                   1    354          1.20  small     A       NA
355  355                   2    355          2.20 medium     A       NA
356  356                  11    356         11.20 medium     D       NA
357  357                  17    357         17.20 medium     F       NA
358  358                  18    358         18.20    big     F       NA
359  359                  14    359         14.20 medium     E       NA
360  360                   7    360          7.20  small     C       NA
361  361                  12    361         12.21    big     D       NA
362  362                  11    362         11.21 medium     D       NA
363  363                  17    363         17.21 medium     F       NA
364  364                  16    364         16.21  small     F       NA
365  365                   9    365          9.21    big     C       NA
366  366                   4    366          4.21  small     B       NA
367  367                   7    367          7.21  small     C       NA
368  368                  15    368         15.21    big     E       NA
369  369                   2    369          2.21 medium     A       NA
370  370                   3    370          3.21    big     A       NA
371  371                   8    371          8.21 medium     C       NA
372  372                  10    372         10.21  small     D       NA
373  373                   5    373          5.21 medium     B       NA
374  374                  18    374         18.21    big     F       NA
375  375                  13    375         13.21  small     E       NA
376  376                  14    376         14.21 medium     E       NA
377  377                   6    377          6.21    big     B       NA
378  378                   1    378          1.21  small     A       NA
379  379                   5    379          5.22 medium     B       NA
380  380                   4    380          4.22  small     B       NA
381  381                  18    381         18.22    big     F       NA
382  382                   2    382          2.22 medium     A       NA
383  383                  12    383         12.22    big     D       NA
384  384                   9    384          9.22    big     C       NA
385  385                   7    385          7.22  small     C       NA
386  386                  16    386         16.22  small     F       NA
387  387                  14    387         14.22 medium     E       NA
388  388                  17    388         17.22 medium     F       NA
389  389                  15    389         15.22    big     E       NA
390  390                  10    390         10.22  small     D       NA
391  391                   6    391          6.22    big     B       NA
392  392                   3    392          3.22    big     A       NA
393  393                   8    393          8.22 medium     C       NA
394  394                   1    394          1.22  small     A       NA
395  395                  11    395         11.22 medium     D       NA
396  396                  13    396         13.22  small     E       NA
397  397                  14    397         14.23 medium     E       NA
398  398                   3    398          3.23    big     A       NA
399  399                   8    399          8.23 medium     C       NA
400  400                   1    400          1.23  small     A       NA
401  401                   4    401          4.23  small     B       NA
402  402                  12    402         12.23    big     D       NA
403  403                  16    403         16.23  small     F       NA
404  404                   7    404          7.23  small     C       NA
405  405                   6    405          6.23    big     B       NA
406  406                   9    406          9.23    big     C       NA
407  407                  18    407         18.23    big     F       NA
408  408                   5    408          5.23 medium     B       NA
409  409                  15    409         15.23    big     E       NA
410  410                   2    410          2.23 medium     A       NA
411  411                  13    411         13.23  small     E       NA
412  412                  17    412         17.23 medium     F       NA
413  413                  11    413         11.23 medium     D       NA
414  414                  10    414         10.23  small     D       NA
415  415                   3    415          3.24    big     A       NA
416  416                  14    416         14.24 medium     E       NA
417  417                  16    417         16.24  small     F       NA
418  418                  17    418         17.24 medium     F       NA
419  419                   8    419          8.24 medium     C       NA
420  420                  18    420         18.24    big     F       NA
421  421                   5    421          5.24 medium     B       NA
422  422                   6    422          6.24    big     B       NA
423  423                   9    423          9.24    big     C       NA
424  424                   2    424          2.24 medium     A       NA
425  425                   4    425          4.24  small     B       NA
426  426                  15    426         15.24    big     E       NA
427  427                   1    427          1.24  small     A       NA
428  428                   7    428          7.24  small     C       NA
429  429                  13    429         13.24  small     E       NA
430  430                  12    430         12.24    big     D       NA
431  431                  10    431         10.24  small     D       NA
432  432                  11    432         11.24 medium     D       NA
433  433                  13    433         13.25  small     E       NA
434  434                   4    434          4.25  small     B       NA
435  435                   3    435          3.25    big     A       NA
436  436                   9    436          9.25    big     C       NA
437  437                  11    437         11.25 medium     D       NA
438  438                   8    438          8.25 medium     C       NA
439  439                   7    439          7.25  small     C       NA
440  440                   1    440          1.25  small     A       NA
441  441                  14    441         14.25 medium     E       NA
442  442                  15    442         15.25    big     E       NA
443  443                  16    443         16.25  small     F       NA
444  444                  17    444         17.25 medium     F       NA
445  445                   6    445          6.25    big     B       NA
446  446                  10    446         10.25  small     D       NA
447  447                  18    447         18.25    big     F       NA
448  448                  12    448         12.25    big     D       NA
449  449                   5    449          5.25 medium     B       NA
450  450                   2    450          2.25 medium     A       NA
451  451                   7    451          7.26  small     C       NA
452  452                  11    452         11.26 medium     D       NA
453  453                   6    453          6.26    big     B       NA
454  454                  15    454         15.26    big     E       NA
455  455                   8    455          8.26 medium     C       NA
456  456                   4    456          4.26  small     B       NA
457  457                  14    457         14.26 medium     E       NA
458  458                  16    458         16.26  small     F       NA
459  459                   1    459          1.26  small     A       NA
460  460                  10    460         10.26  small     D       NA
461  461                  18    461         18.26    big     F       NA
462  462                   5    462          5.26 medium     B       NA
463  463                   2    463          2.26 medium     A       NA
464  464                  12    464         12.26    big     D       NA
465  465                   9    465          9.26    big     C       NA
466  466                  13    466         13.26  small     E       NA
467  467                   3    467          3.26    big     A       NA
468  468                  17    468         17.26 medium     F       NA
469  469                  15    469         15.27    big     E       NA
470  470                   6    470          6.27    big     B       NA
471  471                   4    471          4.27  small     B       NA
472  472                   9    472          9.27    big     C       NA
473  473                  14    473         14.27 medium     E       NA
474  474                  10    474         10.27  small     D       NA
475  475                   3    475          3.27    big     A       NA
476  476                  18    476         18.27    big     F       NA
477  477                  17    477         17.27 medium     F       NA
478  478                   2    478          2.27 medium     A       NA
479  479                   7    479          7.27  small     C       NA
480  480                   5    480          5.27 medium     B       NA
481  481                  11    481         11.27 medium     D       NA
482  482                  12    482         12.27    big     D       NA
483  483                   8    483          8.27 medium     C       NA
484  484                   1    484          1.27  small     A       NA
485  485                  13    485         13.27  small     E       NA
486  486                  16    486         16.27  small     F       NA
487  487                   2    487          2.28 medium     A       NA
488  488                   7    488          7.28  small     C       NA
489  489                  10    489         10.28  small     D       NA
490  490                   8    490          8.28 medium     C       NA
491  491                   9    491          9.28    big     C       NA
492  492                  11    492         11.28 medium     D       NA
493  493                   3    493          3.28    big     A       NA
494  494                  12    494         12.28    big     D       NA
495  495                   5    495          5.28 medium     B       NA
496  496                   4    496          4.28  small     B       NA
497  497                  16    497         16.28  small     F       NA
498  498                  14    498         14.28 medium     E       NA
499  499                  13    499         13.28  small     E       NA
500  500                   1    500          1.28  small     A       NA
501  501                  18    501         18.28    big     F       NA
502  502                  15    502         15.28    big     E       NA
503  503                  17    503         17.28 medium     F       NA
504  504                   6    504          6.28    big     B       NA
505  505                  16    505         16.29  small     F       NA
506  506                  15    506         15.29    big     E       NA
507  507                  12    507         12.29    big     D       NA
508  508                   8    508          8.29 medium     C       NA
509  509                   7    509          7.29  small     C       NA
510  510                   5    510          5.29 medium     B       NA
511  511                  18    511         18.29    big     F       NA
512  512                   1    512          1.29  small     A       NA
513  513                   3    513          3.29    big     A       NA
514  514                  17    514         17.29 medium     F       NA
515  515                  11    515         11.29 medium     D       NA
516  516                   6    516          6.29    big     B       NA
517  517                   4    517          4.29  small     B       NA
518  518                   2    518          2.29 medium     A       NA
519  519                  10    519         10.29  small     D       NA
520  520                  13    520         13.29  small     E       NA
521  521                  14    521         14.29 medium     E       NA
522  522                   9    522          9.29    big     C       NA
523  523                  13    523         13.30  small     E       NA
524  524                   4    524          4.30  small     B       NA
525  525                   6    525          6.30    big     B       NA
526  526                   5    526          5.30 medium     B       NA
527  527                  18    527         18.30    big     F       NA
528  528                   2    528          2.30 medium     A       NA
529  529                   8    529          8.30 medium     C       NA
530  530                  17    530         17.30 medium     F       NA
531  531                  12    531         12.30    big     D       NA
532  532                   1    532          1.30  small     A       NA
533  533                   3    533          3.30    big     A       NA
534  534                  15    534         15.30    big     E       NA
535  535                  10    535         10.30  small     D       NA
536  536                  14    536         14.30 medium     E       NA
537  537                   9    537          9.30    big     C       NA
538  538                   7    538          7.30  small     C       NA
539  539                  16    539         16.30  small     F       NA
540  540                  11    540         11.30 medium     D       NA
    insert_time max_mem platform
1            NA      NA       NA
2            NA      NA       NA
3            NA      NA       NA
4            NA      NA       NA
5            NA      NA       NA
6            NA      NA       NA
7            NA      NA       NA
8            NA      NA       NA
9            NA      NA       NA
10           NA      NA       NA
11           NA      NA       NA
12           NA      NA       NA
13           NA      NA       NA
14           NA      NA       NA
15           NA      NA       NA
16           NA      NA       NA
17           NA      NA       NA
18           NA      NA       NA
19           NA      NA       NA
20           NA      NA       NA
21           NA      NA       NA
22           NA      NA       NA
23           NA      NA       NA
24           NA      NA       NA
25           NA      NA       NA
26           NA      NA       NA
27           NA      NA       NA
28           NA      NA       NA
29           NA      NA       NA
30           NA      NA       NA
31           NA      NA       NA
32           NA      NA       NA
33           NA      NA       NA
34           NA      NA       NA
35           NA      NA       NA
36           NA      NA       NA
37           NA      NA       NA
38           NA      NA       NA
39           NA      NA       NA
40           NA      NA       NA
41           NA      NA       NA
42           NA      NA       NA
43           NA      NA       NA
44           NA      NA       NA
45           NA      NA       NA
46           NA      NA       NA
47           NA      NA       NA
48           NA      NA       NA
49           NA      NA       NA
50           NA      NA       NA
51           NA      NA       NA
52           NA      NA       NA
53           NA      NA       NA
54           NA      NA       NA
55           NA      NA       NA
56           NA      NA       NA
57           NA      NA       NA
58           NA      NA       NA
59           NA      NA       NA
60           NA      NA       NA
61           NA      NA       NA
62           NA      NA       NA
63           NA      NA       NA
64           NA      NA       NA
65           NA      NA       NA
66           NA      NA       NA
67           NA      NA       NA
68           NA      NA       NA
69           NA      NA       NA
70           NA      NA       NA
71           NA      NA       NA
72           NA      NA       NA
73           NA      NA       NA
74           NA      NA       NA
75           NA      NA       NA
76           NA      NA       NA
77           NA      NA       NA
78           NA      NA       NA
79           NA      NA       NA
80           NA      NA       NA
81           NA      NA       NA
82           NA      NA       NA
83           NA      NA       NA
84           NA      NA       NA
85           NA      NA       NA
86           NA      NA       NA
87           NA      NA       NA
88           NA      NA       NA
89           NA      NA       NA
90           NA      NA       NA
91           NA      NA       NA
92           NA      NA       NA
93           NA      NA       NA
94           NA      NA       NA
95           NA      NA       NA
96           NA      NA       NA
97           NA      NA       NA
98           NA      NA       NA
99           NA      NA       NA
100          NA      NA       NA
101          NA      NA       NA
102          NA      NA       NA
103          NA      NA       NA
104          NA      NA       NA
105          NA      NA       NA
106          NA      NA       NA
107          NA      NA       NA
108          NA      NA       NA
109          NA      NA       NA
110          NA      NA       NA
111          NA      NA       NA
112          NA      NA       NA
113          NA      NA       NA
114          NA      NA       NA
115          NA      NA       NA
116          NA      NA       NA
117          NA      NA       NA
118          NA      NA       NA
119          NA      NA       NA
120          NA      NA       NA
121          NA      NA       NA
122          NA      NA       NA
123          NA      NA       NA
124          NA      NA       NA
125          NA      NA       NA
126          NA      NA       NA
127          NA      NA       NA
128          NA      NA       NA
129          NA      NA       NA
130          NA      NA       NA
131          NA      NA       NA
132          NA      NA       NA
133          NA      NA       NA
134          NA      NA       NA
135          NA      NA       NA
136          NA      NA       NA
137          NA      NA       NA
138          NA      NA       NA
139          NA      NA       NA
140          NA      NA       NA
141          NA      NA       NA
142          NA      NA       NA
143          NA      NA       NA
144          NA      NA       NA
145          NA      NA       NA
146          NA      NA       NA
147          NA      NA       NA
148          NA      NA       NA
149          NA      NA       NA
150          NA      NA       NA
151          NA      NA       NA
152          NA      NA       NA
153          NA      NA       NA
154          NA      NA       NA
155          NA      NA       NA
156          NA      NA       NA
157          NA      NA       NA
158          NA      NA       NA
159          NA      NA       NA
160          NA      NA       NA
161          NA      NA       NA
162          NA      NA       NA
163          NA      NA       NA
164          NA      NA       NA
165          NA      NA       NA
166          NA      NA       NA
167          NA      NA       NA
168          NA      NA       NA
169          NA      NA       NA
170          NA      NA       NA
171          NA      NA       NA
172          NA      NA       NA
173          NA      NA       NA
174          NA      NA       NA
175          NA      NA       NA
176          NA      NA       NA
177          NA      NA       NA
178          NA      NA       NA
179          NA      NA       NA
180          NA      NA       NA
181          NA      NA       NA
182          NA      NA       NA
183          NA      NA       NA
184          NA      NA       NA
185          NA      NA       NA
186          NA      NA       NA
187          NA      NA       NA
188          NA      NA       NA
189          NA      NA       NA
190          NA      NA       NA
191          NA      NA       NA
192          NA      NA       NA
193          NA      NA       NA
194          NA      NA       NA
195          NA      NA       NA
196          NA      NA       NA
197          NA      NA       NA
198          NA      NA       NA
199          NA      NA       NA
200          NA      NA       NA
201          NA      NA       NA
202          NA      NA       NA
203          NA      NA       NA
204          NA      NA       NA
205          NA      NA       NA
206          NA      NA       NA
207          NA      NA       NA
208          NA      NA       NA
209          NA      NA       NA
210          NA      NA       NA
211          NA      NA       NA
212          NA      NA       NA
213          NA      NA       NA
214          NA      NA       NA
215          NA      NA       NA
216          NA      NA       NA
217          NA      NA       NA
218          NA      NA       NA
219          NA      NA       NA
220          NA      NA       NA
221          NA      NA       NA
222          NA      NA       NA
223          NA      NA       NA
224          NA      NA       NA
225          NA      NA       NA
226          NA      NA       NA
227          NA      NA       NA
228          NA      NA       NA
229          NA      NA       NA
230          NA      NA       NA
231          NA      NA       NA
232          NA      NA       NA
233          NA      NA       NA
234          NA      NA       NA
235          NA      NA       NA
236          NA      NA       NA
237          NA      NA       NA
238          NA      NA       NA
239          NA      NA       NA
240          NA      NA       NA
241          NA      NA       NA
242          NA      NA       NA
243          NA      NA       NA
244          NA      NA       NA
245          NA      NA       NA
246          NA      NA       NA
247          NA      NA       NA
248          NA      NA       NA
249          NA      NA       NA
250          NA      NA       NA
251          NA      NA       NA
252          NA      NA       NA
253          NA      NA       NA
254          NA      NA       NA
255          NA      NA       NA
256          NA      NA       NA
257          NA      NA       NA
258          NA      NA       NA
259          NA      NA       NA
260          NA      NA       NA
261          NA      NA       NA
262          NA      NA       NA
263          NA      NA       NA
264          NA      NA       NA
265          NA      NA       NA
266          NA      NA       NA
267          NA      NA       NA
268          NA      NA       NA
269          NA      NA       NA
270          NA      NA       NA
271          NA      NA       NA
272          NA      NA       NA
273          NA      NA       NA
274          NA      NA       NA
275          NA      NA       NA
276          NA      NA       NA
277          NA      NA       NA
278          NA      NA       NA
279          NA      NA       NA
280          NA      NA       NA
281          NA      NA       NA
282          NA      NA       NA
283          NA      NA       NA
284          NA      NA       NA
285          NA      NA       NA
286          NA      NA       NA
287          NA      NA       NA
288          NA      NA       NA
289          NA      NA       NA
290          NA      NA       NA
291          NA      NA       NA
292          NA      NA       NA
293          NA      NA       NA
294          NA      NA       NA
295          NA      NA       NA
296          NA      NA       NA
297          NA      NA       NA
298          NA      NA       NA
299          NA      NA       NA
300          NA      NA       NA
301          NA      NA       NA
302          NA      NA       NA
303          NA      NA       NA
304          NA      NA       NA
305          NA      NA       NA
306          NA      NA       NA
307          NA      NA       NA
308          NA      NA       NA
309          NA      NA       NA
310          NA      NA       NA
311          NA      NA       NA
312          NA      NA       NA
313          NA      NA       NA
314          NA      NA       NA
315          NA      NA       NA
316          NA      NA       NA
317          NA      NA       NA
318          NA      NA       NA
319          NA      NA       NA
320          NA      NA       NA
321          NA      NA       NA
322          NA      NA       NA
323          NA      NA       NA
324          NA      NA       NA
325          NA      NA       NA
326          NA      NA       NA
327          NA      NA       NA
328          NA      NA       NA
329          NA      NA       NA
330          NA      NA       NA
331          NA      NA       NA
332          NA      NA       NA
333          NA      NA       NA
334          NA      NA       NA
335          NA      NA       NA
336          NA      NA       NA
337          NA      NA       NA
338          NA      NA       NA
339          NA      NA       NA
340          NA      NA       NA
341          NA      NA       NA
342          NA      NA       NA
343          NA      NA       NA
344          NA      NA       NA
345          NA      NA       NA
346          NA      NA       NA
347          NA      NA       NA
348          NA      NA       NA
349          NA      NA       NA
350          NA      NA       NA
351          NA      NA       NA
352          NA      NA       NA
353          NA      NA       NA
354          NA      NA       NA
355          NA      NA       NA
356          NA      NA       NA
357          NA      NA       NA
358          NA      NA       NA
359          NA      NA       NA
360          NA      NA       NA
361          NA      NA       NA
362          NA      NA       NA
363          NA      NA       NA
364          NA      NA       NA
365          NA      NA       NA
366          NA      NA       NA
367          NA      NA       NA
368          NA      NA       NA
369          NA      NA       NA
370          NA      NA       NA
371          NA      NA       NA
372          NA      NA       NA
373          NA      NA       NA
374          NA      NA       NA
375          NA      NA       NA
376          NA      NA       NA
377          NA      NA       NA
378          NA      NA       NA
379          NA      NA       NA
380          NA      NA       NA
381          NA      NA       NA
382          NA      NA       NA
383          NA      NA       NA
384          NA      NA       NA
385          NA      NA       NA
386          NA      NA       NA
387          NA      NA       NA
388          NA      NA       NA
389          NA      NA       NA
390          NA      NA       NA
391          NA      NA       NA
392          NA      NA       NA
393          NA      NA       NA
394          NA      NA       NA
395          NA      NA       NA
396          NA      NA       NA
397          NA      NA       NA
398          NA      NA       NA
399          NA      NA       NA
400          NA      NA       NA
401          NA      NA       NA
402          NA      NA       NA
403          NA      NA       NA
404          NA      NA       NA
405          NA      NA       NA
406          NA      NA       NA
407          NA      NA       NA
408          NA      NA       NA
409          NA      NA       NA
410          NA      NA       NA
411          NA      NA       NA
412          NA      NA       NA
413          NA      NA       NA
414          NA      NA       NA
415          NA      NA       NA
416          NA      NA       NA
417          NA      NA       NA
418          NA      NA       NA
419          NA      NA       NA
420          NA      NA       NA
421          NA      NA       NA
422          NA      NA       NA
423          NA      NA       NA
424          NA      NA       NA
425          NA      NA       NA
426          NA      NA       NA
427          NA      NA       NA
428          NA      NA       NA
429          NA      NA       NA
430          NA      NA       NA
431          NA      NA       NA
432          NA      NA       NA
433          NA      NA       NA
434          NA      NA       NA
435          NA      NA       NA
436          NA      NA       NA
437          NA      NA       NA
438          NA      NA       NA
439          NA      NA       NA
440          NA      NA       NA
441          NA      NA       NA
442          NA      NA       NA
443          NA      NA       NA
444          NA      NA       NA
445          NA      NA       NA
446          NA      NA       NA
447          NA      NA       NA
448          NA      NA       NA
449          NA      NA       NA
450          NA      NA       NA
451          NA      NA       NA
452          NA      NA       NA
453          NA      NA       NA
454          NA      NA       NA
455          NA      NA       NA
456          NA      NA       NA
457          NA      NA       NA
458          NA      NA       NA
459          NA      NA       NA
460          NA      NA       NA
461          NA      NA       NA
462          NA      NA       NA
463          NA      NA       NA
464          NA      NA       NA
465          NA      NA       NA
466          NA      NA       NA
467          NA      NA       NA
468          NA      NA       NA
469          NA      NA       NA
470          NA      NA       NA
471          NA      NA       NA
472          NA      NA       NA
473          NA      NA       NA
474          NA      NA       NA
475          NA      NA       NA
476          NA      NA       NA
477          NA      NA       NA
478          NA      NA       NA
479          NA      NA       NA
480          NA      NA       NA
481          NA      NA       NA
482          NA      NA       NA
483          NA      NA       NA
484          NA      NA       NA
485          NA      NA       NA
486          NA      NA       NA
487          NA      NA       NA
488          NA      NA       NA
489          NA      NA       NA
490          NA      NA       NA
491          NA      NA       NA
492          NA      NA       NA
493          NA      NA       NA
494          NA      NA       NA
495          NA      NA       NA
496          NA      NA       NA
497          NA      NA       NA
498          NA      NA       NA
499          NA      NA       NA
500          NA      NA       NA
501          NA      NA       NA
502          NA      NA       NA
503          NA      NA       NA
504          NA      NA       NA
505          NA      NA       NA
506          NA      NA       NA
507          NA      NA       NA
508          NA      NA       NA
509          NA      NA       NA
510          NA      NA       NA
511          NA      NA       NA
512          NA      NA       NA
513          NA      NA       NA
514          NA      NA       NA
515          NA      NA       NA
516          NA      NA       NA
517          NA      NA       NA
518          NA      NA       NA
519          NA      NA       NA
520          NA      NA       NA
521          NA      NA       NA
522          NA      NA       NA
523          NA      NA       NA
524          NA      NA       NA
525          NA      NA       NA
526          NA      NA       NA
527          NA      NA       NA
528          NA      NA       NA
529          NA      NA       NA
530          NA      NA       NA
531          NA      NA       NA
532          NA      NA       NA
533          NA      NA       NA
534          NA      NA       NA
535          NA      NA       NA
536          NA      NA       NA
537          NA      NA       NA
538          NA      NA       NA
539          NA      NA       NA
540          NA      NA       NA

2016-03-29 Tracing batch executions along time

This follows the same design as in:

The difference is that ayira should be instrumented to measure, at the end of every batch execution, the following information:

  • batch number (starting at 1)
  • start time (in microseconds)
  • end time (in microseconds)
  • size in bytes of the batch
  • size in number of operations
  • Platform

Ideally, each line holds all this information. Note that there might be many of these lines since it gets registered at every batch execution, specially when size is 128 operations and file is big. So, I strongly recommend you buffer this information in a vector and dump only at the end. The name of the file should be something unique, combining information above, such as input_batch_platform.csv, where input is big/medium/small, batch is A, B, C and so on. You could use all columns values as in the design described in 2016-03-29 DoE for Batch Size Investigation except for those that you measure (simulation and insertion times, memory utilization).

2016-03-31 Adding the Batch Size manipulation

Implementation

  • New argument to “mysql” plugin: -batch <max_size>
    • size is in number of operations, not bytes.
  • We have 4 different types of batches, but it is all counted as it was one.

Batch information data structure

  • It is a Map<Long, Long[]>, where the key is the time when the batch was executed.
  • There is no option of finding out the batch size in bytes.
  • Key: start time. Value: end time, duration, size in operations
    • size in operations is always the same, but the last one, that will be what is left.

2016-04-01 Meeting with Tais (Scripts)

Batch size is measured in two ways:

  • number of operations
  • in bytes (doubt below)

How to measure the batch size in bytes?

  • Count the number of bytes in prepared statements
  • Count the number of bytes when customizing statements
  • Sum everything to reach the number of bytes for each batch

Problem measuring memory utilization in Java. Let’s do like this:

  • Before each batch execution, we measure the amount of memory in bytes like this:
    (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory())
        
  • The result is compared against a global maxMemory variable. It the results is greated, we update maxMemory, otherwise we do nothing.
  • Doing this we will obtain the maximum memory utilization of the program, which we can use to run comparisons according to the batch size. We are not sure yet about the resolution of such functions, so let’s use the data with care.

Measuring time in nanoseconds for everything

System.nanoTime();

Execution time (OKAY)

  • Should not consider time spent for DB connection

Print batch size (the argument) to the output

  • When no batch is provided, you print the initilization’s MAXINT

What is missing for launching the experiments?

  • Three input files
  • Bash script to run the experiment
    • INPUT: CSV design
    • OUTPUT: CSV design with measurements
#!/bin/bash

# CPP_Comparison.sh

# For each line
  # get the name of the program (column 5)
  # get the input size (column 6)
  # Launch the experiment, get execution time
  # print the line, with the time.

function usage()
{
  echo "./Cpp_Comparison.sh <design> <platform>";
}


DESIGN=$1
if [ -z "$DESIGN" ]; then
    usage;
    exit;
fi
PLATFORM=$2
if [ -z "$PLATFORM" ]; then
    usage;
    exit
fi

while read -r line; do
    PROGRAM=`echo $line | cut -d, -f5 | sed -e "s/\"//g"`
    INPUTSIZE=`echo $line | cut -d, -f6 | sed -e "s/\"//g"`
    if [[ $PROGRAM = "version" ]]; then
       echo $line
       continue
    fi
    if [[ $INPUTSIZE = "small" ]]; then
      INPUT="trace-128K.paje"
    elif [[ $INPUTSIZE = "medium" ]]; then
      INPUT="trace-128M.paje"
    else 
      INPUT="trace-1G.paje"
    fi
    if [[ $PROGRAM = "aiyra" ]]; then
      # execute aiyra 
      EXEC="$(java br.ufrgs.inf.tlbellini.PajeGrammar -f $INPUT | cut -d: -f2)"
    elif [[ $PROGRAM = "pj" ]]; then
	# execute pj_dump -q -t
	EXEC="$(pj_dump $INPUT --quiet -t)"
    elif [[ $PROGRAM = "pjflex" ]]; then
	# execute pj_dump -q -t -f
      EXEC="$(pj_dump $INPUT --quiet -t -f)"
    fi
    echo $line | cut -d, -f1-6 | tr '\n' ','
    echo "$EXEC,$PLATFORM"
done < "$DESIGN"
#!/bin/bash

# CPP_BatchSize_Investigation.sh

# For each line
  # get the name of the program (column 5)
  # get the input size (column 6)
  # Launch the experiment, get execution time
  # print the line, with the time.

function usage()
{
  echo "./Cpp_Batch_Size_Investigation.sh <design> <platform>";
}


DESIGN=$1
if [ -z "$DESIGN" ]; then
    usage;
    exit;
fi
PLATFORM=$2
if [ -z "$PLATFORM" ]; then
    usage;
    exit
fi

while read -r line; do
    INPUTSIZE=`echo $line | cut -d, -f5 | sed -e "s/\"//g"`
    BATCHSIZE=`echo $line | cut -d, -f6 | sed -e "s/\"//g"`
    EXEC=" "
    if [[ $INPUTSIZE = "input" ]]; then
       echo $line
       continue
    fi
    if [[ $INPUTSIZE = "small" ]]; then
      INPUT="trace-128K.paje"
      MAXBATCH=2727
    elif [[ $INPUTSIZE = "medium" ]]; then
      INPUT="trace-128M.paje"
      MAXBATCH=2526628
    elif [[ $INPUTSIZE = "big" ]]; then 
      INPUT="trace-1G.paje"
      MAXBATCH=19570722
    fi
    if [[ $BATCHSIZE = "A" ]]; then 
      BATCH=$MAXBATCH
    elif [[ $BATCHSIZE = "B" ]]; then
      BATCH=$((MAXBATCH/2))
    elif [[ $BATCHSIZE = "C" ]]; then
      BATCH=$((MAXBATCH/4))
    elif [[ $BATCHSIZE = "D" ]]; then
      BATCH=$((MAXBATCH/8))
    elif [[ $BATCHSIZE = "E" ]]; then
      BATCH=$((MAXBATCH/16))
    elif [[ $BATCHSIZE = "F" ]]; then
      BATCH=$((MAXBATCH/32))
    fi
    EXEC="$(java br.ufrgs.inf.tlbellini.PajeGrammar -f $INPUT -p mysql -batch $BATCH -test)"
    TIME=`echo $EXEC | cut -d" " -f18`
    TIME_INSERTION=`echo $EXEC | cut -d" " -f8`
    MAX_MEM=`echo $EXEC | cut -d" " -f10`
    if [[ $MAX_MEM = "exceeded" ]]; then
      EXEC=NA
      TIME=NA 
      TIME_INSERTION=NA
      MAX_MEM=NA
    fi
    echo $line | cut -d, -f1-6 | tr '\n' ','
    echo "$TIME,$TIME_INSERTION,$MAX_MEM,$PLATFORM"
done < "$DESIGN"
./Cpp_Batch_Size_Investigation.sh <design> <platform>

Until Tuesday, you sent me both scripts. Next Friday, we analyze some data.

2016-04-01 Generating traces with different sizes

For input, the levels are small (128K), medium (131072K, or 128M), and big (1048576K or 1G). So, to generate these files, you can use an example tool called tracegen that I just developed with poti. To quickly introduce yourself, do:

Clone and Compile

git clone https://github.com/schnorr/poti.git
mkdir -p poti/build; cd poti/build
cmake ..; make VERBOSE=yes

Get to know tracegen

./poti/build/examples/tracegen --help

As you can see you just have to provide an obligatory filename and an optional target size. If size is not provided, it will generate the minimal trace file for this example (see tracegen source code [1] for details).

[1]: https://github.com/schnorr/poti/blob/master/examples/tracegen.c

Generating the trace files

So, to get you trace files, just run the following:

./poti/build/examples/tracegen --filename trace-128K.paje --size `echo 128*1024 | bc -l`
./poti/build/examples/tracegen --filename trace-128M.paje --size `echo 128*1024*1024 | bc -l`
./poti/build/examples/tracegen --filename trace-1G.paje --size `echo 1024*1024*1024 | bc -l`

Great. Let’s it. I’m gzipping them to send to you.

gzip trace-128K.paje trace-128M.paje trace-1G.paje; ls -lh *.paje.gz
-rw-r--r-- 1 schnorr schnorr  40K Apr  1 16:51 trace-128K.paje.gz
-rw-r--r-- 1 schnorr schnorr  36M Apr  1 16:51 trace-128M.paje.gz
-rw-r--r-- 1 schnorr schnorr 277M Apr  1 16:52 trace-1G.paje.gz

They are all available here:

You can gunzip them.

2016-04-02 - Batch Size in bytes

The size of the string is being considered.

2016-04-02 21:00 - Status

What is missing for launching the experiments?

  • Three input files OK
  • Bash script to run the experiment
    • INPUT: CSV design
    • OUTPUT: CSV design with measurements

2016-04-04 - Bash Scripts

Cpp_Comparison:

  • need -t parameter
  • only linux

Batch_Size_Investigation:

-Mac: max number for “big” exceeds gc overhead limit

2016-04-05 - Mac pj_dump

  • CMake Error: The following variables are used in this project, but they are set to NOTFOUND.

Please set them or make sure they are set and tested correctly in the CMake files:

FL_LIBRARY (ADVANCED)

linked by target “paje_library” in directory

This happens beacause the ccmake . is not finding the Flex file libfl.a

If you installed Flex using Brew: brew list flex find the path to the “lib” folder: THIS will be the path to the missing file. Go to ccmake . find FL_LIBRARY put the path , adding libfl.a at the end

Compilation errors: brew install argp-standalone libargp.a is not found brew list argp-standalone find path to libargp.a make VERBOSE=yes execute the full path and instead of -largp put what you found in brew list argp-standalone

2016-04-05 Batch size for each input trace

for i in *.paje; do echo -n "$i "; cat $i | grep ^13 | wc -l; done
trace-128K.paje 2727
trace-128M.paje 2526628
trace-1G.paje 19570722

2016-04-06 Quick analysis of data results/guarani/1_tentativa.csv

library(dplyr);
df <- read.csv ("results/guarani/1_tentativa.csv");
k <- df %>%
     select(version, input, time, platform) %>%
     group_by(version, input, platform) %>%
     summarize(N = n(),
               time_avg = mean(time),
               time_se = 3*sd(time)/sqrt(N))
              
k;
Source: local data frame [9 x 6]
Groups: version, input

  version  input platform  N     time_avg      time_se
1   aiyra    big  guarani 30           NA           NA
2   aiyra medium  guarani 30           NA           NA
3   aiyra  small  guarani 30           NA           NA
4      pj    big  guarani 30 248.34171373 1.1231438892
5      pj medium  guarani 30  31.99264657 0.1493861801
6      pj  small  guarani 30   0.03451830 0.0002804379
7  pjflex    big  guarani 30 266.65148143 1.6170200167
8  pjflex medium  guarani 30  34.08444947 0.1362978432
9  pjflex  small  guarani 30   0.03641293 0.0003676874

2016-04-07 First attempts mac

Batch

  1. reached full memory before completed.
  2. Added -test option that erases the data from DB
  3. initial memory available: 55 GB

2016-04-10 First atempts windows

Batch

  1. initial memory available 398GB

2016-04-12 Meeting

Batch size

  • results/batch_size/bateria_01_batch_luiza.csv

Cpp Comparison

  • results/cpp_comparison/bateria_01_orion1.csv

Let’s analyze the second one.

Remember that aiyra’s time is in microseconds.

df <- read.csv("results/cpp_comparison/bateria_01_orion1.csv");
df[df$version == "aiyra",]$time = df[df$version == "aiyra",]$time/1000000;
head(df);
  name run.no.in.std.order run.no run.no.std.rp version  input       time
1    1                   4      1           4.1   aiyra medium  13.199061
2    2                   9      2           9.1  pjflex    big 280.152443
3    3                   2      3           2.1      pj  small   0.049384
4    4                   5      4           5.1      pj medium  33.616784
5    5                   7      5           7.1   aiyra    big 111.191072
6    6                   6      6           6.1  pjflex medium  35.570650
  platform
1   orion1
2   orion1
3   orion1
4   orion1
5   orion1
6   orion1

Great, we load them into the df data frame.

library(dplyr);
k <- df %>%
     select(version, input, time, platform) %>%
     group_by(version, input, platform) %>%
     summarize(N = n(),
               time_avg = mean(time),
               time_se = 3*sd(time)/sqrt(N)) %>%
     as.data.frame();
k
  version  input platform  N     time_avg     time_se
1   aiyra    big   orion1 30 111.78373253 1.012192396
2   aiyra medium   orion1 30  13.08560363 0.123976860
3   aiyra  small   orion1 30   0.22622977 0.009292423
4      pj    big   orion1 30 259.72360747 1.115892686
5      pj medium   orion1 30  33.47163073 0.125393689
6      pj  small   orion1 30   0.04607153 0.002206699
7  pjflex    big   orion1 30 279.74462510 0.924957991
8  pjflex medium   orion1 30  36.08434247 0.158896225
9  pjflex  small   orion1 30   0.04819160 0.002599552

Let’s plot this with a simple plot.

Learn ggplot2:

How to install gpplot2:

install.packages("ggplot2");

Let’s finally plot.

(Literate Programming)

library(ggplot2);
ggplot(k, aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + ylim(0, NA);

img/first_plot_1.png

Strange, CPP is much slower for big and medium. A possible explanation is that aiyra keeps a minimum memory footprint, while the cpp version buffers everything (but do not dump at the end). So, execution times of pj and pjflex considers all the burden of allocating memory.

Let’s plot again using bars.

library(ggplot2);
ggplot(k, aes(x=version, fill=time_avg)) +
  geom_bar(position="dodge") +
#  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + ylim(0, NA);

img/first_plot_2.png

Didn’t work, to be improved.

Let’s look to the other batch size data.

But that has some strange errors, see:

cat results/batch_size/bateria_01_batch_luiza.csv | grep overhead
"6","6",6,"6.1","big","B",,overhead,exceeded,luiza
"23","6",23,"6.2","big","B",,overhead,exceeded,luiza
"40","6",40,"6.3","big","B",,overhead,exceeded,luiza
"67","6",67,"6.4","big","B",,overhead,exceeded,luiza
"73","6",73,"6.5","big","B",,overhead,exceeded,luiza
"105","6",105,"6.6","big","B",,overhead,exceeded,luiza
"109","6",109,"6.7","big","B",,overhead,exceeded,luiza
"144","6",144,"6.8","big","B",,overhead,exceeded,luiza
"159","6",159,"6.9","big","B",,overhead,exceeded,luiza
"169","6",169,"6.10","big","B",,overhead,exceeded,luiza
"190","6",190,"6.11","big","B",,overhead,exceeded,luiza
"210","6",210,"6.12","big","B",,overhead,exceeded,luiza
"226","6",226,"6.13","big","B",,overhead,exceeded,luiza
"251","6",251,"6.14","big","B",,overhead,exceeded,luiza
"254","6",254,"6.15","big","B",,overhead,exceeded,luiza
"276","6",276,"6.16","big","B",,overhead,exceeded,luiza
"297","6",297,"6.17","big","B",,overhead,exceeded,luiza
"319","6",319,"6.18","big","B",,overhead,exceeded,luiza
"338","6",338,"6.19","big","B",,overhead,exceeded,luiza
"349","6",349,"6.20","big","B",,overhead,exceeded,luiza
"377","6",377,"6.21","big","B",,overhead,exceeded,luiza
"391","6",391,"6.22","big","B",,overhead,exceeded,luiza
"405","6",405,"6.23","big","B",,overhead,exceeded,luiza
"422","6",422,"6.24","big","B",,overhead,exceeded,luiza
"445","6",445,"6.25","big","B",,overhead,exceeded,luiza
"453","6",453,"6.26","big","B",,overhead,exceeded,luiza
"470","6",470,"6.27","big","B",,overhead,exceeded,luiza
"504","6",504,"6.28","big","B",,overhead,exceeded,luiza
"516","6",516,"6.29","big","B",,overhead,exceeded,luiza
"525","6",525,"6.30","big","B",,overhead,exceeded,luiza

All this should be ignored, so, let’s remove with sed.

sed -i "/overhead/d" results/batch_size/bateria_01_batch_luiza.csv

I’ll commit this by hand. Tais will check what’s the problems with these executions (see above the parameters), and try to improve the script to avoid these kind of problems to be dumped to the resulting CSV. It is okay to not work, but NA should be there instead.

Read the data.

dfbs <- read.csv("results/batch_size/bateria_01_batch_luiza.csv");
dfbs$sim_time = dfbs$sim_time / 1000000;
dfbs$insert_time = dfbs$insert_time / 1000000;
head(dfbs);
  name run.no.in.std.order run.no run.no.std.rp  input batch   sim_time
1    1                   8      1           8.1 medium     C  91.388223
2    2                  18      2          18.1    big     F 729.791324
3    3                   5      3           5.1 medium     B  96.152690
4    4                  12      4          12.1    big     D 759.280008
5    5                  16      5          16.1  small     F   0.783357
6    7                  15      7          15.1    big     E 732.142820
  insert_time    max_mem platform
1   56.979417  277594552    luiza
2  435.170061  269932680    luiza
3   57.000087  554541128    luiza
4  434.433669 1070246512    luiza
5    0.157434    3218096    luiza
6  431.922201  538188168    luiza

Let’s get better columns names:

dfbsc <- dfbs;
names(dfbsc)[7] = "ExecutionTime";
names(dfbsc)[8] = "InsertTime";
dfbsc$SimulationTime = dfbsc$ExecutionTime - dfbsc$InsertTime;
head(dfbsc)
  name run.no.in.std.order run.no run.no.std.rp  input batch ExecutionTime
1    1                   8      1           8.1 medium     C     91.388223
2    2                  18      2          18.1    big     F    729.791324
3    3                   5      3           5.1 medium     B     96.152690
4    4                  12      4          12.1    big     D    759.280008
5    5                  16      5          16.1  small     F      0.783357
6    7                  15      7          15.1    big     E    732.142820
  InsertTime    max_mem platform SimulationTime
1  56.979417  277594552    luiza      34.408806
2 435.170061  269932680    luiza     294.621263
3  57.000087  554541128    luiza      39.152603
4 434.433669 1070246512    luiza     324.846339
5   0.157434    3218096    luiza       0.625923
6 431.922201  538188168    luiza     300.220619

What we want to do now:

  • When changing the batch size for each input, we want to see the (E/S/I) times and memory
library(dplyr);
g <- dfbsc %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),

               et_av = mean(ExecutionTime),
               et_se = 3*sd(ExecutionTime)/sqrt(N),

               it_av = mean(InsertTime),
               it_se = 3*sd(InsertTime)/sqrt(N),

               st_av = mean(SimulationTime),
               st_se = 3*sd(SimulationTime)/sqrt(N),

               me_av = mean(max_mem),
               me_se = 3*sd(max_mem)/sqrt(N)) %>%
     as.data.frame();
g
    input batch platform  N       et_av       et_se       it_av       it_se
1     big     A    luiza 30          NA          NA          NA          NA
2     big     C    luiza 30 795.2285569 2.218592773 437.5494863 1.267252406
3     big     D    luiza 30 757.8684267 2.385838687 437.5375775 1.478032249
4     big     E    luiza 30 740.0890006 2.381837838 436.4725984 1.834735939
5     big     F    luiza 30 732.7663411 2.660387652 435.8199257 1.784366366
6  medium     A    luiza 30  96.8813858 0.385917646  56.5639982 0.269410061
7  medium     B    luiza 30  96.0545934 0.280779158  56.6305572 0.222547405
8  medium     C    luiza 30  91.1218662 0.399244420  56.4963666 0.255846178
9  medium     D    luiza 30  90.4261607 0.315775543  56.3642670 0.242080362
10 medium     E    luiza 30  88.9492936 0.342191492  56.4912891 0.229589689
11 medium     F    luiza 30  85.4662720 0.402239479  56.7788707 0.288559939
12  small     A    luiza 30   0.4561541 0.005422778   0.1267087 0.005748402
13  small     B    luiza 30   0.4557321 0.003834888   0.1064588 0.004098957
14  small     C    luiza 30   0.4735896 0.002656302   0.1086365 0.002453057
15  small     D    luiza 30   0.5168988 0.003670323   0.1154382 0.002838226
16  small     E    luiza 30   0.6006172 0.003741141   0.1249221 0.002336092
17  small     F    luiza 30   0.7626586 0.004948073   0.1527009 0.002402932
         st_av       st_se      me_av     me_se
1           NA          NA         NA        NA
2  357.6790706 1.542601163 2140734030 706.46323
3  320.3308492 1.524688718 1070244381 494.68928
4  303.6164022 1.058449357  538188762 452.58933
5  296.9464154 1.317535584  269933135 721.92381
6   40.3173876 0.192735894 1086477209 748.30554
7   39.4240362 0.151464040  554542096 655.43978
8   34.6254996 0.197764297  277595270 711.53454
9   34.0618938 0.135503437  139863937 776.69469
10  32.4580045 0.166169367   71097261 650.56412
11  28.6874013 0.153427395   37724969 528.99102
12   0.3294454 0.002127062    4363366  47.31092
13   0.3492733 0.002585717    3782599  47.65022
14   0.3649531 0.003181645    3487619  48.21309
15   0.4014607 0.003002626    3326572  47.84435
16   0.4756951 0.003460140    3255328  47.97908
17   0.6099577 0.004742031    3218205  49.33970

We are surprised by these results, again:

  • insertion time is stable no matter the batch size
  • simulation time reduces when batch size is smaller
    • Overhead of the garbage collector
    • Everything is being in kept in memory

Discussion about interfereces in the experimental session.

Let’s melt data.

library(reshape);
h <- melt(g, vars=c("input", "batch", "platform"));
h
Using input, batch, platform as id variables
     input batch platform variable        value
1      big     A    luiza        N 3.000000e+01
2      big     C    luiza        N 3.000000e+01
3      big     D    luiza        N 3.000000e+01
4      big     E    luiza        N 3.000000e+01
5      big     F    luiza        N 3.000000e+01
6   medium     A    luiza        N 3.000000e+01
7   medium     B    luiza        N 3.000000e+01
8   medium     C    luiza        N 3.000000e+01
9   medium     D    luiza        N 3.000000e+01
10  medium     E    luiza        N 3.000000e+01
11  medium     F    luiza        N 3.000000e+01
12   small     A    luiza        N 3.000000e+01
13   small     B    luiza        N 3.000000e+01
14   small     C    luiza        N 3.000000e+01
15   small     D    luiza        N 3.000000e+01
16   small     E    luiza        N 3.000000e+01
17   small     F    luiza        N 3.000000e+01
18     big     A    luiza    et_av           NA
19     big     C    luiza    et_av 7.952286e+02
20     big     D    luiza    et_av 7.578684e+02
21     big     E    luiza    et_av 7.400890e+02
22     big     F    luiza    et_av 7.327663e+02
23  medium     A    luiza    et_av 9.688139e+01
24  medium     B    luiza    et_av 9.605459e+01
25  medium     C    luiza    et_av 9.112187e+01
26  medium     D    luiza    et_av 9.042616e+01
27  medium     E    luiza    et_av 8.894929e+01
28  medium     F    luiza    et_av 8.546627e+01
29   small     A    luiza    et_av 4.561541e-01
30   small     B    luiza    et_av 4.557321e-01
31   small     C    luiza    et_av 4.735896e-01
32   small     D    luiza    et_av 5.168988e-01
33   small     E    luiza    et_av 6.006172e-01
34   small     F    luiza    et_av 7.626586e-01
35     big     A    luiza    et_se           NA
36     big     C    luiza    et_se 2.218593e+00
37     big     D    luiza    et_se 2.385839e+00
38     big     E    luiza    et_se 2.381838e+00
39     big     F    luiza    et_se 2.660388e+00
40  medium     A    luiza    et_se 3.859176e-01
41  medium     B    luiza    et_se 2.807792e-01
42  medium     C    luiza    et_se 3.992444e-01
43  medium     D    luiza    et_se 3.157755e-01
44  medium     E    luiza    et_se 3.421915e-01
45  medium     F    luiza    et_se 4.022395e-01
46   small     A    luiza    et_se 5.422778e-03
47   small     B    luiza    et_se 3.834888e-03
48   small     C    luiza    et_se 2.656302e-03
49   small     D    luiza    et_se 3.670323e-03
50   small     E    luiza    et_se 3.741141e-03
51   small     F    luiza    et_se 4.948073e-03
52     big     A    luiza    it_av           NA
53     big     C    luiza    it_av 4.375495e+02
54     big     D    luiza    it_av 4.375376e+02
55     big     E    luiza    it_av 4.364726e+02
56     big     F    luiza    it_av 4.358199e+02
57  medium     A    luiza    it_av 5.656400e+01
58  medium     B    luiza    it_av 5.663056e+01
59  medium     C    luiza    it_av 5.649637e+01
60  medium     D    luiza    it_av 5.636427e+01
61  medium     E    luiza    it_av 5.649129e+01
62  medium     F    luiza    it_av 5.677887e+01
63   small     A    luiza    it_av 1.267087e-01
64   small     B    luiza    it_av 1.064588e-01
65   small     C    luiza    it_av 1.086365e-01
66   small     D    luiza    it_av 1.154382e-01
67   small     E    luiza    it_av 1.249221e-01
68   small     F    luiza    it_av 1.527009e-01
69     big     A    luiza    it_se           NA
70     big     C    luiza    it_se 1.267252e+00
71     big     D    luiza    it_se 1.478032e+00
72     big     E    luiza    it_se 1.834736e+00
73     big     F    luiza    it_se 1.784366e+00
74  medium     A    luiza    it_se 2.694101e-01
75  medium     B    luiza    it_se 2.225474e-01
76  medium     C    luiza    it_se 2.558462e-01
77  medium     D    luiza    it_se 2.420804e-01
78  medium     E    luiza    it_se 2.295897e-01
79  medium     F    luiza    it_se 2.885599e-01
80   small     A    luiza    it_se 5.748402e-03
81   small     B    luiza    it_se 4.098957e-03
82   small     C    luiza    it_se 2.453057e-03
83   small     D    luiza    it_se 2.838226e-03
84   small     E    luiza    it_se 2.336092e-03
85   small     F    luiza    it_se 2.402932e-03
86     big     A    luiza    st_av           NA
87     big     C    luiza    st_av 3.576791e+02
88     big     D    luiza    st_av 3.203308e+02
89     big     E    luiza    st_av 3.036164e+02
90     big     F    luiza    st_av 2.969464e+02
91  medium     A    luiza    st_av 4.031739e+01
92  medium     B    luiza    st_av 3.942404e+01
93  medium     C    luiza    st_av 3.462550e+01
94  medium     D    luiza    st_av 3.406189e+01
95  medium     E    luiza    st_av 3.245800e+01
96  medium     F    luiza    st_av 2.868740e+01
97   small     A    luiza    st_av 3.294454e-01
98   small     B    luiza    st_av 3.492733e-01
99   small     C    luiza    st_av 3.649531e-01
100  small     D    luiza    st_av 4.014607e-01
101  small     E    luiza    st_av 4.756951e-01
102  small     F    luiza    st_av 6.099577e-01
103    big     A    luiza    st_se           NA
104    big     C    luiza    st_se 1.542601e+00
105    big     D    luiza    st_se 1.524689e+00
106    big     E    luiza    st_se 1.058449e+00
107    big     F    luiza    st_se 1.317536e+00
108 medium     A    luiza    st_se 1.927359e-01
109 medium     B    luiza    st_se 1.514640e-01
110 medium     C    luiza    st_se 1.977643e-01
111 medium     D    luiza    st_se 1.355034e-01
112 medium     E    luiza    st_se 1.661694e-01
113 medium     F    luiza    st_se 1.534274e-01
114  small     A    luiza    st_se 2.127062e-03
115  small     B    luiza    st_se 2.585717e-03
116  small     C    luiza    st_se 3.181645e-03
117  small     D    luiza    st_se 3.002626e-03
118  small     E    luiza    st_se 3.460140e-03
119  small     F    luiza    st_se 4.742031e-03
120    big     A    luiza    me_av           NA
121    big     C    luiza    me_av 2.140734e+09
122    big     D    luiza    me_av 1.070244e+09
123    big     E    luiza    me_av 5.381888e+08
124    big     F    luiza    me_av 2.699331e+08
125 medium     A    luiza    me_av 1.086477e+09
126 medium     B    luiza    me_av 5.545421e+08
127 medium     C    luiza    me_av 2.775953e+08
128 medium     D    luiza    me_av 1.398639e+08
129 medium     E    luiza    me_av 7.109726e+07
130 medium     F    luiza    me_av 3.772497e+07
131  small     A    luiza    me_av 4.363366e+06
132  small     B    luiza    me_av 3.782599e+06
133  small     C    luiza    me_av 3.487619e+06
134  small     D    luiza    me_av 3.326572e+06
135  small     E    luiza    me_av 3.255328e+06
136  small     F    luiza    me_av 3.218205e+06
137    big     A    luiza    me_se           NA
138    big     C    luiza    me_se 7.064632e+02
139    big     D    luiza    me_se 4.946893e+02
140    big     E    luiza    me_se 4.525893e+02
141    big     F    luiza    me_se 7.219238e+02
142 medium     A    luiza    me_se 7.483055e+02
143 medium     B    luiza    me_se 6.554398e+02
144 medium     C    luiza    me_se 7.115345e+02
145 medium     D    luiza    me_se 7.766947e+02
146 medium     E    luiza    me_se 6.505641e+02
147 medium     F    luiza    me_se 5.289910e+02
148  small     A    luiza    me_se 4.731092e+01
149  small     B    luiza    me_se 4.765022e+01
150  small     C    luiza    me_se 4.821309e+01
151  small     D    luiza    me_se 4.784435e+01
152  small     E    luiza    me_se 4.797908e+01
153  small     F    luiza    me_se 4.933970e+01

Let’s plot this:

library(ggplot2);
ggplot(g, aes(x=input, y=time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + ylim(0, NA);

img/first_plot_3.png

It didn’t work.

pj_dump in windows

Let’s try to compile. Cmake didn’t work in windows because it cannot detect MinGW. So, let’s compile by hand (first time this is pursued):

touch libpaje_config.h
bison --verbose -d -o parser.cc ../src/libpaje/parser.y
flex -F -o lexer.cc ../src/libpaje/scanner.l
g++ -static -I. -DLIBPAJE_VERSION_STRING="\"\"" -I../src/libpaje/ `find ../src/libpaje/ ../src/tools/ | grep -v equals | grep -v "#" | grep -e cc$` parser.cc lexer.cc -o x -lfl

2016-04-20 Batch size investigation specific for orion1

Save all traces from all experiments, no matter what.

Change experimental design for batch size investigation.

2016-04-20 Meeting

We have discussed about the TCC structure.

What is missing in the (please complete):

  • Guarani
    • cpp comparison
    • batch size and time
  • Luiza
    • batch size and time - 27/04
    • cpp comparison - after batch finishes
  • Orion

Networked.

  • Do not find connection.

Create the bitbucket repo.

2016-04-25 MySQL remote access problem

/etc/my/cnf - > remove bind-address. I don`t have sudo access.

http://stackoverflow.com/questions/14779104/how-to-allow-remote-connection-to-mysql

2016-04-26 Experimental Design

  • Range of factor

2016-05-03 Meeting

Missing _Experiments

  • batch_size and batch_time
    • luiza
    • guarani
  • cpp_comparison
    • luiza
df <- read.csv("results/cpp_comparison/bateria_01_orion1.csv");
df[df$version == "aiyra",]$time = df[df$version == "aiyra",]$time/1000000;
dff <- df;
df <- read.csv("results/cpp_comparison/cpp_guarani_bateria01.csv");
df[df$version == "aiyra",]$time = df[df$version == "aiyra",]$time/1000000;
dff <- rbind (dff, df);
df <- read.csv("results/cpp_comparison/bateria01_cpp_comparison_luiza.csv");
df[df$version == "aiyra",]$time = df[df$version == "aiyra",]$time/1000000;
dff <- rbind (dff, df);
library(dplyr);
k <- dff %>%
     select(version, input, time, platform) %>%
     group_by(version, input, platform) %>%
     summarize(N = n(),
               time_avg = mean(time),
               time_se = 3*sd(time)/sqrt(N)) %>%
     as.data.frame();
k;
   version  input platform  N     time_avg     time_se
1    aiyra    big   orion1 30 111.78373253 1.012192396
2    aiyra    big  guarani 30 124.24501307 1.152942942
3    aiyra    big    luiza 30  86.22067403 1.158466007
4    aiyra medium   orion1 30  13.08560363 0.123976860
5    aiyra medium  guarani 30  13.62979877 0.230900350
6    aiyra medium    luiza 30   9.76933207 0.088475701
7    aiyra  small   orion1 30   0.22622977 0.009292423
8    aiyra  small  guarani 30   0.17240930 0.001343752
9    aiyra  small    luiza 30   0.18771763 0.001778820
10      pj    big   orion1 30 259.72360747 1.115892686
11      pj    big  guarani 30 232.04630737 0.595609380
12      pj    big    luiza 30 220.61451357 2.242830636
13      pj medium   orion1 30  33.47163073 0.125393689
14      pj medium  guarani 30  29.97373843 0.120739316
15      pj medium    luiza 30  26.61536637 0.233190435
16      pj  small   orion1 30   0.04607153 0.002206699
17      pj  small  guarani 30   0.04223553 0.006087253
18      pj  small    luiza 30   0.02630227 0.000777068
19  pjflex    big   orion1 30 279.74462510 0.924957991
20  pjflex    big  guarani 30 248.65192840 1.108033095
21  pjflex    big    luiza 30 235.54202920 3.159039361
22  pjflex medium   orion1 30  36.08434247 0.158896225
23  pjflex medium  guarani 30  32.14986230 0.142211291
24  pjflex medium    luiza 30  27.84721657 0.368436614
25  pjflex  small   orion1 30   0.04819160 0.002599552
26  pjflex  small  guarani 30   0.04453450 0.005719485
27  pjflex  small    luiza 30   0.02688117 0.001202209
library(ggplot2);
ggplot(k, aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + ylim(0, NA) + facet_wrap (~platform);

img/cpp_comparison_v1.png

Do another experiment with the dump plugin, but send the stdout to /dev/null.

batch_size

df <- read.csv("results/batch_size/bateria01_batch_guarani_orion.csv");
df2 <- read.csv("results/batch_size/bateria03_orion1_batch.csv");
df <- rbind (df, df2);
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$name <- NULL;
df$run.no.in.std.order <- NULL;
df$run.no <- NULL;
df$run.no.std.rp <- NULL;
head(df,n=30);
    input batch   sim_time insert_time    max_mem      platform
1  medium     C  95.729191   63.876499  277502560 guarani_orion
2     big     F 765.609348  480.095904  269839192 guarani_orion
3  medium     B  98.475845   62.738128  553416808 guarani_orion
4     big     D 792.449055  482.476239 1070150800 guarani_orion
5   small     F   0.904440    0.209350    3114600 guarani_orion
6     big     B         NA          NA         NA guarani_orion
7     big     E 775.009690  482.444082  538095504 guarani_orion
8     big     A         NA          NA         NA guarani_orion
9   small     E   0.648814    0.159654    3152000 guarani_orion
10  small     D   0.573042    0.150589    3223264 guarani_orion
11  small     C   0.551141    0.138318    3384336 guarani_orion
12 medium     F  93.896372   63.062088   36805648 guarani_orion
13  small     B   0.521541    0.148063    3679016 guarani_orion
14 medium     D  95.989211   62.421635  139769888 guarani_orion
15 medium     E  91.172608   61.063337   71003784 guarani_orion
16 medium     A 100.174817   62.795728 1086384296 guarani_orion
17    big     C 817.663388  479.458389 2140641384 guarani_orion
18  small     A   0.490466    0.157788    4259984 guarani_orion
19    big     A         NA          NA         NA guarani_orion
20 medium     B  98.632146   62.454971  553416808 guarani_orion
21 medium     E  91.534062   60.796546   71003784 guarani_orion
22  small     A   0.488172    0.155130    4259984 guarani_orion
23    big     B         NA          NA         NA guarani_orion
24  small     D   0.615405    0.174422    3223176 guarani_orion
25  small     E   0.670647    0.167575    3152040 guarani_orion
26 medium     F  91.334075   60.392192   36805736 guarani_orion
27    big     D 791.245305  483.278472 1070150984 guarani_orion
28  small     F   0.888700    0.216120    3114600 guarani_orion
29  small     C   0.596495    0.155477    3384152 guarani_orion
30 medium     D  97.194017   63.143370  139769800 guarani_orion

Specific to the insert database plugin.

Hypothesis:

  • few bigger batches leads to more performance
    • but with preliminary results we have seen that makes no difference to have many small batches than few bigger batches. At the end, few bigger batches have worse performance than lots of smaller batches because since they are small, their memory footprint is also small.

Let’s see if that conclusion holds.

library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               sim_time_avg = mean(sim_time),
               insert_time_avg = mean(insert_time),
               max_mem_avg = mean(max_mem),
               sim_time_se = 3*sd(sim_time)/sqrt(N),
               insert_time_se = 3*sd(insert_time)/sqrt(N),
               max_mem_se = 3*sd(max_mem)/sqrt(N)) %>%
     as.data.frame();
k;
    input batch      platform  N sim_time_avg insert_time_avg max_mem_avg
1     big     A guarani_orion 30           NA              NA          NA
2     big     B guarani_orion 30           NA              NA          NA
3     big     B        orion1 10 1107.3227606     394.8574468  4284406334
4     big     C guarani_orion 30  822.9541185     485.1070332  2140641230
5     big     C        orion1 10 1061.8112710     396.1577301  2140641789
6     big     D guarani_orion 30  793.6288847     483.6540466  1070150901
7     big     D        orion1 10 1026.6392053     395.8077851  1070151396
8     big     E guarani_orion 30  776.9857812     483.3852063   538095390
9     big     E        orion1 10  997.8075534     398.4135196   538095911
10    big     F guarani_orion 30  765.5748142     482.0589860   269839281
11    big     F        orion1 10  946.2533760     397.1500097   269839754
12 medium     A guarani_orion 30  100.0850588      62.7520164  1086384385
13 medium     B guarani_orion 30   98.6565419      62.5883192   553416906
14 medium     B        orion1 10  106.2926635      52.5826163   553417419
15 medium     C guarani_orion 30   94.7818702      62.7476814   277502649
16 medium     C        orion1 10  111.3229354      52.3318907   277503154
17 medium     D guarani_orion 30   96.5232222      62.7275693   139769855
18 medium     D        orion1 10  117.2014197      52.7724318   139769880
19 medium     E guarani_orion 30   92.0910381      61.5051609    71003661
20 medium     E        orion1 10   97.4553447      51.7208634    71003500
21 medium     F guarani_orion 30   92.0036549      61.2045346    36805687
22 medium     F        orion1 10  109.6810972      51.1558702    36805627
23  small     A guarani_orion 30    0.4990605       0.1609683     4260085
24  small     B guarani_orion 30    0.5296416       0.1471723     3733615
25  small     B        orion1 10    0.7557264       0.1761141     3678566
26  small     C guarani_orion 30    0.5577120       0.1470663     3384203
27  small     C        orion1 10    0.8611910       0.2005328     3383734
28  small     D guarani_orion 30    0.5934754       0.1613638     3223446
29  small     D        orion1 10    0.9929203       0.2145494     3222753
30  small     E guarani_orion 30    0.6592223       0.1684508     3347033
31  small     E        orion1 10    1.2279947       0.2582695     3151409
32  small     F guarani_orion 30    0.8761198       0.2166778     3091533
33  small     F        orion1 10    1.6951738       0.2952458     3114199
   sim_time_se insert_time_se   max_mem_se
1           NA             NA           NA
2           NA             NA           NA
3  6.658931757    3.557103806     53.67159
4  3.072009855    3.198853271     35.69804
5  5.241098578    3.274400300     27.52744
6  2.793609013    2.613030588     48.37919
7  5.417082452    3.636207259     39.55755
8  3.234269351    2.772145835     47.07640
9  5.549207079    3.627192856     37.25802
10 2.850922209    2.812338889     50.36058
11 3.886923906    2.759267457      7.20000
12 0.438621535    0.413417430     48.58858
13 0.657277641    0.623628613     49.40942
14 3.040111400    0.972669537     70.94336
15 0.528655405    0.507644999     50.36058
16 0.911257568    0.587854008     58.90195
17 0.502179305    0.440970858     42.94921
18 0.883692783    0.502898558    381.42470
19 0.393890608    0.376421887     51.52997
20 7.025394616    0.789976725    237.72926
21 0.480806287    0.489505423     33.77949
22 0.940375391    0.556399760    307.66631
23 0.007578415    0.007121396     50.13835
24 0.009928985    0.005195072 163599.84092
25 0.022256931    0.013618157     74.43010
26 0.010474202    0.004888757     44.18394
27 0.028683959    0.019515474     68.14573
28 0.016890922    0.008557890 194965.58391
29 0.029567395    0.014322845     57.51661
30 0.014014033    0.006232125 322269.11322
31 0.029832810    0.023829933     76.01684
32 0.015386276    0.009960192  69372.89713
33 0.036605546    0.015599726     62.93616
library(ggplot2);
ggplot(k[k$input == "big",], aes(x=batch, y=sim_time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=sim_time_avg-sim_time_se, ymax=sim_time_avg+sim_time_se), width=.1) +
  theme_bw() + facet_wrap (~platform) + ylim(0,NA);

img/batch_size_big_v1.png

Clearly there is a pattern: as the number of batches increases, the simulation time decreases.

library(ggplot2);
ggplot(k[k$input == "medium",], aes(x=batch, y=sim_time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=sim_time_avg-sim_time_se, ymax=sim_time_avg+sim_time_se), width=.1) +
  theme_bw() + facet_wrap (~platform) + ylim(0,NA);

img/batch_size_medium_v1.png

Inconclusive.

library(ggplot2);
ggplot(k[k$input == "small",], aes(x=batch, y=sim_time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=sim_time_avg-sim_time_se, ymax=sim_time_avg+sim_time_se), width=.1) +
  theme_bw() + facet_wrap (~platform) + ylim(0,NA);

img/batch_size_small_v1.png

As the number of batches increases, performance decreases.

2016-05-03 Script to merge all CSV of batch_time

cd results/batch_time/
head -n1 `ls -1 *.csv | head -n1` | sed -e "s/(microseconds)//g" -e "s/Size (Operations)/op_count/" -e "s/Size (Bytes)/size/" -e "s/$/,input,batchsize,platform,execution/" -e "s/ //g" | tr '[:upper:]' '[:lower:]' > ../batch_time.csv
for file in `ls -1 *.csv`;
do
    INPUT=`echo $file | cut -d"_" -f1 | cut -d"." -f1 | sed "s/trace-//"`
    BATCHSIZE=`echo $file | cut -d"_" -f2`
    PLATFORM=`echo $file | sed "s/guarani_orion/guarani-orion1/" | cut -d"_" -f3` # | cut -d"." -f1`
    UNIQUE=`echo $file | sed "s/guarani_orion/guarani-orion1/" | cut -d"_" -f4 | cut -d"." -f1`
    COLUMNS=`echo $INPUT,$BATCHSIZE,$PLATFORM,$UNIQUE | sed -e "s/1G/big/" -e "s/128M/medium/" -e "s/128K/small/"`

    cat $file | tail -n+2 | sed "s/$/,$COLUMNS/"
done >> ../batch_time.csv

Great, it worked.

Now, let’s load with R.

df <- read.csv ("results/batch_time.csv");
df$starttime = df$starttime / 1000000;
df$endtime = df$endtime / 1000000;
df$duration = df$duration / 1000000;
head(df);
  batchnumber starttime  endtime duration op_count  size input batchsize
1           1  0.260947 0.325175 0.064228     1363 47115 small      1363
2           2  0.438277 0.494230 0.055953     1363 49281 small      1363
3           3  0.514935 0.520468 0.005533        2   579 small      1363
4           1  0.250129 0.313937 0.063808     1363 47115 small      1363
5           2  0.450192 0.502917 0.052725     1363 49281 small      1363
6           3  0.524306 0.528925 0.004619        2   579 small      1363
        platform  execution
1 guarani-orion1 1675797067
2 guarani-orion1 1675797067
3 guarani-orion1 1675797067
4 guarani-orion1 1680230725
5 guarani-orion1 1680230725
6 guarani-orion1 1680230725
summary(df);
 batchnumber      starttime            endtime             duration        
Min.   : 1.00   Min.   :   0.0688   Min.   :   0.0750   Min.   :  0.00163  
1st Qu.: 4.00   1st Qu.:   0.6728   1st Qu.:   0.6859   1st Qu.:  0.00938  
Median : 9.00   Median :  41.8361   Median :  47.8734   Median :  1.91862  
Mean   :11.59   Mean   : 157.9904   Mean   : 169.3395   Mean   : 11.34909  
3rd Qu.:17.00   3rd Qu.: 183.8669   3rd Qu.: 209.2987   3rd Qu.: 14.96168  
Max.   :33.00   Max.   :1119.4321   Max.   :1119.4347   Max.   :204.44305  
   op_count            size              input                platform   
Min.   :      1   Min.   :      543   big   :2590   guarani-orion1:6060  
1st Qu.:    170   1st Qu.:     6262   medium:2740   orion1        :2019  
Median :  78957   Median :  3299183   small :2749                        
Mean   : 483820   Mean   : 21241860                                      
3rd Qu.: 611585   3rd Qu.: 25551883                                      
Max.   :9785361   Max.   :463208087                                      
  execution        
Min.   :9.622e+08  
1st Qu.:1.672e+09  
Median :1.723e+09  
Mean   :1.558e+09  
3rd Qu.:1.772e+09  
Max.   :1.822e+09
df[df$input == "small" & df$batchnumber == 1 & df$platform != "orion1",]
     batchnumber starttime  endtime duration op_count  size input batchsize
1              1  0.260947 0.325175 0.064228     1363 47115 small      1363
4              1  0.250129 0.313937 0.063808     1363 47115 small      1363
7              1  0.248299 0.310239 0.061940     1363 47115 small      1363
10             1  0.257183 0.324293 0.067110     1363 47115 small      1363
13             1  0.273497 0.336475 0.062978     1363 47115 small      1363
16             1  0.253036 0.315262 0.062226     1363 47115 small      1363
19             1  0.255901 0.320205 0.064304     1363 47115 small      1363
22             1  0.253300 0.315583 0.062283     1363 47115 small      1363
25             1  0.251402 0.316613 0.065211     1363 47115 small      1363
28             1  0.259033 0.320449 0.061416     1363 47115 small      1363
31             1  0.264858 0.331881 0.067023     1363 47115 small      1363
34             1  0.260197 0.322896 0.062699     1363 47115 small      1363
37             1  0.277637 0.342484 0.064847     1363 47115 small      1363
40             1  0.267532 0.332812 0.065280     1363 47115 small      1363
43             1  0.255276 0.318686 0.063410     1363 47115 small      1363
46             1  0.250359 0.314062 0.063703     1363 47115 small      1363
49             1  0.251041 0.314261 0.063220     1363 47115 small      1363
52             1  0.242527 0.306729 0.064202     1363 47115 small      1363
55             1  0.260315 0.323826 0.063511     1363 47115 small      1363
58             1  0.250519 0.312685 0.062166     1363 47115 small      1363
61             1  0.247476 0.311956 0.064480     1363 47115 small      1363
64             1  0.248585 0.312853 0.064268     1363 47115 small      1363
67             1  0.309358 0.384269 0.074911     1363 47115 small      1363
70             1  0.245734 0.334204 0.088470     1363 47115 small      1363
73             1  0.251926 0.315660 0.063734     1363 47115 small      1363
76             1  0.245549 0.307753 0.062204     1363 47115 small      1363
79             1  0.259887 0.324880 0.064993     1363 47115 small      1363
82             1  0.255746 0.317639 0.061893     1363 47115 small      1363
85             1  0.254501 0.320647 0.066146     1363 47115 small      1363
88             1  0.252678 0.314825 0.062147     1363 47115 small      1363
121            1  0.102706 0.113168 0.010462      170  6056 small       170
138            1  0.097173 0.107291 0.010118      170  6056 small       170
155            1  0.101976 0.112034 0.010058      170  6056 small       170
172            1  0.099378 0.109366 0.009988      170  6056 small       170
189            1  0.119521 0.129508 0.009987      170  6056 small       170
206            1  0.109040 0.119026 0.009986      170  6056 small       170
223            1  0.094324 0.104086 0.009762      170  6056 small       170
240            1  0.094739 0.104646 0.009907      170  6056 small       170
257            1  0.096193 0.105932 0.009739      170  6056 small       170
274            1  0.094419 0.105561 0.011142      170  6056 small       170
291            1  0.095802 0.105740 0.009938      170  6056 small       170
308            1  0.115041 0.124993 0.009952      170  6056 small       170
325            1  0.107166 0.117286 0.010120      170  6056 small       170
342            1  0.095424 0.105552 0.010128      170  6056 small       170
359            1  0.107392 0.117314 0.009922      170  6056 small       170
376            1  0.110775 0.120708 0.009933      170  6056 small       170
393            1  0.101006 0.111128 0.010122      170  6056 small       170
410            1  0.095495 0.105337 0.009842      170  6056 small       170
427            1  0.094980 0.104866 0.009886      170  6056 small       170
444            1  0.101667 0.111726 0.010059      170  6056 small       170
461            1  0.113697 0.123510 0.009813      170  6056 small       170
478            1  0.108654 0.118624 0.009970      170  6056 small       170
495            1  0.096355 0.106346 0.009991      170  6056 small       170
512            1  0.094470 0.104139 0.009669      170  6056 small       170
529            1  0.095549 0.105446 0.009897      170  6056 small       170
546            1  0.125691 0.136445 0.010754      170  6056 small       170
563            1  0.096654 0.107011 0.010357      170  6056 small       170
580            1  0.097030 0.107236 0.010206      170  6056 small       170
597            1  0.100114 0.110073 0.009959      170  6056 small       170
614            1  0.094137 0.103720 0.009583      170  6056 small       170
801            1  0.327633 0.458439 0.130806     2727 95914 small      2727
803            1  0.322435 0.454014 0.131579     2727 95914 small      2727
805            1  0.325214 0.512092 0.186878     2727 95914 small      2727
807            1  0.322176 0.453841 0.131665     2727 95914 small      2727
809            1  0.335010 0.465702 0.130692     2727 95914 small      2727
811            1  0.327602 0.457524 0.129922     2727 95914 small      2727
813            1  0.325034 0.456413 0.131379     2727 95914 small      2727
815            1  0.321860 0.452983 0.131123     2727 95914 small      2727
817            1  0.321930 0.453527 0.131597     2727 95914 small      2727
819            1  0.339958 0.469854 0.129896     2727 95914 small      2727
821            1  0.328478 0.457298 0.128820     2727 95914 small      2727
823            1  0.325443 0.454340 0.128897     2727 95914 small      2727
825            1  0.322145 0.452492 0.130347     2727 95914 small      2727
827            1  0.327743 0.457261 0.129518     2727 95914 small      2727
829            1  0.321621 0.456728 0.135107     2727 95914 small      2727
831            1  0.322076 0.455030 0.132954     2727 95914 small      2727
833            1  0.322951 0.488514 0.165563     2727 95914 small      2727
835            1  0.347428 0.479367 0.131939     2727 95914 small      2727
837            1  0.324668 0.454993 0.130325     2727 95914 small      2727
839            1  0.325622 0.459809 0.134187     2727 95914 small      2727
841            1  0.331279 0.463779 0.132500     2727 95914 small      2727
843            1  0.328716 0.465095 0.136379     2727 95914 small      2727
845            1  0.319629 0.449644 0.130015     2727 95914 small      2727
847            1  0.329689 0.462710 0.133021     2727 95914 small      2727
849            1  0.333090 0.463796 0.130706     2727 95914 small      2727
851            1  0.346588 0.478676 0.132088     2727 95914 small      2727
853            1  0.340306 0.472578 0.132272     2727 95914 small      2727
855            1  0.354546 0.483186 0.128640     2727 95914 small      2727
857            1  0.319886 0.450770 0.130884     2727 95914 small      2727
859            1  0.320302 0.462172 0.141870     2727 95914 small      2727
861            1  0.140550 0.158314 0.017764      340 11801 small       340
870            1  0.177868 0.195649 0.017781      340 11801 small       340
879            1  0.171488 0.189171 0.017683      340 11801 small       340
888            1  0.152983 0.170538 0.017555      340 11801 small       340
897            1  0.147051 0.164721 0.017670      340 11801 small       340
906            1  0.143912 0.161426 0.017514      340 11801 small       340
915            1  0.147236 0.164813 0.017577      340 11801 small       340
924            1  0.138077 0.155637 0.017560      340 11801 small       340
933            1  0.176919 0.194408 0.017489      340 11801 small       340
942            1  0.140037 0.157431 0.017394      340 11801 small       340
951            1  0.151587 0.172798 0.021211      340 11801 small       340
960            1  0.136561 0.153730 0.017169      340 11801 small       340
969            1  0.160996 0.178665 0.017669      340 11801 small       340
978            1  0.147977 0.166529 0.018552      340 11801 small       340
987            1  0.134656 0.152314 0.017658      340 11801 small       340
996            1  0.141566 0.158940 0.017374      340 11801 small       340
1005           1  0.137746 0.155471 0.017725      340 11801 small       340
1014           1  0.158834 0.176431 0.017597      340 11801 small       340
1023           1  0.152053 0.169582 0.017529      340 11801 small       340
1032           1  0.134633 0.152338 0.017705      340 11801 small       340
1041           1  0.134657 0.152167 0.017510      340 11801 small       340
1050           1  0.143881 0.164866 0.020985      340 11801 small       340
1059           1  0.145405 0.162979 0.017574      340 11801 small       340
1068           1  0.130057 0.147745 0.017688      340 11801 small       340
1077           1  0.128343 0.146516 0.018173      340 11801 small       340
1086           1  0.129925 0.147719 0.017794      340 11801 small       340
1095           1  0.147447 0.165022 0.017575      340 11801 small       340
1104           1  0.148243 0.165681 0.017438      340 11801 small       340
1113           1  0.134714 0.152180 0.017466      340 11801 small       340
1122           1  0.136597 0.154022 0.017425      340 11801 small       340
1230           1  0.187522 0.218946 0.031424      681 23322 small       681
1235           1  0.207619 0.239247 0.031628      681 23322 small       681
1240           1  0.197176 0.231730 0.034554      681 23322 small       681
1245           1  0.178109 0.210519 0.032410      681 23322 small       681
1250           1  0.192685 0.225138 0.032453      681 23322 small       681
1255           1  0.177635 0.210090 0.032455      681 23322 small       681
1260           1  0.178646 0.210132 0.031486      681 23322 small       681
1265           1  0.175493 0.207409 0.031916      681 23322 small       681
1270           1  0.206574 0.238499 0.031925      681 23322 small       681
1275           1  0.195778 0.228237 0.032459      681 23322 small       681
1280           1  0.193363 0.225164 0.031801      681 23322 small       681
1285           1  0.176221 0.207828 0.031607      681 23322 small       681
1290           1  0.178486 0.209967 0.031481      681 23322 small       681
1295           1  0.181313 0.212818 0.031505      681 23322 small       681
1300           1  0.205239 0.237207 0.031968      681 23322 small       681
1305           1  0.183108 0.215620 0.032512      681 23322 small       681
1310           1  0.197095 0.229320 0.032225      681 23322 small       681
1315           1  0.179714 0.211314 0.031600      681 23322 small       681
1320           1  0.187697 0.219758 0.032061      681 23322 small       681
1325           1  0.176721 0.208092 0.031371      681 23322 small       681
1330           1  0.181204 0.213745 0.032541      681 23322 small       681
1335           1  0.189644 0.221490 0.031846      681 23322 small       681
1340           1  0.194346 0.226148 0.031802      681 23322 small       681
1345           1  0.186593 0.218292 0.031699      681 23322 small       681
1350           1  0.176912 0.209169 0.032257      681 23322 small       681
1355           1  0.184237 0.215942 0.031705      681 23322 small       681
1360           1  0.183621 0.215274 0.031653      681 23322 small       681
1365           1  0.192220 0.224777 0.032557      681 23322 small       681
1370           1  0.214212 0.245802 0.031590      681 23322 small       681
1375           1  0.194517 0.226731 0.032214      681 23322 small       681
1430           1  0.109498 0.116452 0.006954       85  3200 small        85
1463           1  0.071465 0.077793 0.006328       85  3200 small        85
1496           1  0.070386 0.076808 0.006422       85  3200 small        85
1529           1  0.087084 0.093399 0.006315       85  3200 small        85
1562           1  0.069054 0.075417 0.006363       85  3200 small        85
1595           1  0.069734 0.076154 0.006420       85  3200 small        85
1628           1  0.079036 0.085429 0.006393       85  3200 small        85
1661           1  0.073661 0.079796 0.006135       85  3200 small        85
1694           1  0.084381 0.091221 0.006840       85  3200 small        85
1727           1  0.089337 0.095611 0.006274       85  3200 small        85
1760           1  0.092171 0.098628 0.006457       85  3200 small        85
1793           1  0.089158 0.095262 0.006104       85  3200 small        85
1826           1  0.081770 0.088006 0.006236       85  3200 small        85
1859           1  0.085676 0.092258 0.006582       85  3200 small        85
1892           1  0.074547 0.080766 0.006219       85  3200 small        85
1925           1  0.068764 0.074953 0.006189       85  3200 small        85
1958           1  0.088080 0.095060 0.006980       85  3200 small        85
1991           1  0.068930 0.075165 0.006235       85  3200 small        85
2024           1  0.074180 0.080832 0.006652       85  3200 small        85
2057           1  0.079009 0.085881 0.006872       85  3200 small        85
2090           1  0.070202 0.076538 0.006336       85  3200 small        85
2123           1  0.069065 0.075550 0.006485       85  3200 small        85
2156           1  0.080025 0.086833 0.006808       85  3200 small        85
2189           1  0.075889 0.082181 0.006292       85  3200 small        85
2222           1  0.084039 0.090371 0.006332       85  3200 small        85
2255           1  0.075244 0.081194 0.005950       85  3200 small        85
2288           1  0.069607 0.085937 0.016330       85  3200 small        85
2321           1  0.073219 0.079381 0.006162       85  3200 small        85
2354           1  0.078698 0.085040 0.006342       85  3200 small        85
2387           1  0.070442 0.076672 0.006230       85  3200 small        85
           platform  execution
1    guarani-orion1 1675797067
4    guarani-orion1 1680230725
7    guarani-orion1 1684400669
10   guarani-orion1 1691878618
13   guarani-orion1 1695137278
16   guarani-orion1 1700423900
19   guarani-orion1 1702672641
22   guarani-orion1 1709048507
25   guarani-orion1 1712054748
28   guarani-orion1 1722078245
31   guarani-orion1 1722714931
34   guarani-orion1 1727184990
37   guarani-orion1 1732386762
40   guarani-orion1 1741841349
43   guarani-orion1 1745793144
46   guarani-orion1 1750935916
49   guarani-orion1 1753517996
52   guarani-orion1 1760907125
55   guarani-orion1 1762745387
58   guarani-orion1 1767780178
61   guarani-orion1 1773748175
64   guarani-orion1 1777055998
67   guarani-orion1 1782801721
70   guarani-orion1 1790372173
73   guarani-orion1 1792050057
76   guarani-orion1 1798679912
79   guarani-orion1 1803502969
82   guarani-orion1 1809757321
85   guarani-orion1 1816111360
88   guarani-orion1 1817134362
121  guarani-orion1 1675699437
138  guarani-orion1 1678372738
155  guarani-orion1 1686099622
172  guarani-orion1 1688843073
189  guarani-orion1 1693446683
206  guarani-orion1 1699787190
223  guarani-orion1 1705398883
240  guarani-orion1 1708256373
257  guarani-orion1 1716420051
274  guarani-orion1 1717058098
291  guarani-orion1 1725444765
308  guarani-orion1 1727183249
325  guarani-orion1 1732096395
342  guarani-orion1 1739990282
359  guarani-orion1 1743504464
376  guarani-orion1 1749951533
393  guarani-orion1 1755813022
410  guarani-orion1 1760270361
427  guarani-orion1 1762945062
444  guarani-orion1 1767780586
461  guarani-orion1 1776229605
478  guarani-orion1 1781976603
495  guarani-orion1 1786827453
512  guarani-orion1 1791154010
529  guarani-orion1 1792049108
546  guarani-orion1 1801362793
563  guarani-orion1 1807108650
580  guarani-orion1 1809853161
597  guarani-orion1 1816213035
614  guarani-orion1 1817133392
801  guarani-orion1 1676905651
803  guarani-orion1 1677734664
805  guarani-orion1 1686972328
807  guarani-orion1 1687065487
809  guarani-orion1 1695229978
811  guarani-orion1 1701944121
813  guarani-orion1 1702673764
815  guarani-orion1 1707240719
817  guarani-orion1 1717057176
819  guarani-orion1 1720401844
821  guarani-orion1 1722911004
823  guarani-orion1 1732095084
825  guarani-orion1 1732190703
827  guarani-orion1 1738376391
829  guarani-orion1 1744274190
831  guarani-orion1 1747579091
833  guarani-orion1 1755812680
835  guarani-orion1 1760269540
837  guarani-orion1 1762746430
839  guarani-orion1 1770781183
841  guarani-orion1 1776956594
843  guarani-orion1 1781880090
845  guarani-orion1 1782800771
847  guarani-orion1 1791152068
849  guarani-orion1 1793704016
851  guarani-orion1 1798776137
853  guarani-orion1 1807107717
855  guarani-orion1 1809854123
857  guarani-orion1 1814649138
859  guarani-orion1 1819717508
861  guarani-orion1 1675701079
870  guarani-orion1 1678371728
879  guarani-orion1 1684306794
888  guarani-orion1 1691147321
897  guarani-orion1 1693446079
906  guarani-orion1 1701945178
915  guarani-orion1 1703537380
924  guarani-orion1 1707040031
933  guarani-orion1 1714544729
942  guarani-orion1 1720402934
951  guarani-orion1 1725346689
960  guarani-orion1 1727083351
969  guarani-orion1 1732191543
978  guarani-orion1 1740285053
987  guarani-orion1 1744275268
996  guarani-orion1 1750841471
1005 guarani-orion1 1754154536
1014 guarani-orion1 1756970103
1023 guarani-orion1 1764518937
1032 guarani-orion1 1767782212
1041 guarani-orion1 1775364166
1050 guarani-orion1 1780508336
1059 guarani-orion1 1787019224
1068 guarani-orion1 1791950506
1077 guarani-orion1 1795303521
1086 guarani-orion1 1798776448
1095 guarani-orion1 1804412969
1104 guarani-orion1 1807213534
1113 guarani-orion1 1816212031
1122 guarani-orion1 1821133304
1230 guarani-orion1 1675701422
1235 guarani-orion1 1679258730
1240 guarani-orion1 1683405773
1245 guarani-orion1 1689755706
1250 guarani-orion1 1694499886
1255 guarani-orion1 1700424734
1260 guarani-orion1 1702774884
1265 guarani-orion1 1708162331
1270 guarani-orion1 1717055443
1275 guarani-orion1 1719405964
1280 guarani-orion1 1722713870
1285 guarani-orion1 1727183957
1290 guarani-orion1 1732387628
1295 guarani-orion1 1739895771
1300 guarani-orion1 1743505619
1305 guarani-orion1 1747676452
1310 guarani-orion1 1756866222
1315 guarani-orion1 1758590416
1320 guarani-orion1 1765249326
1325 guarani-orion1 1771940286
1330 guarani-orion1 1773749023
1335 guarani-orion1 1779548002
1340 guarani-orion1 1783596668
1345 guarani-orion1 1791153017
1350 guarani-orion1 1793703037
1355 guarani-orion1 1797070405
1360 guarani-orion1 1806024429
1365 guarani-orion1 1807212324
1370 guarani-orion1 1813784523
1375 guarani-orion1 1822050168
1430 guarani-orion1 1673655686
1463 guarani-orion1 1679257266
1496 guarani-orion1 1684401743
1529 guarani-orion1 1688841310
1562 guarani-orion1 1696911897
1595 guarani-orion1 1698765013
1628 guarani-orion1 1703637609
1661 guarani-orion1 1710654612
1694 guarani-orion1 1717056506
1727 guarani-orion1 1720200718
1760 guarani-orion1 1725347397
1793 guarani-orion1 1728899875
1826 guarani-orion1 1732384889
1859 guarani-orion1 1739896619
1892 guarani-orion1 1742579606
1925 guarani-orion1 1750936237
1958 guarani-orion1 1753425313
1991 guarani-orion1 1760905656
2024 guarani-orion1 1766710882
2057 guarani-orion1 1770779141
2090 guarani-orion1 1772929037
2123 guarani-orion1 1779549002
2156 guarani-orion1 1783595193
2189 guarani-orion1 1787748109
2222 guarani-orion1 1794573384
2255 guarani-orion1 1798774552
2288 guarani-orion1 1807109863
2321 guarani-orion1 1809758383
2354 guarani-orion1 1812113496
2387 guarani-orion1 1822051185
library(ggplot2);
ggplot(df[df$input == "big" & df$platform != "orion1",], aes(x=starttime, xend=endtime, y=as.factor(execution), yend=as.factor(execution), color=input)) +
   geom_segment(size=1.1) +
  theme_bw() + facet_grid (batchsize~platform, scales="free_y") #+ xlim(90,110)# + ylim(0,NA);

img/batch_time_v1.pdf

2016-05-16 CPP Comparison (figures)

Merge everything in a single CSV file:

cd results/cpp_comparison
head -n1 cpp_guarani_bateria01.csv > all_but_first.csv
for i in `ls -1 *.csv | grep -v _cpp_c`; do
  tail -n+2 $i
done >> all_but_first.csv

Now, let’s read the file all_but_first.csv.

df <- read.csv ("results/cpp_comparison/all_but_first.csv");
df[df$version == "aiyra",]$time = df[df$version == "aiyra",]$time/1000000;
summary(df);
head(df);
      name       run.no.in.std.order     run.no      run.no.std.rp  
 Min.   :  1.0   Min.   :1           Min.   :  1.0   Min.   :1.100  
 1st Qu.: 68.0   1st Qu.:3           1st Qu.: 68.0   1st Qu.:3.160  
 Median :135.5   Median :5           Median :135.5   Median :5.225  
 Mean   :135.5   Mean   :5           Mean   :135.5   Mean   :5.290  
 3rd Qu.:203.0   3rd Qu.:7           3rd Qu.:203.0   3rd Qu.:7.300  
 Max.   :270.0   Max.   :9           Max.   :270.0   Max.   :9.900  
   version       input          time              platform  
 aiyra :270   big   :270   Min.   :   0.0245   guarani:270  
 pj    :270   medium:270   1st Qu.:   0.1738   luiza  :270  
 pjflex:270   small :270   Median :  25.7898   orion1 :270  
                           Mean   :  76.8609                
                           3rd Qu.: 123.2365                
                           Max.   :2888.0565
  name run.no.in.std.order run.no run.no.std.rp version  input       time
1    1                   4      1           4.1   aiyra medium  13.199061
2    2                   9      2           9.1  pjflex    big 280.152443
3    3                   2      3           2.1      pj  small   0.049384
4    4                   5      4           5.1      pj medium  33.616784
5    5                   7      5           7.1   aiyra    big 111.191072
6    6                   6      6           6.1  pjflex medium  35.570650
  platform
1   orion1
2   orion1
3   orion1
4   orion1
5   orion1
6   orion1

Let’s average measurements:

library(dplyr);
k <- df %>%
     select(version, input, time, platform) %>%
     group_by(version, input, platform) %>%
     summarize(N = n(),
               time_avg = mean(time),
               time_se = 3*sd(time)/sqrt(N)) %>%
     as.data.frame();
k
Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
   version  input platform  N     time_avg      time_se
1    aiyra    big  guarani 30 124.24501307 1.152943e+00
2    aiyra    big    luiza 30 176.45010810 2.805133e+02
3    aiyra    big   orion1 30 111.78373253 1.012192e+00
4    aiyra medium  guarani 30  13.62979877 2.309004e-01
5    aiyra medium    luiza 30   9.38572857 1.159134e-01
6    aiyra medium   orion1 30  13.08560363 1.239769e-01
7    aiyra  small  guarani 30   0.17240930 1.343752e-03
8    aiyra  small    luiza 30   0.18415197 1.450459e-03
9    aiyra  small   orion1 30   0.22622977 9.292423e-03
10      pj    big  guarani 30 232.04630737 5.956094e-01
11      pj    big    luiza 30 207.43421557 9.992199e+00
12      pj    big   orion1 30 259.72360747 1.115893e+00
13      pj medium  guarani 30  29.97373843 1.207393e-01
14      pj medium    luiza 30  24.92862427 2.842269e-01
15      pj medium   orion1 30  33.47163073 1.253937e-01
16      pj  small  guarani 30   0.04223553 6.087253e-03
17      pj  small    luiza 30   0.02562413 7.886749e-04
18      pj  small   orion1 30   0.04607153 2.206699e-03
19  pjflex    big  guarani 30 248.65192840 1.108033e+00
20  pjflex    big    luiza 30 215.79006593 2.822790e+00
21  pjflex    big   orion1 30 279.74462510 9.249580e-01
22  pjflex medium  guarani 30  32.14986230 1.422113e-01
23  pjflex medium    luiza 30  25.85007707 2.419624e-01
24  pjflex medium   orion1 30  36.08434247 1.588962e-01
25  pjflex  small  guarani 30   0.04453450 5.719485e-03
26  pjflex  small    luiza 30   0.02540013 4.037376e-04
27  pjflex  small   orion1 30   0.04819160 2.599552e-03
library(ggplot2);
ggplot(k, aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  scale_colour_manual(values = c("red","blue", "green")) +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + facet_wrap(~platform);

img/cpp/first_plot_1.png

There is something really strange with luiza aiyra big, let’s look to the measurements.

df[df$platform == "luiza" & df$version == "aiyra" & df$input == "big",];
    name run.no.in.std.order run.no run.no.std.rp version input       time
275    5                   7      5          7.10   aiyra   big   82.81775
281   11                   7     11          7.20   aiyra   big   86.17196
296   26                   7     26          7.30   aiyra   big 2888.05648
299   29                   7     29          7.40   aiyra   big   80.86330
314   44                   7     44          7.50   aiyra   big   82.80871
321   51                   7     51          7.60   aiyra   big   80.60814
325   55                   7     55          7.70   aiyra   big   81.70027
335   65                   7     65          7.80   aiyra   big   81.60717
346   76                   7     76          7.90   aiyra   big   81.75732
359   89                   7     89          7.10   aiyra   big   81.35299
364   94                   7     94          7.11   aiyra   big   83.07451
372  102                   7    102          7.12   aiyra   big   86.38635
382  112                   7    112          7.13   aiyra   big   84.73248
394  124                   7    124          7.14   aiyra   big   84.30511
403  133                   7    133          7.15   aiyra   big   83.61456
406  136                   7    136          7.16   aiyra   big   87.41428
422  152                   7    152          7.17   aiyra   big   82.55839
430  160                   7    160          7.18   aiyra   big   83.14233
433  163                   7    163          7.19   aiyra   big   82.74059
443  173                   7    173          7.20   aiyra   big   80.96755
455  185                   7    185          7.21   aiyra   big   83.47831
461  191                   7    191          7.22   aiyra   big   82.58781
474  204                   7    204          7.23   aiyra   big   80.05653
483  213                   7    213          7.24   aiyra   big   80.95360
487  217                   7    217          7.25   aiyra   big   84.05692
500  230                   7    230          7.26   aiyra   big   84.04072
509  239                   7    239          7.27   aiyra   big   88.34312
518  248                   7    248          7.28   aiyra   big   81.86359
525  255                   7    255          7.29   aiyra   big   80.94467
533  263                   7    263          7.30   aiyra   big   80.49776
    platform
275    luiza
281    luiza
296    luiza
299    luiza
314    luiza
321    luiza
325    luiza
335    luiza
346    luiza
359    luiza
364    luiza
372    luiza
382    luiza
394    luiza
403    luiza
406    luiza
422    luiza
430    luiza
433    luiza
443    luiza
455    luiza
461    luiza
474    luiza
483    luiza
487    luiza
500    luiza
509    luiza
518    luiza
525    luiza
533    luiza

We need to remove the outlier where execution time took $2888.05648$ seconds. To do so, we modify the df dataframe:

df2 <- df[df$time < 2000,]
df2[df2$platform == "luiza" & df2$version == "aiyra" & df2$input == "big",];
    name run.no.in.std.order run.no run.no.std.rp version input     time
275    5                   7      5          7.10   aiyra   big 82.81775
281   11                   7     11          7.20   aiyra   big 86.17196
299   29                   7     29          7.40   aiyra   big 80.86330
314   44                   7     44          7.50   aiyra   big 82.80871
321   51                   7     51          7.60   aiyra   big 80.60814
325   55                   7     55          7.70   aiyra   big 81.70027
335   65                   7     65          7.80   aiyra   big 81.60717
346   76                   7     76          7.90   aiyra   big 81.75732
359   89                   7     89          7.10   aiyra   big 81.35299
364   94                   7     94          7.11   aiyra   big 83.07451
372  102                   7    102          7.12   aiyra   big 86.38635
382  112                   7    112          7.13   aiyra   big 84.73248
394  124                   7    124          7.14   aiyra   big 84.30511
403  133                   7    133          7.15   aiyra   big 83.61456
406  136                   7    136          7.16   aiyra   big 87.41428
422  152                   7    152          7.17   aiyra   big 82.55839
430  160                   7    160          7.18   aiyra   big 83.14233
433  163                   7    163          7.19   aiyra   big 82.74059
443  173                   7    173          7.20   aiyra   big 80.96755
455  185                   7    185          7.21   aiyra   big 83.47831
461  191                   7    191          7.22   aiyra   big 82.58781
474  204                   7    204          7.23   aiyra   big 80.05653
483  213                   7    213          7.24   aiyra   big 80.95360
487  217                   7    217          7.25   aiyra   big 84.05692
500  230                   7    230          7.26   aiyra   big 84.04072
509  239                   7    239          7.27   aiyra   big 88.34312
518  248                   7    248          7.28   aiyra   big 81.86359
525  255                   7    255          7.29   aiyra   big 80.94467
533  263                   7    263          7.30   aiyra   big 80.49776
    platform
275    luiza
281    luiza
299    luiza
314    luiza
321    luiza
325    luiza
335    luiza
346    luiza
359    luiza
364    luiza
372    luiza
382    luiza
394    luiza
403    luiza
406    luiza
422    luiza
430    luiza
433    luiza
443    luiza
455    luiza
461    luiza
474    luiza
483    luiza
487    luiza
500    luiza
509    luiza
518    luiza
525    luiza
533    luiza

Great, it has worked.

Recalculate the averages, do the plot again.

library(dplyr);
k2 <- df2 %>%
     select(version, input, time, platform) %>%
     group_by(version, input, platform) %>%
     summarize(N = n(),
               time_avg = mean(time),
               time_se = 3*sd(time)/sqrt(N)) %>%
     as.data.frame();
k2
   version  input platform  N     time_avg      time_se
1    aiyra    big  guarani 30 124.24501307 1.1529429416
2    aiyra    big    luiza 29  82.94644007 1.1744753970
3    aiyra    big   orion1 30 111.78373253 1.0121923958
4    aiyra medium  guarani 30  13.62979877 0.2309003501
5    aiyra medium    luiza 30   9.38572857 0.1159133555
6    aiyra medium   orion1 30  13.08560363 0.1239768599
7    aiyra  small  guarani 30   0.17240930 0.0013437523
8    aiyra  small    luiza 30   0.18415197 0.0014504593
9    aiyra  small   orion1 30   0.22622977 0.0092924233
10      pj    big  guarani 30 232.04630737 0.5956093796
11      pj    big    luiza 30 207.43421557 9.9921988357
12      pj    big   orion1 30 259.72360747 1.1158926857
13      pj medium  guarani 30  29.97373843 0.1207393162
14      pj medium    luiza 30  24.92862427 0.2842269263
15      pj medium   orion1 30  33.47163073 0.1253936889
16      pj  small  guarani 30   0.04223553 0.0060872534
17      pj  small    luiza 30   0.02562413 0.0007886749
18      pj  small   orion1 30   0.04607153 0.0022066994
19  pjflex    big  guarani 30 248.65192840 1.1080330952
20  pjflex    big    luiza 30 215.79006593 2.8227903219
21  pjflex    big   orion1 30 279.74462510 0.9249579906
22  pjflex medium  guarani 30  32.14986230 0.1422112908
23  pjflex medium    luiza 30  25.85007707 0.2419624421
24  pjflex medium   orion1 30  36.08434247 0.1588962252
25  pjflex  small  guarani 30   0.04453450 0.0057194849
26  pjflex  small    luiza 30   0.02540013 0.0004037376
27  pjflex  small   orion1 30   0.04819160 0.0025995518
library(ggplot2);
ggplot(k2, aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  scale_colour_manual(values = c("red","blue", "green"), name="Input Size") +
  theme_bw() + facet_wrap(~platform) +
  ylab("Average Execution Time (s)") +
  xlab("Version") +
  theme_bw() + ylim(0,NA) + 
  theme(legend.position="right"); # no legend since we have only small

img/cpp/aiyra-pj-pjflex_overview.pdf

You have to explain the larger variability during the luiza, pj, big. Maybe because something was disturbing the experiment during that time. But remember we have randomize the design, so, it should have affected other configurations as well.

df[df$platform == "luiza" & df$version == "pj" & df$input == "big",];
    name run.no.in.std.order run.no run.no.std.rp version input     time
279    9                   8      9          8.10      pj   big 203.8820
282   12                   8     12          8.20      pj   big 303.2675
297   27                   8     27          8.30      pj   big 208.4385
301   31                   8     31          8.40      pj   big 202.1743
311   41                   8     41          8.50      pj   big 203.3942
322   52                   8     52          8.60      pj   big 204.3809
330   60                   8     60          8.70      pj   big 205.7884
334   64                   8     64          8.80      pj   big 202.2098
351   81                   8     81          8.90      pj   big 205.4249
354   84                   8     84          8.10      pj   big 203.5652
367   97                   8     97          8.11      pj   big 206.1473
377  107                   8    107          8.12      pj   big 203.0834
384  114                   8    114          8.13      pj   big 205.3515
393  123                   8    123          8.14      pj   big 204.3712
401  131                   8    131          8.15      pj   big 203.8469
408  138                   8    138          8.16      pj   big 203.6601
419  149                   8    149          8.17      pj   big 204.7679
425  155                   8    155          8.18      pj   big 200.8791
440  170                   8    170          8.19      pj   big 199.3703
449  179                   8    179          8.20      pj   big 204.9851
454  184                   8    184          8.21      pj   big 206.1524
466  196                   8    196          8.22      pj   big 202.3681
472  202                   8    202          8.23      pj   big 206.3542
479  209                   8    209          8.24      pj   big 204.6642
491  221                   8    221          8.25      pj   big 205.7807
503  233                   8    233          8.26      pj   big 209.4978
505  235                   8    235          8.27      pj   big 198.3630
515  245                   8    245          8.28      pj   big 203.5347
530  260                   8    260          8.29      pj   big 203.6857
539  269                   8    269          8.30      pj   big 203.6370
    platform
279    luiza
282    luiza
297    luiza
301    luiza
311    luiza
322    luiza
330    luiza
334    luiza
351    luiza
354    luiza
367    luiza
377    luiza
384    luiza
393    luiza
401    luiza
408    luiza
419    luiza
425    luiza
440    luiza
449    luiza
454    luiza
466    luiza
472    luiza
479    luiza
491    luiza
503    luiza
505    luiza
515    luiza
530    luiza
539    luiza

You can see a 300 seconds measurement that probably is the explanation for the higher standard error. We don’t know, we could remove it, but you can keep it and explain with something. Maybe mention that is unrelated, because the experimental execution order was 12 for this one…, with the other outlier you have seen also in luiza (see above).

Ok.

Aiyra wins for big and medium. What about small? We can’t see, let’s do a plot only about it.

library(ggplot2);
ggplot(k2[k2$input == "small",], aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  scale_colour_manual(values = c("green")) +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + facet_wrap(~platform);

img/cpp/first_plot_3.png

Aiyra loses. But pay the attention to the colors now (also .

Let’s do this same plot in PDF so you can include in your dissertation.

library(ggplot2);
ggplot(k2[k2$input == "small",], aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Version") +
  theme_bw() + ylim(0,NA) + 
  facet_wrap(~platform) +
#  title ("teste") +
  theme(legend.position="none"); # no legend since we have only small

img/cpp/aiyra-pj-pjflex_small.pdf

2016-05-27 CPP Comparison Version 2 (O3) (figures)

Merge everything in a single CSV file:

cd results/cpp_comparison
head -n1 cpp_guarani_bateria01.csv > all_but_first_v2.csv
for i in cpp_guarani_o3_v1.csv cpp_orion_o3_v1.csv cpp_luiza_o3_v1.csv; do
  tail -n+2 $i
done >> all_but_first_v2.csv

Now, let’s read the file all_but_first_v2.csv.

df <- read.csv ("results/cpp_comparison/all_but_first_v2.csv");
df[df$version == "aiyra",]$time = df[df$version == "aiyra",]$time/1000000;
summary(df);
head(df);
      name       run.no.in.std.order     run.no      run.no.std.rp  
 Min.   :  1.0   Min.   :1           Min.   :  1.0   Min.   :1.100  
 1st Qu.: 68.0   1st Qu.:3           1st Qu.: 68.0   1st Qu.:3.160  
 Median :135.5   Median :5           Median :135.5   Median :5.225  
 Mean   :135.5   Mean   :5           Mean   :135.5   Mean   :5.290  
 3rd Qu.:203.0   3rd Qu.:7           3rd Qu.:203.0   3rd Qu.:7.300  
 Max.   :270.0   Max.   :9           Max.   :270.0   Max.   :9.900  
   version       input          time              platform  
 aiyra :270   big   :270   Min.   :  0.00667   guarani:270  
 pj    :270   medium:270   1st Qu.:  0.17389   luiza  :270  
 pjflex:270   small :270   Median :  9.56102   orion1 :270  
                           Mean   : 30.59050                
                           3rd Qu.: 65.37543                
                           Max.   :117.99605
  name run.no.in.std.order run.no run.no.std.rp version  input       time
1    1                   4      1           4.1   aiyra medium  12.370982
2    2                   9      2           9.1  pjflex    big  74.821614
3    3                   2      3           2.1      pj  small   0.007406
4    4                   5      4           5.1      pj medium   7.876383
5    5                   7      5           7.1   aiyra    big 114.246921
6    6                   6      6           6.1  pjflex medium   9.719990
  platform
1  guarani
2  guarani
3  guarani
4  guarani
5  guarani
6  guarani

Let’s average measurements:

library(dplyr);
k <- df %>%
     select(version, input, time, platform) %>%
     group_by(version, input, platform) %>%
     summarize(N = n(),
               time_avg = mean(time),
               time_se = 3*sd(time)/sqrt(N)) %>%
     as.data.frame();
k
Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
   version  input platform  N     time_avg      time_se
1    aiyra    big  guarani 30 1.133892e+02 1.064124e+00
2    aiyra    big    luiza 30 8.342057e+01 1.016463e+00
3    aiyra    big   orion1 30 1.101503e+02 1.287671e+00
4    aiyra medium  guarani 30 1.252370e+01 2.178345e-01
5    aiyra medium    luiza 30 9.514203e+00 1.006092e-01
6    aiyra medium   orion1 30 1.281804e+01 1.351626e-01
7    aiyra  small  guarani 30 1.732774e-01 1.222508e-03
8    aiyra  small    luiza 30 1.785216e-01 8.042493e-04
9    aiyra  small   orion1 30 2.043394e-01 1.132887e-03
10      pj    big  guarani 30 6.039525e+01 1.897985e-01
11      pj    big    luiza 30 6.632887e+01 1.189113e+00
12      pj    big   orion1 30 6.207255e+01 5.356949e-02
13      pj medium  guarani 30 7.758835e+00 2.400090e-02
14      pj medium    luiza 30 7.625045e+00 9.210520e-02
15      pj medium   orion1 30 7.968472e+00 8.787838e-03
16      pj  small  guarani 30 9.583233e-03 1.537028e-03
17      pj  small    luiza 30 7.103467e-03 2.672619e-04
18      pj  small   orion1 30 9.067700e-03 2.600868e-05
19  pjflex    big  guarani 30 7.519103e+01 1.411454e-01
20  pjflex    big    luiza 30 8.511038e+01 2.203729e+00
21  pjflex    big   orion1 30 8.152386e+01 6.579497e-01
22  pjflex medium  guarani 30 9.678624e+00 2.369348e-02
23  pjflex medium    luiza 30 9.401344e+00 1.129723e-01
24  pjflex medium   orion1 30 1.045980e+01 8.515382e-02
25  pjflex  small  guarani 30 1.137983e-02 1.341304e-03
26  pjflex  small    luiza 30 8.607067e-03 4.886444e-04
27  pjflex  small   orion1 30 1.154073e-02 9.178358e-05
library(ggplot2);
ggplot(k, aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  scale_colour_manual(values = c("red","blue", "green"), name="Input Size") +
  theme_bw() + facet_wrap(~platform) +
  ylab("Average Execution Time (s)") +
  xlab("Version") +
  theme_bw() + ylim(0,NA) + 
  theme(legend.position="right"); # no legend since we have only small

img/cpp/aiyra-pj-pjflex_overview_v2_o3.pdf

What about small? We can’t see, let’s do a plot only about it.

library(ggplot2);
ggplot(k[k$input == "small",], aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  scale_colour_manual(values = c("green")) +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  theme_bw() + facet_wrap(~platform);

img/cpp/first_plot_3.png

Let’s do this same plot in PDF so you can include in your dissertation.

library(ggplot2);
ggplot(k[k$input == "small",], aes(x=version, y=time_avg, color=input)) +
  geom_point() +
  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Version") +
  theme_bw() + ylim(0,NA) + 
  facet_wrap(~platform) +
#  title ("teste") +
  theme(legend.position="none"); # no legend since we have only small

img/cpp/aiyra-pj-pjflex_small_v2_o3.pdf

2016-05-17 Batch size (figures)

For local analysis:

head -n1 results/batch_size/bateria03_orion1_batch.csv > results/batch_size/local.csv
for file in "results/batch_size/bateria03_orion1_batch.csv results/batch_size/last_guarani_batch.csv results/batch_size/ultimabat_luiza_batch.csv"; do
    tail -n+2 $file
done >> results/batch_size/local.csv

For remote against local analysis:

head -n1 results/batch_size/bateria03_orion1_batch.csv > results/batch_size/remote.csv
for file in "results/batch_size/bateria03_orion1_batch.csv results/batch_size/last_guarani_batch.csv results/batch_size/ultimabat_luiza_batch.csv results/batch_size/bateria01_batch_guarani_orion.csv"; do
    tail -n+2 $file
done >> results/batch_size/remote.csv

Local

Now, let’s read the file local.csv.

df <- read.csv ("results/batch_size/local.csv");
df <- df[complete.cases(df),];
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;
head(df);
  name run.no.in.std.order run.no run.no.std.rp  input batch   sim_time
2    1                   7      1           7.1  small     D   0.983336
3    2                  15      2          15.1    big     F 950.015134
4    3                   4      3           4.1  small     C   0.872588
5    4                  10      4          10.1  small     E   1.231774
6    5                  13      5          13.1  small     F   1.633785
7    6                  11      6          11.1 medium     E  93.070116
  insert_time    max_mem platform
2    0.202030   3.073425   orion1
3  401.292651 257.339241   orion1
4    0.227659   3.226997   orion1
5    0.256816   3.005371   orion1
6    0.285011   2.969864   orion1
7   52.138141  67.714127   orion1

Let’s average measurements:

library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               time_avg = mean(sim_time),
               time_se = 3*sd(sim_time)/sqrt(N)) %>%
     as.data.frame();
k
Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
    input batch platform  N     time_avg      time_se
1     big     B   orion1 10 1107.3227606 6.658932e+00
2     big     C  guarani 30  824.7546429 2.852808e+00
3     big     C    luiza 30  772.6744859 2.081221e+00
4     big     C   orion1 10 1061.8112710 5.241099e+00
5     big     D  guarani 30  798.8587828 2.957737e+00
6     big     D    luiza 30  796.5267459 1.858931e+02
7     big     D   orion1 10 1026.6392053 5.417082e+00
8     big     E  guarani 30  780.5554274 2.658481e+00
9     big     E    luiza 30  718.6369158 1.532428e+00
10    big     E   orion1 10  997.8075534 5.549207e+00
11    big     F  guarani 30  771.4025417 2.914140e+00
12    big     F    luiza 30  711.4194856 1.529460e+00
13    big     F   orion1 10  946.2533760 3.886924e+00
14 medium     A  guarani 30  100.3892080 4.840424e-01
15 medium     A    luiza 30   93.9088443 1.835017e-01
16 medium     B  guarani 30   99.2527799 4.786725e-01
17 medium     B    luiza 30   93.1713426 1.855400e-01
18 medium     B   orion1 10  106.2926635 3.040111e+00
19 medium     C  guarani 30   95.0724998 5.849231e-01
20 medium     C    luiza 30   88.3279745 2.877727e-01
21 medium     C   orion1 10  111.3229354 9.112576e-01
22 medium     D  guarani 30   96.9286011 5.696124e-01
23 medium     D    luiza 30   87.5662485 2.059891e-01
24 medium     D   orion1 10  117.2014197 8.836928e-01
25 medium     E  guarani 30   92.5820661 5.123305e-01
26 medium     E    luiza 30   86.1899532 1.962095e-01
27 medium     E   orion1 10   97.4553447 7.025395e+00
28 medium     F  guarani 30   92.4204530 4.873207e-01
29 medium     F    luiza 30   82.4328944 2.428668e-01
30 medium     F   orion1 10  109.6810972 9.403754e-01
31  small     A  guarani 30    0.4956575 6.162893e-03
32  small     A    luiza 30    0.4502067 3.531458e-03
33  small     B  guarani 30    0.5281257 9.840373e-03
34  small     B    luiza 30    0.4528172 2.666560e-03
35  small     B   orion1 10    0.7557264 2.225693e-02
36  small     C  guarani 30    0.5544556 1.253729e-02
37  small     C    luiza 30    0.4709659 2.894954e-03
38  small     C   orion1 10    0.8611910 2.868396e-02
39  small     D  guarani 30    0.5818487 1.325817e-02
40  small     D    luiza 30    0.5158810 3.757003e-03
41  small     D   orion1 10    0.9929203 2.956740e-02
42  small     E  guarani 30    0.6681839 1.538822e-02
43  small     E    luiza 30    0.6011176 3.310887e-03
44  small     E   orion1 10    1.2279947 2.983281e-02
45  small     F  guarani 30    0.8698209 9.727556e-03
46  small     F    luiza 30    0.7695288 8.467496e-03
47  small     F   orion1 10    1.6951738 3.660555e-02
library(ggplot2);
ggplot(k[k$input == "big",], aes(x=batch, y=time_avg)) +
  geom_point(size=.3) +
#  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(input~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_v1.pdf

Very strange behavior on luiza, batch D because of the high variability. Let’s see the data:

df[df$input == "big" & df$platform == "luiza" & df$batch == "D",];
     name run.no.in.std.order run.no run.no.std.rp input batch  sim_time
697     4                  12      4         12.10   big     D  734.4769
720    27                  12     27         12.20   big     D  731.5318
736    43                  12     43         12.30   big     D  732.5503
751    58                  12     58         12.40   big     D  739.5146
780    87                  12     87         12.50   big     D  727.4080
784    91                  12     91         12.60   big     D  733.0804
813   120                  12    120         12.70   big     D  737.1977
829   136                  12    136         12.80   big     D  732.3844
843   150                  12    150         12.90   big     D  734.6302
864   171                  12    171         12.10   big     D  730.9828
884   191                  12    191         12.11   big     D  736.7343
901   208                  12    208         12.12   big     D  732.6899
922   229                  12    229         12.13   big     D  732.8134
931   238                  12    238         12.14   big     D  734.9025
956   263                  12    263         12.15   big     D  739.9419
974   281                  12    281         12.16   big     D  738.1100
982   289                  12    289         12.17   big     D  732.5317
1017  324                  12    324         12.18   big     D  737.1109
1026  333                  12    333         12.19   big     D 2593.4167
1044  351                  12    351         12.20   big     D  736.5624
1054  361                  12    361         12.21   big     D  734.8892
1076  383                  12    383         12.22   big     D  727.3046
1095  402                  12    402         12.23   big     D  734.8349
1123  430                  12    430         12.24   big     D  733.0345
1141  448                  12    448         12.25   big     D  740.3488
1157  464                  12    464         12.26   big     D  735.1516
1175  482                  12    482         12.27   big     D  736.5852
1187  494                  12    494         12.28   big     D  734.3452
1200  507                  12    507         12.29   big     D  737.0423
1224  531                  12    531         12.30   big     D  733.6951
     insert_time  max_mem platform
697     423.1795 1020.664    luiza
720     419.9057 1020.664    luiza
736     419.1899 1020.664    luiza
751     422.4795 1020.666    luiza
780     415.7618 1020.663    luiza
784     420.4074 1020.666    luiza
813     422.9430 1020.663    luiza
829     419.7894 1020.663    luiza
843     420.1070 1020.665    luiza
864     419.8785 1020.667    luiza
884     422.2966 1020.664    luiza
901     421.6495 1020.664    luiza
922     421.8006 1020.664    luiza
931     420.3353 1020.663    luiza
956     426.4439 1020.665    luiza
974     419.1994 1020.664    luiza
982     421.6567 1020.663    luiza
1017    420.3747 1020.664    luiza
1026   2279.5546 1020.664    luiza
1044    423.0635 1020.663    luiza
1054    421.1642 1020.667    luiza
1076    417.9471 1020.664    luiza
1095    424.2948 1020.663    luiza
1123    416.0131 1020.664    luiza
1141    421.1605 1020.663    luiza
1157    422.0469 1020.663    luiza
1175    419.2871 1020.663    luiza
1187    421.2706 1020.664    luiza
1200    420.1514 1020.663    luiza
1224    421.4753 1020.664    luiza

Looks like there was a problem when inserting in the database. We’ll manually remove this outlier. You should mention this on the text.

dff <- df[df$name != 333,];
k2 <- dff %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               time_avg = mean(sim_time),
               time_se = 3*sd(sim_time)/sqrt(N)) %>%
     as.data.frame();
k2
    input batch platform  N     time_avg     time_se
1     big     B   orion1 10 1107.3227606 6.658931757
2     big     C  guarani 30  824.7546429 2.852807810
3     big     C    luiza 30  772.6744859 2.081221420
4     big     C   orion1 10 1061.8112710 5.241098578
5     big     D  guarani 29  798.5691785 2.926487070
6     big     D    luiza 29  734.5650227 1.771873261
7     big     D   orion1 10 1026.6392053 5.417082452
8     big     E  guarani 30  780.5554274 2.658481151
9     big     E    luiza 30  718.6369158 1.532427809
10    big     E   orion1 10  997.8075534 5.549207079
11    big     F  guarani 30  771.4025417 2.914139674
12    big     F    luiza 30  711.4194856 1.529459777
13    big     F   orion1 10  946.2533760 3.886923906
14 medium     A  guarani 30  100.3892080 0.484042387
15 medium     A    luiza 30   93.9088443 0.183501686
16 medium     B  guarani 30   99.2527799 0.478672535
17 medium     B    luiza 30   93.1713426 0.185540020
18 medium     B   orion1 10  106.2926635 3.040111400
19 medium     C  guarani 30   95.0724998 0.584923080
20 medium     C    luiza 30   88.3279745 0.287772713
21 medium     C   orion1 10  111.3229354 0.911257568
22 medium     D  guarani 30   96.9286011 0.569612415
23 medium     D    luiza 30   87.5662485 0.205989135
24 medium     D   orion1 10  117.2014197 0.883692783
25 medium     E  guarani 30   92.5820661 0.512330528
26 medium     E    luiza 30   86.1899532 0.196209532
27 medium     E   orion1 10   97.4553447 7.025394616
28 medium     F  guarani 30   92.4204530 0.487320713
29 medium     F    luiza 30   82.4328944 0.242866845
30 medium     F   orion1 10  109.6810972 0.940375391
31  small     A  guarani 30    0.4956575 0.006162893
32  small     A    luiza 30    0.4502067 0.003531458
33  small     B  guarani 30    0.5281257 0.009840373
34  small     B    luiza 30    0.4528172 0.002666560
35  small     B   orion1 10    0.7557264 0.022256931
36  small     C  guarani 30    0.5544556 0.012537292
37  small     C    luiza 30    0.4709659 0.002894954
38  small     C   orion1 10    0.8611910 0.028683959
39  small     D  guarani 30    0.5818487 0.013258166
40  small     D    luiza 30    0.5158810 0.003757003
41  small     D   orion1 10    0.9929203 0.029567395
42  small     E  guarani 30    0.6681839 0.015388218
43  small     E    luiza 30    0.6011176 0.003310887
44  small     E   orion1 10    1.2279947 0.029832810
45  small     F  guarani 30    0.8698209 0.009727556
46  small     F    luiza 30    0.7695288 0.008467496
47  small     F   orion1 10    1.6951738 0.036605546
library(ggplot2);
ggplot(k2[k2$input == "big",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("red","blue", "green"), name="Input Size") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_wrap(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_big_v2.pdf

Much better now.

Let’s look to the medium size.

library(ggplot2);
ggplot(k2[k2$input == "medium",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("blue"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_medium_v1.pdf

library(ggplot2);
ggplot(k2[k2$input == "small",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_small_v1.pdf

It’s different for small. Maybe it should be interesting to plot in the graph the three inputs, so we can see the different trend for small.

library(ggplot2);
ggplot(k2, aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Input Size") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(input~platform, scales="free_y") +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/local_all_inputs_v1.pdf

Great, Tais will write in the text that she explains all three first plots, then to show the different trend, she uses local_all_inputs_v1 plot.

Remember pdfcrop.

Remote

Now, let’s read the file remote.csv.

df <- read.csv ("results/batch_size/remote.csv");
df <- df[complete.cases(df),];
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;
head(df);

dft1 <- df[df$platform != "guarani_orion",];
dft2 <- df[df$platform == "guarani_orion",];
dft2$platform <- "remote";
df <- rbind(dft2, dft1);
head(df);
  name run.no.in.std.order run.no run.no.std.rp  input batch   sim_time
2    1                   7      1           7.1  small     D   0.983336
3    2                  15      2          15.1    big     F 950.015134
4    3                   4      3           4.1  small     C   0.872588
5    4                  10      4          10.1  small     E   1.231774
6    5                  13      5          13.1  small     F   1.633785
7    6                  11      6          11.1 medium     E  93.070116
  insert_time    max_mem platform
2    0.202030   3.073425   orion1
3  401.292651 257.339241   orion1
4    0.227659   3.226997   orion1
5    0.256816   3.005371   orion1
6    0.285011   2.969864   orion1
7   52.138141  67.714127   orion1
     name run.no.in.std.order run.no run.no.std.rp  input batch  sim_time
1235    1                   8      1           8.1 medium     C  95.72919
1236    2                  18      2          18.1    big     F 765.60935
1237    3                   5      3           5.1 medium     B  98.47585
1238    4                  12      4          12.1    big     D 792.44906
1239    5                  16      5          16.1  small     F   0.90444
1241    7                  15      7          15.1    big     E 775.00969
     insert_time     max_mem platform
1235    63.87650  264.647064   remote
1236   480.09590  257.338707   remote
1237    62.73813  527.779396   remote
1238   482.47624 1020.575333   remote
1239     0.20935    2.970314   remote
1241   482.44408  513.167862   remote

Let’s average measurements:

library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               time_avg = mean(sim_time),
               time_se = 3*sd(sim_time)/sqrt(N)) %>%
     as.data.frame();
k[k$platform =="remote",]
    input batch platform  N    time_avg     time_se
5     big     C   remote 30 822.9541185 3.072009855
9     big     D   remote 30 793.6288847 2.793609013
13    big     E   remote 30 776.9857812 3.234269351
17    big     F   remote 30 765.5748142 2.850922209
20 medium     A   remote 30 100.0850588 0.438621535
24 medium     B   remote 30  98.6565419 0.657277641
28 medium     C   remote 30  94.7818702 0.528655405
32 medium     D   remote 30  96.5232222 0.502179305
36 medium     E   remote 30  92.0910381 0.393890608
40 medium     F   remote 30  92.0036549 0.480806287
43  small     A   remote 30   0.4990605 0.007578415
47  small     B   remote 30   0.5296416 0.009928985
51  small     C   remote 30   0.5577120 0.010474202
55  small     D   remote 30   0.5934754 0.016890922
59  small     E   remote 30   0.6592223 0.014014033
63  small     F   remote 30   0.8761198 0.015386276

For remote, guarani had aiyra; orion1 had the database. Tais will check the scripts to confirm and check since this is very important.

So, since guarani was involved, let’s compare remote agains it because it means we took out the database from guarani and put in another machine (in this case orion). The objective of our analysis here is to see the network impact in the execution comparing with a situation where everything resides in the same machine.

Future work idea:

  • create a producer/consumer version where insert requests are put in a queue; another thread will be responsible for insertion. We could implement another plugin that is threaded.

So, let’s take only guarani and remote measurements.

k2 <- k[k$platform == "guarani" | k$platform == "remote",];
k2
    input batch platform  N    time_avg     time_se
2     big     C  guarani 30 824.7546429 2.852807810
5     big     C   remote 30 822.9541185 3.072009855
6     big     D  guarani 30 798.8587828 2.957736903
9     big     D   remote 30 793.6288847 2.793609013
10    big     E  guarani 30 780.5554274 2.658481151
13    big     E   remote 30 776.9857812 3.234269351
14    big     F  guarani 30 771.4025417 2.914139674
17    big     F   remote 30 765.5748142 2.850922209
18 medium     A  guarani 30 100.3892080 0.484042387
20 medium     A   remote 30 100.0850588 0.438621535
21 medium     B  guarani 30  99.2527799 0.478672535
24 medium     B   remote 30  98.6565419 0.657277641
25 medium     C  guarani 30  95.0724998 0.584923080
28 medium     C   remote 30  94.7818702 0.528655405
29 medium     D  guarani 30  96.9286011 0.569612415
32 medium     D   remote 30  96.5232222 0.502179305
33 medium     E  guarani 30  92.5820661 0.512330528
36 medium     E   remote 30  92.0910381 0.393890608
37 medium     F  guarani 30  92.4204530 0.487320713
40 medium     F   remote 30  92.0036549 0.480806287
41  small     A  guarani 30   0.4956575 0.006162893
43  small     A   remote 30   0.4990605 0.007578415
44  small     B  guarani 30   0.5281257 0.009840373
47  small     B   remote 30   0.5296416 0.009928985
48  small     C  guarani 30   0.5544556 0.012537292
51  small     C   remote 30   0.5577120 0.010474202
52  small     D  guarani 30   0.5818487 0.013258166
55  small     D   remote 30   0.5934754 0.016890922
56  small     E  guarani 30   0.6681839 0.015388218
59  small     E   remote 30   0.6592223 0.014014033
60  small     F  guarani 30   0.8698209 0.009727556
63  small     F   remote 30   0.8761198 0.015386276
library(ggplot2);
ggplot(k2[k2$input == "big",], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_big_v1.pdf

library(ggplot2);
ggplot(k2[k2$input == "medium",], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_medium_v1.pdf

library(ggplot2);
ggplot(k2[k2$input == "small",], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_small_v1.pdf

Tais just found out that guarani is fact remote. She will do the experiment again so we can correctly compare a local against a remove experiment. All plots regarding batch size will have to be re-plotted.

k2[k2$platform == "guarani",];
sum(10 * k2$time_avg)/60/60/24;
    input batch platform  N    time_avg     time_se
2     big     C  guarani 30 824.7546429 2.852807810
6     big     D  guarani 30 798.8587828 2.957736903
10    big     E  guarani 30 780.5554274 2.658481151
14    big     F  guarani 30 771.4025417 2.914139674
18 medium     A  guarani 30 100.3892080 0.484042387
21 medium     B  guarani 30  99.2527799 0.478672535
25 medium     C  guarani 30  95.0724998 0.584923080
29 medium     D  guarani 30  96.9286011 0.569612415
33 medium     E  guarani 30  92.5820661 0.512330528
37 medium     F  guarani 30  92.4204530 0.487320713
41  small     A  guarani 30   0.4956575 0.006162893
44  small     B  guarani 30   0.5281257 0.009840373
48  small     C  guarani 30   0.5544556 0.012537292
52  small     D  guarani 30   0.5818487 0.013258166
56  small     E  guarani 30   0.6681839 0.015388218
60  small     F  guarani 30   0.8698209 0.009727556
[1] 0.8672356

2016-05-17 Batch traces (figures)

Ok.

cd results/batch_time
#for each scenario
for scenario in `ls -1 | cut -d"_" -f1-3 | sort | uniq`; do
  #for each execution in temporal order
  CONTADOR=0
  for execution in `ls -1 ${scenario}* | sort --version-sort`; do
     NOME=`echo $execution | cut -d"_" -f1-3`_$CONTADOR.csv
     echo "$execution => $NOME"
     git mv $execution $NOME
     CONTADOR=$(($CONTADOR + 1))
  done
done

Let’s commit this.

cd results/batch_time
#for each scenario/execution
head -n1 trace-1G.paje_611585_luiza_19.csv | \
   sed -e "s/(microseconds)//g" \
       -e "s/(Operations)//" \
       -e "s/(Bytes)//" \
       -e "s/Size/Operations/" \
       -e "s/$/, input, parsize, platform, execution/" \
       -e "s/Time//g" \
       -e "s/Number//" \
       -e "s/ //g" | \
       tr '[:upper:]' '[:lower:]' > batch_traces.csv
for i in `ls -1 trace-*.csv`; do
  #input
  INPUT=`echo $i | cut -d"." -f1 | sed -e "s/trace-//"`
  if [ $INPUT = "1G" ]; then
    INPUT="big"
  elif [ $INPUT = "128M" ]; then
    INPUT="medium"
  elif [ $INPUT = "128K" ]; then
    INPUT="small"
  else
    echo "Error #1"
    exit
  fi

  #size
  SIZE=`echo $i | cut -d"_" -f2`
  
  #platform
  PLATFORM=`echo $i | cut -d"_" -f3`

  #platform
  EXECUTION=`echo $i | cut -d"_" -f4 | sed "s/\.csv//"`

  #echo $i $INPUT $SIZE $PLATFORM $EXECUTION

  tail -n+2 $i | sed -e "s/$/,$INPUT,$SIZE,$PLATFORM,$EXECUTION/" \
                     -e "s/small,85,/small,F,/" \
                     -e "s/small,170,/small,E,/" \
                     -e "s/small,340,/small,D,/" \
                     -e "s/small,681,/small,C,/" \
                     -e "s/small,1363,/small,B,/" \
                     -e "s/small,2727,/small,A,/" \
                     -e "s/medium,78957,/medium,F,/" \
                     -e "s/medium,157914,/medium,E,/" \
                     -e "s/medium,315828,/medium,D,/" \
                     -e "s/medium,631657,/medium,C,/" \
                     -e "s/medium,1263314,/medium,B,/" \
                     -e "s/medium,2526628,/medium,A,/" \
                     -e "s/big,611585,/big,F,/" \
                     -e "s/big,1223170,/big,E,/" \
                     -e "s/big,2446340,/big,D,/" \
                     -e "s/big,4892680,/big,C,/" \
                     -e "s/big,9785361,/big,B,/"
done >> batch_traces.csv

Great, finally.

Let’s read the batch_traces.csv with R.

df <- read.csv ("results/batch_time/batch_traces.csv");
df$start = df$start / 1000000;
df$end = df$end / 1000000;
head(df);
  batch    start      end duration operations  size input parsize      platform
1     1 0.260947 0.325175    64228       1363 47115 small       B guarani-orion
2     2 0.438277 0.494230    55953       1363 49281 small       B guarani-orion
3     3 0.514935 0.520468     5533          2   579 small       B guarani-orion
4     1 0.264858 0.331881    67023       1363 47115 small       B guarani-orion
5     2 0.455022 0.511381    56359       1363 49281 small       B guarani-orion
6     3 0.534869 0.536691     1822          2   579 small       B guarani-orion
  execution
1         0
2         0
3         0
4        10
5        10
6        10
library(ggplot2);
ggplot(df[df$input == "small" & df$execution <= 9,], aes(x=start, xend=end, y=execution, yend=execution, color=input)) +
  geom_segment(size=1.1) +
  theme_bw() + facet_grid (parsize~platform) +
  scale_colour_manual(values = c("green"), name="Input Size") +
  scale_y_continuous(breaks=c(0,3,6,9)) +
  ylab("Run (order)") +
  xlab("Time (s)") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="none"); # no legend since we have only small

img/batch_time/batch_time_small_v1.pdf

library(ggplot2);
ggplot(df[df$input == "medium" & df$execution <= 9,], aes(x=start, xend=end, y=execution, yend=execution, color=input)) +
  geom_segment(size=1.1) +
  theme_bw() + facet_grid (parsize~platform) +
  scale_colour_manual(values = c("blue"), name="Input Size") +
  scale_y_continuous(breaks=c(0,3,6,9)) +
  ylab("Run (order)") +
  xlab("Time (s)") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="none"); # no legend since we have only small

img/batch_time/batch_time_medium_v1.pdf

library(ggplot2);
ggplot(df[df$input == "big" & df$execution <= 9,], aes(x=start, xend=end, y=execution, yend=execution, color=input)) +
  geom_segment(size=1.1) +
  theme_bw() + facet_grid (parsize~platform) +
  scale_colour_manual(values = c("red"), name="Input Size") +
  scale_y_continuous(breaks=c(0,3,6,9)) +
  ylab("Run (order)") +
  xlab("Time (s)") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="none"); # no legend since we have only small

img/batch_time/batch_time_big_v1.pdf

2016-05-17 Batch mem (figures)

Let’s use local measurements.

df <- read.csv ("results/batch_size/local.csv");
df <- df[complete.cases(df),];
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;
head(df);
  name run.no.in.std.order run.no run.no.std.rp  input batch   sim_time
2    1                   7      1           7.1  small     D   0.983336
3    2                  15      2          15.1    big     F 950.015134
4    3                   4      3           4.1  small     C   0.872588
5    4                  10      4          10.1  small     E   1.231774
6    5                  13      5          13.1  small     F   1.633785
7    6                  11      6          11.1 medium     E  93.070116
  insert_time    max_mem platform
2    0.202030   3.073425   orion1
3  401.292651 257.339241   orion1
4    0.227659   3.226997   orion1
5    0.256816   3.005371   orion1
6    0.285011   2.969864   orion1
7   52.138141  67.714127   orion1

Let’s take the max of the max:

library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               mem_max = max(max_mem)) %>%
     as.data.frame();
k
    input batch platform  N     mem_max
1     big     B   orion1 10 4085.928391
2     big     C  guarani 30 2041.474709
3     big     C    luiza 30 2041.564964
4     big     C   orion1 10 2041.475166
5     big     D  guarani 30 1020.575508
6     big     D    luiza 30 1020.667267
7     big     D   orion1 10 1020.575981
8     big     E  guarani 30  513.167862
9     big     E    luiza 30  513.259621
10    big     E   orion1 10  513.168320
11    big     F  guarani 30  257.338882
12    big     F    luiza 30  257.430183
13    big     F   orion1 10  257.339264
14 medium     A  guarani 30 1036.056984
15 medium     A    luiza 30 1036.147446
16 medium     B  guarani 30  527.779579
17 medium     B    luiza 30  528.854622
18 medium     B   orion1 10  527.780113
19 medium     C  guarani 30  264.647240
20 medium     C    luiza 30  264.737885
21 medium     C   orion1 10  264.647789
22 medium     D  guarani 30  133.295044
23 medium     D    luiza 30  133.386436
24 medium     D   orion1 10  133.295593
25 medium     E  guarani 30   67.714485
26 medium     E    luiza 30   67.806183
27 medium     E   orion1 10   67.714890
28 medium     F  guarani 30   35.100777
29 medium     F    luiza 30   35.979355
30 medium     F   orion1 10   35.101166
31  small     A  guarani 30    4.062813
32  small     A    luiza 30    4.160812
33  small     B  guarani 30    5.068680
34  small     B    luiza 30    3.606918
35  small     B   orion1 10    3.508286
36  small     C  guarani 30    3.227562
37  small     C    luiza 30    3.325630
38  small     C   orion1 10    3.227089
39  small     D  guarani 30    4.634178
40  small     D    luiza 30    3.172089
41  small     D   orion1 10    3.073601
42  small     E  guarani 30    4.566124
43  small     E    luiza 30    3.104179
44  small     E   orion1 10    3.005554
45  small     F  guarani 30    2.970490
46  small     F    luiza 30    3.068680
47  small     F   orion1 10    2.970024
library(ggplot2);
ggplot(k[k$input == "big",], aes(x=batch, y=mem_max, fill=input)) +
  geom_bar(stat="identity") +
  scale_fill_manual(values = c("red","blue", "green"), name="Input Size") +
  ylab("Maximum Memory Utilization (Megabytes)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_mem_big_v1.pdf

library(ggplot2);
ggplot(k[k$input == "medium",], aes(x=batch, y=mem_max, fill=input)) +
  geom_bar(stat="identity") +
  scale_fill_manual(values = c("blue"), name="Input Size") +
  ylab("Maximum Memory Utilization (Megabytes)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_mem_medium_v1.pdf

library(ggplot2);
ggplot(k[k$input == "small",], aes(x=batch, y=mem_max, fill=input)) +
  geom_bar(stat="identity") +
  scale_fill_manual(values = c("green"), name="Input Size") +
  ylab("Maximum Memory Utilization (Megabytes)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_mem_small_v1.pdf

2016-05-18 LOCAL Batch size (figures) with GUARANI

Tais has rerun the guarani local experiment:

head -n1 results/batch_size/bateria03_orion1_batch.csv > results/batch_size/local.csv
for file in "results/batch_size/bateria03_orion1_batch.csv results/batch_size/guarani_batch_local.csv results/batch_size/ultimabat_luiza_batch.csv"; do
    tail -n+2 $file
done >> results/batch_size/local.csv

Local

Now, let’s read the file local.csv.

df <- read.csv ("results/batch_size/local.csv");
df <- df[complete.cases(df),];
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;
head(df);
  name run.no.in.std.order run.no run.no.std.rp  input batch   sim_time
2    1                   7      1           7.1  small     D   0.983336
3    2                  15      2          15.1    big     F 950.015134
4    3                   4      3           4.1  small     C   0.872588
5    4                  10      4          10.1  small     E   1.231774
6    5                  13      5          13.1  small     F   1.633785
7    6                  11      6          11.1 medium     E  93.070116
  insert_time    max_mem platform
2    0.202030   3.073425   orion1
3  401.292651 257.339241   orion1
4    0.227659   3.226997   orion1
5    0.256816   3.005371   orion1
6    0.285011   2.969864   orion1
7   52.138141  67.714127   orion1
library(dplyr);
dff <- df[df$name != 333,];
k2 <- dff %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               time_avg = mean(sim_time),
               time_se = 3*sd(sim_time)/sqrt(N)) %>%
     as.data.frame();
k2
Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
    input batch platform  N     time_avg     time_se
1     big     B   orion1 10 1107.3227606 6.658931757
2     big     C  guarani 10  872.7257096 4.335206802
3     big     C    luiza 30  772.6744859 2.081221420
4     big     C   orion1 10 1061.8112710 5.241098578
5     big     D  guarani 10  847.1885084 5.042743795
6     big     D    luiza 29  734.5650227 1.771873261
7     big     D   orion1 10 1026.6392053 5.417082452
8     big     E  guarani 10  823.6926476 4.388586020
9     big     E    luiza 30  718.6369158 1.532427809
10    big     E   orion1 10  997.8075534 5.549207079
11    big     F  guarani 10  812.7665036 5.007015306
12    big     F    luiza 30  711.4194856 1.529459777
13    big     F   orion1 10  946.2533760 3.886923906
14 medium     A    luiza 30   93.9088443 0.183501686
15 medium     B  guarani 10  108.8424911 2.057405430
16 medium     B    luiza 30   93.1713426 0.185540020
17 medium     B   orion1 10  106.2926635 3.040111400
18 medium     C  guarani 10  103.9807552 0.874128073
19 medium     C    luiza 30   88.3279745 0.287772713
20 medium     C   orion1 10  111.3229354 0.911257568
21 medium     D  guarani 10  105.3337499 1.950162087
22 medium     D    luiza 30   87.5662485 0.205989135
23 medium     D   orion1 10  117.2014197 0.883692783
24 medium     E  guarani 10  100.5071578 1.524136028
25 medium     E    luiza 30   86.1899532 0.196209532
26 medium     E   orion1 10   97.4553447 7.025394616
27 medium     F  guarani 10   96.9679865 1.292743754
28 medium     F    luiza 30   82.4328944 0.242866845
29 medium     F   orion1 10  109.6810972 0.940375391
30  small     A    luiza 30    0.4502067 0.003531458
31  small     B  guarani 10    2.2563874 0.160313064
32  small     B    luiza 30    0.4528172 0.002666560
33  small     B   orion1 10    0.7557264 0.022256931
34  small     C  guarani 10    2.1407330 0.286351284
35  small     C    luiza 30    0.4709659 0.002894954
36  small     C   orion1 10    0.8611910 0.028683959
37  small     D  guarani 10    2.3501718 0.221163624
38  small     D    luiza 30    0.5158810 0.003757003
39  small     D   orion1 10    0.9929203 0.029567395
40  small     E  guarani 10    2.6943502 0.183633814
41  small     E    luiza 30    0.6011176 0.003310887
42  small     E   orion1 10    1.2279947 0.029832810
43  small     F  guarani 10    3.6281551 0.275158027
44  small     F    luiza 30    0.7695288 0.008467496
45  small     F   orion1 10    1.6951738 0.036605546
library(ggplot2);
ggplot(k2[k2$input == "big",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("red","blue", "green"), name="Input Size") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_wrap(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_big_v2_2.pdf

Much better now.

Let’s look to the medium size.

library(ggplot2);
ggplot(k2[k2$input == "medium",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("blue"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_medium_v1_2.pdf

library(ggplot2);
ggplot(k2[k2$input == "small",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_small_v1_2.pdf

2016-05-18 REMOTE vs LOCAL Batch size (figures) with GUARANI

Tais has rerun the guarani local experiment:

head -n1 results/batch_size/bateria03_orion1_batch.csv > results/batch_size/remote.csv
for file in "results/batch_size/guarani_batch_local.csv results/batch_size/bateria01_batch_guarani_orion.csv"; do
    tail -n+2 $file
done >> results/batch_size/remote.csv

Remote

df <- read.csv ("results/batch_size/remote.csv");
df <- df[complete.cases(df),];
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;
head(df);

dft1 <- df[df$platform != "guarani_orion",];
dft2 <- df[df$platform == "guarani_orion",];
dft2$platform <- "remote";
df <- rbind(dft2, dft1);
head(df);
  name run.no.in.std.order run.no run.no.std.rp  input batch   sim_time
2    1                   7      1           7.1  small     D   1.977549
3    2                  15      2          15.1    big     F 808.547815
4    3                   4      3           4.1  small     C   2.346436
5    4                  10      4          10.1  small     E   2.539559
6    5                  13      5          13.1  small     F   3.815750
7    6                  11      6          11.1 medium     E  99.211871
  insert_time    max_mem platform
2    1.367572   3.074310  guarani
3  524.050177 257.339867  guarani
4    1.833535   3.227829  guarani
5    1.835741   4.566216  guarani
6    2.854051   2.970589  guarani
7   68.003143  67.715302  guarani
    name run.no.in.std.order run.no run.no.std.rp  input batch  sim_time
153    1                   8      1           8.1 medium     C  95.72919
154    2                  18      2          18.1    big     F 765.60935
155    3                   5      3           5.1 medium     B  98.47585
156    4                  12      4          12.1    big     D 792.44906
157    5                  16      5          16.1  small     F   0.90444
159    7                  15      7          15.1    big     E 775.00969
    insert_time     max_mem platform
153    63.87650  264.647064   remote
154   480.09590  257.338707   remote
155    62.73813  527.779396   remote
156   482.47624 1020.575333   remote
157     0.20935    2.970314   remote
159   482.44408  513.167862   remote

Let’s average measurements:

library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               time_avg = mean(sim_time),
               time_se = 3*sd(sim_time)/sqrt(N)) %>%
     as.data.frame();
k[k$platform =="remote",]
    input batch platform  N    time_avg     time_se
2     big     C   remote 30 822.9541185 3.072009855
4     big     D   remote 30 793.6288847 2.793609013
6     big     E   remote 30 776.9857812 3.234269351
8     big     F   remote 30 765.5748142 2.850922209
9  medium     A   remote 30 100.0850588 0.438621535
11 medium     B   remote 30  98.6565419 0.657277641
13 medium     C   remote 30  94.7818702 0.528655405
15 medium     D   remote 30  96.5232222 0.502179305
17 medium     E   remote 30  92.0910381 0.393890608
19 medium     F   remote 30  92.0036549 0.480806287
20  small     A   remote 30   0.4990605 0.007578415
22  small     B   remote 30   0.5296416 0.009928985
24  small     C   remote 30   0.5577120 0.010474202
26  small     D   remote 30   0.5934754 0.016890922
28  small     E   remote 30   0.6592223 0.014014033
30  small     F   remote 30   0.8761198 0.015386276
k2 <- k; # just copy
library(ggplot2);
ggplot(k2[k2$input == "big",], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_big_v1_2.pdf

library(ggplot2);
ggplot(k2[k2$input == "medium",], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_medium_v1_2.pdf

library(ggplot2);
ggplot(k2[k2$input == "small",], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ggtitle ("TBD") +
  theme_bw() +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_small_v1_2.pdf

2016-05-19 Batch size - SimulTime =local experiment

head -n1 results/batch_size/bateria03_orion1_batch.csv > results/batch_size.csv
for file in "results/batch_size/bateria03_orion1_batch.csv
             results/batch_size/guarani_batch_local.csv
             results/batch_size/ultimabat_luiza_batch.csv
             results/batch_size/remote_batch_v2.csv"; do
    tail -n+2 $file
done >> results/batch_size.csv
# read data
df <- read.csv ("results/batch_size.csv");

# remove outliers
df <- df[df$name != 333,];

# get rid of rows with NA
df <- df[complete.cases(df),];

# transform data types
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));

# transform to metrics we adopt
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;

# average things
library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               time_avg = mean(sim_time),
               time_se = 3*sd(sim_time)/sqrt(N)) %>%
     as.data.frame();
k
    input batch platform  N     time_avg     time_se
1     big     B   orion1 10 1107.3227606 6.658931757
2     big     C  guarani 10  872.7257096 4.335206802
3     big     C    luiza 30  772.6744859 2.081221420
4     big     C   orion1 10 1061.8112710 5.241098578
5     big     C   remote 10  933.2141182 2.988267836
6     big     D  guarani 10  847.1885084 5.042743795
7     big     D    luiza 29  734.5650227 1.771873261
8     big     D   orion1 10 1026.6392053 5.417082452
9     big     D   remote 10  907.5752921 5.282923988
10    big     E  guarani 10  823.6926476 4.388586020
11    big     E    luiza 30  718.6369158 1.532427809
12    big     E   orion1 10  997.8075534 5.549207079
13    big     E   remote 10  894.5280890 3.665965731
14    big     F  guarani 10  812.7665036 5.007015306
15    big     F    luiza 30  711.4194856 1.529459777
16    big     F   orion1 10  946.2533760 3.886923906
17    big     F   remote 10  885.2769965 5.915787926
18 medium     A    luiza 30   93.9088443 0.183501686
19 medium     B  guarani 10  108.8424911 2.057405430
20 medium     B    luiza 30   93.1713426 0.185540020
21 medium     B   orion1 10  106.2926635 3.040111400
22 medium     B   remote 10  113.9462601 1.431539231
23 medium     C  guarani 10  103.9807552 0.874128073
24 medium     C    luiza 30   88.3279745 0.287772713
25 medium     C   orion1 10  111.3229354 0.911257568
26 medium     C   remote 10  109.6834731 1.211775217
27 medium     D  guarani 10  105.3337499 1.950162087
28 medium     D    luiza 30   87.5662485 0.205989135
29 medium     D   orion1 10  117.2014197 0.883692783
30 medium     D   remote 10  111.3265373 1.068041247
31 medium     E  guarani 10  100.5071578 1.524136028
32 medium     E    luiza 30   86.1899532 0.196209532
33 medium     E   orion1 10   97.4553447 7.025394616
34 medium     E   remote 10  106.8520182 0.958682964
35 medium     F  guarani 10   96.9679865 1.292743754
36 medium     F    luiza 30   82.4328944 0.242866845
37 medium     F   orion1 10  109.6810972 0.940375391
38 medium     F   remote 10  107.1742940 0.875799714
39  small     A    luiza 30    0.4502067 0.003531458
40  small     B  guarani 10    2.2563874 0.160313064
41  small     B    luiza 30    0.4528172 0.002666560
42  small     B   orion1 10    0.7557264 0.022256931
43  small     B   remote 10    0.5661220 0.012160020
44  small     C  guarani 10    2.1407330 0.286351284
45  small     C    luiza 30    0.4709659 0.002894954
46  small     C   orion1 10    0.8611910 0.028683959
47  small     C   remote 10    0.6111423 0.025367505
48  small     D  guarani 10    2.3501718 0.221163624
49  small     D    luiza 30    0.5158810 0.003757003
50  small     D   orion1 10    0.9929203 0.029567395
51  small     D   remote 10    0.6209425 0.030249187
52  small     E  guarani 10    2.6943502 0.183633814
53  small     E    luiza 30    0.6011176 0.003310887
54  small     E   orion1 10    1.2279947 0.029832810
55  small     E   remote 10    0.6972343 0.017800904
56  small     F  guarani 10    3.6281551 0.275158027
57  small     F    luiza 30    0.7695288 0.008467496
58  small     F   orion1 10    1.6951738 0.036605546
59  small     F   remote 10    0.9144406 0.025488399
library(ggplot2);
ggplot(k[k$input == "big" & k$platform != "remote",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("red","blue", "green"), name="Input Size") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_wrap(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_big_v2_3.pdf

Much better now.

Let’s look to the medium size.

library(ggplot2);
ggplot(k[k$input == "medium" & k$platform != "remote",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("blue"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_medium_v1_3.pdf

library(ggplot2);
ggplot(k[k$input == "small" & k$platform != "remote",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_small_v1_3.pdf

2016-05-19 Batch size - SimulTime =remote experiment

See previous entry to define the k dataframe.

library(ggplot2);
ggplot(k[k$input == "big" & (k$platform == "remote" | k$platform == "guarani"),], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +  
  ylim(0,NA) +
  theme_bw() +
  facet_wrap (~input) +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_big_v1_3.pdf

library(ggplot2);
ggplot(k[k$input == "medium" & (k$platform == "remote" | k$platform == "guarani"),], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ylim(0,NA) +
  theme_bw() +
  facet_wrap (~input) +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_medium_v1_3.pdf

library(ggplot2);
ggplot(k[k$input == "small" & (k$platform == "remote" | k$platform == "guarani"),], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Execution Time (s)") +
  xlab("Batch Size") +
  ylim(0,NA) +
  theme_bw() +
  facet_wrap (~input) +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_small_v1_3.pdf

2016-05-19 Batch size - InsertTim =local experiment

# read data
df <- read.csv ("results/batch_size.csv");

# remove outliers
df <- df[df$name != 333,];

# get rid of rows with NA
df <- df[complete.cases(df),];

# transform data types
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));

# transform to metrics we adopt
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;

# average things
library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               insert = TRUE,
               time_avg = mean(insert_time),
               time_se = 3*sd(insert_time)/sqrt(N)) %>%
     as.data.frame();
k
    input batch platform  N insert    time_avg     time_se
1     big     B   orion1 10   TRUE 394.8574468 3.557103806
2     big     C  guarani 10   TRUE 534.3269313 4.824749241
3     big     C    luiza 30   TRUE 421.6144614 1.255230524
4     big     C   orion1 10   TRUE 396.1577301 3.274400300
5     big     C   remote 10   TRUE 599.6967210 3.868457861
6     big     D  guarani 10   TRUE 535.1566772 4.101957774
7     big     D    luiza 29   TRUE 420.8714772 1.233664350
8     big     D   orion1 10   TRUE 395.8077851 3.636207259
9     big     D   remote 10   TRUE 601.7075269 4.910124719
10    big     E  guarani 10   TRUE 529.6509552 4.826386198
11    big     E    luiza 30   TRUE 421.1397088 1.184888718
12    big     E   orion1 10   TRUE 398.4135196 3.627192856
13    big     E   remote 10   TRUE 604.4190996 3.840935282
14    big     F  guarani 10   TRUE 529.0666648 4.550948396
15    big     F    luiza 30   TRUE 420.4738213 1.088237307
16    big     F   orion1 10   TRUE 397.1500097 2.759267457
17    big     F   remote 10   TRUE 603.9960911 5.624003120
18 medium     A    luiza 30   TRUE  54.4073008 0.143803895
19 medium     B  guarani 10   TRUE  72.4099307 1.797782468
20 medium     B    luiza 30   TRUE  54.4708129 0.176676627
21 medium     B   orion1 10   TRUE  52.5826163 0.972669537
22 medium     B   remote 10   TRUE  78.5856955 1.409435932
23 medium     C  guarani 10   TRUE  71.9708902 0.853753903
24 medium     C    luiza 30   TRUE  54.4178920 0.160935005
25 medium     C   orion1 10   TRUE  52.3318907 0.587854008
26 medium     C   remote 10   TRUE  78.5598947 1.187744100
27 medium     D  guarani 10   TRUE  71.6063284 1.771691841
28 medium     D    luiza 30   TRUE  54.2420366 0.168531566
29 medium     D   orion1 10   TRUE  52.7724318 0.502898558
30 medium     D   remote 10   TRUE  78.2928266 0.836466508
31 medium     E  guarani 10   TRUE  69.4424091 1.596215916
32 medium     E    luiza 30   TRUE  54.4505100 0.160299959
33 medium     E   orion1 10   TRUE  51.7208634 0.789976725
34 medium     E   remote 10   TRUE  76.9269929 0.919332353
35 medium     F  guarani 10   TRUE  66.3695474 1.258704485
36 medium     F    luiza 30   TRUE  54.5286881 0.175224310
37 medium     F   orion1 10   TRUE  51.1558702 0.556399760
38 medium     F   remote 10   TRUE  76.8346945 0.933692843
39  small     A    luiza 30   TRUE   0.1247766 0.003816498
40  small     B  guarani 10   TRUE   1.8388487 0.166666337
41  small     B    luiza 30   TRUE   0.1075142 0.002161951
42  small     B   orion1 10   TRUE   0.1761141 0.013618157
43  small     B   remote 10   TRUE   0.1768186 0.005281896
44  small     C  guarani 10   TRUE   1.6548571 0.282081289
45  small     C    luiza 30   TRUE   0.1097431 0.002943911
46  small     C   orion1 10   TRUE   0.2005328 0.019515474
47  small     C   remote 10   TRUE   0.1817250 0.010894049
48  small     D  guarani 10   TRUE   1.7480169 0.216861958
49  small     D    luiza 30   TRUE   0.1165228 0.001832728
50  small     D   orion1 10   TRUE   0.2145494 0.014322845
51  small     D   remote 10   TRUE   0.1891820 0.011723498
52  small     E  guarani 10   TRUE   1.9830234 0.177730872
53  small     E    luiza 30   TRUE   0.1282831 0.003133362
54  small     E   orion1 10   TRUE   0.2582695 0.023829933
55  small     E   remote 10   TRUE   0.2007782 0.009725051
56  small     F  guarani 10   TRUE   2.6429075 0.268164182
57  small     F    luiza 30   TRUE   0.1565653 0.006299670
58  small     F   orion1 10   TRUE   0.2952458 0.015599726
59  small     F   remote 10   TRUE   0.2583806 0.016153458
library(ggplot2);
ggplot(k[k$input == "big" & k$platform != "remote",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("red","blue", "green"), name="Input Size") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Insertion Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_wrap(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_insert_big_v1_1.pdf

Much better now.

Let’s look to the medium size.

library(ggplot2);
ggplot(k[k$input == "medium" & k$platform != "remote",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("blue"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Insertion Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_insert_medium_v1_1.pdf

library(ggplot2);
ggplot(k[k$input == "small" & k$platform != "remote",], aes(x=batch, y=time_avg, color=input)) +
  geom_point(size=.3) +
  scale_colour_manual(values = c("green"), name="Input") +
  geom_errorbar(aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Insertion Time (s)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_insert_small_v1_1.pdf

2016-05-19 Batch size - InsertTim =remote experiment

See previous entry to load data and define the dataframe k.

    input batch platform  N insert    time_avg     time_se
1     big     B   orion1 10   TRUE 394.8574468 3.557103806
2     big     C  guarani 10   TRUE 534.3269313 4.824749241
3     big     C    luiza 30   TRUE 421.6144614 1.255230524
4     big     C   orion1 10   TRUE 396.1577301 3.274400300
5     big     C   remote 10   TRUE 599.6967210 3.868457861
6     big     D  guarani 10   TRUE 535.1566772 4.101957774
7     big     D    luiza 29   TRUE 420.8714772 1.233664350
8     big     D   orion1 10   TRUE 395.8077851 3.636207259
9     big     D   remote 10   TRUE 601.7075269 4.910124719
10    big     E  guarani 10   TRUE 529.6509552 4.826386198
11    big     E    luiza 30   TRUE 421.1397088 1.184888718
12    big     E   orion1 10   TRUE 398.4135196 3.627192856
13    big     E   remote 10   TRUE 604.4190996 3.840935282
14    big     F  guarani 10   TRUE 529.0666648 4.550948396
15    big     F    luiza 30   TRUE 420.4738213 1.088237307
16    big     F   orion1 10   TRUE 397.1500097 2.759267457
17    big     F   remote 10   TRUE 603.9960911 5.624003120
18 medium     A    luiza 30   TRUE  54.4073008 0.143803895
19 medium     B  guarani 10   TRUE  72.4099307 1.797782468
20 medium     B    luiza 30   TRUE  54.4708129 0.176676627
21 medium     B   orion1 10   TRUE  52.5826163 0.972669537
22 medium     B   remote 10   TRUE  78.5856955 1.409435932
23 medium     C  guarani 10   TRUE  71.9708902 0.853753903
24 medium     C    luiza 30   TRUE  54.4178920 0.160935005
25 medium     C   orion1 10   TRUE  52.3318907 0.587854008
26 medium     C   remote 10   TRUE  78.5598947 1.187744100
27 medium     D  guarani 10   TRUE  71.6063284 1.771691841
28 medium     D    luiza 30   TRUE  54.2420366 0.168531566
29 medium     D   orion1 10   TRUE  52.7724318 0.502898558
30 medium     D   remote 10   TRUE  78.2928266 0.836466508
31 medium     E  guarani 10   TRUE  69.4424091 1.596215916
32 medium     E    luiza 30   TRUE  54.4505100 0.160299959
33 medium     E   orion1 10   TRUE  51.7208634 0.789976725
34 medium     E   remote 10   TRUE  76.9269929 0.919332353
35 medium     F  guarani 10   TRUE  66.3695474 1.258704485
36 medium     F    luiza 30   TRUE  54.5286881 0.175224310
37 medium     F   orion1 10   TRUE  51.1558702 0.556399760
38 medium     F   remote 10   TRUE  76.8346945 0.933692843
39  small     A    luiza 30   TRUE   0.1247766 0.003816498
40  small     B  guarani 10   TRUE   1.8388487 0.166666337
41  small     B    luiza 30   TRUE   0.1075142 0.002161951
42  small     B   orion1 10   TRUE   0.1761141 0.013618157
43  small     B   remote 10   TRUE   0.1768186 0.005281896
44  small     C  guarani 10   TRUE   1.6548571 0.282081289
45  small     C    luiza 30   TRUE   0.1097431 0.002943911
46  small     C   orion1 10   TRUE   0.2005328 0.019515474
47  small     C   remote 10   TRUE   0.1817250 0.010894049
48  small     D  guarani 10   TRUE   1.7480169 0.216861958
49  small     D    luiza 30   TRUE   0.1165228 0.001832728
50  small     D   orion1 10   TRUE   0.2145494 0.014322845
51  small     D   remote 10   TRUE   0.1891820 0.011723498
52  small     E  guarani 10   TRUE   1.9830234 0.177730872
53  small     E    luiza 30   TRUE   0.1282831 0.003133362
54  small     E   orion1 10   TRUE   0.2582695 0.023829933
55  small     E   remote 10   TRUE   0.2007782 0.009725051
56  small     F  guarani 10   TRUE   2.6429075 0.268164182
57  small     F    luiza 30   TRUE   0.1565653 0.006299670
58  small     F   orion1 10   TRUE   0.2952458 0.015599726
59  small     F   remote 10   TRUE   0.2583806 0.016153458
library(ggplot2);
ggplot(k[k$input == "big" & (k$platform == "remote" | k$platform == "guarani"),], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Insertion Time (s)") +
  xlab("Batch Size") +  
  ylim(0,NA) +
  theme_bw() +
  facet_wrap (~input) +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_insert_big_v1_3.pdf

library(ggplot2);
ggplot(k[k$input == "medium" & (k$platform == "remote" | k$platform == "guarani"),], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Insertion Time (s)") +
  xlab("Batch Size") +
  ylim(0,NA) +
  theme_bw() +
  facet_wrap (~input) +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_insert_medium_v1_3.pdf

library(ggplot2);
ggplot(k[k$input == "small" & (k$platform == "remote" | k$platform == "guarani"),], aes(x=batch, y=time_avg, color=platform)) +
  geom_point(position=position_dodge(width=0.3)) +
  scale_colour_manual(values = c("red", "blue", "green"), name="Platform") +
  geom_errorbar(position=position_dodge(width=0.3), aes(ymin=time_avg-time_se, ymax=time_avg+time_se), width=.1) +
  ylab("Average Insertion Time (s)") +
  xlab("Batch Size") +
  ylim(0,NA) +
  theme_bw() +
  facet_wrap (~input) +
  theme(legend.position="right"); # no legend since we have only small

img/batch_size/remote_insert_small_v1_3.pdf

2016-05-19 Batch traces again

cd results/batch_time
#for each scenario
for scenario in `ls -1 *_guarani_*.csv | cut -d"_" -f1-3 | sort | uniq`; do
  #for each execution in temporal order
  CONTADOR=0
  for execution in `ls -1 ${scenario}* | grep -v orion | sort --version-sort`; do
     NOME=`echo $execution | cut -d"_" -f1-3`_$CONTADOR.csv
     echo "$execution => $NOME"
     git mv $execution $NOME
     CONTADOR=$(($CONTADOR + 1))
  done
done
unset CONTADOR
cd results/batch_time
#for each scenario/execution
head -n1 trace-1G.paje_611585_luiza_19.csv | \
   sed -e "s/(microseconds)//g" \
       -e "s/(Operations)//" \
       -e "s/(Bytes)//" \
       -e "s/Size/Operations/" \
       -e "s/$/, input, parsize, platform, execution/" \
       -e "s/Time//g" \
       -e "s/Number//" \
       -e "s/ //g" | \
       tr '[:upper:]' '[:lower:]' > batch_traces.csv
for i in `ls -1 trace-*.csv`; do
  #input
  INPUT=`echo $i | cut -d"." -f1 | sed -e "s/trace-//"`
  if [ $INPUT = "1G" ]; then
    INPUT="big"
  elif [ $INPUT = "128M" ]; then
    INPUT="medium"
  elif [ $INPUT = "128K" ]; then
    INPUT="small"
  else
    echo "Error #1"
    exit
  fi

  #size
  SIZE=`echo $i | cut -d"_" -f2`
  
  #platform
  PLATFORM=`echo $i | cut -d"_" -f3`

  #platform
  EXECUTION=`echo $i | cut -d"_" -f4 | sed "s/\.csv//"`

  #echo $i $INPUT $SIZE $PLATFORM $EXECUTION

  tail -n+2 $i | sed -e "s/$/,$INPUT,$SIZE,$PLATFORM,$EXECUTION/" \
                     -e "s/small,85,/small,F,/" \
                     -e "s/small,170,/small,E,/" \
                     -e "s/small,340,/small,D,/" \
                     -e "s/small,681,/small,C,/" \
                     -e "s/small,1363,/small,B,/" \
                     -e "s/small,2727,/small,A,/" \
                     -e "s/medium,78957,/medium,F,/" \
                     -e "s/medium,157914,/medium,E,/" \
                     -e "s/medium,315828,/medium,D,/" \
                     -e "s/medium,631657,/medium,C,/" \
                     -e "s/medium,1263314,/medium,B,/" \
                     -e "s/medium,2526628,/medium,A,/" \
                     -e "s/big,611585,/big,F,/" \
                     -e "s/big,1223170,/big,E,/" \
                     -e "s/big,2446340,/big,D,/" \
                     -e "s/big,4892680,/big,C,/" \
                     -e "s/big,9785361,/big,B,/"
done >> batch_traces.csv

Let’s read the batch_traces.csv with R.

df <- read.csv ("results/batch_time/batch_traces.csv");
df$start = df$start / 1000000;
df$end = df$end / 1000000;
head(df);
summary(df);
  batch    start      end duration operations  size input parsize platform
1     1 0.804555 1.309217   504662       1363 47115 small       B  guarani
2     2 1.443537 1.727040   283503       1363 49281 small       B  guarani
3     3 1.818504 1.960465   141961          2   579 small       B  guarani
4     1 0.627891 1.735577  1107686       1363 47115 small       B  guarani
5     2 1.875481 2.228531   353050       1363 49281 small       B  guarani
6     3 2.320698 2.353420    32722          2   579 small       B  guarani
  execution
1         0
2         0
3         0
4         1
5         1
6         1
     batch           start                end              duration        
 Min.   : 1.00   Min.   :   0.0666   Min.   :   0.073   Min.   :1.505e+03  
 1st Qu.: 4.00   1st Qu.:   0.7507   1st Qu.:   0.768   1st Qu.:1.155e+04  
 Median : 9.00   Median :  41.4935   Median :  48.848   Median :1.828e+06  
 Mean   :11.66   Mean   : 155.1063   Mean   : 166.524   Mean   :1.142e+07  
 3rd Qu.:17.00   3rd Qu.: 181.5546   3rd Qu.: 216.433   3rd Qu.:1.324e+07  
 Max.   :33.00   Max.   :2593.4133   Max.   :2593.415   Max.   :1.911e+09  
   operations           size              input      parsize     platform   
 Min.   :      1   Min.   :      543   big   :3870   A: 120   guarani:1980  
 1st Qu.:    170   1st Qu.:     6262   medium:4080   B: 390   luiza  :6060  
 Median :  78957   Median :  3299183   small :4089   C: 900   orion1 :2019  
 Mean   : 475735   Mean   : 20892125                 D:1629   remote :1980  
 3rd Qu.: 611585   3rd Qu.: 25551883                 E:3060                 
 Max.   :9785361   Max.   :463208087                 F:5940                 
   execution     
 Min.   : 0.000  
 1st Qu.: 3.000  
 Median : 7.000  
 Mean   : 9.538  
 3rd Qu.:15.000  
 Max.   :29.000
library(ggplot2);
ggplot(df[df$input == "small" & df$execution <= 9 & df$parsize != "A" & df$platform != "remote",], aes(x=start, xend=end, y=execution, yend=execution, color=input)) +
  geom_segment(size=1.1) +
  theme_bw() + facet_grid (parsize~platform) +
  scale_colour_manual(values = c("green"), name="Input Size") +
  scale_y_continuous(breaks=c(0,3,6,9)) +
  ylab("Replications") +
  xlab("Time (s)") +
  theme_bw() +
  theme(legend.position="none"); # no legend since we have only small

img/batch_time/batch_time_small_v2.pdf

library(ggplot2);
ggplot(df[df$input == "medium" & df$execution <= 9 & df$parsize != "A" & df$platform != "remote",], aes(x=start, xend=end, y=execution, yend=execution, color=input)) +
  geom_segment(size=1.1) +
  theme_bw() + facet_grid (parsize~platform) +
  scale_colour_manual(values = c("blue"), name="Input Size") +
  scale_y_continuous(breaks=c(0,3,6,9)) +
  ylab("Replications") +
  xlab("Time (s)") +
  theme_bw() +
  theme(legend.position="none"); # no legend since we have only small

img/batch_time/batch_time_medium_v2.pdf

library(ggplot2);
ggplot(df[df$input == "big" & df$execution <= 9 & df$parsize != "A" & df$platform != "remote",], aes(x=start, xend=end, y=execution, yend=execution, color=input)) +
  geom_segment(size=1.1) +
  theme_bw() + facet_grid (parsize~platform) +
  scale_colour_manual(values = c("red"), name="Input Size") +
  scale_y_continuous(breaks=c(0,3,6,9)) +
  ylab("Replications") +
  xlab("Time (s)") +
  theme_bw() +
  theme(legend.position="none"); # no legend since we have only small

img/batch_time/batch_time_big_v2.pdf

2016-05-19 MEM plots again

df <- read.csv ("results/batch_size.csv");
df <- df[complete.cases(df),];
df$sim_time <- as.numeric(as.character(df$sim_time));
df$insert_time <- as.numeric(as.character(df$insert_time));
df$sim_time = df$sim_time/1000000;
df$insert_time = df$insert_time/1000000;
df$max_mem = df$max_mem/1024/1024;
library(dplyr);
k <- df %>%
     group_by(input, batch, platform) %>%
     summarize(N = n(),
               mem_max = max(max_mem)) %>%
     as.data.frame();
k
    input batch platform  N     mem_max
1     big     B   orion1 10 4085.928391
2     big     C  guarani 10 2041.474648
3     big     C    luiza 30 2041.564964
4     big     C   orion1 10 2041.475166
5     big     C   remote 10 2041.474663
6     big     D  guarani 10 1020.576500
7     big     D    luiza 30 1020.667267
8     big     D   orion1 10 1020.575981
9     big     D   remote 10 1020.575462
10    big     E  guarani 10  513.168854
11    big     E    luiza 30  513.259621
12    big     E   orion1 10  513.168320
13    big     E   remote 10  513.167816
14    big     F  guarani 10  257.339867
15    big     F    luiza 30  257.430183
16    big     F   orion1 10  257.339264
17    big     F   remote 10  257.338837
18 medium     A    luiza 30 1036.147446
19 medium     B  guarani 10  527.779510
20 medium     B    luiza 30  528.854622
21 medium     B   orion1 10  527.780113
22 medium     B   remote 10  527.779533
23 medium     C  guarani 10  264.647179
24 medium     C    luiza 30  264.737885
25 medium     C   orion1 10  264.647789
26 medium     C   remote 10  264.647202
27 medium     D  guarani 10  133.296036
28 medium     D    luiza 30  133.386436
29 medium     D   orion1 10  133.295593
30 medium     D   remote 10  133.295006
31 medium     E  guarani 10   67.715477
32 medium     E    luiza 30   67.806183
33 medium     E   orion1 10   67.714890
34 medium     E   remote 10   67.714447
35 medium     F  guarani 10   35.101768
36 medium     F    luiza 30   35.979355
37 medium     F   orion1 10   35.101166
38 medium     F   remote 10   35.100731
39  small     A    luiza 30    4.160812
40  small     B  guarani 10    3.509026
41  small     B    luiza 30    3.606918
42  small     B   orion1 10    3.508286
43  small     B   remote 10    3.508720
44  small     C  guarani 10    3.227829
45  small     C    luiza 30    3.325630
46  small     C   orion1 10    3.227089
47  small     C   remote 10    3.227524
48  small     D  guarani 10    3.074348
49  small     D    luiza 30    3.172089
50  small     D   orion1 10    3.073601
51  small     D   remote 10    3.074043
52  small     E  guarani 10    4.566216
53  small     E    luiza 30    3.104179
54  small     E   orion1 10    3.005554
55  small     E   remote 10    3.005989
56  small     F  guarani 10    2.970764
57  small     F    luiza 30    3.068680
58  small     F   orion1 10    2.970024
59  small     F   remote 10    2.970367
library(ggplot2);
ggplot(k[k$input == "big" & k$platform != "remote",], aes(x=batch, y=mem_max, fill=input)) +
  geom_bar(stat="identity") +
  scale_fill_manual(values = c("red","blue", "green"), name="Input Size") +
  ylab("Maximum Memory Utilization (Megabytes)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_mem_big_v2.pdf

library(ggplot2);
ggplot(k[k$input == "medium" & k$platform != "remote",], aes(x=batch, y=mem_max, fill=input)) +
  geom_bar(stat="identity") +
  scale_fill_manual(values = c("blue"), name="Input Size") +
  ylab("Maximum Memory Utilization (Megabytes)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_mem_medium_v2.pdf

library(ggplot2);
ggplot(k[k$input == "small" & k$platform != "remote",], aes(x=batch, y=mem_max, fill=input)) +
  geom_bar(stat="identity") +
  scale_fill_manual(values = c("green"), name="Input Size") +
  ylab("Maximum Memory Utilization (Megabytes)") +
  xlab("Batch Size") +
  theme_bw() + ylim(0,NA) + 
  facet_grid(~platform) +
  theme(legend.position="none"); # no legend since we have only small

img/batch_size/local_mem_small_v2.pdf

2016-05-30 Comments about the bachelor defense

First member of the jury

  • Well elaborated structured
  • Should send your work to a conference (maybe WSCAD-WIC)
  • Lack of overhead analysis (explain with details)
  • In the conclusion
    • Better delineate your conclusions
    • Reinforcement of what are the main conclusion
    • Perhaps you should make an itemized list in the conclusion
  • Lack of extra fields, should be

Second member of the jury

  • Very good bachelor thesis
  • Should improve the English style at some points
    • Long and too-much elaborated phrases
  • Avoid things such as
    • Orphan and Widows lines
    • Figures should have the source below
    • Should use shapes in plots for black/white prints
  • Only relevant information should be added
    • Chapter 2 adds nothing considering your work
  • Good presentation, liked
    • Direct to the point
  • “Sell the fish”
    • Better delineate the objective
      • It is the simulator, and plugins are proof of concept
  • Say that your simulator is sequential
  • Comment about slide 23
    • I/O bandwidth should limit the disk
    • Could also be memory. Or both.
  • Table with exact values are missing (slide 11, 19, others)
    • Network is I/O
    • In the I/O experiment, the thing is synchronous
      • Should mention all OS configurations
  • Text should be correct
    • Mb/s -> Mbps
    • Remove this phrase: “We believe this is due the existence of a 100Mb/s switch between both endpoints limiting the network speed.”