-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[simulator][machine] Simulator config, command and improve logic #13
Conversation
- alias sim - No real logic is added
- use cobra to bind command line flag - use logrus for logging
which introduced a lot of packages
An example snippet of machine simulator config simulator:
machine:
num: 100
# start time in ISO 8601
start: 1997-07-16T19:20:30.45+01:00
# end time in ISO 8601x
end: 1997-07-16T19:20:31.45+01:00
# in second
step: 10
|
- Add machine simulator config struct, which supports: num, start, end, step - add util function for read config file in test (TODO: don't know if it will have sideffect if is called multiple times)
- the machine.go is already 150+ lines with current dumbest simulation logic, so should put it in sub package and split to several files - Add format script in Ayi, which will call gofmt
- machine is the pojo and factory for machine - simulator is the real machine simulator
The plan for refactor the generator logic is like the following
And the machine simulator logic used to be
Now it will be
|
- intial time and value sub package under generator - add fixed interval - add constant - TestNewGenerator shows it is working
- add functions ending with New, in order to have old code as example, will remove old and change name after migration - add value generator interface - when generate default machine, add seriesWithValueGenerater into machine - machine has different array for keep int and double series
- see TestMachineSimulatorNew - TODO: why I saw more cpu_core=3 in log ouput - TODO: remove old generator code
- generate machine - add default machine - start - related property and methods in simulator interface
- delete interface for point generator - delete constant int point generator - delete series with point generator
The plan for the simulator command is like the following
some changes need to be made
|
- the simulator config struct is actually never used - TODO: not output in current default setting, might be problem with config file
It seems the serializer is not properly initialized ```` DEBU[0000] enable debug logging pkg=x.util DEBU[0000] config file xephon-b.yml is loaded pkg=x.cmd DEBU[0000] triggered simulator command pkg=x.cmd DEBU[0000] machine simulator start pkg=x.s.machine DEBU[0000] need to serialize pkg=x.s.machine panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x498eed] goroutine 21 [running]: panic(0x7bc3e0, 0xc420016120) /home/at15/app/go/src/runtime/panic.go:500 +0x1a1 github.com/xephonhq/xephon-b/pkg/simulator/machine.(*MachineSimulator).Start.func2(0xc42007cde0, 0xc4200aa630, 0xc42007ce40, 0xc42017c290) /home/at15/workspace/src/github.com/xephonhq/xephon-b/pkg/simulator/machine/simulator.go:84 +0x11d created by github.com/xephonhq/xephon-b/pkg/simulator/machine.(*MachineSimulator).Start /home/at15/workspace/src/github.com/xephonhq/xephon-b/pkg/simulator/machine/simulator.go:92 +0x20e ````
- treated output as encoding and cased the null pointer problem - change example num to 3 for less output when try the command - run `xb sim --debug --type json` to see json output - TODO: json serialize should have `\n` in order to be read by others
- ignore *.dat file as its the default output file suffix
- TODO: use append may not be the most efficient way. But adding a wrapper for writer might be more messy since the writer should know little(nothing) about the serializer
- though it seems file without calling f.Close
now a small problem for json ouput, it has an extra empty line
I think the reader could have check when deserialize the data, so I will just ignore it. PTAL @czheo I will merge it tomorrow if no response from u. |
one series + one data point
instead ofone series + multiple data points
when generating data. When loading the data, it really depends on how the database provide interface for bulk loading, batches can be done in loader, so this should not be the concern for simulator, using database independent format can allow generated be used on multiple databases, which is useful if the data generation is resource consuming.deserialize file (the part is more used in loader, but it should also be used in serializer's test)leave it it to loaderThere are a lot more that could be configured for the machine, like we can add higher level things like rack, data center, for micro benchmark, they are just tags, for real bechmark, those information can be used for deploy probes to machines in different rack, dc. However, they are just too much for now.