Dreamhack changes #1
Conversation
+ optimize filter list enabled + minor changes
This comment has been minimized.
This comment has been minimized.
|
With disabled optimization I receive a mem alloc error if using a certain amount of ports.
Enabled pcap filter optimiization and everything is fine. |
This comment has been minimized.
This comment has been minimized.
|
Got a strange behavior here, sometimes on a debian machine pcap did not return -1 as required |
| void writeLogStats() | ||
| { | ||
| // don't spam! just write the log once every X seconds | ||
| if((logtimer + 10) < (int)time(NULL)) { |
sirsquidness
Jan 16, 2017
•
Collaborator
I think this is missing the value being update after each loop (eg, logtimer = time(NULL)), so this will run every time a packet is forwarded?
I think this is missing the value being update after each loop (eg, logtimer = time(NULL)), so this will run every time a packet is forwarded?
solariz
Jan 17, 2017
Author
Contributor
Actually it is missing but somehow it run correctly :) Will check.
Actually it is missing but somehow it run correctly :) Will check.
| if (fp) | ||
| { | ||
| // General Stats | ||
| fprintf(fp,"## GENERAL\nRX:%ld\nTX:%ld\nDROP:%ld\n## PORT STATS\n",(long) pkt_rx, (long) pkt_tx, (long) pkt_drop); |
sirsquidness
Jan 16, 2017
Collaborator
Does your use case with influxDB support single line entries? I'd prefer single line entries (eg, CSV, JSON, or similar) so that it's more of a traditional log, but I don't mind leaving this as is if you can't use single line :)
Does your use case with influxDB support single line entries? I'd prefer single line entries (eg, CSV, JSON, or similar) so that it's more of a traditional log, but I don't mind leaving this as is if you can't use single line :)
solariz
Jan 17, 2017
Author
Contributor
I choose this way of logfile because it'S still human readable. For pushing it to influxDB I used a sperate bash Script to convert this into a one line insert every 10s.
I choose this way of logfile because it'S still human readable. For pushing it to influxDB I used a sperate bash Script to convert this into a one line insert every 10s.
solariz
Jan 18, 2017
Author
Contributor
I put the Bash Script I'm using to pusinh this stats into a influxDB at gist:
https://gist.github.com/solariz/29362abbcf45605ab700df6f6e6be141
It's pretty simple, influx can handle a huge amount of posts per second so we're totally fine with that currently. On the last event the influx run on a raspberry pi at a mean CPU cycle of less then 10% :)
I put the Bash Script I'm using to pusinh this stats into a influxDB at gist:
https://gist.github.com/solariz/29362abbcf45605ab700df6f6e6be141
It's pretty simple, influx can handle a huge amount of posts per second so we're totally fine with that currently. On the last event the influx run on a raspberry pi at a mean CPU cycle of less then 10% :)
| // Does nothing at the moment. Will enable rewriting subnet info if relevant. | ||
| int do_network_rewrite = 0; // Rewrite IP broadcast address for the new | ||
| // network interface | ||
| int32_t pkt_rx = 0; |
sirsquidness
Jan 16, 2017
Collaborator
How many packets did you see at your event? Did an int32 get close to overflowing? Probably better to make it an int64 just in case?
How many packets did you see at your event? Did an int32 get close to overflowing? Probably better to make it an int64 just in case?
| int32_t pkt_rx = 0; | ||
| int32_t pkt_tx = 0; | ||
| int32_t pkt_drop = 0; | ||
| unsigned short int pkt_stats[65535]; |
sirsquidness
Jan 16, 2017
Collaborator
Ditto for this - I expect the short would overflow pretty regularly? Maybe in the future we should make this a hash table so we don't need a giant array of mostly 0s, or somehow index the array based on the ports listed in the ports file. Not important though, just thinking out loud
Ditto for this - I expect the short would overflow pretty regularly? Maybe in the future we should make this a hash table so we don't need a giant array of mostly 0s, or somehow index the array based on the ports listed in the ports file. Not important though, just thinking out loud
solariz
Jan 17, 2017
Author
Contributor
Hey, indeed on the event the counter overflow several times, especially 27015 :)
For my purpose it was ok because I just tracked difference between previeous poll in a 10s resolution. But could be changed to int64
Hey, indeed on the event the counter overflow several times, especially 27015 :)
For my purpose it was ok because I just tracked difference between previeous poll in a 10s resolution. But could be changed to int64
|
Thank you so much for submitting this PR! It's so exciting to know that Dreamhack used something I wrote :D I am happy to merge this PR as is, but I left a few comments to get your thoughts. And thanks especially for the BPF optimisation thing - I had that problem last time I used it, and worked around by running multiple instances of the app. This is a much nicer solution :) |
|
We've been using it at epic.LAN for over a year now as well. Thanks adding the extra games to it as well. I'm looking forward to getting some stats from our next event! |
+ Added logtimer update to run properly every loop
|
Hey, pushed back latest fixes. Guess I'm out for now until next LAN :) Ready for Merge |
|
Merged! Thanks again, both for finding the tool useful and for contributing back :) |
We succesfully used it at the Dreamhack DE, this is my commit some minutes before our dismantling starts :)
Added several games and added statistic log function to the code. This stat file was pushed to a influxDB so we directly could get some very interesting insights of what was received and forwarded exactly. SDH works pretty well, just had a few problems.