RamSize plugin

[pop]

fixes issue #75

Changes in this PR.

• Spec Tests
• Migrations
• Plugin Implementation

Testing this PR.

1. Log into development environment (docker-compose run dev bash)
2. Run tests rake spec

Expected Output.

$ rake spec
[...]
RamSize plugin
  .store method
    does not fail with valid data
    fails when not passed node name
    does not crash when passed improperly formatted data
[...]
Finished in 0.61158 seconds (files took 0.5293 seconds to load)
63 examples, 0 failures

@osuosl/devs @Kennric

[Kennric]

Looks pretty good. +1

[alxngyn]

looks like this doesn't belong here

[alxngyn]

aside from the cache -> util changes that don't look like they belong in this PR
+1

[pop]

@alxngyn @Kennric ready to merge this?

[alxngyn]

I'm testing this right now and it seems like the ram_size plugin doens't store to the DB.

sqlite> select * from disk_size_measurements limit 1;
1||2016-06-20 19:44:18.339366|alembic-java.osuosl.org|0|active

sqlite> .tables
clients                     projects                  
disk_size_measurements      ram_size_measurements     
disk_template_measurements  schema_info               
node_resources              vcpu_count_measurements   
plugins                   

sqlite> select * from vcpu_count_measurements limit 1;
1||2016-06-20 19:45:05.069686|alembic-java.osuosl.org|0|active

sqlite> select * from ram_size_measurements limit 1;
sqlite> 

I'm investigating it now

[alxngyn]

found it,

No total_ram_meas information for deluge.osuosl.org:
node_info : {"disk_sizes": "[30720]",
 "disk_usage": "30848",
 "disk_template": "drbd",
 "num_cpus": "8",
 "total_ram": "2048",
 "active": true
 }

[alxngyn]

TL,DR: we need to change total_ram_meas to total_ram

So it appears that you (and I, too) got confused from collectors.rb where

collectors.rb:
40 total_ram_meas     = node['beparams']['memory']       || 'unknown'

total_ram_meas looks like the json variable in the cache but in reality it is passed to datastruct.erb to create the object that is stored into the cache

datastruct.erb:
{"disk_sizes": "<%= disk_sizes_meas %>",
 "disk_usage": "<%= disk_usage_meas %>",
 "disk_template": "<%= disk_template_meas %>",
 "num_cpus": "<%= num_cpus_meas %>",
 "total_ram": "<%= total_ram_meas %>",
 "active": <%= active_meas %>
 }

so to fix this bug, we need to change total_ram_meas to total_ram, then it should store into the DB like the other plugins

[pop]

@alxngyn Good catch, I didn't think to look for that. I'll also add a test for that somewhere...

[pop]

@alxngyn I'm keeping the cache.rb -> util.rb in this PR. I ended up adding some util functionality that actually gets used in a test. I could add a separate util.rb from the cache.rb but I think it's best to keep them in the same file.

[alxngyn]

@ElijahCaine, okay as long as the change (cache.rb -> utils.rb) is consistent throughout the app

[pop]

@alxngyn I checked to make sure it was consistent.

[alxngyn]

Won't run because collectors.rb still has the cache reference:

require_relative 'lib/cache'

[pop]

@alxngyn Made that change. I swear I thought I had completely converted to the new lib/util. I ran the tests and it works great. Force pull and let me know what you think.

[alxngyn]

I just tested it. It runs but now the DB isn't being written to again

sqlite> .tables
clients                     projects                  
disk_size_measurements      ram_size_measurements     
disk_template_measurements  schema_info               
node_resources              vcpu_count_measurements   
plugins                   

sqlite> .schema ram_size_measurements
CREATE TABLE `ram_size_measurements` (`id` integer NOT NULL PRIMARY KEY AUTOINCREMENT, `node_resource` integer, `created` timestamp NOT NULL, `node` varchar(255), `value` integer, `active` Boolean);

sqlite> select * from ram_size_measurements limit 1;

sqlite> select * from vcpu_count_measurements limit 1;
1||2016-06-29 17:41:39.748265|alembic-java.osuosl.org|0|active

sqlite> select * from ram_size_measurements;

sqlite> select * from disk_size_measurements limit 1;
1||2016-06-29 17:41:13.003854|alembic-java.osuosl.org|0|active

sqlite> 

[pop]

@alxngyn Weeeeird. I'll take another look at it. Thanks for the heads up.

[pop]

@alxngyn I pushed a fix. Turns out our collector was not parsing the JSON it was collecting. Honestly I don't know how anything was working. Should be good now though.

[alxngyn]

Good fix! Expected output is good,

sqlite> .tables
clients                     projects                  
disk_size_measurements      ram_size_measurements     
disk_template_measurements  schema_info               
node_resources              vcpu_count_measurements   
plugins                   

sqlite> select * ram_size_measurements limit 1;
Error: near "ram_size_measurements": syntax error

sqlite> select * from ram_size_measurements limit 1;
1||2016-06-30 18:22:57.330686|alembic-java.osuosl.org|1024|1

sqlite> select * from vcpu_count_measurements limit 1;
1||2016-06-30 18:22:03.593952|alembic-java.osuosl.org|2|1

sqlite> select * from disk_size_measurements limit 1;
1||2016-06-30 18:21:38.595226|alembic-java.osuosl.org|10240|1

sqlite> select * from disk_template_measurements limit 1;
1||2016-06-30 18:22:29.980749|alembic-java.osuosl.org|drbd|1

sqlite> select * from plugins;
1|DiskSize|node||bytes
2|VCPUCount|node||vcpu
3|DiskTemplate|node||type
4|RamSize|node||mb
sqlite> 

[Kennric]

+1

[alxngyn]

+1 for me too