Skip to content

Performance Manager IOPS Average Metric returning negative values #191

@zukeru

Description

@zukeru

Hello,

I'm using a performance manager to collect iops data for VMs. What I have noticed is that vCenter returns a negative average for some VMs. I didn't think it was possible to return negative values for IOPS, is this a bug in pyvmomi or the way I'm getting the data.

I'm using a sample interval of 1 minute, thus if it takes 3 values over that one minute and we divide the average by 3 we should get a value as close as possible to a realtime IOPS value. I got this from the community samples. At the time I wrote it I didn't quite understand it. I just need to know why its returning a negative and what I can do to fix it?

Here is a sample output of data:

  2014-07-30 11:14:45.085235 |          1 | 27535 |          802 |   12.66666666666666666666666667 |                               0
  2014-07-30 11:30:38.153511 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 11:46:01.573311 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 12:02:27.679907 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 12:18:27.932628 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 12:34:15.535328 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 12:50:50.178003 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 13:06:44.797447 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 13:22:57.345443 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 13:38:29.005453 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 13:54:27.5787   |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 14:10:19.191901 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 14:26:15.321434 |          1 | 27535 |          802 |                               0 |                               0
   2014-07-30 14:42:06.629368 |          1 | 27535 |          802 |                               0 |                               0
   2014-07-30 14:57:41.566514 |          1 | 27535 |          802 |                               2 |  0.6666666666666666666666666667
   2014-07-30 15:13:17.66591  |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 15:29:04.774211 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 15:45:06.767868 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 16:01:30.503479 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 16:17:22.913474 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 16:33:25.220693 |          1 | 27535 |          802 |                               0 |                               0
  2014-07-30 16:49:22.731897 |          1 | 27535 |          802 |                               2 |  0.6666666666666666666666666667
  2014-07-30 17:04:42.9296   |          1 | 27535 |          802 | -0.3333333333333333333333333333 | -0.3333333333333333333333333333

     atexit.register(Disconnect, vcenter_connection)             
     content = vcenter_connection.RetrieveContent()       
     perf_dict = {} 
     perfList = content.perfManager.perfCounter

     for counter in perfList: #build the vcenter counters for the objects
        counter_full = "{}.{}.{}".format(counter.groupInfo.key,counter.nameInfo.key,counter.rollupType)
        perf_dict[counter_full] = counter.key

     viewType = [vim.VirtualMachine]
     props = ['name','runtime.powerState', 'datastore']
     specType = vim.VirtualMachine
     objView = content.viewManager.CreateContainerView(content.rootFolder,viewType,True)  
     tSpec = vim.PropertyCollector.TraversalSpec(name='tSpecName', path='view', skip=False, type=vim.view.ContainerView)    
     pSpec = vim.PropertyCollector.PropertySpec(all=False, pathSet=props,type=specType)   
     oSpec = vim.PropertyCollector.ObjectSpec(obj=objView,selectSet=[tSpec],skip=False)   
     pfSpec = vim.PropertyCollector.FilterSpec(objectSet=[oSpec], propSet=[pSpec], reportMissingObjectsInResults=False)   
     vm_properties = content.propertyCollector.RetrieveProperties(specSet=[pfSpec])  
     objView.Destroy()     

     for vm_property in vm_properties: #loop through the list built from vcenter and build dictonaries.
        property_dic = {}
        for prop in vm_property.propSet:
           property_dic[prop.name] = prop.val 

        vm = vm_property.obj 
        vm_mor = vm._moId
        if "poweredOn" in vm.runtime.powerState:
           try:
              #Get Values of IOPS which are 20 second averages more accurate you can get
              counter_name = 'datastore.numberReadAveraged.average'
              perfManager = content.perfManager  
              counterId = perf_dict[counter_name]
              metricId = vim.PerformanceManager.MetricId(counterId=counterId, instance="*") 
              query = vim.PerformanceManager.QuerySpec(intervalId=20, entity=vm, metricId=[metricId], startTime=start_time, endTime=end_time) 
              statDatastoreIoRead = perfManager.QueryPerf(querySpec=[query])
              DatastoreIoRead = (Decimal(sum(statDatastoreIoRead[0].value[0].value)) / Decimal(3.0)) #dividing by 3 to get the closest to real time value as possible
              iopsr = DatastoreIoRead

              counter_name = 'datastore.numberWriteAveraged.average'
              perfManager = content.perfManager  
              counterId = perf_dict[counter_name]
              metricId = vim.PerformanceManager.MetricId(counterId=counterId, instance="*") 
              query = vim.PerformanceManager.QuerySpec(intervalId=20, entity=vm, metricId=[metricId], startTime=start_time, endTime=end_time) 
              statDatastoreIoWrite = perfManager.QueryPerf(querySpec=[query])
              DatastoreIoWrite = (Decimal(sum(statDatastoreIoWrite[0].value[0].value)) / Decimal(3.0)) #dividing by 3 to get the closest to real time value as possible
              iopsw = DatastoreIoWrite

              '''
              Check the vm_mor from Vcenter which is set in the getProperties function to see if 
              exist in the VMS dictonary that was built from the oracle database.
              '''
              if vm_mor in vms:                
                 vm_id = vms[vm_mor]
                 for datastore_vm in property_dic['datastore']:                                                                  
                    datastore_mor = str(datastore_vm).split(':')[1][:-1]
                    if datastore_mor in datastores_db:                         
                       datastore_id = datastores_db[datastore_mor]      
                       pg_cursor.execute("""insert into storage_perf_samples
                                                         (datetime,
                                                         vcenter_id,
                                                         vm_id,
                                                         datastore_id,
                                                         read,
                                                         write)
                                                  values (%s,
                                                         %s,
                                                         %s,
                                                         %s,
                                                         %s,
                                                         %s) """,
                                                         (rundate,
                                                         vcenter_id,
                                                         vm_id,
                                                         datastore_id,
                                                         iopsr,
                                                         iopsw))         

                    else:                          
                       logging.warning("VM %d has a datastore that is not in the Datastores table %s vm_mor doest show" % vm_id, vm_mor)                                      
              else:             
                 logging.warning("The vm_mor %s was not found in the VMS database from Oracle, skipping...." %  vm_mor)          
           except: 
              pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions