Inference multiple images

[b04902036]

Thanks again for your great work!
I'm now trying to Inference multiple images without recreate FPGA handles or ExecData in ml-suite/xfdnn/rt/xdnn.py, so I change the script ml-suite/examples/classification/test_classify.py to something like this

#!/usr/bin/python

import os.path
import math
import sys
import timeit
import xdnn, xdnn_io
import numpy as np
import types

def main():
  args = xdnn_io.processCommandLine()
  ret = xdnn.createHandle(args['xclbin'], "kernelSxdnn_0", args['xlnxlib'])
  if ret != 0:
    sys.exit(1)
  (weightsBlob, fcWeight, fcBias ) = xdnn_io.loadWeights( args )
  for i in range(300):
    (fpgaInputs, batch_sz) = xdnn_io.prepareInput( args )
    fpgaOutput = xdnn_io.prepareOutput(args['fpgaoutsz'], batch_sz)

    startTime = timeit.default_timer()
    xdnn.execute(args['netcfg'],
      weightsBlob, fpgaInputs, fpgaOutput,
      batch_sz, # num batches
      args['quantizecfg'], args['scaleB'], args['PE'])

    elapsedTime = timeit.default_timer() - startTime
    print "\nAfter FPGA (%f ms)" % (elapsedTime*1000)

    startTime = timeit.default_timer()
    if(fcWeight is None):
      startTime = timeit.default_timer()
      softmaxOut = xdnn.computeSoftmax(fpgaOutput, batch_sz)
      elapsedTime = timeit.default_timer() - startTime
      print "\nAfter Softmax (%f ms)" % (elapsedTime*1000)
    else:
      fcOut = xdnn.computeFC(fcWeight, fcBias, fpgaOutput,
      batch_sz, args['outsz'], args['fpgaoutsz'], args['useblas'])
      elapsedTime = timeit.default_timer() - startTime
      print "\nAfter FC (%f ms)" % (elapsedTime*1000)
      #for i in range(10):
      #  print "%f" % fpgaOutput[i],
      startTime = timeit.default_timer()
      softmaxOut = xdnn.computeSoftmax(fcOut, batch_sz)
      elapsedTime = timeit.default_timer() - startTime
      #print "\nAfter Softmax (%f ms)" % (elapsedTime*1000)

    #for i in range(10):
    #  print "%f" % fpgaOutput[i],

    xdnn_io.printClassification(softmaxOut, args);
    if(i % 3 == 1):
      args["images"] = ['./flower.jpg']
    elif(i % 3 == 2):
      args["images"] = ['./cat.jpg']
    else:
      args["images"] = ['./dog.jpg']

  #print "\nSuccess!\n"
  xdnn.closeHandle()

if __name__ == '__main__':
  main()

I check fcWeight if it is None or not since there are some models that do not have fc (which is instead implemented by Conv layer).

Then use commad ./run.sh aws test_classify large 8 to run the program.
However, the for loop which run 300 iterations crashed at 256th iteration, at line xdnn.execute(), which seems that a uint8 resource isn't freed after use, and generate the error message:

python: xmlrt.cpp:106: std::pair<_cl_mem*, int> XComputeUnit::getCreateBuffer(void*, int, cl_mem_flags): Assertion `_numFreeMemSlots > 0' failed.

Then I look into ml-suite/xfdnn/rt/xdnn.py, it crash at last line of function self.execution, namely self._lib.XDNNExecute(execData._executor, inputs, outputPtr, numFpgaBatches, blocking)
, however I have no access to the source code of the dynamic linked library...
Could you please help me with this?
Or, if possible, publish source code of the library ml-suite/xfdnn/rt/xdnn_cpp/lib/libxfdnn.so please.
Thank you!

[wilderfield]

Hey,

The problem you are facing looks to be because you are calling prepareOutput iteratively.

Ideally, prepareOutput should be called once.

Please look to the batch_classify.py script for inspiration. It is in examples/classification

You can also look at apps/yolo - yolo.py, xyolo.py

For the time being we are not publishing our C++ code, because it is still evolving along with our hardware kernel.