Multithread predictions

[vegas2ny]

Hello i am having memory access violation when i attempt to multi thread image predictions. this is happening at line

// Start evaluation on the device
modelFunc.Evaluate(inputDataMap, outputDataMap, GlobalParameters.Device);

within ImageNet.cs is there a way to multithread the functionality ?

Thanks

[deepakkumar1984]

Trying to make the basics working :) I will try to find the root cause. Could you please share the code snippet?

…

On Fri, Nov 24, 2017 at 10:53 AM, vegas2ny ***@***.***> wrote: Hello i am having memory access violation when i attempt to multi thread image predictions. this is happening at line // Start evaluation on the device modelFunc.Evaluate(inputDataMap, outputDataMap, GlobalParameters.Device); within ImageNet.cs is there a way to multithread the functionality ? Thanks — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKfygm02u987PE04ujpF41TGJ4LEaks5s5gySgaJpZM4QpQp5> .

-- Regards, Deepak

[vegas2ny]

sure, do you have gmail, maybe we can google chat? and ill update here with resolution?

[deepakkumar1984]

Lets use this chat, I created this for public conversation so we can start our discussion there. https://gitter.im/sia-cog/SiaNet?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge

…

On Fri, Nov 24, 2017 at 11:11 AM, vegas2ny ***@***.***> wrote: sure, do you have gmail, maybe we can google chat? and ill update here with resolution? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#11 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCQKXIa9ChdRtA_RgRRXHNpkrRzybU3ks5s5hDLgaJpZM4QpQp5> .

-- Regards, Deepak

[deepakkumar1984]

Or here is my gmail: deepakkumar1984@gmail.com

[deepakkumar1984]

Try using lock to avoid access violation issue:

static ImageNet app;
private static Object thisLock = new Object();

static void Main(string[] args)
{
    app = new ImageNet(ImageNetModel.ResNet50);
    app.LoadModel();

    for (int i = 0; i < 5; i++)
    {
        System.Threading.ThreadPool.QueueUserWorkItem(InvokeCls);
    }
}

static void InvokeCls(Object stateInfo)
{
    Console.WriteLine("started");
    string imagePath = string.Format("{0}images\\dog_cls.jpg", AppDomain.CurrentDomain.BaseDirectory);
    lock (thisLock)
    {
        var p = app.Predict(imagePath);
        Console.WriteLine("Predicted: " + p[0].Name);
    }
}

[vegas2ny]

hello i have found a resolution, by cloning the module func it makes predictions thread safe, there for no more access violations.

//clone model func for multithread
var newmodelfunc = modelFunc.Clone(ParameterCloningMethod.Share);

full method code below:

   //clone model func for multihread
            var newmodelfunc  = modelFunc.Clone(ParameterCloningMethod.Share);

            Variable inputVar = newmodelfunc.Arguments[0];

            NDShape inputShape = inputVar.Shape;
            int imageWidth = inputShape[0];
            int imageHeight = inputShape[1];

            var resized = bmp.Resize(imageWidth, imageHeight, true);
            List<float> resizedCHW = resized.ParallelExtractCHW();
            
            // Create input data map
            var inputDataMap = new Dictionary<Variable, Value>();
            var inputVal = Value.CreateBatch(inputShape, resizedCHW, GlobalParameters.Device);
            inputDataMap.Add(inputVar, inputVal);
            inputVar = newmodelfunc.Arguments[1];
            //inputDataMap.Add(inputVar, null);

            Variable outputVar = newmodelfunc.Outputs.Where(x => (x.Shape.TotalSize == 1000)).ToList()[0];

            // Create output data map. Using null as Value to indicate using system allocated memory.
            // Alternatively, create a Value object and add it to the data map.
            var outputDataMap = new Dictionary<Variable, Value>();
            outputDataMap.Add(outputVar, null);


            // Start evaluation on the device
            newmodelfunc.Evaluate(inputDataMap, outputDataMap, GlobalParameters.Device);

            // Get evaluate result as dense output
            var outputVal = outputDataMap[outputVar];
            var outputData = outputVal.GetDenseData<float>(outputVar);
            Dictionary<int, float> outputPred = new Dictionary<int, float>();

            for (int i = 0; i < outputData[0].Count; i++)
            {
                outputPred.Add(i, outputData[0][i]);
            }

            var topList = outputPred.OrderByDescending(x => (x.Value)).Take(topK).ToList();
            List<PredResult> result = new List<PredResult>();
            float sumpredresult = outputPred.Sum(x => (x.Value));
            float avgpredresult = outputPred.Average(x => (x.Value));
            float min = outputPred.Min(x=>(x.Value));
            float max = outputPred.Max(x=>(x.Value));

            foreach (var item in topList)
            {
                result.Add(new PredResult()
                {
                    Score = item.Value,
                    Name = actualValues[item.Key]
                });
            }

            Logging.WriteTrace("Prediction Completed");

            return result;

[vegas2ny]

Hello will close this bug now since the issue is resolved.

Thanks

[deepakkumar1984]

Thanks for your help