Memory leak

[pyed]

Sending large files with SendDocument or SendVideo will result in high memory usage, and after the file done sending, this memory won't get freed

consider the following code

package main

import (
    "log"

    "github.com/Syfaro/telegram-bot-api"
)

func main() {
    bot, err := tgbotapi.NewBotAPI("MyAwesomeToken")
    if err != nil {
        log.Panic(err)
    }

    // bot.Debug = true

    log.Printf("Authorized on account %s", bot.Self.UserName)

    u := tgbotapi.NewUpdate(0)
    u.Timeout = 60

    err = bot.UpdatesChan(u)
    if err != nil {
        log.Panic(err)
    }

    filename := "file.dat" // ~33Mb, should be fairly a big file to notice
    for update := range bot.Updates {
        log.Printf("[%s] %s", update.Message.From.UserName, update.Message.Text)

        msg := tgbotapi.NewMessage(update.Message.Chat.ID, "the file on its way")
        msg.ReplyToMessageID = update.Message.MessageID
        bot.SendMessage(msg)

        newDoc := tgbotapi.NewDocumentUpload(update.Message.Chat.ID, filename)
        if _, err := bot.SendDocument(newDoc); err != nil {
            log.Panic(err)
        }

    }
}

Now running this bot on OS X will start with 8.2 MB of Real Mem "according to Activity Monitor"
after sending the first message, the bot will start sending file.dat, Real Mem will jump to 107.6 MB it will finish sending but it will not free the memory, requesting the file again will rise Real Mem again to 180 MB ... etc

I took a look at the code and I figured it has to be in the UploadFile method, and I found that a usage to os.Open() bond to a variable called f hasn't get closed, I tried closing it, but it did not solve the problem.

[Syfaro]

Hmm... you're definitely right that there's a memory leak somewhere.

I think it has something to do with the form, but I'm not entirely sure. I'll work on figuring out what's happening.

[Syfaro]

Well, this is what's using the most memory.

(pprof) top
26184.78kB of 26184.78kB total (  100%)
Dropped 33 nodes (cum <= 130.92kB)
      flat  flat%   sum%        cum   cum%
   24648kB 94.13% 94.13%    24648kB 94.13%  bytes.makeSlice

[Syfaro]

This might be a limitation in Go.

moby/moby#1922 (comment)

[pyed]

so nothing we can do about it ?

I can confirm that github.com/rockneurotiko/go-tgbot has the same issue

[Syfaro]

Well, this commit will reduce memory usage, a 30MiB test file increases my memory usage by ~2MB.

That upload function is pretty much identical to mine, so I don't know what the difference could be.

[pyed]

I just tried it out, it is a great improvement.

[Syfaro]

If in the future anyone comes up with a better solution, I'd be happy to reopen this. Until then, I'm going to close it.