Add pre-populate option to avoid flooding channel on instantiation

[leifmadsen]

When starting the bot for the first time using the feeds.sql.sample file, it will pull all data from the various feeds provided, then proceed to flood the channel with 10 lines per feed. On Freenode, this will cause the bot to be kicked from the channel due to flood protection.

It should be possible to cause the bot to instantiate the data in the database for the various feeds without outputting them to the channel on start to avoid flooding the channel. Only new values would then be posted to the channel.

[gehaxelt]

Yeah, I initially was too lazy to address this, because I didn't think that the bot would get so many users. Maybe we can write a separate python script (e.g. containing most of the __fetch_feed function) which could be called separately.

The best solution would be to provide a run.sh which first runs a update-db.py and then the main.py

[leifmadsen]

+1 that sounds reasonable. I was also confused a bit at first on the documentation as it wasn't clear to me at first glance (it became more obvious later) that starting the bot would populate the database and create the tables, etc.

If we can break out the appropriate functions so they can be called separately, then an initialization phase could be run independent of the bot, if that was desired.

It's probably pretty minor, and I wonder if just not writing to a channel on initial start up would be a simple enough work around? I'll see if I can find some time to try that out, because then the documentation can mostly just be updated.

[gehaxelt]

Yep, maybe that wasn't the right design decision back then. I'll think of a solution to this next week when I'll have some spare time.

I don't know if your approach will work, because you don't really want to miss new updates when you restart the bot and the db is "up to date". However, that might work if we skip the first update-iteration in every thread...

[leifmadsen]

Yea, I was thinking it would only be when the database hadn't been initialized, so only when you have to CREATE the database and tables would you ever perform a pre-join update.

Or like you said earlier, break out the functionality into a separate library, and make that available for anyone to run independent of the bot setup.

Or to simplify this, leave everything as-is, detect whether you're initializing the database (CREATE table, etc), and if so, set a variable database_initialized=True and then just wrap the post_news function in an if statement to just skip posting the news?

I guess it depends how elaborate the solution needs to be for an IRC bot :)