v1.14.0 - The Bot Keeps a Record

• Faster: a finished send no longer rewrites the whole download-count file. Each one rewrote every count the bot had ever kept - most of a second on a bot that has sent many different files, with other saves waiting behind it. The counts move into data/download_counts.db on the first start; data/download_counts.json is left exactly as it was, so going back to an older version still shows the counts up to the upgrade.
• Faster: the Stats and Live Transfers pages no longer re-read whole files every few seconds. On a bot that has sent many different files, each refresh re-read and sorted the download counts and recounted the RAR list - seconds each time, and finished sends waited on it. Live Transfers now asks only for its own figures, and the rest is worked out again only when the files change.
• Changed: the bot keeps up better when many people join or leave channels at once. Work that grew with the square of the number of people now grows with the number.
• Changed: Live Transfers is its own page, and the Stats page follows one period. Speed, slots and the queue moved from Stats to a new Live Transfers page. On Stats, pick 24 hours, 7 days, 30 days or all time and the whole page answers for it: one row of Sent figures (files, size, lists, top and average speed, average wait, files and size received), Most downloaded files and albums, and the nicks. The two Most downloaded tables are one now, and the old Total/Today/Yesterday Sent cards are gone. All time still counts everything the bot sent before the record began.
• New: the bot keeps a record of every finished transfer, and the Stats page shows it. One row per transfer goes into data/transfers.db: what it was (a file, a packed folder or a list), its size, its speed, how long it waited in the queue and the nick it went to or came from (in lower case; no host and no channel is stored). On the Stats page, pick 24 hours, 7 days, 30 days or all time to see what was sent and received, the speeds, the average wait in the queue, the files sent most and the nicks you sent to and received from most. Look up one nick, Forget it when somebody asks (every row with that nick is wiped from the file), or Forget everyone; both ask first. Export CSV saves the period as a spreadsheet. TRANSFER_LOG_FILE sets where the record is kept; empty turns it off.
• Changed (mIRC): the @dccore right-click menu is grouped by what you do. Script Settings (the old Options) and Console command come first, then Info (status, slots, queue, uptime, version, command list), Lists, Library, User control (bans, ban, unban, clear a queue) and Control (update check, console feed, reload the bot, and last, behind a line, Stop the bot). Connection and Window follow; DCCore Chat and Downloads window are now inside Window, and Clear finished downloads is called Clear finished... and now asks first (only the list is cleared, no files are deleted). Everything that was in the menu is still there, and Stop the bot and Reload still ask first. Update dccore.mrc (1.11) and reload it with /reload -rs.
• New: KeepTrack's per-nick history can be imported. Choose KTData.txt on the Stats page: you see how many nicks and files it holds, the top ten each way and any lines it could not read, then import it. Each nick's KeepTrack totals are added to what the bot has recorded for them since. Only nicks and totals are kept - the hosts in the file are not.
• New: KeepTrack's totals can be imported too. The Stats page's import reads KeepTrack's files and bytes sent and received from the same mIRC vars.ini as OmenServe's. If both counted your sends - the same sends twice - you choose which totals to keep. KeepTrack's received totals go into the new transfer record, and the preview says when KeepTrack began counting and which file types it counted (its default list leaves out .rar and .flac).
• New: on Windows the bot can run minimised, or with no window at all. Settings > Debug & logging > The bot's window: normal (as before), minimised (in the taskbar) or hidden (no window; what it says goes to data\logs\dccore.log). The first run always has its window, for the setup.
• New: ways to stop the bot without its window. start-dccore stop (on Windows scripts\windows\start-dccore.bat stop) asks the bot to stop and waits until it has; the dashboard's Tools page has Stop the bot, the admin console has shutdown now, and the DCCore menu in dccore.mrc has Stop the bot.... All of them stop it the way Ctrl-C in its window does.
• Fixed: the List Browser's filter showed a list from a bot whose nick is only symbols (such as ^_^) as holding no match. It is found now.
• Fixed: a Save of the on-connect commands that was refused put the saved ones back over what you had typed. What you type now stays in the box until it is saved.
• New: the bot keeps a log file. Everything its window shows is also saved in data/logs/dccore.log, with the date on every line, so it is still there after the window is closed. At 5 MB a new file is started and five old ones are kept; Settings > Debug & logging changes that, or turns it off.
• Fixed: asking for the same file again no longer queues it again. A request for something already waiting in your own queue, or already being sent to you, is not added a second time; the bot tells you once that it is already queued and where, or already on its way. Before, asking over and over could fill your whole queue with copies of one file, and each copy was then sent.
• Fixed: the bot could take most of a minute to start when it held many fetched lists. On every start it read the whole cross-list search index (several GB on a busy bot) just to see which lists were in it. It now asks about each list directly, which takes a fraction of a second.
• Changed: DCCore's files are tidied into folders. The program's modules are now in src/, and your own settings.conf and admin_config.py in conf/. Nothing to do: the first time the new version starts it moves your two files into conf/ by itself, and the start scripts find them there. oserve.py, configure.py and update_list.py stay where they were, so the commands that start and set up the bot do not change; the one that makes the admin console's password hash is now python src/adminchat.py. If a guide or a script of yours opens settings.conf by its path, point it at conf/settings.conf.
• New: the bot checks that its on-connect commands worked, and sends them again if not. In a net split the X login can go nowhere, and the bot used to sit in its channels with its real host until it reconnected. Now, if the commands set a user mode such as +x, the bot checks a minute after joining and then every 5 minutes (Settings > Identity & network; 0 turns it off) - for +x, that the host really is hidden - and sends them all again if not, up to six times per connection. Resend commands, beside Save on-connect commands, sends them right away. It works whether or not a channel let the bot in - on Undernet a +r channel refuses it exactly when the X login did not take.
• Fixed: a send slot no longer stands idle while a folder (!rar) is being packed. When the person who had just finished had a folder next in their queue, the freed slot was left unused for as long as the packing took - up to half an hour - and a new request was queued instead of sent. It now goes to whoever has waited longest.
• Fixed (security): in the DCCore Chat window, an answer to someone who wrote to you privately could go to a channel instead. If a line arrived from a channel while you were typing a private reply, the reply was sent to that channel. A private conversation now stays private until you pick another target yourself.
• Fixed: Cancel on the Downloads page could delete a file that had just finished downloading, if it finished while the "Nothing has been downloaded yet" question was open. Cancel now only ever lets go of a request that has not started; a finished one stays, and the page says so.
• Fixed: @<nick>-remove <file> did nothing for someone who had just left the channel - no reply, and their queue on disk was not updated. It removes the file and says so now.
• Fixed: with an older dccore.mrc, the DCCore window filled with [PEERS] ... lines after the bot was updated - on every connect, every ten minutes and whenever another DCCore bot came or went. The bot now sends those lines only to a script that can read them (1.7 or later).
• Fixed: the DCCore Chat window's title always said your next line would go "privately", and named the wrong place when you had picked a channel or a person to talk to. It now names where the line really goes.
• Fixed: with DCCore Chat on, your own bot's [ServersChat] lines showed in the channel window. Every line your bot relayed - what you typed, or what it passed on from another bot - also appeared raw in the channel. They are hidden there again.
• Fixed: "Download again" on a failed folder (!rar) download now asks for the folder again. From the dashboard or the mIRC Downloads window, it asked in a way that made DCCore refuse the album when it arrived.
• Fixed: a download that waited a long time in another bot's queue, then had to wait for disk space, could be given up on too soon after it was queued again. Its new place in that bot's queue is now timed from when it got it.
• Fixed (security): many people joining a channel at once could make dashboard searches come back empty. DCCore Chat checks each newcomer to see whether it is another DCCore bot, and those checks queued up ahead of the search. They are now limited and kept out of the search's way.
• Fixed: a large download that had to wait for disk space, or was cut off by a restart, could be asked for again and again without ever starting while your disk had less than twice its size free. It now starts as soon as it fits.
• Fixed: a folder (!rar) request made while every slot was busy could wait for ever. It was only tried again when that person's own next download finished - so for someone whose only request was the folder, never. It now gets the next free slot when its turn comes, in the same longest-waiting-first order as single files.
• Fixed: a free send slot now goes to whoever has waited longest. The person who had just finished got their own next file straight away, so someone with a long queue took every slot in turn while everyone else waited for that queue to run dry. Now they go to the back of the line, and a new request no longer jumps ahead of people already waiting. Still one send per person, and the short pause after a finished send is unchanged.
• Fixed: a download request is sent once, and its wait starts when it is sent. On a busy bot a request waited in the send queue behind other users' replies while its timer was already running, so it timed out and was queued again - a file could be asked for four or five times, and the other bot sent finished files twice. Requests now have a queue of their own that is sent before everything else, the channel advert included, the timer starts when the request leaves, and a request asked again replaces the one still waiting instead of adding a second. A request that had not gone out when the connection dropped goes out once the bot is back.
• New: a Downloads window in dccore.mrc. /dccore downloads (or the DCCore menu) opens a window with what your bot is downloading from other bots. The main area logs each request, the place in the other bot's queue, the transfer and the result as they happen; the list on the right shows how things stand: files coming in with progress and speed, requests waiting and why, and the latest finished and failed ones. Right-click a waiting request to cancel it, or a failed one to download it again; Clear finished empties the finished list, and Open the dashboard goes to the web page for searching other bots. Needs dccore.mrc 1.10: save the new file over the old one and /reload -rs dccore.mrc.
• New: the dccore.mrc window shows how far a list rebuild has got. Typing update in the DCCore window used to say it could take minutes and then nothing until the result. While a rebuild runs - started from the window, with !update, from the dashboard or by LIST_REBUILD_SCHEDULE - the side panel shows a Rebuilding section (the phase, folder 7/20, the files so far and how long it has run), or the same in the title bar when the panel is off, refreshed every few seconds and gone when it ends. Needs dccore.mrc 1.9: save the new file over the old one and /reload -rs dccore.mrc.
• Fixed: the dashboard's Tools page now shows a list rebuild started from anywhere. A rebuild started from mIRC (update in the DCCore window), with !update, or by the rebuild schedule used to run with no progress on the Tools page, whose bar only followed its own button. Now opening Tools - or leaving it open - shows the progress of whatever rebuild is running.
• New: the Downloads page sorts, pages and clears. Click a column heading to sort (again to reverse), choose 10, 15, 20, 50 or 100 rows a page, and use the two Clear buttons to forget every downloaded row, or every failed one, in a click. Only finished rows are removed; the files themselves stay on disk. Clearing a failed request another DCCore bot still holds tells that bot to drop it too - unless a newer request for the same file is waiting there.
• New: the dccore.mrc window shows what the bot downloads. Each file the bot fetches from another bot is reported as it is asked for, queued, received, finished or failed, and the side panel has a Downloading section under Sending with progress and speed. Reload dccore.mrc (version 1.8) to get it.
• New: the Downloads page counts your download queue, and Cancel works on every request that has not started. A strip above the table shows how many downloads are in the queue, running, waiting for the other bot's answer, queued at other bots and still to be asked. A request that had been sent but not answered had no Cancel button; it has one now. Cancelling one also tells the other bot to drop that file, with @<bot>-remove <file> - but only a bot known to be another DCCore, since some other file servers would read that as "remove everything" and clear your whole queue there. DCCore learned that form too: @<nick>-remove still clears your whole queue, and @<nick>-remove <file> (or CTCP REMOVE <file>) removes just that file. The list is now three boxes - in the queue (with Cancel), finished and failed - each sorted and paged on its own, and it no longer redraws when nothing has changed, which could swallow a click on Cancel. A bot that queued your request and sent the file a little after DCCore had given up waiting no longer has it refused: a late answer within 30 minutes is taken. A request that gets no answer is now sent up to three times before it fails; when it does, DCCore tells the other bot to drop it (a DCCore bot only, as above) and keeps the row under Failed so you can ask again.
• Fixed: a bot no longer offers you a file again after you have fetched it. DCCore never told the sending bot how much of a file it had received, so a DCCore sender took a finished send for a failed one and offered the file again a few seconds later. DCCore now acknowledges what it receives, as the DCC protocol asks.
• Fixed: downloads waiting in other bots' queues can no longer tie up the ports your own sends need. When several of them came due at once, each one that used a passive (reverse) connection opened a listening port, past your MAX_FETCH_SLOTS, from the same port range your sends to your own users use. Past the limit, such a download is now asked for again once a slot is free.
• Fixed: the admin console now trusts a local-network address only when you share the bot's router. When you ask the bot for its console, it waits for your connection. It also accepted one from any private-network address, so on a shared network, or behind a proxy that hides the real address, someone else could get there first and take your console's place (they still needed the password). Now a private address is accepted only when you and the bot are behind the same router.
• Fixed: the DCCore window's button in mIRC now stays red (or blue, for a failure) until you look. It went back to black within about half a minute, when the side panel refreshed, so new activity was easy to miss. And with the side panel on, the window no longer prints a [STATUS] line every few minutes - the panel already shows those numbers, live. Without the panel you still get them.
• Update dccore.mrc. The new windows and several fixes above are in the script: save this release's scripts/mirc/dccore.mrc over yours and type /reload -rs dccore.mrc.