Asagi

What is Asagi?

Asagi was a reimplementation of Fuuka's dumper side in Java. It allowed you to dump and archive posts, images and all relevant data from imageboards into a local database and local image store. The original project only supported 4chan.

Using Ena as an Asagi drop-in replacement

I've added support to use Asagi's schema for Ena. All you need to change is the config file. Improvements have been made such as the overall lower memory usage and bandwidth. Unlike Asagi, things aren't kept in memory and the database does all the hard work, so it may feel a bit slower because we're relying on a questionable DBMS to do consistent UPSERTs.

Did you change anything?

I added an archivedOn value in the exif column because I believe it was originally supposed to be there.
The exif column is a TEXT field and now looks something like {"uniqueIps": "32", "archivedOn": 1234567}.

Runtime dependencies

• MySQL^* >= 8
  _{*As long as it has the JSON_TABLE function}

Example config file

The engine can be mysql|innodb|tokudb, with mysql synonymous to innodb.
Notice the asagiMode and strictMode fields. Both of these need to be set true.

{
  "settings": {
    "engine": "mysql",
    "database": "asagi",
    "schema": "asagi",
    "host": "localhost",
    "port": 3306,
    "username": "root",
    "password": "zxc",
    "charset": "utf8mb4",
    "path": "./archive",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0",
    "apiUrl": "https://a.4cdn.org",
    "mediaUrl": "https://i.4cdn.org",
    "asagiMode": true,
    "strictMode": true
  },
  "boardSettings": {
    "retryAttempts": 3,
    "refreshDelay": 20,
    "throttleMillisec": 300,
    "downloadArchives": false,
    "downloadMedia": false,
    "downloadThumbnails": true,
    "keepMedia": false,
    "keepThumbnails": true
  },
  "boards": [
    { "board": "cm" },
    { "board": "co" },
    { "board": "d" },
    { "board": "diy" },
    { "board": "e" },
    { "board": "f" },
    { "board": "fa" }
  ]
}

boardSettings is the default settings for all boards listed in the boards array. To override any attribute, just specify them in the specific board.

Note:
You could still use the main schema with Postgres as it has an Asagi schema through VIEWs. That way you get the best of both worlds!