Skip to content

Configuration Files

Yukimi Kazari edited this page Jan 19, 2022 · 1 revision

Configuration Options

Table Name: global_parameters You only need to insert and configure required values to start
When configuring values please note if the value is set in param_value or param_data, param_data is for JSON objects while param_value is for text

  • Required - Must be defined to operate
  • Dynamic - These are refreshed or are pulled when needed and do not require restart
  • Database-Only - Can only be set from Database
  • Server ID Required - Requires you to set the Server ID in the config

To filter configuration values down you can use the following options (and prioritized in this order):

  1. system_name - Exact host
  2. application - Exact application
  3. account - Account ID for the instance
  4. serverid - Server ID
  5. config.json

Required config.json Options

  • System Name
  • SQL Login
  • User Account Selections
  • Temp Folder
{
  "SystemName": "Master",
  "SQLServer" : "127.0.0.1",
  "SQLUsername" : "eiga",
  "SQLPassword" : "",
  "SQLDatabase" : "kanmi_system",
  "AuthwareUser": "1",
  "DiscordUser": "1",
  "TwitterUser": "1",
  "PixivUser": "1",
  "YouTubeUser": "1",
  "RSSAccount": "1",
  "FlickrAccount": "1",
  "TempFolder": "./temp"
}

Common

  • MQ Login Required
  • Watchdog Check-in
    • Sets Hostname:Port or IP:Port and Group ID for up/down monitoring
  • FFMPEG Encoder Configuration Database Only
  • Home Server ID Required
  • Message Queue Names Required
  • Sequenzia CDN URL Base (if used)
  • Sequenzia CDS Pickup URL Base Required
{
"MQServer": "192.168.250.X",
"MQUsername": "eiga",
"MQPassword": "",

"Watchdog_Host": "192.168.100.X",
"Watchdog_ID": "main",

"DiscordHomeGuild": "1234567890",

"Discord_Out": "outbox.discord",
"Sequenzia_In": "inbox.sequenzia",
"Twitter_In": "inbox.twitter",
"Pixiv_In": "inbox.pixiv",

"Cache_Base_URL": "https://cdn.seq.moe/",
"Pickup_Base_URL": "https://seq.moe/cds/files/"
}
param_key param_value param_data
mq.login { "host" : "192.168.250.X", "username" : "eiga", "password" : "" }
watchdog.host 192.168.100.X
watchdog.id main
ffmpeg.preview {"exec": "ffmpeg", "vcrf": "30", "scale": "640:-1", "acodec": "aac", "vcodec": "h264", "abitrate": "128K", "vbitrate": "500K"}
discord.home_guild 1234567890
mq.discord.out outbox.discord
mq.fileworker.in inbox.fileworker
mq.twitter.in inbox.twitter
mq.pixiv.in inbox.pixiv
seq.common {"cache_base_url": "https://cdn.seq.moe/", "pickup_base_url": "https://seq.moe/cds/files/"}

Automagic Updater

Below is the suggested configuration values, "DisablePatching" should be set to true on all hosts that are not the main server. Only one host ideally should do patching.

{
"AutomaticUpdates": true,
"AutomaticRestart": true,
"AlwaysRestart": ["Sequenzia"]
}

Advanced Multi-Project Updates:

  • Automatically Download Updates
  • Automatically Install & Restart Affected Apps
  • List of projects that are updated
    • name: Folder Name in reference to ../ of sequenzia-framework
      • Do not set a name for sequenzia-framework or it will block patching
    • Cron schedule for updates
    • Branch name
  • Soft pull repos instead of hard pull
  • Do not install any updates that will cause adverse downtime to apply due to patches
  • Do not restart these PM2 applications
  • Always restart there PM2 application on any commit
{
"AutomaticUpdates": true,
"AutomaticRestart": true,
"DisablePatching": false,
"UpdateProjects" : [
  {
    "name": "sequenzia-web",
    "schedule": "0 * * * *",
    "branch": "main"
  },
  {
    "schedule": "0 * * * *",
    "branch": "main"
  }
],
"SoftUpdateRepos": false,
"NoDowntimeAllowed": false,
"DontRestart": [],
"AlwaysRestart": ["Sequenzia"]
}

Discord I/O

Note: Account information is sensitive and it is up to you if you want to store these values in the database

  • Discord Bot Token Key Required
  • Discord Undelivered Messages Channel Dynamic
  • Acceptable Files to Cache Database Only Dynamic
  • Enable File Management Reaction for Discord Dynamic
  • Discord Bot Info (Shown in Help)
  • Rate Limiters 1 - Priority Message
  • Rate Limiters 2 - Standard Message
  • Rate Limiters 3 - Backlog Message
  • Interval Timers - Refresh Cache Data
  • Interval Timers - Refresh Counts
  • Interval Timers - Refresh SQL Cache / Parameters
  • MQ Prefetch 1 Count - Priority Messages
  • MQ Prefetch 2 Count - Standard Messages
  • MQ Prefetch 3 Count - Backlog Messages
  • Discord Thread Manager (for CMS)
  • Discord CMS Thread Creation Settings
  • Discord Overrides
    • List of IDs of Bots to consider users
    • Consider Websockets Bots to be users and import messages
    • List of Bots to allow reactions from
  • Discord Insights Panel Requires Server ID
{
"Discord_Key": "DISCORDBOTKEY",

"Discord_Recycling_Bin": "1234567890",

"Discord_FSMgr_Enable": true,

"DiscordOwner": "Yukimi Kazari",
"DiscordDescription": "My Discord Bot",
"DiscordPrefix": "!juzo",

"Limiter_1_Tokens": 15,
"Limiter_1_Interval": 60000,

"Limiter_2_Tokens": 10,
"Limiter_2_Interval": 60000,

"Limiter_3_Tokens": 5,
"Limiter_3_Interval": 60000,

"Prefetch_1_Count": 10,
"Prefetch_2_Count": 10,
"Prefetch_3_Count": 5,

"Discord_Timer_Refresh": 300000,
"Discord_Timer_Counts": 900000,
"Discord_Timer_SQLCache": 1200000,

"Discord_Cron_Thread_Rollover": "58 23 * * *",
"Discord_Cron_Thread_Peak": "0 2 * * *",
"Discord_Cron_Thread_Offpeak": "0 12 * * *",
"Discord_Thread_Delete_After_Days": 16,
"Discord_Thread_Unarchive_CMS_Channels": ["829601558680698920"],
"Discord_Thread_Unarchive_CMS_Exclude": [" RT"],

"CMS_Timeline_Parent": 1234567890,
"CMS_New_Thread_Reactions": ["DownloadToArt", "DownloadToNSFW", "DownloadToFat"],

"Discord_Overided_Bots": [],
"Discord_Websockets_As_Users": true,
"Discord_Allow_Reactions_From_Bots": [],

"Discord_Insights_Custom_Image_URL": { "default" : "https://...", "1234567890" : "https://..." }
}
param_key param_value param_data
discord.login DISCORDBOTKEY
discord.undelivered 1234567890
cache.accepted_formats { "images" : ['jpeg','jpg','jiff', 'png', 'webp', 'tiff'], "videos" : [ "mov","mp4","avi","ts","mkv" ] }
discord.mgmt_enabled true
discord.info { "owner" : "Yukimi Kazari", "description" : "Discord Bot", "prefix" : "!juzo" }
discord.limiter.priority { "tokens" : 15, "interval" : 60000 }
discord.limiter.standard { "tokens" : 10, "interval" : 60000 }
discord.limiter.backlog { "tokens" : 5, "interval" : 60000 }
discord.prefetch.priority 10
discord.prefetch.standard 10
discord.prefetch.backlog 5
discord.timers { "refresh_discord_cache" : 15, "refresh_counts" : 60, "refresh_sql_cache" : 10 }
discord.threads {"thread_delete_after": 16, "thread_unarchive_exclude": [" RT"], "thread_unarchive_channels": ["829601558680698920"], "thread_rollover" : "58 23 * * *", "thread_tc_peak" : "0 2 * * *", "thread_tc_offpeak" : "0 12 * * *" }
cms { "timeline_chid": "829601558680698920", "new_thread_reactions": ["DownloadToArt", "DownloadToNSFW", "DownloadToFat"] }
discord.overides { "users" : [], "websocket_is_user": true, "allow_bot_reactions" : [] }
discord.insights {"custom_image_url": "https://media.discordapp.net/..."}

AuthWare

Note: Account information is sensitive and it is up to you if you want to store these values in the database

  • AuthWare Bot Token Required
  • Discord Bot Info (Shown in Help)
{
"Authware_Key": "DISCORDBOTKEY",

"DiscordOwner": "Yukimi Kazari",
"DiscordDescription": "My AuthBot",
"DiscordPrefix": "!auth"
}
param_key param_value param_data
authware.login DISCORDBOTKEY
authware.info { "owner" : "Yukimi Kazari", "description" : "Auth Bot", "prefix" : "!auth" }

FileWorker

Note: If a FileWorker is only use for Sequenzia Uploads, simply do not set a Watch Folder or a Pickup Folder as these requests are handled internally. This will disable folder watching and pickup requests. If the FileWorker will use used for CDS (Content Delivery aka Pickup) you only need to set a Pickup Folder. Otherwise a normal FileWorker only needs a Watch folder set to upload files.

  • Accepted File Formats
    • Files
    • Images
    • Videos
  • Watch Folder
  • Pickup Directory
  • Use Legacy JS Split (Required on Windows)
  • Always Keep Original Image
  • Watch all usable channels
    • Overrides Watch Folder setting in Channels table and will enable all usable channels as a folder
{
"FW_Accepted_Files": ["jpg","png","jpeg","jiff","tiff","mov","mp4","avi","mkv","zip","rar","tar","exe","unitypackage","txt","blend"],
"FW_Accepted_Images": ["jpg","png","jpeg","jiff","tiff"],
"FW_Accepted_Videos": ["mov","mp4","avi","mkv"],

"WatchFolder_1":  "./upload",
"PickupFolder": "./download",
"UseJSSplit": false,
"FW_Always_Keep_Original_Images": true,
"FW_Enable_All_Channels": false
}
param_key param_data
fileworker.accepted_types { "files" : [ "jpg","png","jpeg","jiff","tiff","mov","mp4","avi","mkv","zip","rar","tar","exe","unitypackage","txt","blend" ], "images" : [ "jpg","png","jpeg","jiff","tiff" ], "videos" : [ "mov","mp4","avi","mkv" ] }
fileworker {"watch_dir" : "./upload", "pickup_dir" : "./download", "classic_split": false, "keep_original_images": true, "enable_all_channels": false}

Backup and Sync

  • Disable Backup of Parity File Parts
  • Interval Between Backups Required
  • N items to backup each run Required
  • Sequenzia CDN Base Folder Path (Use SMB or NFS if not on host)
  • FileWorker/CDS Base Folder Path (Use SMB or NFS if not on host)
  • Backup Directory Path (Where files are saved to) Required
{
"Backup_Parts_Disabled": false,
"Backup_Interval_Min": 15,
"Backup_N_Per_Interval": 2400,
"Cache_Base_Path": "/mnt/path",
"Pickup_Base_Path": "/mnt/path",
"Backup_Base_Path": "/mnt/path"
}
param_key param_data
backup {"backup_parts": true, "interval_min": 5, "items_per_backup" : 2400, "backup_base_path": "/mnt/backup/", "pickup_base_path": "/mnt/data/kanmi-files/", "cache_base_path": "/mnt/cache/"}

Twitter CMS

Note: Account information is sensitive and it is up to you if you want to store these values in the database

  • List of Twitter Account Objects
    • Bot Application Login
      • ID of account
      • Consumer Token
      • Consumer Secret
      • Access Token
      • Access Secret
      • Stream Env
      • Flowcontrol Settings (Remove to disable)
        • Status Record Name (Add to status table to link to a channel name indicator, otherwise it will only go to the insights display)
        • Normal - Release Schedule
        • Underflow - Release Schedule
        • Overflow - Release Schedule
        • Overflow Release N of tweets per schedule trigger
        • Overflow Divide by Max Volume (Only applies when no Overflow Schedule is set)
        • Enable Warning in Insights when in Underflow condition
        • Enable Warning in Insights when in Overflow condition
        • Volume Control Settings (Water marks for the state of the release queue)
          • Queue Empty (Indicator only)
          • Queue Underflow
          • Queue Warning (Indicator only)
          • Queue Overflow
  • Rate Limiter - Get Tweet (Seconds)
  • Rate Limiter - Get Mentions (Seconds)
  • Rate Limiter - Media Uploads (Seconds)
{
"Twitter_Accounts" : [
  {
    "id": 1
    "consumer_key": "",
    "consumer_secret": "",
    "access_token": "",
    "access_token_secret": "",
    "flowcontrol": {
      "status_name": "tflow_1",
      "schedule": "*/30 * * * *",
      "schedule_min": "0 * * * *",
      "schedule_max": "*/15 * * * *",
      "max_rate": 1,
      "max_divide": 500,
      "min_alert": true,
      "max_alert": false,
      "volume": {
          "empty": 4,
          "min": 64,
          "warning": 128,
          "max": 1500
      }
    }
  }
]

"Twitter_Timeline_Pull": 1,
"Twitter_MediaUpload_Delay": 90,
"Twitter_Mention_Pull": 1
}
param_key param_value param_data
twitter.account {"access_token": "", "consumer_key": "", "access_secret": "", "consumer_secret": ""}
twitter.limiter.get_timeline 1
twitter.limiter.upload_media 90
twitter.limiter.get_mention 1

Pixiv CMS

  • Disable History and use Bookmarks
{
"Pixiv_No_History": false
}
param_key param_value
pixiv { "disable_history":false }

Feed Parser

  • Flickr API Key
{
"Flickr_Key": "APIKEY"
}
param_key param_value
flickr.login APIKEY

Webcrawler

Any Timer and Settings that is undefined will not be enabled

  • Refresh Timers (Hours)
    • Mixcloud
    • MyFigureColection
    • MPZero
    • Sankaku Complex
  • MyFigureCollection Settings
  • Sankaku Complex Settings
  • MPZero Settings
{
"Mixcloud_Interval": 8,
"MFC_Interval": 1,
"MPZero_Interval": 8,
"SankakuComplex_Interval": 8,

"MFC_Channel": 1234567890,

"SankakuComplex_Pages": [
  {"url": "https://www.sankakucomplex.com/tag/cosplay/",
  "channel": "1234567890"}
],

"MPZero_Channel": 1234567890,
"MPZero_Deep_Crawl": false,
"MPZero_Backlog": false,
"MPZero_Pages": [
  "https://mpzerocos.exblog.jp/page/1/", 
  "https://mpzerocos.exblog.jp/page/2/"
]
}
param_key param_data
webparser.timers {"mpzero": 8, "mixcloud": 8, "sankakucomplex": 8, "myfigurecollection": 1}
webparser.myfigurecollection {"channel": "886085760841834516"}
webparser.sankakucomplex {"pages": [{"url": "https://www.sankakucomplex.com/tag/cosplay/", "channel": "806544860311846933"}]}
webparser.mpzero {"pages": ["https://mpzerocos.exblog.jp/page/1/", "https://mpzerocos.exblog.jp/page/2/"], "backlog": false, "channel": "806544860311846933", "deepcrawl": false}