Skip to content

Providers & Plugins

matty edited this page Jan 10, 2023 · 36 revisions

Note: Data here only applies to v3.0.0. Plugins are not supported on previous versions of aphrodite.

Plugin development

This application supports plugins for other boorus hosted with DanBooru or Philomena. This page should give a basic rundown on developing a plugin.

While plugin support is very initial, it may change per release. If any changes to the provider interfaces occur, the creator of the provider should be responsible for updating the provider.

Providers also have the option to create their own userscript for the booru of their choice. All arguments outlined in the arguments page will apply.

All providers are required to have a valid GUID associated with it, and that it is a unique GUID. GUIDs are used to identify providers, and if the GUID is invalid or not present, the provider will not be loaded.

The following GUIDs are not allowed to be used:

  • 00000000-0000-0000-0000-000000000000 -> An invalid GUID, which is the default value. Defers to the internal e621 provider.
  • 0fb5a737-ea09-4615-8048-0dc2bacede18 -> aphrodites GUID. Defers to the internal e621 provider.

Finally, your provider should be as hands-off to the main application as possible. The program will call the provider when it requires data, but the provider should only call the program for the tag filtering system and saving the infos.

You should create your own logic to parse & save. Although the internal e621 providers can be used as a reference for what you should do, do NOT use it as a copy/paste for your own provider.

Requirements for developing a provider

I tried to make developing a provider as easy as possible, and hopefully it's as simple as possible.

Aphrodite is built on C# 11 and .NET Framework 4.7.2. It's recommended to use Visual Studio 2022 17.4, a higher version, or any IDE that supports C# 11 and .NET Framework 4.7.2 to build for the compiler flags and runtime additions to work properly. All in all, that will be all that is required to develop a provider. To debug your provider, you can use a build of Aphrodite to verify and test outputs.

You will need to reference System.Runtime.Serialization for your project, as the way JSON is serialized and deserialized is using System.Runtime.Serilization.Json.DataContractJsonSerializer. Be sure to use Edit -> Paste Special -> Paste JSON as Classes available in Visual Studio 2022 to easily create API-ready classes, marking all classes with the DataContract attribute, as well as marking all properties/fields that are serialized with the DataMember(Name="") attribute. I won't give you a tutorial on how to use DataContractJsonSerializer, but it's pretty easy. Your classes MUST inherit aphrodite.Post in order to be applicable to be used.

Additionally to the DataMember attribute, you will need to set a custom Api attribute to specific members to gather data for the provider. The following ApiValue values are required:

  • DownloadUrl
  • Extension
  • PostId
  • Rating
  • Tags, not required but extremely recommended.
  • PoolOrderedPostIds
  • PoolSingleApiParse
  • PoolPageCount

The rest of the ApiValue values are optional, but it's recommended to fill in as much of the values as possible. Some boorus may not have all the data that is there, so you can skip them.

The info below should help you understand what you should expect when building a provider.

(Notice: Json.Net may take over DataContractJsonSerializer after more practice is made. It's not happening now due to the fact I wish to maintain as low of a size as possible, but the switch over may occur in the future.)

IProvider

The default provider interface. It is not used generally for much, but every provider (aside from ICustomProvider) implement this interface, and it's methods and values.

Properties

  • string Name
    The name of the provider. This will appear in the log and the drop-down buttons on the main form.
  • string SiteAccessName
    The short and sweet URL to access the booru. An example would be e621.net. There is no https or subdomains, just the sub domain (if required), domain name, and TLD.
  • Image Icon
    The System.Drawing.Image object to represent the booru. This can be the favicon. If this value is NULL, it will use a grayed-out application icon.
  • string BaseUrl
    The base URL that will be used to connect to the API, with format parameters following
    • {0} must be the tag or pool id or the image id, or whatever necessary identifier.
    • {1} should the the page number for page downloads, if applicable.
    • {2} should be the max amount of files displayed per page, if applicable.
  • string FolderName
    The name of the output folder that the downloader will save the files to. I usually use acronyms for boorus that have words, like furbooru would become fb, but you can choose whatever.
  • int PostsPerPage
    The amount of posts that will appear per-page on the API. Image downloads, and some pool boorus will not require this value.
  • string DateTimeFormat
    If the booru has a DateTime object, this should be the format of that value. Unix epoch timestamps are not currently supported.
  • bool SupportsFileSizes
    The flag for the downloader to display a total download size, if the API contains a value for the file size for each file.

Methods

  • void Initialize()
    Loads the provider instance, and runs any necessary code to make the provider work.
  • void PrepareDownloadClient(HttpClientAddons)
    Occurs when the download client of the download instance is created. This will allow you to add required data to the client before it is used. It currently only allows cookies to be added.

ITagProvider

The provider that is used for downloading files using tags. It's the main attraction.

Sub-providers

ITagProvider isn't used as the main provider interface, instead there are sub-providers to differentiate them and pass them to their proper provider. Currently supported sub-providers are IDanBooruTagProvider and IPhilomenaTagProvider. No unique methods are currently used.

Derived Objects

ITagProvider heavily uses a base Post object to maintain a certain level of ambiguity. The provider must fill in the following properties when parsing:

  • string ParsingPostId
  • string GeneratedFileName
  • string DownloadURL
  • string ParsingFileExtension
  • int ParsingScore
  • int ParsingFavoriteCount
  • List<string> ParsingTags
  • long ParsingFileSize

If any of the value names above are desired or are available by the API when deserializing, you may use new or override. However you must use base.Property in the get/set operations of the property so the downloader can handle the data from the API. All other properties in the object are handled by the downloader.

Properties

  • bool SupportsPageDownload
    Whether the provider supports downloading single pages from the API. In general, most tag providers do have support for this, but the program does extra logic for page downloads to ensure it's successful.

Methods

  • bool ValidPageUrl(string)
    Returns true if the input string is a valid page URL for the provider; otherwise, false. This is used for page downloading.
  • bool ValidPageWithTagsUrl(string)
    Returns true if the input string is a valid page URL that contains tags; otherwise, false. This is used for extracting tags from page urls.
  • Uri GetApiUrl(Uri)
    Returns a new Uri that converts the input Uri into an API Uri. This should not mess with important data, such as tags or page number, and should only change necessary data to access the API with the values in the input Uri. This should always begin with "https://" or "http://" or it will throw.
  • string ExtractTagsFromUri(Uri)
    Returns the tags extracted from the input Uri. They can be cleaned or just what is available in the Uri.
  • int ExtractPageNumberFromUri(Uri)
    Returns the current page number extracted from the input Uri. If no page number is present, it might be page 1. If a booru does not display that value, who knows what to do.
  • string SanitizeTags(string, bool)
    Returns the input tags that get sanitized for use based on the Identifier bool value:
    • true -> The tags should appear as pretty as possible, to display to the user on the form.
    • false -> The tags should be as friendly as possible to the API, which may require encoding into HTML. It's up to the provider to format them.
  • Post[] LoadExistingPosts(string Json);
    Returns a Post array that deserializes the Json value, which is the previously saved info data.
  • Post[] DeserializePage(string Json)
    Returns a Post array of pages from an API download. The provider is required to give the right array or to return an empty array (which will break the loop).
  • bool Parse(Post, string)
    Parses the Post object, using the string file name schema to generate a file name for that post that is supported by the provider. Return true if the file is able to be downloaded at all, otherwise `false.
  • bool PostUpdated(Post, Post)
    Returns true if the NewPost is newer than ExistingPost.
  • PostReplaced(Post, Post)
    Returns true if the NewPost has a file that replaced a previous version compared to ExistingPost.
  • string GetInfoString(List<object>, List<object>)
    Pre-downloading retrieves the info string for the lists of New and Existing posts. This should return a string of the two lists joined together. This is also the method where you can order the lists or do any last-minute modifications to the post objects, if required.

IPoolProvider

The provider that is used for downloading pools from boorus. The less used, but equally appreciated part.

Sub-providers

IPoolProvider isn't used as the main provider interface, instead there are sub-providers to differentiate them and pass them to their proper provider. Currently supported sub-providers are IDanBooruPoolProvider and IPhilomenaPoolProvider. No unique methods are currently used.

Derived Objects

IPoolProvider heavily uses a base Pool object to maintain a certain level of ambiguity. The provider must fill in the following properties:

  • bool SingleApiParse
  • int TotalPagesToDownload (Only if SingleApiParse is false or requires another API download.)
  • List<uint> OrderedPagePostIDs
  • string ParsingPoolId
  • bool PageDeleted
  • string PoolName
  • Post[] PoolPosts (Only if SingleApiParse is true and the JSON from extracting the pool info has all the page information required.)

If any of the value names above are desired or are available by the API when deserializing, you may use new or override. However you must use base.Property in the get/set operations of the property so the downloader can handle the data from the API. string FormattedPoolName is filled in by the downloader.

In addition to the Post objects (which can be read above).

Properties

  • string BasePoolUrl
    The base URL for the Pool API. This should include data such as the pool name, pool description, pool post IDs, etc.

Methods

  • bool ValidPoolUrl(string)
    Returns true if the input string is a valid pool URL for the provider; otherwise, false.
  • bool ValidId(string)
    Returns true if the input string is a valid pool ID for the provider; otherwise, false.
  • string ExtractPoolId(string)
    Returns the pool id that is extracted from the pool url in the string parameter.
  • string CleanUrl(string)
    Returns the pool url from the string parameter that has any unnecessary data removed from it.
  • Pool GetPool(string)
    Returns a Pool object that is the provider-specific Pool object with its required values.
  • Post[] LoadExistingPosts(string Json);
    Returns a Post array that deserializes the Json value, which is the previously saved info data.
  • Post[] DeserializePage(string Json)
    Returns a Post array of pages from an API download. The provider is required to give the right array or to return an empty array (which will break the loop).
  • bool Parse(Post, string, out uint)
    Parses the Post object, using the string file name schema to generate a file name for that post that is supported by the provider. Return true if the file is able to be downloaded at all, otherwise false. The uint` value must be the PostID of that post, so the downloader can find the number of the page.
  • bool PostUpdated(Post, Post)
    Returns true if the NewPost is newer than ExistingPost.
  • PostReplaced(Post, Post)
    Returns true if the NewPost has a file that replaced a previous version compared to ExistingPost.
  • bool PoolInfoChanged(string, object)
    Returns true if the string (which is the previous pool.json file, you should deserialize it to compare) is different to the base pool info in object. If the string is invalid, return true; if the object is invalid, return false.
  • string GetPoolInfoString(object)
    Returns a string of the pool info object serialized.
  • string GetInfoString(List<object>, List<object>)
    Pre-downloading retrieves the info string for the lists of New and Existing posts. This should return a string of the two lists joined together. This is also the method where you can order the lists or do any last-minute modifications to the post objects, if required.

IImageProvider

The provider for downloading single images. This is a very simple provider and is 100% ambiguous when it comes to data. All of it is basically handled by the provider, which means any site is technically supported, if implemented properly.

Properties

There are no specific properties for the image provider.

Methods

  • bool ValidImageLink(string)
    Returns true if the input string value is a valid post link for the provider; otherwise, false.
  • bool ValidId(string)
    Returns true if the input string value is a valid post ID for the provider; otherwise, false.
  • string ExtractImageId(string)
    Returns the image ID that gets extracted from the input string value.
  • void PreParse(DownloadInfo, ImageData, ref bool)
    This will occur before the main API parsing begins.
  • void Parse(DownloadInfo, ImageData, ref bool)
    The main parsing logic. Here you can serialize the DownloadInfo.CurrentJson string into an API object,using DownloadInfo.CurrentJson.JsonDeserialize<T>([string DateTimeFormat]). Here you are responsible for generating required data for the file.
  • void SaveInfo(DownloadInfo, ImageData)
    The pre-download step that saves the post infos as well as the main pool information in the Data.PoolInfo object. This is handled by the provider due to, again, each API being unique. You are responsible for using DownloadHelpers.SaveInfo(Post) to save each info.

ITagScannerProvider

This provider was created for a simple way to check individual posts that may have been filtered that the user wishes to NOT have lost. It doesn't overwrite the tag filters and it doesn't download the images, only the API data.

This interface is not required to implement everything, only the things below are used.

Properties

  • string Name
    The name of the provider. This will appear in the log and the drop-down buttons on the main form.
  • string SiteAccessName
    The short and sweet URL to access the booru. An example would be e621.net. There is no https or subdomains, just the domain name and TLD.
  • Image Icon
    The System.Drawing.Image object to represent the booru. This can be the favicon. If this value is NULL, it will use a grayed-out application icon.
  • bool Available
    The flag that the provider is available to be loaded and used. This should not be true unless it's a release built library, and is finished and safe for using.
  • string BasePostUrl
    The base URL for a post URL that contains "{0}" for the Post ID.
  • string BaseSearchUrl`
    The base URL for a Search URL that contains the "{0}" for the tags to be formatted in.

Methods

  • string[] GetPostTags(string)
    The method that parses the string JSON and returns a string array of the tags for that post.
  • string GetPostId(string)
    The helper method to get the ID of the post for the main form to handle with. It's passed to GetPostTags(...).

Delegates

  • string ApiDownload(string)
    This is called back to the offending tag form to download the API data from the input string. The return value is the downloaded data as a string.

Every other implemented field or function can be safely ignored as it is not used by the form.

ICustomProvider

Custom providers are not supported quite yet, due to concerns. I will hopefully find a way to solve those concerns, so I can remove the InkBunny and Imgur downloader from the main application and have them be their own custom provider. Until then, this section will remain as is.

IInvalidProvider

An empty interface that is used for testing. This should be ignored, if it is visible outside of the aphrodite application.

Tag filtering

aphrodite will handle tag filtering for your provider, using the tag filtering system.

Your provided booru must either conform to the DanBooru tagging style, or you must account for spaces in tags by replacing them with_underscores per file.

When you are ready to get the designation of the parsed post, use TagFilters.GetDesignation(...) to retrieve the designation.

Arguments

aphrodite will forward any arguments to your provider if the GUID is to the right of the argument id. Currently, tags, pages, pools, and image are supported for forwarding, while poolwishlist will continue to be handled by aphrodite.

When using arguments with providers, make sure users can access them by following the schema for the argument system: <argument>:<your GUID> .... An example of this would e tags:1234... "checkmark".

The pool wishlist is supported by using poolwl:<your GUID> <pool url> [pool name]. The pool URL is required, while the pool name is optional. If no pool name is given, the name will be saved as empty pool name.

Finally, arguments with spaces should be "in doubble-quotes". If you require using double-quotes, you can \"escape\" them or use the internal double-quote marker {{%DQ%}}. If you require that marker for other purposes, it is up to you to work around it.

Json and XML

aphrodite uses DataContractJsonSerializer to serialize and deserialize. If your provider uses XML, you can use the string.XmlToJsonDeserialize(string) method to convert it to a JSON object and then deserialize. You can use string XmlConversion.XmlToJson(string) to get a string version of the Json to paste as a json class.

Hosting your provider

aphrodite will host the provider, if you wish to push it to the repo. However, issues opened regarding your provider will be marked as invalid. If you wish to handle issues, you can open your own repo and either open a new issue asking to be added to the wiki, or if it's supported, open a wiki edit pull request to add yours to the list.


Available providers

No providers are available at this time.

Clone this wiki locally