Skip to content

Providers & Plugins

Jump to bottom
matty edited this page Jan 14, 2023 · 36 revisions

Note: Data here only applies to v3.0.0. Plugins are not supported on previous versions of aphrodite.

Plugin development

This application supports plugins for other boorus hosted with DanBooru or Philomena. This page should give a basic rundown on developing a plugin.

While plugin support is very initial, it may change per release. If any changes to the provider interfaces occur, the creator of the provider should be responsible for updating the provider.

Providers also have the option to create their own userscript for the booru of their choice. All arguments outlined in the arguments page will apply.

All providers are required to have a valid GUID associated with it, and that it is a unique GUID. GUIDs are used to identify providers, and if the GUID is invalid or not present, the provider will not be loaded.

The following GUIDs are not allowed to be used:

  • 00000000-0000-0000-0000-000000000000 -> The default GUID. Defers to the default provider selected by the user.
  • 0fb5a737-ea09-4615-8048-0dc2bacede18 -> aphrodites GUID. Defers to the internal e621 provider.

Although the internal e621 providers can be used as a reference for what you should do, do NOT use it as a copy/paste for your own provider.

Requirements for developing a provider

I tried to make developing a provider as easy as possible, and hopefully it's as simple as possible.

Aphrodite is built on C# 11 and .NET Framework 4.7.2. It's recommended to use Visual Studio 2022 17.4, a higher version, or any IDE that supports C# 11 and .NET Framework 4.7.2 to build for the compiler flags and runtime additions to work properly. All in all, that will be all that is required to develop a provider. To debug your provider, you can use the debug configuration of aphrodite.

Currently, json.net is used for serializing and deserializing with a desire to revert back to using DataContractJsonSerializer. Your classes should use DataContract and DataMember attributes instead of json.net specific attributes for compatibility with DataContractJsonSerializer if it changes back to that system. The attributes are available in the System.Runtime.Serialization assembly. In addition to JSON-based attributes, you will need to specify attributes for Pool and Post derived classes used by the parse and the provider system.

Posts & PostApiAttribute

PostApi is used to find Properties within your Post class. This attribute is first-come first-serve, whichever property that has a PostApiValue first will be the property that is for that value. There are some require properties for this attribute or the provider will not function. Any value marked with a ! is required, any value marked with ? is a conditional requirement.

  • ArtistTagsArray -> A string[] that contains string-values of artist(s) for that post.
  • Author -> A string of the ID or Username of the person who uploaded the post.
  • DownloadUrl -> A string of the URL to download the file. This isn't required because some boorus may hide the URL but it may still be accessible. Providers will be able to fix this before parsing begins for that post.
  • ! Extension -> A string of the extension of the file. This will be trimmed before being used.
  • FavoriteCount -> An int? of the amount of favorites on the post.
  • FileSize -> A long of the size of the file (in bytes). If your booru does not support it, you can just return 0.
  • FileHash -> A string of the has of the file, MD5, SHA1, etc...
  • ! PostId -> A uint of the ID of the post.
  • ! Rating -> A string of the rating of the post. This should follow suit for the booru provided.
  • Score -> A int? of the total score for the post.
  • ScoreUp -> A int? of the total positive score for the post.
  • ScoreDown -> A int? of the total negative score for the post.
  • AllTagsArray -> A string[] of all the tags for the post. Highly recommended to have this.
  • ?Root -> A Post or Post[] that represents the post object or array within a post search object. See #PostRoot for information on when to use this.

Pools, PoolTypeAttribute, and PoolApiAttribute

Pool classes require a class-specific attribute PoolTypeAttribute which will determine if the pool requires multiple api downloads to parse. This is required or the provider will not load.

  • SingleParse -> The pool has all required post data within the pool API data.
  • MultiParse -> The pool needs to download more API pages to parse the posts within the pool.

The PoolApiAttribute is for Properties in the Pool class, akin to the PostApiAttribute.

  • !Name -> The string name of the pool.
  • ?Posts -> The Post[] of all the posts within the pool, if PooType is set to SingleParse this will be required.
  • !Deleted -> The bool of whether the pool was deleted. Some boorus may delete pools but not the files, so this should be used per-booru. Do your research.
  • !OrderedPostIds -> The uint[] of the post IDs in chronological order.
  • !PageCount -> The int of the amount of posts within the pool, in total. If this is wrong, it may affect the MultiParse type boorus.

ApiSerializedFormatAttribute with JSON and XML APIs

By default, the parser will automatically parse JSON. If the booru API only has XML, you can specify a ApiSerializedFormat attribute on a Pool or Post inherited class, and you can choose between ApiSerializedFormat.JSON and ApiSerializedFormat.XML. You don't have to specify ApiSerializedFormatAttribute and ApiSerializeFormat.JSON if the API is formatted with JSON, it's just there for brevity.

If ApiSerializedFormat.XML is specified, it will attempt to convert the XML into a JSON string and use that. You can get the XML-to-JSON string that the API sees by using XmlConversion.XmlToJson(string). The XML converter is derived from another implementation online, but was modified to handle value types such as ints and bools.

PostRoot

Some boorus may have a "special" way of returning API data, in which it gives you an object of an array of posts instead of just an array of posts. This is annoying, and it is supported for fixing.

Let's say, for example, your booru returns an api like this:

{
    Post:... (array or object)
}

You would have to implement a class that can handle that object itself, since it's the object that holds that value. If the posts are within a sub-object (or sub-sub object, or even more layered), you can add a getter property for the nested post object or array, since the attribute only scans the object itself and not sub-objects.

Your post root class would need to inherit the PostRoot class for it to catch on properly, and then implement a Post array or object property. For example:

public sealed class BooruPostData : PostRoot {
    [PostApi(PostApiValue.Root)]
    public Post[] PostData { get; }
    ...

The end goal

The goal was to make the provider be as hands-off as possible, to have the downloaders handle as much work as possible for you.

IProvider

The default provider interface. It is not used generally for much, but every provider (aside from ICustomProvider) implement this interface.

Properties

  • string Name
    The name of the provider. This will appear in the log and the drop-down buttons on the main form.
  • string SiteAccessName
    The short and sweet URL to access the booru. An example would be e621.net. There is no https or subdomains, just the sub domain (if required), domain name, and TLD.
  • Image Icon
    The System.Drawing.Image object to represent the booru. This can be the favicon. If this value is NULL, it will use a grayed-out application icon.
  • string BaseApiUrl
    The base URL that will be used to connect to the API, with format parameters following
    • {0} must be the tag or pool id or the image id, or whatever necessary identifier.
    • {1} should the the page number for page downloads, if the provider is a ITagProvider or MultiPost IPoolProvider.
    • {2} should be the max amount of files displayed per page, if the provider is a ITagProvider or MultiPost IPoolProvider.
  • string BaseFrontendUrl
    The base URL that will be used to connect to the frontend of the booru, with the format parameters followed
    • {0} must be the tag, pool id, image id, or whatever identifier used with that booru.
    • {1} must be the page number, if the provider is a ITagProvider.
  • string DateTimeFormat
    If the booru has a DateTime object, this should be the format of that value. Unix epoch timestamps are not currently supported.

Methods

  • void Initialize()
    Loads the provider instance, and runs any necessary code to make the provider work.
  • void PrepareDownloadClient(HttpClientAddons)
    Occurs when the download client of the download instance is created. This will allow you to add required data to the client before it is used. It currently only allows cookies to be added.

ITagProvider

The provider that is used for downloading files using tags. It's the main attraction.

Sub-providers

ITagProvider isn't used as the main provider interface, instead there are sub-providers to differentiate them and pass them to their proper provider. Currently supported sub-providers are IDanBooruTagProvider and IPhilomenaTagProvider. No unique methods are currently used.

Properties

  • string FolderName
    The name of the output folder that the downloader will save the files to. I usually use acronyms for boorus that have words, like furbooru would become fb, but you can choose whatever.
  • int PostsPerPage
    The amount of posts that will appear per-page on the API. Image downloads, and some pool boorus will not require this value.
  • bool SupportsFileSizes
    The flag for the downloader to display a total download size, if the API contains a value for the file size for each file.
  • string BasePageApiUrl
    The base url for the API page for a page that the user usually sees. This should not have a changed posts-per-page query string. The following format parameters are required:
    • {0} -> The requested page number.
    • {1} -> The tags that are included with the page.
  • string BasePageApiUrlPageOnly
    The This is the same as above, but it does not include the tags format parameter. The flag for the downloader to display a total download size, if the API contains a value for the file size for each file.
  • bool SupportsPageDownload
    Whether the provider supports downloading single pages from the API. In general, most tag providers do have support for this, but the program does extra logic for page downloads to ensure it's successful.

Methods

  • bool ValidPageLink(string)
    Returns true if the input string is a valid page URL for the provider; otherwise, false. This is used for page downloading.
  • bool ValidPageWithTagsLink(string)
    Returns true if the input string is a valid page URL that contains tags; otherwise, false. This is used for extracting tags from page urls.
  • string GetTags(string)
    Returns the tags extracted from the input string. They can be cleaned or just what is available in the Uri.
  • int GetPageNumber(string)
    Returns the current page number extracted from the input string. If no page number is present, it might be page 1. If a booru does not display that value, who knows what to do.
  • string SanitizeTags(string, bool)
    Returns the input tags that get sanitized for use based on the Identifier bool value:
    • true -> The tags should appear as pretty as possible, to display to the user on the form.
    • false -> The tags should be as friendly as possible to the API, which may require encoding into HTML. It's up to the provider to format them.
  • bool PostUnavailable(Post)
    Returns true if the NewPost is newer than ExistingPost.
  • bool PostReplaced(Post, Post)
    Returns true if the NewPost has a file that replaced a previous version compared to ExistingPost.

IPoolProvider

The provider that is used for downloading pools from boorus. The less used, but equally appreciated part.

Sub-providers

IPoolProvider isn't used as the main provider interface, instead there are sub-providers to differentiate them and pass them to their proper provider. Currently supported sub-providers are IDanBooruPoolProvider and IPhilomenaPoolProvider. No unique methods are currently used.

Properties

  • string FolderName
    The name of the output folder that the downloader will save the files to. I usually use acronyms for boorus that have words, like furbooru would become fb, but you can choose whatever.
  • int PostsPerPage
    The amount of posts that will appear per-page on the API. Image downloads, and some pool boorus will not require this value.
  • bool SupportsFileSizes
    The flag for the downloader to display a total download size, if the API contains a value for the file size for each file.
  • string BasePoolSearchApiUrl
    The base URL for multiparse pools that will be repeatedly downloaded. This 3 format parameters:
    • {0} -> The pool post search query string, for posts within that pool only. This will be replaced with the pool id.
    • {1} -> The page number query string.
    • {2} -> The posts per page query string.

Methods

  • bool ValidId(string)
    Returns true if the input string is a valid pool ID for the provider; otherwise, false.
  • bool ValidPoolLink(string)
    Returns true if the input string is a valid pool URL for the provider; otherwise, false.
  • string GetPoolId(string)
    Returns the pool id that is extracted from the pool url in the string parameter.
  • string CleanUrl(string)
    Returns the pool url from the string parameter that has any unnecessary data removed from it.
  • bool PostUnavailable(Post)
    Returns true if the NewPost is newer than ExistingPost.
  • bool PostReplaced(Post, Post)
    Returns true if the NewPost has a file that replaced a previous version compared to ExistingPost.

IImageProvider

The provider for downloading single images. This is a very simple provider and is 100% ambiguous when it comes to data. All of it is basically handled by the provider, which means any site is technically supported, if implemented properly.

Properties

  • string FolderName
    The name of the output folder that the downloader will save the files to. I usually use acronyms for boorus that have words, like furbooru would become fb, but you can choose whatever.

Methods

  • bool ValidId(string)
    Returns true if the input string value is a valid post ID for the provider; otherwise, false.
  • bool ValidImageLink(string)
    Returns true if the input string value is a valid post link for the provider; otherwise, false.
  • string GetImageId(string)
    Returns the image ID that gets extracted from the input string value.
  • bool PostUnavailable(Post)
    Returns true if the NewPost is newer than ExistingPost.
  • bool PostReplaced(Post, Post)
    Returns true if the NewPost has a file that replaced a previous version compared to ExistingPost.

ITagScannerProvider

This provider was created for a simple way to check individual posts that may have been filtered that the user wishes to NOT have lost. It doesn't overwrite the tag filters and it doesn't download the images, only the API data.

Properties

  • string BaseFrontendTagSearchUrl
    The frontend url for the tag search site.

Methods

  • string GetPostId(string)
    The helper method to get the ID of the post for the main form to handle with. It's passed to GetPostTags(...).

ICustomProvider

Custom providers are not supported quite yet, due to concerns. I will hopefully find a way to solve those concerns, so I can remove the InkBunny and Imgur downloader from the main application and have them be their own custom provider. Until then, this section will remain as is.

Arguments

aphrodite will forward any arguments to your provider if the GUID is to the right of the argument id. Currently, tags, pages, pools, and image are supported for forwarding, while poolwishlist will continue to be handled by aphrodite.

When using arguments with providers, make sure users can access them by following the schema for the argument system: <argument>:<your GUID> .... An example of this would e tags:1234... "checkmark".

The pool wishlist is supported by using poolwl:<your GUID> <pool url> [pool name]. The pool URL is required, while the pool name is optional. If no pool name is given, the name will be saved as empty pool name.

Finally, arguments with spaces should be "in doubble-quotes". If you require using double-quotes, you can \"escape\" them or use the internal double-quote marker {{%DQ%}}. If you require that marker for other purposes, it is up to you to work around it.

Hosting your provider

aphrodite will host the provider, if you wish to push it to the repo. However, issues opened regarding your provider will be marked as invalid. If you wish to handle issues, you can open your own repo and either open a new issue asking to be added to the wiki, or if it's supported, open a wiki edit pull request to add yours to the list.

Available providers

No providers are available at this time.

Clone this wiki locally