-
Notifications
You must be signed in to change notification settings - Fork 0
The downloader
The downloader has two modes of operation: new manga, and "update" and existing one. While mostly similar, I will highlight the differences.
Every mention of "API" refers to the MangaDex API.
To download a new manga, the user will click on the Download a Manga button. This will hide the main menu form and open the downloader, which will have all controls locked except for the link TextBox and the search Button. The program expects the user to paste (or write) in the MangaDex link to the manga to be downloaded, then click the search Button.
A typical (and valid) MangaDex link has the following structure: https://mangadex.org/title/MANGA-ID/MANGA-TITLE (OPTIONAL)
The search button will first validate the link by doing several checks:
- Is the TextBox empty?
- Is the domain correct? - "mangadex.org" in the correct part of the link
- Is the link a manga? - this checks that the link points to "title", as opposed to, for example, "chapter"
- Is the link a link? - tries to parse the text as a "https" uri
- Is the manga ID present in the link?
Once the link is considered valid, the program will attempt to get the chapters of the manga by sending a request to the /feed API endpoint with the extracted manga ID, with the Scanlation Group reference expanded. From this call, four data points are taken:
- The chapter numbers
- The chapter IDs
- The number of pages in each chapter
- The scanlation group (if there is no group then the value
Anonymous (No Group)will be logged)
A second API call is made, just with the manga ID, and the references to the cover art, author and artist expanded. From this call, the following data is taken:
- The title
- All alt-titles - the program first attempts to get the alt-titles in the user's preferred language, but should there be none then it defaults to English
- The cover file name
- The author
- The artist
- The "creation date" of the manga
- The status of the manga - "ongoing" and "hiatus" are considered as ongoing (
true), "completed" and "cancelled" are not (false) - The content rating
- All the tags of the "genre" and "theme" types
The chapters and all their associated data are sorted by chapter number, and a "heatmap" of the scanlation groups is made. This keeps track of how many times each scanlation group has appeared up to the processed chapter. The chapters are then added to the CheckedListBox in order, and the appropriate chapters are selected. Each chapter that was only scanlated once is selected; of the ones that appear more than once, the program will try and prioritize the scanlators that translated most chapters up to this one to try and maintain the same scanlation style / quality etc.
The user can now select which chapters they want to download with complete freedom (chapters with multiple scanlators are listed with their respective scanlators) - the user can also select a chapter, hold shift and select another chapter to invert the selection of everything in-between to allow for faster bulk (de)selection, which title to use from the gathered ones, whether to download the pages in original quality or data-saver, the desired format of the output file (.epub or .cbz), where to download the manga (defaulting to where the executable is located; the chosen path is displayed in the ToolTip) and whether to add the downloaded manga to the Library Manager once it's done (if successful). The program also allows the user to quickly unselect or subsequently reselect "extras". As said in the warning that appears, the program has no way to know the actual content of the chapters, so it simply (un/re)selects chapters with a decimal place (i.e. "Chapter 15.5").
Once the user is satisfied with their selection, they can click the Start Downloading button. If they failed to select a title, the title (not alt-titles) will be selected. All the controls except the Download button (it becomes a cancel button) are disabled, with the chapter list being functionally disabled, and the right panel is filled with ProgressBars corresponding to each selected chapter number (while this is happening the window is prevented from refreshing to make the process faster). Please keep in mind the ProgressBars are not very accurate in terms of exactly how far along the process is, but they should give a good idea, not much I can do about that sadly.
The program starts by creating a temporary folder in the selected download location. If the selected file format is .epub, the program will now create the folder structure, as well as the css files, the container.xml file, the mimetype file, the cover.xhtml file, the content.opf with a preliminary structure and empty nodes, and the toc.ncx file with a preliminary structure and empty nodes. Once this is done, the download itself begins.
For each chapter, the program will first call the /at-home/server/CHAPTER-ID endpoint of the API, which returns the base URL and file names of the chapter pages. The program will then iterate through each page link and download it, reporting successes and failures in accordance to the MangaDex API instructions. Each page is saved in its appropriate location and with its appropriate name, and its corresponding .xhtml file is also generated if the target format is .epub. Once all the pages of all the chapters have been downloaded, the program will download the cover using the cover image name obtained in the second API call.
Once that is completed, if the target format is .cbz the files are added to a zip archive with the .cbz extension, the temporary folder is deleted and the download is completed. The form will now close.
If the target format is .epub, the program goes on to add the correct nodes to the content.opf and toc.ncx files. First, the description node of the file is filled with the downloaded chapters' numbers, one per line, formatted as "Ch.NNNN" where N is padding to the last chapter's digits (i.e. if the last chapter is 123, having 3 digits, the list will start as "Ch.001", "Ch.001.5" etc.), followed by the title, creator, date, language and identifier (set to the manga link) nodes. Next the manifest node is filled with the appropriate child nodes, namely the css files, xhtml, and image files, followed by them being added to the spine node as well. Next up is the toc.ncx file, where the UID is set as it is in the content.opf, and depth node is checked to ensure it is at least 2. The program adds the chapters as level 1, and the pages as level 2. Once that is finished, the files are added to a zip archive with the .epub extension, the temporary folder is deleted and the download is completed. The form will now close.
Here are the differences. To do this, the user is expected to click on the Download New Chapters button enabled after the Check Online button found new chapters online.
The form opens with most controlled locked, disallowing the user to change the book link, the title, the target format, the download location (all are auto-filled with current) and whether to add it to the Library Manager. The program will then simulate a click on the search button, executing the same sequence as above, with two key differences:
- Only the chapter numbers higher than the Last Chapter set for that manga are "looked at"
- There is no second API call since the program doesn't need any of that data, it already has it all
Once the user starts the download, the program will extract the existing file into the temporary folder. In the case of .epub being the target, the program will create sub-folders with the appropriate sequential numbers and generate the css files. In the case of .cbz being the target, the program will get the "prefix" of the image files (i.e. if the images are saved as "pg-001.jpg", "pg-002.jpg" etc. it will get the "pg-" to maintain formatting). The download commences normally. The only exception is that the program will no longer download the cover image.
Once the download is finished, if the target is a .cbz then it will simply finish the download as before. Continuing for the .epub format, the program only edits the description node, appending the downloaded chapters in the same format as above. The program will then also append the appropriate nodes in both the content.opf and toc.ncx files in the same way as above. Once done, the download is finished as above.
Cancelling the download simply leads to the removal of the ProgressBars (the window will again be prevented from updating during this, it may take a moment longer) and the deletion of the temporary folder.