Skip to content

Commit

Permalink
Version 342
Browse files Browse the repository at this point in the history
  • Loading branch information
@hydrusnetwork
hydrusnetwork committed Mar 6, 2019
1 parent 4b6fc54 commit 1a7fe45
Show file tree
Hide file tree
Showing 34 changed files with 915 additions and 325 deletions.
2 changes: 2 additions & 0 deletions client.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
argparser.add_argument( '-d', '--db_dir', help = 'set an external db location' )
argparser.add_argument( '--no_daemons', action='store_true', help = 'run without background daemons' )
argparser.add_argument( '--no_wal', action='store_true', help = 'run without WAL db journalling' )
argparser.add_argument( '--no_db_temp_files', action='store_true', help = 'run the db entirely in memory' )

result = argparser.parse_args()

Expand Down Expand Up @@ -77,6 +78,7 @@

no_daemons = result.no_daemons
no_wal = result.no_wal
HG.no_db_temp_files = result.no_db_temp_files

#

Expand Down
2 changes: 2 additions & 0 deletions client.pyw
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ try:
argparser.add_argument( '-d', '--db_dir', help = 'set an external db location' )
argparser.add_argument( '--no_daemons', action='store_true', help = 'run without background daemons' )
argparser.add_argument( '--no_wal', action='store_true', help = 'run without WAL db journalling' )
argparser.add_argument( '--no_db_temp_files', action='store_true', help = 'run the db entirely in memory' )

result = argparser.parse_args()

Expand Down Expand Up @@ -77,6 +78,7 @@ try:

no_daemons = result.no_daemons
no_wal = result.no_wal
HG.no_db_temp_files = result.no_db_temp_files

#

Expand Down
25 changes: 25 additions & 0 deletions help/changelog.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,31 @@
<div class="content">
<h3>changelog</h3>
<ul>
<li><h3>version 342</h3></li>
<ul>
<li>added support for webp import. it does not yet support animated webps, which, if the local platform supports, will import like apngs used to: just the first frame</li>
<li>added support for tiff import. it works ok for 24bit and 8bit (monochrome) tiffs, but I am not sure how well it will do with 48bit</li>
<li>both webp and tiff should work on the duplicate files system</li>
<li>improved webm detection to include opus audio (previously, these files were falling back to mkv)</li>
<li>fixed an issue where unusual formats with duration but no frames or frames but no duration were being sorted and otherwise presented incorrectly</li>
<li>improved autocomplete job cancelability. this job can now cancel much faster on large jobs, meaning typing searches with large result sets will hit less CPU and return faster on subsequent keystrokes</li>
<li>_all_ of the complicated 'copy url' commands from the thumbnail right-click->known urls menu are now available on the 'open' submenu! if there is more than one url to open (e.g. 'open all of these files blahbooru post urls' on a selection of 50 files), you will be presented with a yes/no dialog to confirm, and it will open one url in your browser every second (with a cancellable popup if num_urls > 5)</li>
<li>by default, system:everything is now hidden if its total files is >10k. you can force it to always show under options->default system predicates</li>
<li>the gallery downloader's list's status column now shows gallery status (deferring to active file status) when appropriate and shows 'done!' when all work is complete</li>
<li>after working back and forth with a user, I _believe_ the linux similar files >0 distance search crash is finally fixed</li>
<li>fixed sorting by media views/viewtiming with collections</li>
<li>a single-selected collection right-click now shows total media views for all files in the collection! you can now see how long you have been viewing an artist!</li>
<li>fixed an issue that lead to export folders not running on always-on clients as often as they should</li>
<li>updated the gelbooru 0.2.5 file page parser to pull rating tag from the correct location (previously, it was pulling from what appears to be a site-wide 'mature' browser hint)</li>
<li>improved memory cleanup stability when animations and other parts of the video rendering pipeline are deleted--this _may_ fix some rare crashes</li>
<li>increased animation rendering aggression overall and particularly in 'future' of frame buffer</li>
<li>if a video renderer that is asked to start some way into the video fails to render anything, it will now fall back to trying to render from the beginning. this is slightly hacky atm and leads to out of phase rendering frames, but it is better than an error</li>
<li>added a '--no_db_temp_files' launch parameter that will force the client or server to return to the recent old behaviour of exclusively using memory for journalling. this is useful if your temp directory is small and/or your available ram is very large. if running in this mode, the client will attempt to check available memory (instead of free space on your temp dir) before performing very large transactions</li>
<li>with the new lighter-weight update transactions, the client now tests for less free space for journalling before running repository update processing</li>
<li>added /get_files/search_files to the client api, which does the first half of file searching. it allows tag search (including -tag negation) and system inbox/archive. since the second half, which will fetch file metadata, is not yet in, this can't do anything interesting yet</li>
<li>updated help and unit tests to support this, client api version is now 3</li>
<li>some misc refactoring</li>
</ul>
<li><h3>version 341</h3></li>
<ul>
<li>client api:</li>
Expand Down
51 changes: 47 additions & 4 deletions help/client_api.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ <h4>Adding URLs</h4>
<li><a href="#add_urls_add_url">POST /add_urls/add_url</a></li>
<li><a href="#add_urls_associate_url">POST /add_urls/associate_url</a></li>
</ul>
<h4>Searching and Fetching Files</h4>
<ul>
<li><a href="#get_files_search_files">POST /get_files/search_files</a></li>
</ul>
</ul>
<h3>Access Management</h3>
<div class="apiborder" id="api_version">
Expand Down Expand Up @@ -471,15 +475,14 @@ <h3><b>POST /add_urls/associate_url</b></h3>
<li>hashes : (a list of SHA256 hashes)</li>
</ul>
</li>
<p>All of these are optional, but you obviously need to have at least one of 'url' arguments and one of the 'hash' arguments. Unless you really know what you are doing with URL Classes, I strongly recommend you stick to associating URLs with just one single 'hash' at a time. Multiple hashes pointing to the same URL is unusual and frequently unhelpful.</p>
<p>All of these are optional, but you obviously need to have at least one of 'url' arguments and one of the 'hash' arguments. The single/multiple arguments work the same--just use whatever is convenient for you. Unless you really know what you are doing with URL Classes, I strongly recommend you stick to associating URLs with just one single 'hash' at a time. Multiple hashes pointing to the same URL is unusual and frequently unhelpful.</p>
<li>
<p>Example request body:</p>
<ul>
<li>
<pre>{
"url_to_add": "https://rule34.xxx/index.php?id=2588418&page=post&s=view",
"hash": "3b820114f658d768550e4e3d4f1dced3ff8db77443472b5ad93700647ad2d3ba"
}
}</pre>
</li>
</ul>
Expand All @@ -488,11 +491,51 @@ <h3><b>POST /add_urls/associate_url</b></h3>
</ul>
</div>
<h3>Searching Files</h3>
<div class="apiborder">
<h3><b>GET file search</b></h3>

<div class="apiborder" id="get_files_search_files">
<h3><b>POST /get_files/search_files</b></h3>
<p><i>Search for the client's files.</i></p>
<ul>
<li>
<p>Headers:</p>
<ul>
<li>Hydrus-Client-API-Access-Key : (Your hexadecimal access key)</li>
</ul>
</li>
<li>
<p>Arguments (in percent-encoded JSON):</p>
<ul>
<li>tags : (a list of tags you wish to search for)</li>
<li>system_inbox : true or false (optional, defaulting to false)
<li>system_archive : true or false (optional, defaulting to false)
</ul>
</li>
<li>
<p>Example request for all files in the inbox with tags "blue eyes", "blonde hair", and "&#1082;&#1080;&#1085;&#1086;":</p>
<ul>
<li><p>/get_files/search_files?system_inbox=true&tags=%5B%22blue%20eyes%22%2C%20%22blonde%20hair%22%2C%20%22%5Cu043a%5Cu0438%5Cu043d%5Cu043e%22%5D</p></li>
</ul>
</li>
<p>If the access key's permissions only permit search for certain tags, at least one whitelisted/non-blacklisted tag must be in the "tags" list or this will 403. Tags can be prepended with a hyphen to make a negated tag (e.g. "-green eyes"), but these will not be eligible for the permissions whitelist check.</p>
<p>Response description: The full list of numerical file ids that match the search.</p>
<li>
<p>Example response:</p>
<ul>
<li>
<pre>{
"file_ids": [ 125462, 4852415, 123, 591415 ]
}</pre>
</li>
</ul>
</li>
<p>The search will be performed on the 'local files' file domain and 'all known tags' tag domain.</p>
<p>Note that most clients will have an invisible system:limit of 10,000 files on all queries. I expect to add more system predicates to help searching for untagged files, but it is tricky to fetch all files under any circumstance. Large queries may take several seconds to respond.</p>
</ul>
</div>
<div class="apiborder">
<h3><b>GET file metadata</b></h3>
# id to info objects
# hashes only arg
</div>
<div class="apiborder">
<h3><b>GET file</b></h3>
Expand Down
24 changes: 16 additions & 8 deletions include/ClientAPI.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -216,18 +216,26 @@ def _InitialiseFromSerialisableInfo( self, serialisable_info ):
self._search_tag_filter = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_search_tag_filter )


def CanSearchThis( self, tags ):

# this is very simple, but a simple tag filter works for our v1.0 purposes
# you say 'only allow "for my script" tag' and then any file tagged with that is one you have allowed, nice
# also, if you blacklist "my secrets", then len filtered_tags reduces
# this doesn't support tag negation or OR
def CheckCanSearchTags( self, tags ):

with self._lock:

filtered_tags = self._search_tag_filter.Filter( tags )
if self._search_tag_filter.AllowsEverything():

return


if len( tags ) > 0:

filtered_tags = self._search_tag_filter.Filter( tags )

if len( filtered_tags ) > 0:

return len( filtered_tags ) == len( tags )



return len( filtered_tags ) == len( tags )
raise HydrusExceptions.InsufficientCredentialsException( 'You do not have permission to do this search. Your tag search permissions are: {}'.format( self._search_tag_filter.ToPermittedString() ) )



Expand Down
2 changes: 2 additions & 0 deletions include/ClientConstants.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,8 @@

media_viewer_capabilities[ HC.IMAGE_JPEG ] = static_full_support
media_viewer_capabilities[ HC.IMAGE_PNG ] = static_full_support
media_viewer_capabilities[ HC.IMAGE_WEBP ] = static_full_support
media_viewer_capabilities[ HC.IMAGE_TIFF ] = static_full_support
media_viewer_capabilities[ HC.IMAGE_APNG ] = animated_full_support
media_viewer_capabilities[ HC.IMAGE_GIF ] = animated_full_support

Expand Down
Loading

0 comments on commit 1a7fe45

Please sign in to comment.