fixing CF cookie instant expiration bug

AAAAAAAAAAAAAHHHHHHHHHHHHHHHHHHHHH

help/changelog.html

@@ -11,7 +11,7 @@ <h3>changelog</h3>
 				<li><h3>version 393</h3></li>
 				<ul>
 					<li>cloudflare and network:</li>
-					<li>the hydrus client now has an experimental hook to the cloudscraper module, which is now an optional pip module for source users and included in all built releases. if a CF challenge page is downloaded, hydrus attempts to detect and solve it with cloudscraper and save the CF cookies back to the session before reattempting the request. all feedback on this working/breaking irl would be welcome. current expectation for this prototype is it can pass the basic 'wait five seconds' javascript challenge, and likely all but a handful of the more complicated captcha ones</li>
+					<li>the hydrus client now has an experimental hook to the cloudscraper module, which is now an optional pip module for source users and included in all built releases. if a CF challenge page is downloaded, hydrus attempts to detect and solve it with cloudscraper and save the CF cookies back to the session before reattempting the request. all feedback on this working/breaking irl would be welcome. current expectation for this prototype is it can pass the basic 'wait five seconds' javascript challenge, but only a handful of the more complicated captcha ones</li>
 					<li>if a CF challenge page is not solvable, the respective fail reason for that URL will be labelled appropriately about CloudFlare and have more technical information</li>
 					<li>.</li>
 					<li>the hydrus network engine now has the capability to remember recent serious network infrastructure errors (no connection, unsolvable cloudflare problem, etc..) on a per domain basis. if many serious errors have happened on a domain, new jobs will now wait until they are clear. this defaults to three or more such errors in the past ten minutes, and is configurable (and disableable) under options->connection. this will be built out to a flexible system in future, with per-domain options+status ui to see what's going on and actions to scrub delays</li>

hydrus/ClientNetworkingDomain.py

@@ -16,19 +16,22 @@
 import time
 import urllib.parse
 
-def AddCookieToSession( session, name, value, domain, path, expires ):
+def AddCookieToSession( session, name, value, domain, path, expires, secure = False, rest = None ):
 
     version = 0
     port = None
     port_specified = False
     domain_specified = True
     domain_initial_dot = domain.startswith( '.' )
     path_specified = True
-    secure = False
     discard = False
     comment = None
     comment_url = None
-    rest = {}
+
+    if rest is None:
+
+        rest = {}
+
 
     cookie = http.cookiejar.Cookie( version, name, value, port, port_specified, domain, domain_specified, domain_initial_dot, path, path_specified, secure, expires, discard, comment, comment_url, rest )
 

hydrus/ClientNetworkingJobs.py

@@ -562,13 +562,22 @@ def _SolveCloudFlare( self, response ):
 
                     session = self.engine.session_manager.GetSession( snc )
 
+                    cf_cookies = [ cookie for cookie in session.cookies if cookie.name.startswith( '__cf' ) ]
+
+                    for cookie in cf_cookies:
+
+                        session.cookies.clear( cookie.domain, cookie.path, cookie.name )
+
+
                     domain = '.{}'.format( ClientNetworkingDomain.ConvertURLIntoSecondLevelDomain( self._url ) )
                     path = '/'
-                    expires = 30 * 86400
+                    expires = HydrusData.GetNow() + 30 * 86400
+                    secure = True
+                    rest = { 'HttpOnly' : None, 'SameSite' : 'None' }
 
                     for ( name, value ) in cf_tokens.items():
 
-                        ClientNetworkingDomain.AddCookieToSession( session, name, value, domain, path, expires )
+                        ClientNetworkingDomain.AddCookieToSession( session, name, value, domain, path, expires, secure = secure, rest = rest )
 
 
                     self.engine.session_manager.SetDirty()