documentation update

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@843 6c8d7289-2bf4-0310-a012-ef5d649a1542

doc/Links.html

@@ -54,7 +54,8 @@ <h2>Links</h2>
 
 <p>Other YaCy Project Sites
 <ul>
-<li><a href="http://www.yacy-websuche.de/"><b>German documentation</b></a> - Wiki, administrated and initiated by Alexander Schier</li>
+<li><a href="http://www.yacy-websuche.de/wiki"><b>YaCy Wiki</b></a> - administrated by Alexander Schier</li>
+<li><a href="http://www.yacy-websuche.de"><b>German documentation</b></a> - initiated and administrated by Alexander Schier</li>
 <li><a href="http://www.yacy-forum.de"><b>Deutsches Forum</b></a>, administrated by Roland Ramthun</li>
 <li><a href="http://sourceforge.net/forum/?group_id=116142"><b>English Forum</b></a></li>
 <li><a href="http://developer.berlios.de/projects/yacy/"><b>YaCy at BerliOS</b> - our SVN hosting service</li>
@@ -79,7 +80,9 @@ <h2>Links</h2>
   page 52 (german): "<a href="http://www.heise.de/ct/05/16/052/"><b>Wider die Monokultur</b></a>"</li>
 <li>Article in <a href="http://www.ccc.de/updates/2005/ds86">Datenschleuder #086</a>,
  magazine of the german 'chaos computer association' (<b>Chaos Computer Club</b>, <b>CCC</b>), german:
- <a href="http://www.yacy.net/yacy/material/YaCy-Datenschleuder086.pdf"><b>"YaCy -- Peer-to-Peer Web-Suchmaschine"</b></a></li>
+ <a href="http://www.yacy.net/yacy/material/YaCy-Datenschleuder086.pdf"><b>"YaCy -- Peer-to-Peer Web-Suchmaschine (Original-PDF)"</b></a>, 
+<a href="https://dsred.ccc.de/086/yacy"><b>"YaCy -- Peer-to-Peer Web-Suchmaschine (Datenschleuder-Redaktion)"</b>
+</li>
 <li>Cited as <b>world-wide unique proof-of-concept for a distributed p2p search engine</b>
  in the German issue of the MIT's Magazine of Innovation
  <a href="http://www.technologyreview.de"><b>Technology Review</b></a> in an

doc/News.html

@@ -50,7 +50,7 @@
 
 <h2>News</h2>
 
-<p>This is essentially the release change-log. We have a <a href="roadmap.txt">release roadmap</a> and releases published here will (hopefully) match the milestones from the roadmap's vision.
+<p>This is essentially the release change-log. We have a <a href="http://www.yacy-websuche.de/wiki/index.php/Dev:Roadmap">release roadmap</a> and releases published here will (hopefully) match the milestones from the roadmap's vision.
 
 <p>Release list in reverse order:
 
@@ -73,6 +73,55 @@ <h2>News</h2>
 </ul>
 -->
 
+<br><p>v0.41_20051004_843
+<ul>
+<li>Prevention of unwanted DDoS effects caused by YaCy crawls by doing a target-server load balancing; further prevention is done by robots.txt</li>
+<li>New setting of proxy cache size and storage path in submenu 'Proxy Indexing'</li>
+<li>Modified cleanup of HTCache</li>
+<li>Sorted config list in Config_p.html and sorted file list in Cache Admin menu</li>
+<li>Code clean-up: added finals and constants</li>
+<li>New Network menu design; showing peer status, Index Receive and Crawl Receive properties as images</li>
+<li>Added ICAP support; now an experimental ICAP Server is embedded and Yacy allows other proxies to use the indexing service via icap response modification requests</li>
+<li>Single-Peer permanent index transfer (full flush to other peer)</li>
+<li>Added a templateCache to httpFileHandler</li>
+<li>Added blacklist support for https requests and crawler</li>
+<li>Adding functionality to delete entries from Indexing and Crawler Queue</li>
+<li>Adding functionality to clear the whole indexing queue</li>
+<li>Proxy now supports the X-Forwarded-For Header</li>
+<li>Indexing queue now displays total size of enqueued content in kb</li>
+<li>Remembering Crawler-isPaused setting by storing status into config file</li>
+<li>Splitting of status page into a private and a public accessible part</li>
+<li>Adding Queue overview to status page</li>
+<li>New symbols for Peer Status, connection status, Index-Receive-Granted and Crawl-Receive-Granted in Network menu, replacing old separate columns</li>
+<li>Support for robots.txt</li>
+  <ul>
+  <li>Implementation of a robots.txt parser</li>
+  <li>Control of remote crawls with robots.txt</li>
+  </ul>
+<li>Performance enhancements</li>
+  <ul>
+  <li>Better Database Caching</li>
+    <ul>
+    <li>Better usage of memory in kelondro Record-Nodes and less IO access</li>
+    <li>New cache-control menu within the performance menu</li>
+    <li>Better cache-size default values</li>
+    </ul>
+  <li>Accelerated Blacklists import; makes big lists possible</li>
+  <li>Content-Encoding GZIP support for http post requests on index transfer/distribution</li>
+  </ul>
+<li>Bugfixes</li>
+  <ul>
+  <li>Normalization problems: prevent URLs with ':80'</li>
+  <li>Many bug fixes for NULL pointer occurrences</li>
+  <li>Display of an proxy error page instead of a white page if the server has closed the connection before yacy was able to receive the http response line</li>
+  <li>Crawler Redirection bug fixed</li>
+  <li>Indexer now gets the mimeType now from the parsed document instead of the responseHeader (this is especially necessary if mimeType has to be detected by the MimeType parser)</li>
+  <li>URLs pointing to a server having a private ip addess will not be indexed anymore</li>
+  <li>Unsupported MimeTypes and fileExtensions will not be queued by the cachemanager in the indexer queue anymore (to reduce unneccesary IO)</li>
+  </ul>
+<li>.. many more small changes and bugfixes. for details please see the <a href="http://svn.berlios.de/wsvn/yacy/?op=log&rev=0&sc=0&isdir=1">SVN history</a></li>
+</ul>
+
 <br><p>v0.40_20050816_548
 <ul>
 <li>Index distribution to DHT now fully active</li>

source/de/anomic/yacy/yacyClient.java

@@ -106,6 +106,7 @@ public static int publishMySeed(String address, String otherHash) {
             obj.put("count", "20");
             obj.put("key", key);
             obj.put("mytime", yacyCore.universalDateShortString(new Date()));
+            obj.put("myUTC", System.currentTimeMillis());
             obj.put("seed", yacyCore.seedDB.mySeed.genSeedStr(key));
             result = nxTools.table(httpc.wput(url,
             105000, null, null,