Skip to content

Commit

Permalink
integrated YaCyNews basis
Browse files Browse the repository at this point in the history
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@455 6c8d7289-2bf4-0310-a012-ef5d649a1542
  • Loading branch information
orbiter committed Jul 29, 2005
1 parent b8e294f commit 60117a2
Show file tree
Hide file tree
Showing 12 changed files with 859 additions and 5 deletions.
6 changes: 3 additions & 3 deletions doc/Download.html
Expand Up @@ -43,12 +43,12 @@ <h3>3rd Step: Download YaCy</h3>
<li>Generic release of YaCy (all platforms with J2SE 1.4.2: Linux, Mac OS X, Windows, Solaris):</li>
<ul>
<li><tt>from yacy.net&nbsp;&nbsp;&nbsp;: <a href="http://www.yacy.net/yacy/release/yacy_v0.391_20050726_434.tar.gz"><tt>yacy_v0.391_20050726_434.tar.gz</tt></a></tt></li>
<li><tt>from BerliOS.de&nbsp;: <a href="http://download.berlios.de/yacy/yacy_v0.39_20050722_425.tar.gz"><tt>yacy_v0.39_20050722_425.tar.gz</tt></a></tt></li>
<li><tt>from BerliOS.de&nbsp;: <a href="http://download.berlios.de/yacy/yacy_v0.391_20050726_434.tar.gz"><tt>yacy_v0.391_20050726_434.tar.gz</tt></a></tt></li>
</ul>
<li>Windows-flavour release of YaCy (same code as generic release, but with convenient Windows-Installer):</li>
<ul>
<li><tt>from yacy.net&nbsp;&nbsp;&nbsp;: <a href="http://www.yacy.net/yacy/release/yacy_v0.39_20050722_425.exe"><tt>yacy_v0.39_20050722_425.exe</tt></a></tt></li>
<li><tt>from BerliOS.de&nbsp;: <a href="http://download.berlios.de/yacy/yacy_v0.39_20050722_425.exe"><tt>yacy_v0.39_20050722_425.exe</tt></a></tt></li>
<li><tt>from yacy.net&nbsp;&nbsp;&nbsp;: <a href="http://www.yacy.net/yacy/release/yacy_v0.391_20050726_434.exe"><tt>yacy_v0.391_20050726_434.exe</tt></a></tt></li>
<li><tt>from BerliOS.de&nbsp;: <a href="http://download.berlios.de/yacy/yacy_v0.391_20050726_434.exe"><tt>yacy_v0.391_20050726_434.exe</tt></a></tt></li>
</ul>
</ul>
</p>
Expand Down
3 changes: 3 additions & 0 deletions doc/Links.html
Expand Up @@ -34,6 +34,9 @@ <h2>Links</h2>

<p>Publications about YaCy
<ul>
<li>Article in the german computer magazine <a href="http://www.heise.de/ct/"><b>c't</b></a>,
issue <a href="http://www.heise.de/ct/05/16/006/">c't 16/2005</a>,
page 52 (german): "<a href="http://www.heise.de/ct/05/16/052/"><b>Wider die Monokultur</b></a>"</li>
<li>Article in <a href="http://www.ccc.de/updates/2005/ds86">Datenschleuder #086</a>,
magazine of the german 'chaos computer association' (<b>Chaos Computer Club</b>, <b>CCC</b>), german:
<a href="http://www.yacy.net/yacy/material/YaCy-Datenschleuder086.pdf"><b>"YaCy -- Peer-to-Peer Web-Suchmaschine"</b></a></li>
Expand Down
3 changes: 2 additions & 1 deletion htroot/Help.html
Expand Up @@ -63,7 +63,8 @@ <h3>Local and Global Search: Options and Functions</h3>
<li> All OS and Amaya: Strg + Accesskey<p>

s --> Search Page<br>
n --> Network<br>
n --> News<br>
w --> Network<br>
t --> Status<br>

#[footer]#
Expand Down
67 changes: 67 additions & 0 deletions htroot/News.html
@@ -0,0 +1,67 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<title>YaCy: News</title>
#[metas]#
</head>
<body marginheight="0" marginwidth="0" leftmargin="0" topmargin="0">
#[header]#
<table width="100%" border="0" cellpadding="0" cellspacing="0" class="SubMenu">
<tr height="10"><td colspan="9" class="MenuHeader">&nbsp;Network Menu</td></tr>
<tr height="2"><td colspan="9"></td></tr>
<tr class="TableHeader">
<td width="20%" class="MenuSubItem">&nbsp;&nbsp;<a href="/News.html" class="MenuItemLink">News&nbsp;Overview</a>&nbsp;</td>
<td class="MenuSubSpacer"></td>
<td width="20%" class="MenuSubItem">&nbsp;&nbsp;<a href="/News.html?page=1" class="MenuItemLink">Incoming&nbsp;News</a>&nbsp;</td>
<td class="MenuSubSpacer"></td>
<td width="20%" class="MenuSubItem">&nbsp;&nbsp;<a href="/News.html?page=2" class="MenuItemLink">Processed&nbsp;News</a>&nbsp;</td>
<td class="MenuSubSpacer"></td>
<td width="20%" class="MenuSubItem">&nbsp;&nbsp;<a href="/News.html?page=3" class="MenuItemLink">Outgoing&nbsp;News</a>&nbsp;</td>
<td class="MenuSubSpacer"></td>
<td width="20%" class="MenuSubItem">&nbsp;&nbsp;<a href="/News.html?page=4" class="MenuItemLink">Published&nbsp;News</a>&nbsp;</td>
</tr>
</table>
<br>
#(page)#
<h2>News Overview</h2>
::
<h2>Incoming&nbsp;News</h2>
::
<h2>Processed&nbsp;News</h2>
::
<h2>Outgoing&nbsp;News</h2>
::
<h2>Published&nbsp;News</h2>
#(/page)#
<p>
#(table)#
<p>
This is the news system (currently under testing).
</p>
::
<p>
<table border="0" cellpadding="2" cellspacing="1">
<tr class="TableHeader" valign="bottom">
<td class="small">Originator</td>
<td class="small">Created</td>
<td class="small">Category</td>
<td class="small">Received</td>
<td class="small">Distributed</td>
<td class="small">Attributes</td>
</tr>
#{list}#
<tr class="TableCell#(dark)#Light::Dark::Summary#(/dark)#">
<td class="small" align="right">#[ori]#</td>
<td class="small" align="right">#[cre]#</td>
<td class="small" align="right">#[cat]#</td>
<td class="small" align="right">#[rec]#</td>
<td class="small" align="right">#[dis]#</td>
<td class="small" align="right">#[att]#</td>
</tr>
#{/list}#
</table>
</p>
#(/table)#
#[footer]#
</body>
</html>
96 changes: 96 additions & 0 deletions htroot/News.java
@@ -0,0 +1,96 @@
// News.java
// -----------------------
// part of YaCy
// (C) by Michael Peter Christen; mc@anomic.de
// first published on http://www.anomic.de
// Frankfurt, Germany, 2005
// last major change: 29.07.2005
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
// Using this software in any meaning (reading, learning, copying, compiling,
// running) means that you agree that the Author(s) is (are) not responsible
// for cost, loss of data or any harm that may be caused directly or indirectly
// by usage of this softare or this documentation. The usage of this software
// is on your own risk. The installation and usage (starting/running) of this
// software may allow other people or application to access your computer and
// any attached devices and is highly dependent on the configuration of the
// software which must be done by the user of the software; the author(s) is
// (are) also not responsible for proper configuration and usage of the
// software, even if provoked by documentation provided together with
// the software.
//
// Any changes to this file according to the GPL as documented in the file
// gpl.txt aside this file in the shipment you received can be done to the
// lines that follows this copyright notice here, but changes must not be
// done inside the copyright notive above. A re-distribution must contain
// the intact and unchanged copyright notice.
// Contributions and changes to the program code must be marked as such.

// You must compile this file with
// javac -classpath .:../classes Network.java
// if the shell's current path is HTROOT

import java.util.Enumeration;
import java.util.HashMap;
import java.io.IOException;

import de.anomic.http.httpHeader;
import de.anomic.server.serverObjects;
import de.anomic.server.serverSwitch;
import de.anomic.server.serverDate;
import de.anomic.yacy.yacyCore;
import de.anomic.yacy.yacyNewsRecord;

public class News {

public static serverObjects respond(httpHeader header, serverObjects post, serverSwitch sb) {
// return variable that accumulates replacements
serverObjects prop = new serverObjects();
boolean overview = (post == null) || (((String) post.get("page", "0")).equals("0"));

if (overview) {
// show overview
prop.put("table", 0);
prop.put("page", 0);
} else {
// generate table
int tableID = Integer.parseInt((String) post.get("page", "1")) - 1;
prop.put("table", 1);
prop.put("page", tableID + 1);

if (yacyCore.seedDB == null) {

} else {
int maxCount = yacyCore.newsPool.size(tableID);
if (maxCount > 500) maxCount = 500;

yacyNewsRecord record;
for (int i = 0; i < maxCount; i++) try {
record = yacyCore.newsPool.get(tableID, i);
prop.put("table_list_" + i + "_ori", record.originator());
prop.put("table_list_" + i + "_cre", yacyCore.universalDateShortString(record.created()));
prop.put("table_list_" + i + "_cat", record.category());
prop.put("table_list_" + i + "_rec", record.received());
prop.put("table_list_" + i + "_dis", record.attributes().toString());
} catch (IOException e) {e.printStackTrace();}
prop.put("table_list", maxCount);
}
}
// return rewrite properties
return prop;
}

}
3 changes: 2 additions & 1 deletion htroot/env/templates/header.template
Expand Up @@ -59,7 +59,8 @@

<tr><td class="MenuHeader">&nbsp;Peer&nbsp;Control</td></tr>
<tr><td class="MenuItem">&nbsp;<a href="/Status.html" accesskey="t" class="MenuItemLink">Status</a></td></tr>
<tr><td class="MenuItem">&nbsp;<a href="/Network.html" accesskey="n" class="MenuItemLink">Network</a></td></tr>
<tr><td class="MenuItem">&nbsp;<a href="/News.html" accesskey="n" class="MenuItemLink">News</a></td></tr>
<tr><td class="MenuItem">&nbsp;<a href="/Network.html" accesskey="w" class="MenuItemLink">Network</a></td></tr>
<tr><td class="MenuItem">&nbsp;<img border="0" src="/env/grafics/lock.gif" align="top">&nbsp;<a href="/ViewLog_p.html" class="MenuItemLink">Log</a></td></tr>
<tr><td class="MenuItem">&nbsp;<img border="0" src="/env/grafics/lock.gif" align="top">&nbsp;<a href="/Settings_p.html" class="MenuItemLink">Settings</a></td></tr>
<tr><td class="MenuItem">&nbsp;<img border="0" src="/env/grafics/lock.gif" align="top">&nbsp;<a href="/Performance_p.html" class="MenuItemLink">Performance</a></td></tr>
Expand Down
15 changes: 15 additions & 0 deletions source/de/anomic/yacy/yacyCore.java
Expand Up @@ -83,6 +83,7 @@ public class yacyCore {
public static ThreadGroup publishThreadGroup = new ThreadGroup("publishThreadGroup");
public static long startupTime = System.currentTimeMillis();
public static yacySeedDB seedDB = null;
public static yacyNewsPool newsPool = null;
public static final Hashtable seedUploadMethods = new Hashtable();
public static yacyPeerActions peerActions = null;
public static yacyDHTAction dhtAgent = null;
Expand Down Expand Up @@ -164,13 +165,18 @@ public yacyCore(plasmaSwitchboard sb) throws IOException {
new File(yacyDBPath, "seed.pot.db"),
mem);

// create or init news database
newsPool = new yacyNewsPool(yacyDBPath, 1024);

loadSeedUploadMethods();

// deploy peer actions
peerActions = new yacyPeerActions(seedDB, switchboard,
new File(sb.getRootPath(), sb.getConfig("superseedFile", "superseed.txt")),
switchboard.getConfig("superseedLocation", "http://www.yacy.net/yacy/superseed.txt"));
dhtAgent = new yacyDHTAction(seedDB);
peerActions.deploy(dhtAgent);
peerActions.deploy(new yacyNewsAction(newsPool));

// create or init index sharing
//shareManager = new yacyShare(switchboard);
Expand Down Expand Up @@ -277,6 +283,7 @@ public void peerPing() {
// before publishing, update some seed data
peerActions.updateMySeed();


// publish own seed to other peer, this can every peer, but makes only sense for senior peers
int oldSize = seedDB.sizeConnected();
if (oldSize == 0) {
Expand Down Expand Up @@ -380,6 +387,14 @@ private int publishMySeed(boolean force) {
}
if (seeds == null) return 0;

// include a YaCyNews record to my seed
try {
yacyNewsRecord record = newsPool.dequeueMyNews();
if (record != null) seedDB.mySeed.put("news", de.anomic.tools.crypt.simpleEncode(record.toString()));
} catch (IOException e) {
e.printStackTrace();
}

// holding a reference to all started threads
int contactedSeedCount = 0;
List syncList = Collections.synchronizedList(new LinkedList()); // memory for threads
Expand Down
70 changes: 70 additions & 0 deletions source/de/anomic/yacy/yacyNewsAction.java
@@ -0,0 +1,70 @@
// yacyNewsAction.java
// -----------------------
// part of YaCy
// (C) by Michael Peter Christen; mc@anomic.de
// first published on http://www.anomic.de
// Frankfurt, Germany, 2005
// last major change: 29.07.2005
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//
// Using this software in any meaning (reading, learning, copying, compiling,
// running) means that you agree that the Author(s) is (are) not responsible
// for cost, loss of data or any harm that may be caused directly or indirectly
// by usage of this softare or this documentation. The usage of this software
// is on your own risk. The installation and usage (starting/running) of this
// software may allow other people or application to access your computer and
// any attached devices and is highly dependent on the configuration of the
// software which must be done by the user of the software; the author(s) is
// (are) also not responsible for proper configuration and usage of the
// software, even if provoked by documentation provided together with
// the software.
//
// Any changes to this file according to the GPL as documented in the file
// gpl.txt aside this file in the shipment you received can be done to the
// lines that follows this copyright notice here, but changes must not be
// done inside the copyright notice above. A re-distribution must contain
// the intact and unchanged copyright notice.
// Contributions and changes to the program code must be marked as such.

package de.anomic.yacy;

import java.io.IOException;

public class yacyNewsAction implements yacyPeerAction {

yacyNewsPool pool;

public yacyNewsAction(yacyNewsPool pool) {
this.pool = pool;
}

public void processPeerArrival(yacySeed peer, boolean direct) {
String recordString = peer.get("news", null);
if (recordString == null) return;
yacyNewsRecord record = new yacyNewsRecord(de.anomic.tools.crypt.simpleDecode(recordString, ""));
try {
this.pool.enqueueGlobalNews(record);
} catch (IOException e) {e.printStackTrace();}
}

public void processPeerDeparture(yacySeed peer) {
}

public void processPeerPing(yacySeed peer) {
processPeerArrival(peer, true);
}

}

0 comments on commit 60117a2

Please sign in to comment.