@luccioman luccioman released this Sep 24, 2018 · 19 commits to master since this release

Assets 6

Attached file yacy_v1.921_20180922_9788.tar.gz can be used as a YaCy system update (available as "dev 1.921/9788" in YaCy page : "Dowload System Update" - /ConfigUpdate_p.html) or as a cross-platform binary distribution.

Contributors by alphabetical order :

Changes included since release 1.921.9701 (2018-06-23) :

  • Bug fixes :
    • Fixed unsafe conccurent access to generic SimpleDateFormat instances
    • Fixed Table_API_p.html current table page loss on row editing
    • Fixed JavaScript sorting of tables with cells containing an input field
    • Fixed removal of URLs from the delegatedURL remote crawl stack
    • Fixed exceeding max size of failreason_s Solr field on large link list
    • Fixed a NPE case on the Solr OpenSearch response writer
    • yacy#209 : fixed a NullPointerException case
    • yacy#187 : fixed crawl queue folder naming for IPv6 hosts on MS Windows filesystems
    • Fixed NullPointerException case on malformed crawl queue folder name
    • Fixed rendering of crawl queues page for URLs with raw IPV6 addresses
    • yacy#216 : Removed unnecessary '+' character URL decoding from search query and fixed encoding of '+' character on search pages links
    • yacy#217 : Added possibility to hide or show image results with rendering errors
    • Fixed logs folder checking and creation
    • Fixed rendering of the YMarks.html page
    • Fixed raw IPV6 addresses snapshots read/write on FAT32 and NTFS fs
    • yacy#227 : Use a constant default target file name for seed SCP upload method
    • yacy#228 : Hide password values from visible HTML in the Advanced Config page
  • Enhancements :
    • Updated the JRE URL from 8u131 to 8u171 for the MS Windows installer
    • Added server side columns sorting on the Process Scheduler table
    • Added and updated hint messages about remote crawler status
    • Properly lock solrInstances for reboot and restoration of embedded Solr
    • Removed some unnecessary synchronization locks
    • Consistently render empty facets and facets having only entries at zero
    • Solr HTML writer :
    • Override qf and df Solr params with defaults only when they are not set
    • Optionally render the response header
    • Improved rendering of relative urls : api link and stylesheet
    • Improved support for external Solr index
    • added support for the OpenSearch and yjson response writers
    • Prevent entering empty OpenSearch URLs in ConfigHeuristics_p.html
    • Added a link to MediaWiki dumps summary in import page for convenience
    • Added a parser for XZ compressed archives.
    • Updated pdf cache clear steps consistently with current pdfbox version
    • Improved Docker image building
    • Replaced RWI ranking JQuery sliders with standard HTML range inputs
    • Search portal widget :
    • Limit search API calls rate
    • Configured local peer as default portal search widget backend
    • Fixed portal search widget results favicon url
    • Fetch result pages one by one when scrolling in portal search widget
    • Made possible to provide an absolute data root path for start script
    • Added a first version of Snapcraft configuration file for Snap packaging
    • Improved support for Java logs file pattern options
    • Updated html input field type for seed upload with file method
    • Enforced access controls to System settings pages
    • Use accessible labels in the Server Access Settings page
  • Translation :
    • Large update to Chinese translations
  • Data sources upgrades
    • Updated bootstrap hosts
    • Upgraded the OpenGeoDB dump URL
  • Dependencies upgrades :
    • icu4j : from 61.1 to 62.1
    • commons-compress : from 1.16.1 to 1.17
    • pdfbox : from 2.0.9 to 2.0.11
    • Solr : from 6.6.3 to 6.6.5
    • Jetty : from 9.4.11.v20180605 to 9.4.12.v20180830
    • BouncyCastle : from jdk15:1.46 to jdk15on:1.60
    • Updated the JRE URL to 8u181 for the MS Windows installer
  • Documentation :
    • Use standard Java annotation syntax instead of custom Javadoc tag
    • Added contributor guidelines
    • Removed deprecated warning comments about imports and Debian installer

@luccioman luccioman released this Jun 23, 2018 · 106 commits to master since this release

Assets 4

Attached file yacy_v1.921_20180622_9701.tar.gz can be used as a YaCy system update (available as "dev 1.921/9701" in YaCy page : "Dowload System Update" - /ConfigUpdate_p.html) or as a cross-platform binary distribution.
A Debian package yacy_1.921.9701_all.deb can also be found at https://bintray.com/luccioman/yacy_search_server/yacy/1.921.9701.

Contributors by alphabetical order :

Changes included since release 1.921.9598 (2018-03-10) :

  • Bug fixes :
    • Fixed error 414 (URI Too Long) when manually selecting to many RSS items
    • yacy#169 : Removed hard-coded patch for Solr 5.0 on ranking boost function
    • Fixed NPE case in Solr select servlet on external Solr only setup
    • Fixed text snippet computation : don't reject ones starting with a space char
    • Properly handle embedded Solr partial results
    • Fixed NullPointerException when omitHeader=true on external Solr server
    • Fixed unresolve-pattern case on old html title element on Connections_p.html page
    • Properly render json string attributes in the crawl profile html editor
    • Do not block whole server startup on persisted crawl profile load error
    • Made CrawlProfile loading tolerant to malformed json string attribute
    • Fixed potential infinite loop case (does not occur in current code base)
    • Fixed a NullPointerException case in the suggest api
  • Enhancements :
    • Parse more audio metadata :
      • URLs, tag fields
    • Handle 'fl' (Field List) parameter when requesting internal Solr with YaCy html and exml writers
    • RSS Loader :
      • Added support for enclosures (media links)
      • Reduced log verbosity on feed items with no link
    • Crawler : - Crawler :
      • Added a new crawler document filter type on documents Media Type (MIME)
      • Added a new crawler document filter type using Solr syntax
    • Keep the selected view mode when typing a new URL in the ViewFile page
    • Table API :
      • Display the total number of matching items when filtering
      • Enable page navigation then using a search query
      • Fixed start record on the last results page
      • Adjusted api table query matching strategies
    • User help :
      • Updated link URL to IANA Media Types with https
      • Updated link to Solr Function Queries documentation page
      • Added hint to clarify results rendered dates and 'Sort by date' switch
      • Adjusted localhost/qualified account admin access informational texts
      • Added hint message inciting to check accounts settings on fresh install
      • Updated YaCy home page embedded links from http to https scheme
    • External Solr index usage :
      • Optionally allow external Solr server with self-signed certificate
      • Adjusted Solr select servlet output when using an external Solr only
      • Ensure no remote Solr is attached before "Shut Down and Re-Start Solr"
      • Ensure an embedded Solr is available for Solr dump/restore operations
      • Improved rendering of remote Solr admin URLs
      • Added controls on mode switch when attached to remote Solr instance(s)
      • Added Solr HTML writer support for responses from remote instances
      • Fixed Solr response header duplication when requesting external Solr
    • Text snippets :
      • Added analysis optional setting to compute statistics on text snippets
      • Reduced text snippet extraction processing time and memory footprint
    • Added a start script option to run as a foreground process without JMX
    • Enforced access controls on basic administration settings pages.
    • Added support for preferred https in peers communication on more operations
    • Added a UI section to configure encryption of peers communications
    • Removed remaining uses of deprecated Seed.getIP()
    • Make sure for image resource url enabled index image pixel size fields are filled if at least one of the image size fields is enabled in index
    • Corrected icons_sizes_sxt to SolrType.string in Solr Schema
    • Upgraded ConfigNetwork_p html doctype, added language attribute and cleaned up html errors
    • Prevented duplication of Solr query highlight fields parameters
    • Suggestions api :
      • Adjusted timeout management
      • Limited the rate of calls to the API when typing in search field
    • Small perf improvement : initialize threads names early when possible
    • Communication with remote peers Solr index :
      • Added missing default Solr http client connection timeout initialization
      • Trace level log message for monitoring remote solr response times
      • Removed use of deprecated ConcurrentUpdateSolrClient constructor
      • Use a common pooled http connection manager for remote solr instances
      • Apply remote solr configured timeout on getting connection from pool
      • Consistently allow gzip compression of remote Solr responses
      • Add a max acceptable limit to the size of Solr responses on p2p search
      • Use the heavy ConcurrentUpdateSolrClient only when necessary
    • Added a search page customization setting to display or not favicons
    • Added gzip compression of responses returned to user-agents accepting it
    • Made outgoing pools max total connections user configurable
    • Reuse expired connections evictors threads provided by apache and solr
    • Allow edtion of scheduled next execution dates for finer control (in Table_API_p.html page)
    • Removed no more used html transformer classes
  • Initial seeds : Removed seed URLs pointing to server low.audioattack.de since it will not be updated anymore
  • Translation :
    • Russian : Removed unnecessary html end line tag with invalid syntax for ConfigNetwork_p.html
    • Added Italian translation.
  • Dependencies upgrades :
    • commons-codec : from 1.10 to 1.11
    • Solr : from 6.6.2 to 6.6.3
    • Jetty : from 9.4.8.v20171121 to 9.4.11.v20180605
    • pdfbox : from 2.0.8 to 2.0.9
    • icu4j : from 60.2 to 61.1
    • jsoup : from 1.11.2 to 1.11.3
    • jwat : from 1.1.0 to 1.1.1
    • slf4j : from 1.7.24 to 1.7.25
  • Documentation :
    • Added some JavaDoc to the RSSMessage class.
    • Updated external Solr setup basic instructions
    • Removed Docker Cloud deploy button as service was shut down on May 21st

@luccioman luccioman released this Feb 28, 2018 · 211 commits to master since this release

Assets 4

Attached file yacy_v1.921_20180228_9596.tar.gz can be used as a YaCy system update (available as "dev 1.921/9596" in YaCy page : "Dowload System Update" - /ConfigUpdate_p.html) or as a cross-platform binary distribution.

Contributors by alphabetical order :

Changes included since release 1.921.9513 :

  • Bug fixes :
    • Fixed internal tables exact value match iterator
    • Fixed stored URL in web cache when redirection(s) occurs.
    • Replaced improper ByteBuffer.equals() implementation by Arrays.equals()
    • Fixed NullPointerException case on Table init with relative file path.
    • Enable results suggestions (Did you Mean) even when RWI is not enabled.
    • Remove old query terms from search results suggestions links.
    • Fixed loss of other modifiers on keywords/tags search navigation links.
    • Do not clear all search modifiers when unselecting one modifier.
    • Fixed loss of "meanCount" search param when using facets or page buttons.
    • Fixed loss of search modifiers on bookmark, recommand or delete result.
    • Fixed issue yacy#158 : completed div CSS class ignore in crawl
    • Fixes issue yacy#160 : handle properly syntax exceptions with a user friendly message
    • Fixes loss of information on multiple blacklist entries editions
    • Fixes loss of entries when moving entries from one list to another
    • Fixed issue yacy#156 : introduced a new option to clean up (or not) search cache on crawl start
    • Added missing vocabulary navigator increment on results from RWI
    • Fixed NPE case when on audio resource parsed with null tag
  • Enhancements :
    • Revised ReIndex and ReCrawl operations :
      • Enforced access controls
      • Made possible to customize selection query and preview results before launching a recrawl
      • Added a stop condition to the Recrawl busy thread
      • More details for recrawl job (running and terminated) reports
      • Record recrawl calls to make them schedulable
      • More comprehensive log on rejected recrawls caused by date constraint
      • Added a specific default crawl profile for the recrawl job.
    • Removed unnecessary reflection usage for workflow tasks.
    • Removed unused duplicated HTML id on header hidden field
    • Revised the RDFaParser main launcher for minimal proper operation.
    • Added an utility to generate/update XLIFF master file from lng files.
    • Added HTML microdata typed items parsing capability.
    • Adjusted and moved Language Navigator to be member of the navigatior plugin
    • Reuse the same Pattern instance when matching multiple key/values
    • Adjusted last blacklist entry example for a more accurate description
    • Improved the autotagging vocabulary editor :
      • Added option to choose field delimiter and line start in import from CSV
      • Handle escaped line breaks and separators in vocabulary import from CSV
      • Enforced access controls
      • Provide user interface messages on vocabulary creation read/write errors
    • Use https rather than http in links and queries to openstreetmap.org
    • Fixed CrawlStartExpert.html and Vocabulary_p.html HTML validation errors
    • Make StreamResponse usable in Java try-with-resources statements
    • Allow creation of vocabularies from remote CSV file URLs.
    • Added HTML5 embedded audio for results playing on supporting browsers
    • Updated the list of audio file formats supported by the audioTagParser
  • Translation :
    • Updated Chinese translation for various pages
    • Updated master and French translation for the IndexReIndexMonitor_p page
  • Dependencies upgrades :
    • metadata-extractor : from 2.10.1 to 2.11.0
    • httpclient : from 4.5.4 to 4.5.5
    • junit : from 4.11 to 4.12
    • commons-compress : from 1.15 to 1.16.1
    • jaudiotagger : from 2.0.3 to 2.2.5
  • Tests :
    • Removed time condition on HostBalancer initialization in JUnit test.
    • Added a manual performance test for the HostBalancer.
    • Shutdown daemon threads at the end of dbtest
  • Documentation :
    • Fixed some typos in the Readme.

@luccioman luccioman released this Jan 5, 2018 · 294 commits to master since this release

Assets 4

Attached file yacy_v1.921_20180102_9513.tar.gz can be used as a YaCy system update (available as "dev 1.921/9513" in YaCy page : "Dowload System Update" - /ConfigUpdate_p.html) or as a cross-platform binary distribution.

Contributors by alphabetical order :

Changes included since release 1.921/9460 :

  • Bug fixes :
    • Fixed url mask filter generated when protocol modifier is not null
    • Fixed URL parsing with fragment and empty path
    • Fixed issue yacy#145 : use HTTP Post operation for resetting memory monitoring state.
    • Fixed issue yacy#147 : removed Java 1.8 no more necessary version checking
    • Fixed loading default settings in Performance Settings of Queues and Processes
  • Enhancements :
    • Do more locale neutrale case conversions when possible, for improved compatibility with systems using Turkish language locale
    • Made possible to use https for remote search on peers with SSL enabled.
    • Removed redundant setting of timeout for remoteinstance
    • Apply tld query modifier on Solr host_s mandatory field.
    • Improved accuracy of URLs search filters : protocol, tld, host, file ext
    • Made "site:" query modifier case insensitive.
    • Improved support for internationalized domain names on "site:" modifier
    • Made "tld:" modifier case insensitive and IDN complient.
    • Removed deprecated jetty continuation class from urlproxyservlet
    • Removed use of deprecated Jetty IPAccessHandler for client filtering.
    • Added a crawl filter based on
      tag class names
    • Show/hide or show public surftip button depending on current config status
    • Started implementing optional https preference for protocol operations
    • Enable full size images preview for users with extended search rights
    • Added optional search parameter/setting to control content domain filter
    • Made TokenizedStringNavigator (used for keyword search facet) active
  • Dependencies upgrades :
    • POI : from 3.16 to 3.17
    • jsoup : from 1.10.3 to 1.11.2
    • httpcore : from 4.4.6 to 4.4.8
    • jetty : from 9.4.7.v20170914 to 9.4.8.v20171121
    • commons-compress : from 1.14 to 1.15
    • icu4j : from 60.1 to 60.2
    • httpclient : from 4.5.3 to 4.5.4
    • commons-io : from 2.5 to 2.6
  • Automated tests :
    • Added a basic JUnit test for the Visio parser (vsdParser)
    • Fixed SegmentTest test case time dependant occasional failures
  • Documentation :
    • Upgraded URLs from HTTP ro HTTPS in README.md
    • Updated links to Java Regular Expressions documentation to version 8

@luccioman luccioman released this Nov 17, 2017 · 347 commits to master since this release

Assets 4

Attached file yacy_v1.921_20171117_9460.tar.gz can be used as a YaCy system update (available as "dev 1.921/9460" in YaCy page : "Dowload System Update" - /ConfigUpdate_p.html) or as a cross-platform binary distribution.

Changes included since release 1.921/9232 :

  • Bug fixes :
    • Fixed loss of index page form values on 'more options' link click.
    • Fixed blacklist returned location URL on empty parameters
  • Enhancements :
    • Added an optional login link/status to the search public top nav bar.
    • Enable HTTP Digest authentication for non admin users.
    • Use accessible labels for search home page radio buttons.
    • Updated search page options descriptions (shortcuts, language and spatial restrictions)
    • Added language HTML attribute to the search home page.
    • Customized Threads with generic name for easier monitoring.
    • Removed old hard-coded holiday dates from DateDection class.
    • Updated log path in informative message of stop script.
    • Updated top-level known domains list.
  • Translation :
    • Renamed Chinese & Greek lng files using ISO639-1 codes.
  • Dependencies upgrades :
    • solr : from 6.6.1 to 6.6.2
    • pdfbox : from 2.0.7 to 2.0.8
    • icu4j : from 59_1 to 60.1
    • com.twelvemonkeys.imageio : from 3.3.1 to 3.3.2
  • Automated tests :
    • Added a html parser charset detection unit test
    • Improved some JUnit tests isolation and resources release

@luccioman luccioman released this Oct 19, 2017 · 375 commits to master since this release

Assets 2

Changes included since august 30 2017 :

  • New Features :
    • Added a JavaScript search results resorting mode for authenticated users to improve accuracy of results sorting in realtime
    • Add a configurable limit to tags initially displayed in search results
    • Made the dates navigator max elements number user configurable.
    • Fixed a NullPointerException case on images encoding errors.
    • Fixed search result Snapshots link.
    • Consistently encode snapshot image with format requested on the API.
    • Fixed NullPointerException cases on snapshot images parsing.
  • Bug fixes :
    • Mantis 766 : ensure private search features are not lost on Digest auth timeout (http://mantis.tokeek.de/view.php?id=766)
    • Prevent integer overflow cases in Chart graphs
    • Fixed array out of bounds in YJsonResponseWriter and OpensearchResponsriter
    • Fixed NullPointerException case on 'Browser' lang selection
    • Fixed RWI distance calculation on multi words search queries.
    • Mantis 772 : fixed JPEG snapshot resizing when running on OpenJDK.
  • Enhancements :
    • Improved search progress bar detailed statistics accuracy
    • Improved search navigators counters accuracy
    • Updated internal ISO 639-1 language codes with latest standards.
    • Improved search for audio, video and application-specific content types
    • Improved paginations buttons refreshing strategy.
    • Improved accessibility of histograms widgets.
    • Restrict Search Result Layout modification to HTTP POST only.
    • Fixed ConfigSearchPage_p HTML validation errors.
    • Added missing accessible labels to ConfigSearchPage_p.html
    • Added links to the optional keyword tags of search result
    • Adjusted filetype: query modifier parameter to lower case
    • Added textual hints to language radio buttons labels
    • Added partial bzip2 stream parsing support
    • Updated deprecated SolrInputDocument.addField() with boost value
    • Properly close resources (even on error) on OS and ThreadDump classes.
  • Translation :
    • Updated master.lng for IndexExport_p.html, ConfigSearchPage_p.html files
    • Added French translation for ConfigSearchPage_p.html
  • Documentation :
    • Updated Java version information on Readme
  • Dependencies upgrades :
    • solr : from 6.6.0 to 6.6.1
    • raphael.js : from 2.1.3 to 2.2.7
    • Jetty : from 9.4.6.v20170531 to 9.4.7.v20170914
  • Automated tests :
    • Added bzipParser Junit test
    • Adjusted ResponseHeaderTest to succeed on slow or highly loaded CPU
    • Fixed and enabled Travis automated build
    • Fixed pdf2image conversion with imagemagick on PDFs having transparency

@luccioman luccioman released this May 5, 2017 · 609 commits to master since this release

Assets 3

Attached file yacy_v1.92_20170504_9199.tar.gz can be used as a YaCy system update (available as "dev 1.92/9199 (unsigned)" in YaCy page : "Dowload System Update" - /ConfigUpdate_p.html) or as a cross-platform binary distribution.

Includes :

  • Bug fixes :
    • Fixed endless loop case in wikicode processing
    • Fixed unresolved_pattern case on missing post parameter api/message.html
    • Fixed surrogates import monitoring page (mantis 740 : http://mantis.tokeek.de/view.php?id=740)
    • Fixed Index Export feature for compatibility with old indexed documents (mantis 682 :
      http://mantis.tokeek.de/view.php?id=682 and issue yacy#116)
    • Copied fix from yacy_grid_parser for wrong array type
    • Fixed blacklist Regex containing '+' characters rendering
  • Enhancements :
    • Activated hosts navigator as a plugin
    • MediaWiki dump import :
      • used a text input for dump file selection
      • improved error reports on various wiki dump import prerequisites failure cases
      • added support for direct import on remote URLs
      • extended WikiCode template inclusion syntax support
      • improved monitoring
      • added the possibility to schedule import
    • Improved support for non ASCII chars in local file system URLs
    • Improved http client close time on stream processing errors
    • Added url input field as source for WarcImporter
    • Took out mailto collected in internal parsed documents
    • Fixed "Unchecked conversion" compilation warnings
    • Crawl results page : apply table lines number limit
    • Improved solr index dump format detection on import
    • Added patch to rewrite altered yacy grid schema into yacy schema
    • Handle '?' and '+' chars as valid wild cards when adding to blacklist.
    • Improved new blacklist entries URL scheme detection
  • New features :
    • new facet using keywords
    • added yacy grid flatjson surrogate parser
    • added a servlet to list users and edit them in quick and easy overview
  • Translation :
    • added missing text from ConfigRobotsTxt_p to master.lng
  • Dependencies upgrades :
    • solr : from 5.5.3 to 5.5.4
    • jwat : from 1.0.4 to 1.0.5
  • Automated tests :
    • Fixed unit test MultiProtocolURL(file) assertion for Windows path with drive letter

@luccioman luccioman released this Apr 4, 2017 · 649 commits to master since this release

Assets 3

Attached file yacy_v1.92_20170404_9159.tar.gz can be used as a YaCy system update or as a cross-platform binary distribution.

Includes :

  • Bug fixes :
    • Removed deprecated "localMissCount" prop from yacysearchlatestinfo.json.
    • Fixed NullPointerException case occurring when local solr index is disabled in search.
    • Fixed NullPointerException in HTMLResponseWriter on missing document title.
    • Fixed a NullPointerException case possible on Index Export.
    • Corrected fromDate init value on missing param in api/timeline_p servlet
    • Corrected dublincore title property text to lowercase in htmlresponsewriter
    • Fixed htmlParser <script> text extraction on code containing expression
    • Fixed call parameter for ConnectionInfo in MonitorHandler
    • Fixed delta time calculation in PerformanceSearch_p for the 1. entry
    • Fixed ReindexSolrBusyThread new and unexpected repeat of same query with low number of found documents
    • Fixed proxyservlet response url to respect http scheme if a relative Location header is returned.
    • Fixed unresolved pattern case on /yacysearchlatestinfo.json api
    • Fixed unresolved pattern case in search results progress bar (fix for mantis 715)
    • Fixed default/httpd.mime Z file extension to lower case
    • Fixed NullPointerException case and API URL link on Solr HTML output for webgraph core.
  • Default configuration :
    • Removed suche.sueddeutsche.de and faroo new from opensearch default config.
    • Fixed datacite.org heuristics base url.
    • Updated Archive-It heuristics URL.
  • Enhancements :
    • Modified some threads naming for easier monitoring.
    • Improved termination of timed out remote solr requests to peers.
    • Replaced absolute redirection locations by relative ones when possible for easier reverse proxy integration.
    • Make ConfigParser a protected page, for consistent behavior of locked
    • Used java.net.URLDecoder to improve Multiprotocol non ASCII characters support
    • Added robots.txt support for heuristics federated search.
    • Reduced self generated content for text_t (visible text index field)
    • Added hint of query syntax in AccessTracker log
    • Refactored and enforced Solr mandatory fields for proper operation
    • Added extract of queries.log in form of top search word cloud (last 7 days) to AccessTracker_p.html
    • Fixed some typos in French translations
    • Deprecated obsolete custom X-...-Scheme header constant.
    • Used existing FORMAT_ANSIC Dateformatter in HeaderFramework.
    • Use pre-defined "Connection" header key, replace depreceated
    • Further avoid to set connect info properties as header value
    • Extended DCEntry.getLanguage convert to ISO639-1 codes for more languages
    • Eliminated some compiler unchecked and deprecation warnings
    • Improved pdf text extraction resource handling.
    • Implemented RequestHeader getRequestURI, getRequestURL for legacy request
    • Removed double occurance of geo:lat in rss tokens
    • Made HTTP dDgest authentication default
    • Updated translation for ConfigNetwork_p.html
    • Added switchboardconstants for server ports config keys
    • Display the local search word statistic in alphabetic order
    • Added hint text to default ServerAcess Port Settings page
    • Updated master lng file with added text in Settings_ServerAccess
    • Updated shell scripts to be compatible with HTTP Digest authentication
    • Updated dump/restore shell scripts to use the new API IndexExport_p.html
    • Extended Apache HTTP Digest Auth. for use of YaCy encoded password
    • Enforced access controls on some administrative actions.
    • Added label text for search word statistic (AccessTracker_p.html) to master.lng
    • Removed seedlist bootstrap target (not working for some longer time)
    • URL Viewer : only display the link to metadata when metadata exists
    • Set Config Portal as a private administration page.
    • Escaped HTML eventually active content from recorded API call comments.
    • Added CookieTest_p.html text to master.lng
  • New features
    • New Debug/Analysis advanced settings subsection.
    • Support for HTML OpenSearch results.
    • Privacy enhancement : added settings to control referrer policy.
    • Introduced the option to configure a shutdown port.
    • Import from Warc archives
    • Export to elasticsearch
  • Dependencies upgrades :
    • ant : from 1.10.0 to 1.10.1 in the Docker alpine flavor image.
    • jetty : from 9.2.19.v20160908 to 9.2.21.v20170120
    • httpclient : from 4.5.2 to 4.5.3
    • pdfbox : from 2.0.3 to 2.0.5
    • metadata-extractor : from 2.9.1 to 2.10.1
    • jsch : from 0.1.53 to 0.1.54
    • jsoup : from 1.10.1 to 1.10.2
    • icu4j : from 57_1 to 58_2
    • slf4j : from 1.7.21 to 1.7.24
    • xmpcore : from 5.1.2 to 5.1.3
  • Automated tests :
    • Adjusted DefaultServlet test case to recent changes
    • Removed test case for Standard_MemoryControl which will always fail
  • Documentation :
    • Added hint how to build with maven (for the first time) to readme

@luccioman luccioman released this Feb 2, 2017 · 742 commits to master since this release

Assets 3

Attached file yacy_v1.92_20170201_9066.tar.gz can be used as a YaCy system update or as a cross-platform binary distribution.

Includes :

  • Bug fixes :
    • webstructure.xml API used with a domain name 'about' parameter (mantis 720)
    • WatchWebStructure_p.html render to include https URLs (mantis 721)
    • Display of crawler pending URLs counts in HostBrowser.html page (mantis 722)
    • unresolved pattern admin parameter in HostBrowser.html "load & index" links
    • Ensured that RWI Index.Segment IODispatcher is not blocking on shudown (mantis 723)
    • local image search pagination regression (related to issue yacy#90)
    • unresolved pattern on directory entries in HostBrowser.xml api (mantis 725)
  • Enhancements :
    • Moved "Augmented Browsing" settings to "URL/Web Proxy Access Settings" in "Advanced settings" subsection
    • Used DigestURL.hosthash() where possible instead of re-implementing
    • Made sure webstructure.xml API produces valid XML.
    • Removed obsolete lastmodified calculation in WebgraphConfig
    • Added an optional parameter to webstructure.xml api.
    • Deleted outdated and unmaintained Netbeans project
    • Added ukr and pol to DCEntry.getLanguage ISO639-2 3-char language code
    • Display Index Browser links requiring auth only when authenticated.
    • Fixed Index Browser page HTML validation errors and switched to HTML5
    • Updated French translation of HostBrowser.html
    • Added dc.date.modified and dc.date.created to date parser
    • Improved Index Browser accessibility with semantically richer html tags.
    • Added LSB Tag section to the Debian package descriptor
    • Adjusted the Field-Reindex Thread to verify and update the document id
    • Added user-friendly controls over disk usage configuration settings (see issue yacy#103)
    • Added consistency check for related index fields upon load and save of index schema.
    • Refactored the DHT-Trigger section in Performance_p.html page.
  • Dependencies upgrades :
    • commons-compress : from 1.12 to 1.13
    • Solr/Lucene : from 5.5.2 to 5.5.3
  • Automated tests :
    • Fixed 2 failing JUnit tests
    • Added automated unit tests and perfs test for WebStructureGraph class
  • Documentation :
    • Cleaned-up some Javadoc warnings
    • Detailed HostBrowser, WebStructureGraph, and webstructure classes Javadoc
    • Updated README.md - removed deprecated URL

@luccioman luccioman released this Jan 9, 2017 · 791 commits to master since this release

Assets 3

Attached file yacy_v1.92_20170109_9018.tar.gz can be used as a YaCy system update or as a cross-platform binary distribution.

Includes :

  • Bug fixes :
    • Proper shutdown on SIGTERM signal
    • fulltext.remove() by id of webgraph document
    • URIMetadataNote instantiation
    • Display result favicons only for http or https resources
    • Concurrency issue with htmlParser using not current scraper data
    • DC.Elements namespace in DublinCore vocabulary class
  • Enhancements :
    • On negative result vote also delete document from fulltext index
    • Complete harmonization RequestHeader
    • Adjusted crawl depth control for FTP crawl start URLs
    • Added descriptive titles to Crawler_p.html speed settings
    • Blacklist import and update performance improvements
  • Dependencies upgrades :
    • Apache Ant to 1.10.0 for the Alpine flavored Docker image
    • jgit build library to version 4.5.0
  • Updated some JavaDoc