src/core/org/apache/hadoop/io/BloomMapFile.java src/core/org/apache/hadoop/util/bloom/BloomFilter.java src/core/org/apache/hadoop/util/bloom/CountingBloomFilter.java src/core/org/apache/hadoop/util/bloom/DynamicBloomFilter.java src/core/org/apache/hadoop/util/bloom/Filter.java src/core/org/apache/hadoop/util/bloom/HashFunction.java src/core/org/apache/hadoop/util/bloom/Key.java src/core/org/apache/hadoop/util/bloom/RemoveScheme.java src/core/org/apache/hadoop/util/bloom/RetouchedBloomFilter.java src/core/org/apache/hadoop/util/hash/Hash.java src/core/org/apache/hadoop/util/hash/JenkinsHash.java src/core/org/apache/hadoop/util/hash/MurmurHash.java src/test/org/apache/hadoop/io/TestBloomMapFile.java @@ -86,6 +86,9 @@ Trunk (unreleased changes) HADOOP-4826. Introduce admin command saveNamespace. (shv) + HADOOP-3063 BloomMapFile - fail-fast version of MapFile for sparsely + populated key space (Andrzej Bialecki via stack) + IMPROVEMENTS HADOOP-4749. Added a new counter REDUCE_INPUT_BYTES. (Yongqiang He via CHANGES.txt @@ -200,3 +200,45 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. + + +APACHE HADOOP SUBCOMPONENTS: + +The Apache Hadoop project contains subcomponents with separate copyright +notices and license terms. Your use of the source code for the these +subcomponents is subject to the terms and conditions of the following +licenses. + +For the org.apache.hadoop.util.bloom.* classes: + +/** + * + * Copyright (c) 2005, European Commission project OneLab under contract + * 034819 (http://www.one-lab.org) + * All rights reserved. + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * - Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * - Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the distribution. + * - Neither the name of the University Catholique de Louvain - UCL + * nor the names of its contributors may be used to endorse or + * promote products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER + * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ LICENSE.txt @@ -1264,6 +1264,34 @@ creations/deletions), or "all".</description> </property> <property> + <name>io.mapfile.bloom.size</name> + <value>1048576</value> + <description>The size of BloomFilter-s used in BloomMapFile. Each time this many + keys is appended the next BloomFilter will be created (inside a DynamicBloomFilter). + Larger values minimize the number of filters, which slightly increases the performance, + but may waste too much space if the total number of keys is usually much smaller + than this number. + </description> +</property> + +<property> + <name>io.mapfile.bloom.error.rate</name> + <value>0.005</value> + <description>The rate of false positives in BloomFilter-s used in BloomMapFile. + As this value decreases, the size of BloomFilter-s increases exponentially. This + value is the probability of encountering false positives (default is 0.5%). + </description> +</property> + +<property> + <name>hadoop.util.hash.type</name> + <value>murmur</value> + <description>The default implementation of Hash. Currently this can take one of the + two values: 'murmur' to select MurmurHash and 'jenkins' to select JenkinsHash. + </description> +</property> + +<property> <name>map.sort.class</name> <value>org.apache.hadoop.util.QuickSort</value> <description>The default sort class for sorting keys. conf/hadoop-default.xml 450f6f26bb0de62cae8f189fbda09d1a9a036edf Michael Stack stack@apache.org http://github.com/yahoo/hadoop/commit/88f57aff394d5a3ee4054469b1647d782586e246 88f57aff394d5a3ee4054469b1647d782586e246 2008-12-15T12:57:34-08:00 2008-12-15T12:57:34-08:00 HBASE-3063 BloomMapFile - fail-fast version of MapFile for sparsely populated key space git-svn-id: https://svn.eu.apache.org/repos/asf/hadoop/core/trunk@726797 13f79535-47bb-0310-9956-ffa450edef68 0c641c493cdb3ccac638a62bf9555bde57741f3d Michael Stack stack@apache.org