Final edition methinks

andradeandrey · Jun 18, 2010 · e4d1245 · e4d1245
1 parent 0d79fe1
commit e4d1245
Show file tree

Hide file tree

Showing 11 changed files with 141 additions and 17 deletions.
diff --git a/!GenCHM.bat b/!GenCHM.bat
@@ -1,5 +1,6 @@
 copy /y Readme.html docs\html\index.html
 copy /y doxygen.css docs\html\doxygen.css
+xcopy /y /s images docs\html\images\
 cd docs\html
 "G:\Program Files\HTML Help Workshop\hhc.exe" index.hhp
 cd ..\..
diff --git a/Readme.html b/Readme.html
@@ -47,7 +47,7 @@
 <body>
 
 <div style="text-align: center">
-	<h1 style="text-decoration: underline">nedtries v1.00 (17th June 2010)</h1>
+	<h1 style="text-decoration: underline">nedtries v1.00 (18th June 2010)</h1>
 	<h2 style="text-decoration: none;">by Niall Douglas</h2>
 	<p>Web site: <a href="http://www.nedprod.com/programs/portable/nedtries/">http://www.nedprod.com/programs/portable/nedtries/</a></p>
 	<hr /></div>
@@ -99,6 +99,8 @@ <h2 style="text-decoration: none;">by Niall Douglas</h2>
 <p>In short, <strong>if 
 you are not already using bitwise tries then you probably ought to be!</strong> 
 Read on for how to add them to your software.</p>
+<div><center>
+<img alt="Bitwise Trees Scaling" height="25%" src="images/BitwiseTreesScaling.png" width="25%" /><img alt="Red Black Trees Scaling" height="25%" src="images/RedBlackTreesScaling.png" width="25%" /><img alt="Hash Table Scaling" height="25%" src="images/HashTableScaling.png" width="25%" /></center></div>
 <h2><a name="implementation">A. Implementation:</a></h2>
 <p>The source makes use of C macros on C and C++ templates on C++ - therefore, 
 unlike typical C-macro-based algorithms it is easy to debug and in fact, the improved 
@@ -110,7 +112,7 @@ <h2><a name="implementation">A. Implementation:</a></h2>
 complete state validation check to be performed after each and every change to 
 the trie which tends to be very good at catching bugs early, but can make debug 
 builds a little slow.</p>
-<p>SSo what is &quot;an in-place bitwise binary Fredkin trie algorithm&quot; then? 
+<p>So what is &quot;an in-place bitwise binary Fredkin trie algorithm&quot; then? 
 Well you ought to start by reading and fully digesting
 <a href="http://en.wikipedia.org/wiki/Trie" target="_blank">the Wikipedia page 
 on Fredkin tries</a> as what comes won&#39;t make much sense otherwise. The 
@@ -142,17 +144,24 @@ <h2><a name="implementation">A. Implementation:</a></h2>
 operations, but if you need to guarantee order then you can bubble sort per MSB 
 bin as bubble sort performs <em>very</em> well on nearly sorted data (as does 
 smooth sort if you have a very large set of data) .</p>
-<p>TThe enclosed benchmark.cpp will run a scalability test comparing the bitwise 
-binary trie implementation from nedtries, the red-black binary tree 
+<p>The enclosed benchmark.cpp will run a series of scalability tests comparing the bitwise 
+binary trie implementation from nedtries with others outputting its results in 
+CSV format:</p>
+<ul>
+	<li>If compiled as C not C++, the C macro version of nedtries is compared to the red-black binary tree 
 implementation from FreeBSD and the O(1) hash table implementation from
-<a href="http://uthash.sourceforge.net/">http://uthash.sourceforge.net/</a> outputting 
-its results in CSV format. You will also find enclosed a set of precomputed 
+	<a href="http://uthash.sourceforge.net/">http://uthash.sourceforge.net/</a>.</li>
+	<li>If compiled as C++, the C++ template version of nedtries is compared to 
+	all of the C tests above as well as the STL associative container classes 
+	std::map&lt;&gt; and std::unordered_map&lt;&gt;. NOTE THAT YOU NEED A TR1 SUPPORTING 
+	COMPILER FOR std::unordered_map&lt;&gt; SUPPORT!</li>
+</ul>
+<p>You will also find enclosed a set of precomputed 
 Microsoft Excel spreadsheets which were generated on a 2.67Ghz Intel Core 2 Quad 
 Windows 7 x64 machine. They should be representative of performance on modern 
 hardware - though note that the Intel Atom has a 17 cycle nedtriebitscanr() 
 which is the only modern CPU to be so slow. See
-<a href="http://gmplib.org/~tege/x86-timing.pdf">
-http://gmplib.org/~tege/x86-timing.pdf</a> for x86 and x64 instruction timings.</p>
+<a href="http://gmplib.org/~tege/x86-timing.pdf">http://gmplib.org/~tege/x86-timing.pdf</a> for x86 and x64 instruction timings.</p>
 <h2><a name="c-usage">B. C Macro Usage:</a></h2>
 <p>Usage via C macros follows the FreeBSD <a href="rbtree.h">rbtree.h</a> 
 format. See the enclosed <a href="nedtries.chm">nedtries.chm</a>  for detailed 
@@ -211,11 +220,51 @@ <h3>Choosing The Nobble Function</h3>
 Start with nobble zeroes which tends to be right in most situations, and then 
 use benchmarking your code to determine the correct setting.</p>
 <h2><a name="changelog0">C. C++ Usage:</a></h2>
-<p>&lt;coming soon&gt;</p>
+<p>C++ usage is even easier than the C macro usage thanks to nedtries::trie_map&lt;&gt; 
+which is API compatible with the std::map&lt;&gt;, std::multimap&lt;&gt; and std::unordered_map&lt;&gt; 
+STL associative containers. trie_map&lt;&gt; 
+makes full use of rvalue construction if either you are running on C++0x 
+according to the value of __cplusplus, or have 
+defined HAVE_CPP0XRVALUEREFS. In the general case, simply drop trie_map&lt;&gt; in where 
+your STL associative container used to be and enjoy the speed benefits!</p>
+<p>Note that insertion and deletion speed in <strong>any</strong> STL container 
+is heavily bound by the speed of your memory allocator. You may wish to consider 
+employing
+<a href="http://www.nedprod.com/programs/portables/nedmalloc/" target="_blank">
+nedmalloc</a> which can deliver some unholy speed benefits if you run it as 
+root, otherwise it will need some small source changes to employ its advanced v2 
+malloc API.</p>
+<p>In case you are not familiar with STL associative containers, they are very 
+simple e.g.:</p>
+<pre>nedtries::trie_map&lt;size_t, Foo&gt; foomap;
+foomap[5]=Foo();
+foomap.erase(foomap.find(5));</pre>
+<p>You can of course iterate through them and do all the normal things you can 
+do with any STL container.</p>
+<h3>The trie_map&lt;&gt; Implementation</h3>
+<p>trie_map&lt;&gt; is actually a STL container <em>wrapper</em> rather than a proper STL 
+container in its own right i.e. it subclasses an existing STL container passing 
+through most of its API, but selectively overrides certain members. Its default 
+parameters point at std::list&lt;&gt; which is its most likely usage model for most 
+people.</p>
+<p>The advantages are mainly that it is quick to implement and can be 
+theoretically applied to any arbitrary STL container, thus taking advantage of 
+that STL container&#39;s optimisations and customisations. The big disadvantage is 
+that it is hacky, dirty and prone to getting bugs into it, and if you look at 
+the source you&#39;ll see what I mean. There is after all a number of places where I 
+am doing a number of very illegal things in C++ which just happen to usually 
+work.</p>
+<p>The chances are that this implementation will be good enough for most people. 
+If however you might like to sponsor the development of a full bitwise trie STL 
+associative container for submission to
+<a href="http://www.boost.org/" target="_blank">the Boost C++ peer reviewed 
+libraries</a> (and thereafter into the standard C++ language itself), I would be 
+very pleased to oblige. <a href="http://www.nedproductions.biz/" target="_blank">
+Please contact ned Productions Consulting Ltd. for further details</a>.</p>
 <h2><a name="changelog">D. ChangeLog:</a></h2>
-<h3>v1.00 beta 1 (17th June 2010):</h3>
+<h3>v1.00 beta 1 (18th June 2010):</h3>
 <ul>
-	<li><span class="gitcommit">[master 89f1806]</span> First release.</li>
+	<li><span class="gitcommit">[master xxxxxxx]</span> First release.</li>
 </ul>
 
 </body>

diff --git a/SConscript b/SConscript
@@ -0,0 +1,10 @@
+import os, sys, platform
+
+Import("env")
+
+# Test program
+sources = [ "benchmark.cpp" ]
+objects = env.Object(source = sources)
+testprogram = env.Program("benchmark", source = objects)
+
+Default(testprogram)
diff --git a/SConstruct b/SConstruct
@@ -0,0 +1,64 @@
+import os, sys, platform
+
+env = Environment()
+
+# Force scons to always use absolute paths in everything (helps debuggers to find source files)
+env['CCCOM']   =    env['CCCOM'].replace('$CHANGED_SOURCES','$SOURCES.abspath')
+env['SHCCCOM'] =  env['SHCCCOM'].replace('$CHANGED_SOURCES','$SOURCES.abspath')
+env['CXXCOM']  =   env['CXXCOM'].replace('$CHANGED_SOURCES','$SOURCES.abspath')
+env['SHCXXCOM']= env['SHCXXCOM'].replace('$CHANGED_SOURCES','$SOURCES.abspath')
+architecture="generic"
+env['CPPDEFINES']=[]
+env['CCFLAGS']=[]
+env['LIBS']=[]
+env['LINKFLAGS']=[]
+env['CCFLAGSFORNEDMALLOC']=[]
+
+# Am I in a 32 or 64 bit environment? Note that not specifying --sse doesn't set any x86 or x64 specific options
+# so it's good to go for ANY platform
+if sys.platform=="win32":
+    # Even the very latest scons still screws this up :(
+    env['ENV']['INCLUDE']=os.environ['INCLUDE']
+    env['ENV']['LIB']=os.environ['LIB']
+    env['ENV']['PATH']=os.environ['PATH']
+
+# Am I building a debug or release build?
+if env.GetOption('debug'):
+    env['CPPDEFINES']+=["DEBUG", "_DEBUG"]
+    variant=architecture+"/Debug"
+else:
+    env['CPPDEFINES']+=["NDEBUG"]
+    variant=architecture+"/Release"
+
+# Am I building for Windows or POSIX?
+if sys.platform=='win32':
+    env['CPPDEFINES']+=["WIN32", "_WINDOWS", "UNICODE", "_UNICODE"]
+    env['CCFLAGS']+=["/EHsc"]           # Turn on exception handling
+    env['CCFLAGS']+=["/GF"]             # Eliminate duplicate strings
+    env['CCFLAGS']+=["/Gy"]             # Seperate COMDATs
+    env['CCFLAGS']+=["/Zi"]             # Program database debug info
+    if env.GetOption('debug'):
+        env['CCFLAGS']+=["/Od", "/MDd"]
+    else:
+        env['CCFLAGS']+=["/O2", "/MD"]
+    env['LIBS']+=["psapi", "user32", "advapi32"]
+    env['LINKFLAGS']+=["/DEBUG"]                # Output debug symbols
+    env['LINKFLAGS']+=["/LARGEADDRESSAWARE"]    # Works past 2Gb
+    env['LINKFLAGS']+=["/NXCOMPAT"]             # Likes no execute
+
+    env['LINKFLAGS']+=["/MANIFEST"]             # Be UAC compatible
+
+    if not env.GetOption('debug'):
+        env['LINKFLAGS']+=["/OPT:REF", "/OPT:ICF"]  # Eliminate redundants
+else:
+    env['CPPDEFINES']+=[]
+    env['CCFLAGS']+=["-Wall", "-std=gnu++0x"]
+    if env.GetOption('debug'):
+        env['CCFLAGS']+=["-O0", "-g"]
+    else:
+        env['CCFLAGS']+=["-O2", "-g"]
+    env['LIBS']+=["rt"]
+    env['LINKFLAGS']+=[]
+
+# Build
+nedmalloclib=SConscript("SConscript", variant_dir=variant, duplicate=False, exports="env")
diff --git a/images/BitwiseTreesScaling.png b/images/BitwiseTreesScaling.png
diff --git a/images/HashTableScaling.png b/images/HashTableScaling.png
diff --git a/images/RedBlackTreesScaling.png b/images/RedBlackTreesScaling.png
diff --git a/nedalloc.chm b/nedalloc.chm
diff --git a/nedtrie.h b/nedtrie.h
@@ -1364,7 +1364,7 @@ namespace nedtries {
     trie_maptype(const type &v) : trie_value(v) { }
     template<class otype, class oittype> trie_maptype(const trie_maptype<otype, oittype> &o) : trie_value(o.trie_value) { }
 #ifdef HAVE_CPP0XRVALUEREFS
-	  template<class otype, class oittype> trie_maptype(trie_maptype<otype, oittype> &&o) : value(std::move(o.trie_value)) { }
+	  template<class otype, class oittype> trie_maptype(trie_maptype<otype, oittype> &&o) : trie_value(std::move(o.trie_value)) { }
 #endif
     //! Silent const lvalue converter for type
     operator const type &() const { return trie_value; }
@@ -1604,7 +1604,7 @@ namespace nedtries {
     {
       memcpy(&triehead, &o.triehead, sizeof(triehead));
     }
-    template<class okeytype, class otype, class oallocator> triemap &operator=(trie_map<okeytype, otype, oallocator> &&o)
+    template<class okeytype, class otype, class oallocator> trie_map &operator=(trie_map<okeytype, otype, oallocator> &&o)
     {
       *static_cast<stlcontainer *>(this)=std::move(static_cast<stlcontainer &&>(o));
       memcpy(&triehead, &o.triehead, sizeof(triehead));

diff --git a/nedtries.vcxproj b/nedtries.vcxproj
@@ -68,7 +68,7 @@
   </PropertyGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
     <ClCompile>
-      <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;HAVE_CPP0XRVALUEREFS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
       <WarningLevel>Level3</WarningLevel>
       <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
@@ -82,7 +82,7 @@
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
     <ClCompile>
-      <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;HAVE_CPP0XRVALUEREFS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
       <WarningLevel>Level3</WarningLevel>
       <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
@@ -95,7 +95,7 @@
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
     <ClCompile>
-      <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;HAVE_CPP0XRVALUEREFS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
       <WarningLevel>Level3</WarningLevel>
       <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
@@ -112,7 +112,7 @@
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
     <ClCompile>
-      <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;HAVE_CPP0XRVALUEREFS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
       <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
       <EnableEnhancedInstructionSet>NotSet</EnableEnhancedInstructionSet>

diff --git a/results64.xlsx b/results64.xlsx