Skip to content

Commit

Permalink
Implement unordered container support, xhash no floating point
Browse files Browse the repository at this point in the history
  • Loading branch information
jxy-s committed Sep 18, 2021
1 parent fefdbf7 commit 0ac1d3b
Show file tree
Hide file tree
Showing 15 changed files with 2,281 additions and 28 deletions.
23 changes: 16 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,10 @@ Below is table of functionality under the `jxy` namespace:
| `jxy::set` | `std::set` | `<jxy/set.hpp>` | |
| `jxy::multiset` | `std::multiset` | `<jxy/set.hpp>` | |
| `jxy::stack` | `std::stack` | `<jxy/stack.hpp>` | |
| `jxy::unordered_map` | `std::unordered_map` | `<jxy/unordered_map.hpp>` | |
| `jxy::unordered_multimap` | `std::unordered_multimap` | `<jxy/unordered_map.hpp>` | |
| `jxy::unordered_set` | `std::unordered_set` | `<jxy/unordered_set.hpp>` | |
| `jxy::unordered_multiset` | `std::unordered_multiset` | `<jxy/unordered_set.hpp>` | |

## Tests - `stltest.sys`

Expand Down Expand Up @@ -208,9 +212,6 @@ Key components of `stlkrn.sys`:
| `jxy::ThreadMap` | Maps shared `jxy::ThreadContext` objects to a TID. | `thread_map.hpp/cpp` | The global thread table (singleton) is accessed via `jxy::GetThreadMap`. Each `jxy::ProcessContext` also has a thread map which is accessed through `jxy::ProcessContext::GetThreads`. Uses `jxy::shared_mutex` and `jxy::map`. |
| `jxy::GetModuleMap` | Maps shared `jxy::ModuleContext` to a loaded image extents (base and end address). | `module_map.hpp/cpp` | Each process context has a module map member. Loaded images for a given process are tracked using this object. Uses `jxy::shared_mutex` and `jxy::map` |

`std::unordered_map` would have been a better choice over the ordered tree (`std::map`)
for the object maps. There is a reason this isn't used (see `TODO` section).

```
stlkrn!jxy::nt::CreateProcessNotifyRoutine+0xa6:
3: kd> dx proc
Expand All @@ -229,10 +230,18 @@ proc : {...} [Type: std::shared_ptr<jxy::ProcessContext>]
[+0x070] m_Modules [Type: jxy::ModuleMap]
```

## TODO
I had wanted to include `std::unordered_map` initially, however it uses `ceilf`.
Floating point arithmetic in the Windows Kernel comes with some challenges.
So, for now it is omitted until an appropriate solution is designed.
## A note on `xhash` and `_XHASH_NO_FLOATING_POINT`
In order to support `unordered_map`, `unordered_set`, and the multi variants
`jxystl.lib` copies `xhash` from the MSVC STL and implements an alternative
bucket calculation algorithm. The STL implementation uses floating point
arithmetic, supporting this in the kernel is non-trivial and ill advised. To
avoid floating point arithmetic `jxystl` patches `xhash` with a conditionally
compiled `_XHASH_NO_FLOATING_POINT`. I expect the `jxystl` implementation here
to be faster than the STL's since it completely avoids floating point arithmetic,
however I have not yet had time run good experiments on this hypothesis.

In the future, I would like to submit a pull request to the MSVC STL to affect
this change natively in the MSVC `xhash` implementation.

## Disclaimer
This solution is a passion project. At this time it is not intended for
Expand Down
32 changes: 28 additions & 4 deletions include/jxy/unordered_map.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,38 @@
//
#pragma once
#include <jxy/memory.hpp>

//
// We make a patch to xhash to eliminate floating point arithmetic. To do this
// jxystl copies the MSVC xhash and modifies it slightly. To enforce this we
// make our new define and include "xhash" directly from our source, then
// include the functionality that will use it.
//
// I would like, in the future, to PR this change to the MSVC STL.
//
#define _XHASH_NO_FLOATING_POINT 1
#include "xhash"
#include <unordered_map>

namespace jxy
{

//
// TODO - std::unordered_map uses some floating point arithmetic that hasn't
// been easy to stub/redirect, needs some more thought/effort to get right.
//
template <typename TKey,
typename T,
POOL_TYPE t_PoolType,
ULONG t_PoolTag,
typename THash = std::hash<TKey>,
typename TKeyEqual = std::equal_to<TKey>,
typename TAllocator = jxy::allocator<std::pair<const TKey, T>, t_PoolType, t_PoolTag>>
using unordered_map = std::unordered_map<TKey, T, THash, TKeyEqual, TAllocator>;

template <typename TKey,
typename T,
POOL_TYPE t_PoolType,
ULONG t_PoolTag,
typename THash = std::hash<TKey>,
typename TKeyEqual = std::equal_to<TKey>,
typename TAllocator = jxy::allocator<std::pair<const TKey, T>, t_PoolType, t_PoolTag>>
using unordered_multimap = std::unordered_multimap<TKey, T, THash, TKeyEqual, TAllocator>;

}
31 changes: 26 additions & 5 deletions include/jxy/unordered_set.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,36 @@
//
#pragma once
#include <jxy/memory.hpp>

//
// We make a patch to xhash to eliminate floating point arithmetic. To do this
// jxystl copies the MSVC xhash and modifies it slightly. To enforce this we
// make our new define and include "xhash" directly from our source, then
// include the functionality that will use it.
//
// I would like, in the future, to PR this change to the MSVC STL.
//
#define _XHASH_NO_FLOATING_POINT 1
#include "xhash"
#include <unordered_set>

namespace jxy
{

//
// TODO - std::unordered_set uses some floating point arithmetic that hasn't
// been easy to stub/redirect, needs some more thought/effort to get right.
//
template <typename TKey,
POOL_TYPE t_PoolType,
ULONG t_PoolTag,
typename THash = std::hash<TKey>,
typename TKeyEqual = std::equal_to<TKey>,
typename TAllocator = jxy::allocator<TKey, t_PoolType, t_PoolTag>>
using unordered_set = std::unordered_set<TKey, THash, TKeyEqual, TAllocator>;

}
template <typename TKey,
POOL_TYPE t_PoolType,
ULONG t_PoolTag,
typename THash = std::hash<TKey>,
typename TKeyEqual = std::equal_to<TKey>,
typename TAllocator = jxy::allocator<TKey, t_PoolType, t_PoolTag>>
using unordered_multiset = std::unordered_multiset<TKey, THash, TKeyEqual, TAllocator>;

}

0 comments on commit 0ac1d3b

Please sign in to comment.