use memory mapped files for the peer buffer #12

Merged
merged 4 commits into from Aug 14, 2016

Projects

None yet

2 participants

@arvidn
Contributor
arvidn commented Aug 13, 2016

to preserve state between restarts with some simple unit tests

@ssiloti ssiloti commented on an outdated diff Aug 13, 2016
mapped_file.hpp
+ ~mapped_file() { munmap(m_buf, m_size); }
+
+private:
+
+ file m_file;
+ void* m_buf;
+ size_t const m_size;
+};
+
+template <typename T>
+struct mapped_vector
+{
+ enum { header_size = 16 };
+ mapped_vector(char const* file, size_t const size)
+ : m_map(file, header_size + size * sizeof(T))
+ , m_size(*static_cast<size_t*>(m_map.data()))
@ssiloti
ssiloti Aug 13, 2016 Contributor

This should be reinterpret_cast since you're transmuting the data rather than downcasting.

@ssiloti
ssiloti Aug 13, 2016 Contributor

Never mind, I double checked the C++ spec and static_cast can indeed be used to convert void* to T*. Learn something new every day.

@arvidn
Contributor
arvidn commented Aug 13, 2016

one questionable thing, I think, is that the header size is not technically guaranteed to be a multiple of alignof(T) in mapped_vector. That should actually be pretty easy to fix.

@arvidn arvidn commented on the diff Aug 13, 2016
{
- if (socks.empty())
- {
- fprintf(stderr, "no interfaces to receive on\n");
- return;
- }
-
- this->threadid = threadid;
@arvidn
arvidn Aug 13, 2016 Contributor

I moved the thread id around a bit here, to have it be available in the constructor, in order to create unique names for the mapped files

@arvidn arvidn commented on the diff Aug 13, 2016
@@ -1247,12 +1259,6 @@ struct router_thread
int threadid;
};
-void launch_router_thread(int threadid, std::vector<address> const& bind_addrs)
@arvidn
arvidn Aug 13, 2016 Contributor

I flattened this function into a lambda

@arvidn arvidn commented on the diff Aug 13, 2016
@@ -1466,15 +1489,16 @@ int main(int argc, char* argv[])
, std::ref(signals), std::ref(stats_timer), std::ref(ios)));
ios.run(ec);
+
+ for (auto& i : threads)
@arvidn
arvidn Aug 13, 2016 Contributor

the threads must be joined before destructed. the return below would cause a std::terminate in the thread destructors

@ssiloti
Contributor
ssiloti commented Aug 13, 2016

lgtm, it would probably be a good idea to check the last modified time of the file and discard it if it is too old.

@arvidn
Contributor
arvidn commented Aug 13, 2016

Yeah, I was thinking about that too. I got a bit lazy and thought that the user could always just delete the file :)
maybe in another patch

@arvidn
Contributor
arvidn commented Aug 13, 2016

ok, fixed the alignment of T too

@ssiloti ssiloti commented on an outdated diff Aug 13, 2016
mapped_file.hpp
@@ -86,7 +87,10 @@ struct mapped_file
template <typename T>
struct mapped_vector
{
- enum { header_size = 16 };
+ // the header must be large enough to make the first element still be
+ // correctly aligned
+ static constexpr size_t header_size = std::max(size_t(16), alignof(T));
@ssiloti
ssiloti Aug 13, 2016 Contributor

std::max isn't constexpr in C++11, I don't think the qualifier is necessary in this case.

@ssiloti
Contributor
ssiloti commented Aug 14, 2016

lgtm

@arvidn
Contributor
arvidn commented Aug 14, 2016

rebased on top of master again

@arvidn arvidn merged commit dded993 into bittorrent:master Aug 14, 2016
@arvidn arvidn deleted the arvidn:memory-mapped-file branch Aug 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment