Skip to content

Wrap beast file_writer type to prevent variant explosion. #803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
evoskuil merged 1 commit into from
May 14, 2026
Merged

Wrap beast file_writer type to prevent variant explosion. #803

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -580,6 +580,7 @@ include_bitcoin_network_messages_HEADERS = \
include_bitcoin_network_messages_httpdir = ${includedir}/bitcoin/network/messages/http
include_bitcoin_network_messages_http_HEADERS = \
include/bitcoin/network/messages/http/fields.hpp \
include/bitcoin/network/messages/http/file_writer.hpp \
include/bitcoin/network/messages/http/http.hpp

include_bitcoin_network_messages_http_enumsdir = ${includedir}/bitcoin/network/messages/http/enums
Expand Down
1 change: 1 addition & 0 deletions builds/msvc/vs2022/libbitcoin-network/libbitcoin-network.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,7 @@
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\enums\target.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\enums\verb.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\fields.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\file_writer.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\http.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http_body.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http_method.hpp" />
Expand Down
3 changes: 3 additions & 0 deletions builds/msvc/vs2022/libbitcoin-network/libbitcoin-network.vcxproj.filters
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -779,6 +779,9 @@
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\fields.hpp">
<Filter>include\bitcoin\network\messages\http</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\file_writer.hpp">
<Filter>include\bitcoin\network\messages\http</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\http.hpp">
<Filter>include\bitcoin\network\messages\http</Filter>
</ClInclude>
Expand Down
1 change: 1 addition & 0 deletions builds/msvc/vs2026/libbitcoin-network/libbitcoin-network.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,7 @@
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\enums\target.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\enums\verb.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\fields.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\file_writer.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\http.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http_body.hpp" />
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http_method.hpp" />
Expand Down
3 changes: 3 additions & 0 deletions builds/msvc/vs2026/libbitcoin-network/libbitcoin-network.vcxproj.filters
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -779,6 +779,9 @@
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\fields.hpp">
<Filter>include\bitcoin\network\messages\http</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\file_writer.hpp">
<Filter>include\bitcoin\network\messages\http</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\include\bitcoin\network\messages\http\http.hpp">
<Filter>include\bitcoin\network\messages\http</Filter>
</ClInclude>
Expand Down
3 changes: 1 addition & 2 deletions include/bitcoin/network.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
#include <bitcoin/network/messages/json_body.hpp>
#include <bitcoin/network/messages/messages.hpp>
#include <bitcoin/network/messages/http/fields.hpp>
#include <bitcoin/network/messages/http/file_writer.hpp>
#include <bitcoin/network/messages/http/http.hpp>
#include <bitcoin/network/messages/http/enums/magic_numbers.hpp>
#include <bitcoin/network/messages/http/enums/media_type.hpp>
Expand Down Expand Up @@ -162,7 +163,6 @@
#include <bitcoin/network/sessions/session_seed.hpp>
#include <bitcoin/network/sessions/session_server.hpp>
#include <bitcoin/network/sessions/sessions.hpp>
#if WITH_SSL
#include <bitcoin/network/ssl/openssl/conf.h>
#include <bitcoin/network/ssl/openssl/dh.h>
#include <bitcoin/network/ssl/openssl/engine.h>
Expand Down Expand Up @@ -291,6 +291,5 @@
#include <bitcoin/network/ssl/wolfssl/wolfcrypt/wc_encrypt.h>
#include <bitcoin/network/ssl/wolfssl/wolfcrypt/wc_port.h>
#include <bitcoin/network/ssl/wolfssl/wolfcrypt/wolfmath.h>
#endif

#endif
5 changes: 3 additions & 2 deletions include/bitcoin/network/beast.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,9 @@ namespace http
using chunk_body = boost::beast::http::vector_body<uint8_t,
allocator<uint8_t>>;

// This is wrapped for http_body.
/// beast::http::file_body
using file_body = boost::beast::http::file_body;
////using file_body = boost::beast::http::file_body;

/// beast::http::span_body<uint8_t>
/// Must cast write span uint8_t* to non-const.
Expand All @@ -67,7 +68,7 @@ namespace http
using string_body = boost::beast::http::string_body;

/// general purpose
using file = file_body::value_type;
using file = boost::beast::http::file_body::value_type;
using field = boost::beast::http::field;
using fields = boost::beast::http::fields;
using flat_buffer = boost::beast::flat_buffer;
Expand Down
90 changes: 90 additions & 0 deletions include/bitcoin/network/messages/http/file_writer.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/**
* Copyright (c) 2011-2026 libbitcoin developers (see AUTHORS)
*
* This file is part of libbitcoin.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef LIBBITCOIN_NETWORK_MESSAGES_HTTP_FILE_WRITER_HPP
#define LIBBITCOIN_NETWORK_MESSAGES_HTTP_FILE_WRITER_HPP

#include <memory>
#include <bitcoin/network/define.hpp>

namespace libbitcoin {
namespace network {
namespace http {

/// BOOST_BEAST_FILE_BUFFER_SIZE reduced from 4k to 1k in beast.hpp.
/// beast_body::writer would eat up 1k in variant so wrap in this custom body.
struct file_body
{
using beast_body = boost::beast::http::file_body;
using value_type = beast_body::value_type;
using reader = beast_body::reader;

struct writer
{
using const_buffers_type = asio::const_buffer;
using out_buffer = http::get_buffer<const_buffers_type>;

template <bool IsRequest, class Fields>
explicit writer(http::message_header<IsRequest, Fields>& header,
beast_body::value_type& value) NOEXCEPT
: writer_(std::make_unique<beast_body::writer>(header, value))
{
}

inline void init(boost_code& ec) NOEXCEPT
{
try
{
writer_->init(ec);
}
catch (...)
{
using namespace error;
ec = to_http_code(http_error_t::end_of_stream);
}
}

inline out_buffer get(boost_code& ec) NOEXCEPT
{
try
{
return writer_->get(ec);
}
catch (...)
{
using namespace error;
ec = to_http_code(http_error_t::end_of_stream);
return {};
}
}

private:
std::unique_ptr<beast_body::writer> writer_;
};

static inline uint64_t size(value_type const& body) NOEXCEPT
{
return beast_body::size(body);
}
};

} // namespace http
} // namespace network
} // namespace libbitcoin

#endif
13 changes: 7 additions & 6 deletions include/bitcoin/network/messages/http/http.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@
#ifndef LIBBITCOIN_NETWORK_MESSAGES_HTTP_HTTP_HPP
#define LIBBITCOIN_NETWORK_MESSAGES_HTTP_HTTP_HPP

#include <bitcoin/network/messages/http/enums/magic_numbers.hpp>
#include <bitcoin/network/messages/http/enums/media_type.hpp>
#include <bitcoin/network/messages/http/enums/status.hpp>
#include <bitcoin/network/messages/http/enums/target.hpp>
#include <bitcoin/network/messages/http/enums/verb.hpp>
#include <bitcoin/network/messages/http/fields.hpp>
#include <bitcoin/network/messages/http/enums/magic_numbers.hpp>
#include <bitcoin/network/messages/http/enums/media_type.hpp>
#include <bitcoin/network/messages/http/enums/status.hpp>
#include <bitcoin/network/messages/http/enums/target.hpp>
#include <bitcoin/network/messages/http/enums/verb.hpp>
#include <bitcoin/network/messages/http/fields.hpp>
#include <bitcoin/network/messages/http/file_writer.hpp>

#endif
22 changes: 10 additions & 12 deletions include/bitcoin/network/messages/http_body.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,17 @@ using string_reader = http::string_body::reader;
using json_reader = http::json_body::reader;
using body_reader = std::variant
<
std::monostate,
empty_reader,
data_reader,
file_reader,
span_reader,
buffer_reader,
string_reader,
json_reader,
rpc::reader
std::monostate, // 1 byte
empty_reader, // 1 bytes
data_reader, // 8 bytes
file_reader, // 8 bytes
span_reader, // 8 bytes
buffer_reader, // 8 bytes
string_reader, // 8 bytes
json_reader, // 320 bytes!
rpc::reader // 328 bytes!
>;

// TODO: file_writer is eating 4k stack for each type.
// BOOST_BEAST_FILE_BUFFER_SIZE set to 1024 in beast.hpp.
using empty_writer = http::empty_body::writer;
using data_writer = http::chunk_body::writer;
using file_writer = http::file_body::writer;
Expand All @@ -65,7 +63,7 @@ using body_writer = std::variant
std::monostate, // 1 byte
empty_writer, // 1 byte
data_writer, // 8 bytes
file_writer, // 1,040 bytes! (4,112 bytes by default)
file_writer, // 8 bytes
span_writer, // 8 bytes
buffer_writer, // 16 bytes
string_writer, // 8 bytes
Expand Down
2 changes: 1 addition & 1 deletion src/messages/http/enums/target.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ http::file get_file_body(const std::filesystem::path& path) NOEXCEPT

// http::file::open accepts a "utf-8 encoded path to the file" on win32.
const auto utf8_path = from_path(path);
http::file_body::value_type file{};
http::file file{};

try
{
Expand Down
Loading