Skip to content
Merged

V0.13.x #1999

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
57d9405
test: update pmr test, allow 4b alignment for test memalign
alfreb Dec 5, 2018
89f4aa4
tcp: Replace global rcv wnd getter with one based on the connections …
AndreasAakesson Dec 5, 2018
97eb09a
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 5, 2018
da87ce7
tcp: avoid using delegate by default for receive window calculation
alfreb Dec 5, 2018
cf1cd14
examples: Add a TCP-to-TCP transfer example
fwsGonzo Dec 5, 2018
dfa9c4e
pmr: add on_non_full event to notify when not full anymore
alfreb Dec 6, 2018
d439ec0
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
AndreasAakesson Dec 6, 2018
8393490
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
alfreb Dec 6, 2018
5512518
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
alfreb Dec 6, 2018
d22f68c
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
alfreb Dec 6, 2018
4ddb164
Merge branch 'v0.13.x' of https://github.com/hioa-cs/IncludeOS into v…
alfreb Dec 6, 2018
2a5af62
tcp: Calc rcv wnd more frequent etc
AndreasAakesson Dec 6, 2018
07b1a3b
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
AndreasAakesson Dec 6, 2018
629e6ca
Merge branch 'v0.13.x' of github.com:alfred-bratterud/IncludeOS into …
alfreb Dec 6, 2018
1e9612f
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 6, 2018
421d897
tcp: WIP move drop on rcv-win closed and allow window updates when cl…
alfreb Dec 6, 2018
a3093fe
test: TCP circle of evil
alfreb Dec 6, 2018
c6e0079
liveupdate: allocate new memory resource on deserialize connection
alfreb Dec 7, 2018
fbbc1a1
microlb: give default to new ctor param to preserve API
alfreb Dec 7, 2018
57d11b8
tcp: update rcv win on all dropped packets
alfreb Dec 7, 2018
26f1b9a
tcp: update receive window on all acks, avoid silly windows
alfreb Dec 7, 2018
b10f96b
microlb: use TCP's internal receive window calculations
alfreb Dec 7, 2018
0b44040
Merge branch 'v0.13.x' of github.com:fwsGonzo/IncludeOS into v0.13.x
fwsGonzo Dec 7, 2018
2a5396b
tcp: SSSE3 checksum algo added alongside AVX need -mssse3 for it to b…
KristianJerpetjon Dec 7, 2018
a32d6eb
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
alfreb Dec 8, 2018
506b3fa
tcp: return early if connection couldn't be established
alfreb Dec 11, 2018
2f44d2b
test: add stats printing for lb test
alfreb Dec 11, 2018
3c8ec30
pmr: avoid giving whole pool to a single resource
alfreb Dec 11, 2018
75497dc
tcp: flush mem on time-wait, abort when no memory etc.
alfreb Dec 11, 2018
1a788bd
test: Link with experimental lib when clang apple
AndreasAakesson Dec 12, 2018
a8b3475
tcp: Only increase rtx attempts on rtx timeout
AndreasAakesson Dec 13, 2018
729f32b
test: update pmr test to reflect division factor
alfreb Dec 13, 2018
ca0bd7d
test: add back MSL timeout to microlb test
alfreb Dec 13, 2018
acbe909
tcp: shared ptr cleanup magic
AndreasAakesson Dec 13, 2018
0caa610
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 13, 2018
c6e9ebf
tcp: Don't use shared ptr when cleaning up
AndreasAakesson Dec 13, 2018
c270d4b
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 13, 2018
6e99a2e
test: update stress test to handle new arping API
alfreb Dec 13, 2018
41631ee
pmr: add on_avail event for notification of allocatable >= N
alfreb Dec 14, 2018
d640be0
tcp: Avoid bug when ts get lost by reparsing if lost, trigger window …
AndreasAakesson Dec 15, 2018
c94cf30
Merge branch 'v0.13.x' of github.com:alfred-bratterud/IncludeOS into …
AndreasAakesson Dec 15, 2018
ebc56aa
tcp: Change trigger send window event to avail
AndreasAakesson Dec 15, 2018
6fa9617
net: remove wrong endif from new checksum.cpp
alfreb Dec 17, 2018
b45230c
Merge branch 'v0.13.x' of github.com:alfred-bratterud/IncludeOS into …
alfreb Dec 17, 2018
6b00d25
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 17, 2018
04deec3
microlb: removed code that closed connection once buffer limit of 8k …
KristianJerpetjon Dec 17, 2018
b01d4de
Merge pull request #9 from KristianJerpetjon/tcp_rcvwnd
AndreasAakesson Dec 18, 2018
c22072d
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 18, 2018
39d538e
tcp: Avoid wrap in seq num check, reparse TS if missing
AndreasAakesson Dec 18, 2018
43d4344
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
AndreasAakesson Dec 18, 2018
4debeac
net: added function to check if a buffer has unhandled data to check …
KristianJerpetjon Dec 18, 2018
d6b28af
microlb: wrong ordering between flush and session create caused out o…
KristianJerpetjon Dec 18, 2018
e1f2fb1
Merge pull request #10 from KristianJerpetjon/tcp_rcvwnd
AndreasAakesson Dec 18, 2018
72cf163
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
alfreb Dec 20, 2018
079e8e4
tcp: added some debugging prints, remove later
AndreasAakesson Dec 20, 2018
9d89358
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
AndreasAakesson Dec 20, 2018
6f7c159
tcp: comment out some debugging
AndreasAakesson Jan 4, 2019
ae0b58e
Correct connection tracking cherry picked from ipv6
KristianJerpetjon Jan 6, 2019
74a5a6e
tcp: add on_data event with sync read. WIP.
alfreb Jan 8, 2019
bc7ad77
net::Stream: add on_data & friends. WIP.
alfreb Jan 8, 2019
3f94857
tcp: allow buffer reuse and pop completed after pushing to on_read
alfreb Jan 8, 2019
7e56f12
refactor: Moved tls from header to source code
KristianJerpetjon Jan 8, 2019
cea1161
Merge remote-tracking branch 'alfred/v0.13.x' into 0.13.x
KristianJerpetjon Jan 8, 2019
7ec4f5b
0.13.x: Created StreamBuffer to wrap Stream and refactor out parts of…
KristianJerpetjon Jan 10, 2019
1b87519
pmr: make default resource throw on failure
alfreb Jan 11, 2019
4317157
typo: violated address
alfreb Jan 11, 2019
6a95e51
microlb: Timer event to handle congestion
KristianJerpetjon Jan 11, 2019
4893912
Merge remote-tracking branch 'alfred/v0.13.x' into 0.13.x
KristianJerpetjon Jan 11, 2019
2c112ee
0.13.x: Cleaned up code
KristianJerpetjon Jan 14, 2019
5d66015
test: improved server.js so that it can generate file size on request
KristianJerpetjon Jan 14, 2019
d0318e7
microlb: deferred destruction of stream elements
KristianJerpetjon Jan 14, 2019
28347fe
tls_streams: further hardening of m_busy.
KristianJerpetjon Jan 14, 2019
2eefdb1
tls_streams: further hardening of m_busy.
KristianJerpetjon Jan 14, 2019
ba7384d
Merge branch 'v0.13.x' of https://github.com/alfred-bratterud/Include…
KristianJerpetjon Jan 15, 2019
1f71b32
Merge branch '0.13.x' of https://github.com/KristianJerpetjon/Include…
KristianJerpetjon Jan 15, 2019
b6b381d
0.13.x: if reset callbacks is called in loop exit
KristianJerpetjon Jan 15, 2019
21c3c28
0.13.x: if reset callbacks is called in loop exit
KristianJerpetjon Jan 15, 2019
8f38456
Merge branch '0.13.x' of https://github.com/KristianJerpetjon/Include…
KristianJerpetjon Jan 16, 2019
1e33fb1
Merge branch '0.13.x' of github.com:KristianJerpetjon/IncludeOS into …
AndreasAakesson Jan 17, 2019
6ae0fc9
pmr: Don't promise more capacity than whats actually remaining in pool
AndreasAakesson Jan 18, 2019
08a8a28
tcp: Throw exception when trying to connect if not enough resource mem
AndreasAakesson Jan 18, 2019
1b7b646
microlb: Handle exception when connect and terminate hack for unused …
AndreasAakesson Jan 18, 2019
27fd8af
misc: removed some dead code
AndreasAakesson Jan 18, 2019
03b3017
Merge pull request #1 from AndreasAakesson/tcp_rcvwnd
KristianJerpetjon Jan 18, 2019
e3a098c
Merge pull request #9 from KristianJerpetjon/0.13.x
alfreb Jan 18, 2019
d4f295f
net: Remove namespace from call to ntohs for building unittest on mac
AndreasAakesson Jan 21, 2019
73e5f23
test: Update tcp read request test to reflect changes to API
AndreasAakesson Jan 21, 2019
0081283
net: removed unicode char in stream buffer code
AndreasAakesson Jan 21, 2019
82b0219
pmr: return subcap if set
AndreasAakesson Jan 21, 2019
1db1523
net: Flush data in stream buffer when handler is set
AndreasAakesson Jan 22, 2019
afc9044
microlb: Replace lambda with member functions
AndreasAakesson Jan 22, 2019
9cfc81e
test: Increase memory for microLB test
AndreasAakesson Jan 22, 2019
a181276
microlb: Add try/catch to avoid silently invalidating the client queue
fwsGonzo Jan 22, 2019
1f59bc0
Merge with Andreas v0.13.x
fwsGonzo Jan 22, 2019
dcfdbaa
microlb: Remove reference to readq
fwsGonzo Jan 22, 2019
6a5f8a5
Test: Set a timeout for microlb get connection
mnordsletten Jan 22, 2019
03fccd6
Merge pull request #11 from mnordsletten/xx
AndreasAakesson Jan 22, 2019
b94c9f7
Merge branch 'tcp_rcvwnd' of github.com:AndreasAakesson/IncludeOS int…
AndreasAakesson Jan 22, 2019
948f230
posix: Fix TCP posix hanging when data and FIN is recv right after ea…
AndreasAakesson Jan 22, 2019
39a3790
Merge pull request #10 from AndreasAakesson/tcp_rcvwnd
alfreb Jan 22, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions api/net/botan/tls_server.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@ class Server : public Botan::TLS::Callbacks, public net::Stream
m_transport->on_read(bs, {this, &Server::tls_read});
this->m_on_read = cb;
}
void on_data(DataCallback cb) override {
// FIXME
throw std::runtime_error("on_data not implemented on botan::server");
}
size_t next_size() override {
// FIXME
throw std::runtime_error("next_size not implemented on botan::server");
}
buffer_t read_next() override {
// FIXME
throw std::runtime_error("read_next not implemented on botan::server");
}
void on_write(WriteCallback cb) override {
this->m_on_write = cb;
}
Expand Down
288 changes: 10 additions & 278 deletions api/net/openssl/tls_stream.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
#include <openssl/err.h>
#include <openssl/pem.h>
#include <openssl/ssl.h>
#include <net/stream.hpp>
#include <net/stream_buffer.hpp>

//#define VERBOSE_OPENSSL
//#define VERBOSE_OPENSSL 0
#ifdef VERBOSE_OPENSSL
#define TLS_PRINT(fmt, ...) printf(fmt, ##__VA_ARGS__)
#define TLS_PRINT(fmt, ...) printf("TLS_Stream");printf(fmt, ##__VA_ARGS__)
#else
#define TLS_PRINT(fmt, ...) /* fmt */
#endif

namespace openssl
{
struct TLS_stream : public net::Stream
struct TLS_stream : public net::StreamBuffer
{
using Stream_ptr = net::Stream_ptr;

Expand All @@ -25,7 +25,6 @@ namespace openssl
void write(const std::string&) override;
void write(const void* buf, size_t n) override;
void close() override;
void reset_callbacks() override;

net::Socket local() const override {
return m_transport->local();
Expand All @@ -37,24 +36,11 @@ namespace openssl
return m_transport->to_string();
}

void on_connect(ConnectCallback cb) override {
m_on_connect = std::move(cb);
}
void on_read(size_t, ReadCallback cb) override {
m_on_read = std::move(cb);
}
void on_close(CloseCallback cb) override {
m_on_close = std::move(cb);
}
void on_write(WriteCallback cb) override {
m_on_write = std::move(cb);
}

bool is_connected() const noexcept override {
return handshake_completed() && m_transport->is_connected();
}
bool is_writable() const noexcept override {
return is_connected() && m_transport->is_writable();
return (not write_congested()) && is_connected() && m_transport->is_writable();
}
bool is_readable() const noexcept override {
return m_transport->is_readable();
Expand All @@ -76,7 +62,12 @@ namespace openssl

size_t serialize_to(void*) const override;

void handle_read_congestion() override;
void handle_write_congestion() override;
private:
void handle_data();
int decrypt(const void *data,int size);
int send_decrypted();
void tls_read(buffer_t);
int tls_perform_stream_write();
int tls_perform_handshake();
Expand All @@ -89,271 +80,12 @@ namespace openssl
STATUS_FAIL
};
status_t status(int n) const noexcept;

Stream_ptr m_transport = nullptr;
SSL* m_ssl = nullptr;
BIO* m_bio_rd = nullptr;
BIO* m_bio_wr = nullptr;
bool m_busy = false;
bool m_deferred_close = false;
ConnectCallback m_on_connect = nullptr;
ReadCallback m_on_read = nullptr;
WriteCallback m_on_write = nullptr;
CloseCallback m_on_close = nullptr;
};

inline TLS_stream::TLS_stream(SSL_CTX* ctx, Stream_ptr t, bool outgoing)
: m_transport(std::move(t))
{
ERR_clear_error(); // prevent old errors from mucking things up
this->m_bio_rd = BIO_new(BIO_s_mem());
this->m_bio_wr = BIO_new(BIO_s_mem());
assert(ERR_get_error() == 0 && "Initializing BIOs");
this->m_ssl = SSL_new(ctx);
assert(this->m_ssl != nullptr);
assert(ERR_get_error() == 0 && "Initializing SSL");
// TLS server-mode
if (outgoing == false)
SSL_set_accept_state(this->m_ssl);
else
SSL_set_connect_state(this->m_ssl);

SSL_set_bio(this->m_ssl, this->m_bio_rd, this->m_bio_wr);
// always-on callbacks
m_transport->on_read(8192, {this, &TLS_stream::tls_read});
m_transport->on_close({this, &TLS_stream::close_callback_once});

// start TLS handshake process
if (outgoing == true)
{
if (this->tls_perform_handshake() < 0) return;
}
}
inline TLS_stream::TLS_stream(Stream_ptr t, SSL* ssl, BIO* rd, BIO* wr)
: m_transport(std::move(t)), m_ssl(ssl), m_bio_rd(rd), m_bio_wr(wr)
{
// always-on callbacks
m_transport->on_read(8192, {this, &TLS_stream::tls_read});
m_transport->on_close({this, &TLS_stream::close_callback_once});
}
inline TLS_stream::~TLS_stream()
{
assert(m_busy == false && "Cannot delete stream while in its call stack");
SSL_free(this->m_ssl);
}

inline void TLS_stream::write(buffer_t buffer)
{
if (UNLIKELY(this->is_connected() == false)) {
TLS_PRINT("TLS_stream::write() called on closed stream\n");
return;
}

int n = SSL_write(this->m_ssl, buffer->data(), buffer->size());
auto status = this->status(n);
if (status == STATUS_FAIL) {
this->close();
return;
}

do {
n = tls_perform_stream_write();
} while (n > 0);
}
inline void TLS_stream::write(const std::string& str)
{
write(net::Stream::construct_buffer(str.data(), str.data() + str.size()));
}
inline void TLS_stream::write(const void* data, const size_t len)
{
auto* buf = static_cast<const uint8_t*> (data);
write(net::Stream::construct_buffer(buf, buf + len));
}

inline void TLS_stream::tls_read(buffer_t buffer)
{
ERR_clear_error();
uint8_t* buf = buffer->data();
int len = buffer->size();

while (len > 0)
{
int n = BIO_write(this->m_bio_rd, buf, len);
if (UNLIKELY(n < 0)) {
this->close();
return;
}
buf += n;
len -= n;

// if we aren't finished initializing session
if (UNLIKELY(!handshake_completed()))
{
int num = SSL_do_handshake(this->m_ssl);
auto status = this->status(num);

// OpenSSL wants to write
if (status == STATUS_WANT_IO)
{
tls_perform_stream_write();
}
else if (status == STATUS_FAIL)
{
if (num < 0) {
TLS_PRINT("TLS_stream::SSL_do_handshake() returned %d\n", num);
#ifdef VERBOSE_OPENSSL
ERR_print_errors_fp(stdout);
#endif
}
this->close();
return;
}
// nothing more to do if still not finished
if (handshake_completed() == false) return;
// handshake success
if (m_on_connect) m_on_connect(*this);
}

// read decrypted data
do {
char temp[8192];
n = SSL_read(this->m_ssl, temp, sizeof(temp));
if (n > 0) {
auto buf = net::Stream::construct_buffer(temp, temp + n);
if (m_on_read) {
this->m_busy = true;
m_on_read(std::move(buf));
this->m_busy = false;
}
}
} while (n > 0);
// this goes here?
if (UNLIKELY(this->is_closing() || this->is_closed())) {
TLS_PRINT("TLS_stream::SSL_read closed during read\n");
return;
}
if (this->m_deferred_close) {
this->close(); return;
}

auto status = this->status(n);
// did peer request stream renegotiation?
if (status == STATUS_WANT_IO)
{
do {
n = tls_perform_stream_write();
} while (n > 0);
}
else if (status == STATUS_FAIL)
{
this->close();
return;
}
// check deferred closing
if (this->m_deferred_close) {
this->close(); return;
}

} // while it < end
} // tls_read()

inline int TLS_stream::tls_perform_stream_write()
{
ERR_clear_error();
int pending = BIO_ctrl_pending(this->m_bio_wr);
//printf("pending: %d\n", pending);
if (pending > 0)
{
auto buffer = net::Stream::construct_buffer(pending);
int n = BIO_read(this->m_bio_wr, buffer->data(), buffer->size());
assert(n == pending);
m_transport->write(buffer);
if (m_on_write) {
this->m_busy = true;
m_on_write(n);
this->m_busy = false;
}
return n;
}
else {
BIO_read(this->m_bio_wr, nullptr, 0);
}
if (!BIO_should_retry(this->m_bio_wr))
{
this->close();
return -1;
}
return 0;
}
inline int TLS_stream::tls_perform_handshake()
{
ERR_clear_error(); // prevent old errors from mucking things up
// will return -1:SSL_ERROR_WANT_WRITE
int ret = SSL_do_handshake(this->m_ssl);
int n = this->status(ret);
ERR_print_errors_fp(stderr);
if (n == STATUS_WANT_IO)
{
do {
n = tls_perform_stream_write();
if (n < 0) {
TLS_PRINT("TLS_stream::tls_perform_handshake() stream write failed\n");
}
} while (n > 0);
return n;
}
else {
TLS_PRINT("TLS_stream::tls_perform_handshake() returned %d\n", ret);
this->close();
return -1;
}
}

inline void TLS_stream::close()
{
//ERR_clear_error();
if (this->m_busy) {
this->m_deferred_close = true; return;
}
CloseCallback func = std::move(this->m_on_close);
this->reset_callbacks();
if (m_transport->is_connected())
m_transport->close();
if (func) func();
}
inline void TLS_stream::close_callback_once()
{
if (this->m_busy) {
this->m_deferred_close = true; return;
}
CloseCallback func = std::move(this->m_on_close);
this->reset_callbacks();
if (func) func();
}
inline void TLS_stream::reset_callbacks()
{
this->m_on_close = nullptr;
this->m_on_connect = nullptr;
this->m_on_read = nullptr;
this->m_on_write = nullptr;
}

inline bool TLS_stream::handshake_completed() const noexcept
{
return SSL_is_init_finished(this->m_ssl);
}
inline TLS_stream::status_t TLS_stream::status(int n) const noexcept
{
int error = SSL_get_error(this->m_ssl, n);
switch (error)
{
case SSL_ERROR_NONE:
return STATUS_OK;
case SSL_ERROR_WANT_WRITE:
case SSL_ERROR_WANT_READ:
return STATUS_WANT_IO;
default:
return STATUS_FAIL;
}
}
} // openssl
21 changes: 20 additions & 1 deletion api/net/stream.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,32 @@ namespace net {
/** Called with a shared buffer and the length of the data when received. */
using ReadCallback = delegate<void(buffer_t)>;
/**
* @brief Event when data is received.
* @brief Event when data is received. Pushes data to the callback.
*
* @param[in] n The size of the receive buffer
* @param[in] cb The read callback
*/
virtual void on_read(size_t n, ReadCallback cb) = 0;

using DataCallback = delegate<void()>;
/**
* @brief Event when data is received.
* Does not push data, just signals its presence.
*
* @param[in] cb The callback
*/
virtual void on_data(DataCallback cb) = 0;

/**
* @return The size of the next available chunk of data if any.
*/
virtual size_t next_size() = 0;

/**
* @return The next available chunk of data if any.
*/
virtual buffer_t read_next() = 0;

/** Called with nothing ¯\_(ツ)_/¯ */
using CloseCallback = delegate<void()>;
/**
Expand Down
Loading