Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 2 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,54 +42,6 @@ The tracepoints `CLLTK_TRACEPOINT` defines first the target tracebuffer, than th
`decoder_tool/python/clltk_decoder.py <path to tracebuffers>`
8. View your traces in `output.csv`

## Build Requirements
### Base Library build
- cmake (>=3.18)
- gcc (>=10.0)
- g++ (>=10.0)
- git
- file
- rsync
- gettext
- libtar-dev

### CLI tool build
- libcli11-dev
- nlohmann-json3-dev

### Library Unit tests
- valgrind
- openjdk-17-jdk
- openssl
- lcov
- gcovr

### Kernel module build
- make
- curl
- rpm
- elfutils
- clang
- ll
- llvm
- libelf-dev
- build-essential
- flex
- bison
- bc
- libncurses-dev
- libssl-dev
- kmod

### Kernel module tests
- busybox-static
- strace
- cpio
- qemu-system

## Runtime Requirements
- non

## Security Considerations
**Tampering with Trace Files**: Any modification or tampering with the trace files can cause the library to **crash or potentially freeze** the system. Ensure the integrity of these files is maintained to avoid instability.

Expand Down Expand Up @@ -159,15 +111,15 @@ It is also possible to cross compile with the container env by using of example:
To build this repository for test purposes or development run:

```bash
./scripts/ci-cd/build_userspace.shs
./scripts/ci-cd/build_userspace.sh
```

### Tests tracing standalone

To run all test build the whole project with cmake and than run:

```bash
./scripts/ci-cd/test_userspace.shs
./scripts/ci-cd/test_userspace.sh
```

This will run all c++ and python tests.
Expand Down
4 changes: 3 additions & 1 deletion VERSION.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
1.2.48
1.2.49

# Change log
## 1.2.49
- fix typos
## 1.2.48
refactor: reorganize include structure and improve project layout
- Renamed project from CommoneLowLevelTracingKit to CommonLowLevelTracingKit (correcting typo).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ namespace CommonLowLevelTracingKit::decoder {
const std::filesystem::path m_path;
};

// Synchonous Tracebuffer
// Synchronous Tracebuffer
struct SyncTracebuffer;
using SyncTracebufferPtr = std::unique_ptr<SyncTracebuffer>;
struct EXPORT SyncTracebuffer : Tracebuffer {
Expand Down
22 changes: 11 additions & 11 deletions decoder_tool/cpp/source/low_level/formatter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

#include "inline.hpp"

namespace formater = CommonLowLevelTracingKit::decoder::source::formatter;
namespace formatter = CommonLowLevelTracingKit::decoder::source::formatter;
using namespace CommonLowLevelTracingKit::decoder::exception;
using namespace std::string_literals;

Expand All @@ -39,7 +39,7 @@ CONST_INLINE static constexpr ffi_type *clltk_type_to_ffi_type(const char clltk_
case 's': return &ffi_type_pointer;
case 'p': return &ffi_type_pointer;
case InvalidStringArgType: return &ffi_type_pointer;
default: CLLTK_DECODER_THROW(FormattingFailed, "unkown type");
default: CLLTK_DECODER_THROW(FormattingFailed, "unknown type");
}
}
CONST_INLINE static constexpr size_t
Expand Down Expand Up @@ -71,13 +71,13 @@ clltk_arg_to_size(const char clltk_type, const uintptr_t clltk_arg, size_t remai
}
case 'p': return sizeof(void *);
case InvalidStringArgType: return sizeof(void *);
default: CLLTK_DECODER_THROW(FormattingFailed, "unkown type");
default: CLLTK_DECODER_THROW(FormattingFailed, "unknown type");
}
}
template <typename T, typename ProxyT = T>
static INLINE const any get_native(uintptr_t p, size_t remaining) {
if (sizeof(T) > remaining) [[unlikely]]
CLLTK_DECODER_THROW(FormattingFailed, "out of range access for formater");
CLLTK_DECODER_THROW(FormattingFailed, "out of range access for formatter");
T value{};
memcpy(&value, std::bit_cast<void *>(p), sizeof(T));
static_assert(sizeof(ProxyT) == sizeof(void *));
Expand Down Expand Up @@ -105,7 +105,7 @@ INLINE static constexpr any clltk_arg_to_native(const char clltk_type, const uin
// the value in clltk_arg is a pointer to a now invalid/unusable memory address.
// replace it with a dummy arg string
return std::bit_cast<uint64_t>(&InvalidStringArg);
default: CLLTK_DECODER_THROW(FormattingFailed, "unkown type");
default: CLLTK_DECODER_THROW(FormattingFailed, "unknown type");
}
}

Expand All @@ -120,7 +120,7 @@ clltk_args_to_native_args(const std::string_view format, const std::span<const c
for (size_t i = 0; i < clltk_types.size(); i++) {
const char type = clltk_types[i];
if (raw_arg_offset >= raw_clltk_args.size()) [[unlikely]]
CLLTK_DECODER_THROW(FormattingFailed, "out of range access for formater");
CLLTK_DECODER_THROW(FormattingFailed, "out of range access for formatter");
const uintptr_t current = std::bit_cast<uintptr_t>(&raw_clltk_args[raw_arg_offset]);
const size_t remaining = raw_clltk_args.size() - raw_arg_offset;
const any value = clltk_arg_to_native(type, current, remaining);
Expand Down Expand Up @@ -183,7 +183,7 @@ INLINE static auto fix_types_based_on_format(const std::string_view format,
}
}
if (arg_count != raw_types.size()) [[unlikely]]
CLLTK_DECODER_THROW(FormattingFailed, "invalid format secifier");
CLLTK_DECODER_THROW(FormattingFailed, "invalid format specifier");
return out;
}

Expand Down Expand Up @@ -244,8 +244,8 @@ static INLINE std::string clean_up_str_view(const std::string_view str) {
}

// call snprintf with ffi
std::string formater::printf(const std::string_view format, const std::span<const char> &types_raw,
const std::span<const uint8_t> &args_raw) {
std::string formatter::printf(const std::string_view format, const std::span<const char> &types_raw,
const std::span<const uint8_t> &args_raw) {
if (format.empty()) return "";
if (*format.end() != '\0') CLLTK_DECODER_THROW(FormattingFailed, "missing format termination");
const auto fixed_typ_array = fix_types_based_on_format(format, types_raw);
Expand Down Expand Up @@ -325,8 +325,8 @@ std::string formater::printf(const std::string_view format, const std::span<cons
return msg;
}

std::string formater::dump(const std::string_view format, const std::span<const char> &types_raw,
const std::span<const uint8_t> &args_raw) {
std::string formatter::dump(const std::string_view format, const std::span<const char> &types_raw,
const std::span<const uint8_t> &args_raw) {
if (types_raw.size() != 1 || types_raw[0] != 'x')
CLLTK_DECODER_THROW(InvalidMeta, "wrong meta for drump tracepoint");
const size_t format_size = format.size();
Expand Down
8 changes: 4 additions & 4 deletions decoder_tool/python/clltk_decoder.py
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@ def __init__(self, entry: Ringbuffer_Entry, meta: Stack_Entry):

self.line = get_int(meta.body, self.metaentry_in_metablob + 6, 4)

# get argument informations
# get argument information
self.argument_count = get_int(
meta.body, self.metaentry_in_metablob + 10, 1)
self.argument_type_array = get(meta.body, self.metaentry_in_metablob +
Expand Down Expand Up @@ -474,7 +474,7 @@ def __init__(self, entry: Ringbuffer_Entry, meta: Stack_Entry):

assert self.argument_count == len(formats), \
f"argument types are {self.arg_types} \n"\
+ f"but formates {formats} for type {self.type} and format \"{self.format}\""\
+ f"but formats {formats} for type {self.type} and format \"{self.format}\""\
+ f" at {self.file}:{self.line}"


Expand Down Expand Up @@ -580,7 +580,7 @@ def get_arg(raw: bytes, offset: int, t: int, format: dict):
value = " ".join(f"{x:02X}" for x in value)
offset += string_length
else:
assert 0, "type unkown"
assert 0, "type unknown"

return (offset, value)

Expand Down Expand Up @@ -666,7 +666,7 @@ def __init__(self, raw: bytes, recover_all:bool=False):
continue

if all(character.isspace() for character in entry.formatted):
logging.info("empty trace entry skiped")
logging.info("empty trace entry skipped")
self.entries.append(entry)
pass

Expand Down
4 changes: 2 additions & 2 deletions docs/diagrams/clltk_flow_tracebuffer.drawio
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@
<mxCell id="108" value="add meta section to stack in tracebuffer" style="rounded=0;whiteSpace=wrap;html=1;fillColor=none;" parent="103" vertex="1">
<mxGeometry x="20" y="260" width="120" height="40" as="geometry"/>
</mxCell>
<mxCell id="107" value="return file offset&lt;br&gt;&amp;nbsp;of meta setion" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;fillColor=none;" parent="103" vertex="1">
<mxCell id="107" value="return file offset&lt;br&gt;&amp;nbsp;of meta section" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;fillColor=none;" parent="103" vertex="1">
<mxGeometry x="20" y="320" width="120" height="40" as="geometry"/>
</mxCell>
<mxCell id="132" style="edgeStyle=none;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="103" source="131" target="104" edge="1">
Expand Down Expand Up @@ -247,7 +247,7 @@
</Array>
</mxGeometry>
</mxCell>
<mxCell id="140" value="called by c runtime&lt;br&gt;befor main" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="139" vertex="1" connectable="0">
<mxCell id="140" value="called by c runtime&lt;br&gt;before main" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="139" vertex="1" connectable="0">
<mxGeometry x="-0.641" y="-2" relative="1" as="geometry">
<mxPoint as="offset"/>
</mxGeometry>
Expand Down
4 changes: 2 additions & 2 deletions docs/diagrams/clltk_flow_tracepoint_compilation.drawio
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@
<mxCell id="71" style="edgeStyle=none;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="48" target="69" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="48" value="creat arument type information for runtime" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
<mxCell id="48" value="creat argument type information for runtime" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
<mxGeometry x="120" y="750" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="49" value="" style="endArrow=classic;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="10" target="3" edge="1">
Expand Down Expand Up @@ -178,7 +178,7 @@
<mxCell id="53" value="" style="ellipse;shape=doubleEllipse;html=1;dashed=0;whitespace=wrap;aspect=fixed;" parent="1" vertex="1">
<mxGeometry x="170" y="90" width="20" height="20" as="geometry"/>
</mxCell>
<mxCell id="55" value="artument types" style="rounded=0;whiteSpace=wrap;html=1;rotation=-90;" parent="1" vertex="1">
<mxCell id="55" value="argument types" style="rounded=0;whiteSpace=wrap;html=1;rotation=-90;" parent="1" vertex="1">
<mxGeometry x="260" y="760" width="80" height="40" as="geometry"/>
</mxCell>
<mxCell id="56" value="argument count" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
Expand Down
4 changes: 2 additions & 2 deletions docs/diagrams/clltk_flow_tracepoint_runtime.drawio
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@
<mxCell id="25" value="return" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="9" vertex="1">
<mxGeometry x="30" y="460" width="120" height="40" as="geometry"/>
</mxCell>
<mxCell id="10" value="progamm resums after tracepoint" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
<mxCell id="10" value="program resums after tracepoint" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
<mxGeometry x="100" y="440" width="160" height="40" as="geometry"/>
</mxCell>
<mxCell id="29" value="add to ringbuffer" style="swimlane;whiteSpace=wrap;html=1;startSize=40;" parent="1" vertex="1">
Expand All @@ -97,7 +97,7 @@
<mxCell id="38" style="edgeStyle=none;html=1;" parent="29" source="32" target="33" edge="1">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="32" value="drop old entries unitl enough space for new one" style="rounded=0;whiteSpace=wrap;html=1;" parent="29" vertex="1">
<mxCell id="32" value="drop old entries until enough space for new one" style="rounded=0;whiteSpace=wrap;html=1;" parent="29" vertex="1">
<mxGeometry x="20" y="160" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="39" style="edgeStyle=none;html=1;" parent="29" source="33" target="34" edge="1">
Expand Down
4 changes: 2 additions & 2 deletions docs/diagrams/clltk_simple_overview.drawio
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@
<mxPoint as="offset"/>
</mxGeometry>
</mxCell>
<mxCell id="67" value="live reader deamon" style="rounded=0;whiteSpace=wrap;html=1;labelBackgroundColor=none;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
<mxCell id="67" value="live reader daemon" style="rounded=0;whiteSpace=wrap;html=1;labelBackgroundColor=none;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
<mxGeometry x="377" y="440" width="280" height="40" as="geometry"/>
</mxCell>
<mxCell id="70" value="extracted" style="edgeStyle=none;html=1;fillColor=#f8cecc;strokeColor=#b85450;strokeWidth=3;" parent="1" source="67" target="74" edge="1">
Expand Down Expand Up @@ -201,7 +201,7 @@
<mxCell id="89" value="Traced Application" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;labelBackgroundColor=none;" parent="1" vertex="1">
<mxGeometry x="110" y="10" width="110" height="20" as="geometry"/>
</mxCell>
<mxCell id="90" value="Reader Deamon" style="text;html=1;strokeColor=none;fillColor=none;align=right;verticalAlign=middle;whiteSpace=wrap;rounded=0;labelBackgroundColor=none;" parent="1" vertex="1">
<mxCell id="90" value="Reader Daemon" style="text;html=1;strokeColor=none;fillColor=none;align=right;verticalAlign=middle;whiteSpace=wrap;rounded=0;labelBackgroundColor=none;" parent="1" vertex="1">
<mxGeometry x="590" y="340" width="100" height="20" as="geometry"/>
</mxCell>
<mxCell id="91" value="Decoder Tool" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;labelBackgroundColor=none;" parent="1" vertex="1">
Expand Down
6 changes: 3 additions & 3 deletions docs/file_specification.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ The following table shows the structure of the ringbuffer head, with all offsets
All write access to a tracebuffer is only permitted after obtaining the semaphore. The valid states of the semaphore are therefore 0 and 1. With 0 meaning some process or thread obtained write access. And 1 that now one is currently writing to the tracebuffer. +

For adding a new traceentry, after obtaining write access, maybe some old entries are been dropped until enough space between next_free and last_valid is available to a the new traceentry. Both values are offset relative to the ringbuffer body location inside the file. And for each dropped entry, the dropped counter is incremented, up to a maximum of UINT64_MAX.
If next_free gets bigger than body size, the wrapped counter is incremented, up to a maximum of UINT64_MAX. And next_free is wrapped back to the beginning of the ringbuffer body. The same is true for next_valid, but with out hte incrementation of any counter. +
If next_free gets bigger than body size, the wrapped counter is incremented, up to a maximum of UINT64_MAX. And next_free is wrapped back to the beginning of the ringbuffer body. The same is true for next_valid, but with out the incrementation of any counter. +
An ringbuffer is thereby empty if next_free == last_valid and full if next_free+1==last_valid. Will moving next_free or last_valid, next_free may never overtake last_valid. Instead last_valid must first be moved by dropping old entries and than next_free moved.

The dropping of old entries may only occur on whole traceentires.
Expand All @@ -81,7 +81,7 @@ Each of the entries is constructed like the following table shows:
|====
All offsets are relative to the beginning of an entry. For the oldest entry the beginning is always the last_valid offset. If the combined offset or index for an access is equal or bigger than the ringbuffer body size than the offset/index needs to be wrapped to the beginning of the ringbuffer body. This could be done by calculate the module of index for ringbuffer body size.

The magic is used to securely detect the start ot an ringbuffer entry while dropping old entries or, if a last_valid and next_free are damaged, to find all entries at decoding time.
The magic is used to securely detect the start of an ringbuffer entry while dropping old entries or, if a last_valid and next_free are damaged, to find all entries at decoding time.

The body size enables the extraction of a entry as a single entity.

Expand Down Expand Up @@ -142,7 +142,7 @@ The layout for the traceentry header is shown in the following table:
>h| 22 >| flex | body (second part of traceentry)
|====

The second part, the same as the traceentry header body, contains the user provided argument values and is therefor only present if any argument was provided.
The second part, the same as the traceentry header body, contains the user provided argument values and is therefore only present if any argument was provided.
If present, the layout of this part is defined by the argument information inside the metab information inside the file at offset `in_file_offset`.
All argument values are stored, ordered by there placement inside the tracepoint call, without any gapes.
To decode the argument values, first the argument type has to be retrieved from the argument type array inside the meta entry. Base on this type the value is access.
Expand Down
2 changes: 1 addition & 1 deletion docs/readme.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ A *Tracebuffer* actually refers to two parts:
NOTE: The `clltk` library does not manage buffer memory itself. Instead you will need a data store backend which can allocate buffer memory and provide access to it.

=== Tracepoint creation
In the application code you can create *Tracepoints* whose entries and metadata will be sent into the asociated *Tracebuffer*. This is done by using the line
In the application code you can create *Tracepoints* whose entries and metadata will be sent into the associated *Tracebuffer*. This is done by using the line

`CLLTK_TRACEPOINT(MyFirstTracebuffer, "Hello %s, you are %d years old", name, age);`

Expand Down
4 changes: 2 additions & 2 deletions docs/technical_documentation.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -72,15 +72,15 @@ printf_like_function(receiver, "%d", (long)1); // gcc warning [-Wformat]
----

=== Type safe access to variable arguments
1. get argument count and array with argument-types as enums, trough link:https://stackoverflow.com/a/11920694[c-macro-magic], at compile-time. Use `_Generic` macro to map types to enum and use meta programing for c++.
1. get argument count and array with argument-types as enums, through link:https://stackoverflow.com/a/11920694[c-macro-magic], at compile-time. Use `_Generic` macro to map types to enum and use meta programming for c++.
2. use argument type array to access var_list with with valid type

---

=== Gathering meta data at compile-time
To get all meta data for one tracebuffer, all relevant information are stored in to a custom elf-section. This is possible due to a
link:https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Function-Attributes.html#:~:text=section%20(%22section%2Dname%22)[gcc section attribute].
By defining all meta data strings and data structures as `static const` and appling the section attribute, gcc will gathering all meta data into the same section.
By defining all meta data strings and data structures as `static const` and applying the section attribute, gcc will gathering all meta data into the same section.

[source,C,linenums]
----
Expand Down
Loading