Skip to content

Conversation

@kcgen
Copy link
Member

@kcgen kcgen commented Jun 5, 2022

The bit_view class is a wrapper around a data value. It provides a view into a subset of its bits allowing them to be read, written, assigned, flipped, cleared, and tested, without the need to for the usual twiddling operations.

Constructing a bit_view is similar to C bitfields, however unlike C bitfields, bit_views are free from undefined behavior and have been proven using GCC's and Clang's undefined behavior sanitizers.

This gives us the benefit of bitfields without their specification downsides:

  • they're succinct and clear to use
  • they're just as fast as bitwise operators (maybe more so, being constexpr)
  • each bit_view is self-documenting by specifying its bit index, number of bits, and logical name
  • the view's range-of-bits are locked-in using explicit template values, so there's no risk of them being "slippery" or changing during runtime.
  • the view's internal data member is always safely sized to accommodate the view's position and size. static-asserts ensure that it isn't under or over-specified.

Before and after comparison

Assume we have a 16-bit audio control register that holds the card's:

  • left volume at bit 1 using 6 bits
  • right volume at bit 7, also 6 bits
  • speaker on/off state at bit 13 with just one bit

The existing DOSBox code would ferret out this information using manual bit twiddling:

if (data & 0x2000)
	enable_speaker();
else
	disable_speaker();

const auto left_percent = ((data & 0x7e) >> 1) / 63.0;
const auto right_percent = ((data & 0x1f80) >> 7) / 63.0;
set_volume(left_percent, right_percent);

The bit_view makes the register's logical elements self-documenting:

union AudioReg {
   uint16_t data = 0;
   bit_view<1, 6> left_volume;
   bit_view<7, 6> right_volume;
   bit_view<13, 1> speaker_on;
};

The views can be used directly, making for much clearer (and safer) code:

AudioReg reg = {data};

if (reg.speaker_on)
	enable_speaker();
else
	disable_speaker();

const auto left_percent = reg.left_volume / 63.0;
const auto right_percent = reg.right_volume / 63.0;
set_volume(left_percent, right_percent);

@kcgen kcgen changed the title Kc/bit view 1 Add bit_view support class Jun 5, 2022
@kcgen
Copy link
Member Author

kcgen commented Jun 5, 2022

Todo:

  • assert sums to catch over/under flow
  • assert right-hand side type is_integral and not floating point
  • add unit tests that traffic bit_view objects for everyday use in function arguments, aliased as references, pointed to by pointers, new-able, smart-pointer-able, and used in fixed arrays.
  • mention a couple other resources regarding "safe bitfields for c++", and their tradeoffs.

Edit: done.

@kcgen
Copy link
Member Author

kcgen commented Jun 5, 2022

Similar concepts have been tried. For example, this approach uses lots of preprocessor macros to build up a safe union and class members:

BEGIN_BITFIELD_TYPE(Status, uint32_t) 
    ADD_BITFIELD_MEMBER(readers, 0, 10)  
    ADD_BITFIELD_MEMBER(waitToRead, 10, 10)
    ADD_BITFIELD_MEMBER(writers, 20, 10)
END_BITFIELD_TYPE()

To me, this is an eye-sore that says "we had to hack the language with macros". Even if it gets the job done, it feels way too much like a custom contraption.

There's also this approach, which adds some C++ meta programming o get rid of the macros:

union manually_done_bitfield {
    using element_t = long;
    using base_t = base<long, 4, 3, 5>;
    BitFieldMember<element_t, base_t::displacement(0), 4> fourBits;
    BitFieldMember<element_t, base_t::displacement(1), 3> threeBits;
    BitFieldMember<element_t, base_t::displacement(2), 5> fiveBits;
};

However, these displacement values feel more like a second layer of meta-programming (given we're already using templates), and their values represent the position in the base<long, 4, 3, 5> as opposed to the logical bit positions of the view. So it's visually carrying a lot of confusing boiler-plate distracting from it's actual purpose.

@kcgen kcgen requested review from johnnovak, kklobe and shermp June 5, 2022 22:16
@kcgen kcgen self-assigned this Jun 5, 2022
@kcgen kcgen added the plumbing Issues related to low-level support functions and classes label Jun 5, 2022
@kcgen
Copy link
Member Author

kcgen commented Jun 5, 2022

UBSAN tests

Build

meson setup -Dbuildtype=debug -Db_sanitize=undefined build/clang-undefined
meson compile -C build/clang-undefined

Results

[==========] Running 20 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 20 tests from bit_view
[ RUN      ] bit_view.assign_from_literal
[       OK ] bit_view.assign_from_literal (0 ms)
[ RUN      ] bit_view.assign_to_data
[       OK ] bit_view.assign_to_data (0 ms)
[ RUN      ] bit_view.assign_from_parts
[       OK ] bit_view.assign_from_parts (0 ms)
[ RUN      ] bit_view.flip
[       OK ] bit_view.flip (0 ms)
[ RUN      ] bit_view.increment
[       OK ] bit_view.increment (0 ms)
[ RUN      ] bit_view.decrement
[       OK ] bit_view.decrement (0 ms)
[ RUN      ] bit_view.clear
[       OK ] bit_view.clear (0 ms)
[ RUN      ] bit_view.boolean_checks
[       OK ] bit_view.boolean_checks (0 ms)
[ RUN      ] bit_view.equality
[       OK ] bit_view.equality (0 ms)
[ RUN      ] bit_view.size_safety
[       OK ] bit_view.size_safety (0 ms)
[ RUN      ] bit_view.illegal_view
[       OK ] bit_view.illegal_view (0 ms)
[ RUN      ] bit_view.writable_via_reference
[       OK ] bit_view.writable_via_reference (0 ms)
[ RUN      ] bit_view.readable_via_reference
[       OK ] bit_view.readable_via_reference (0 ms)
[ RUN      ] bit_view.writable_via_pointer
[       OK ] bit_view.writable_via_pointer (0 ms)
[ RUN      ] bit_view.readable_via_pointer
[       OK ] bit_view.readable_via_pointer (0 ms)
[ RUN      ] bit_view.create_with_new
[       OK ] bit_view.create_with_new (0 ms)
[ RUN      ] bit_view.create_with_make_unique
[       OK ] bit_view.create_with_make_unique (0 ms)
[ RUN      ] bit_view.use_in_array
[       OK ] bit_view.use_in_array (0 ms)
[ RUN      ] bit_view.bare_initialization
[       OK ] bit_view.bare_initialization (0 ms)
[ RUN      ] bit_view.bare_constructor
[       OK ] bit_view.bare_constructor (0 ms)
[----------] 20 tests from bit_view (0 ms total)
[  PASSED  ] 20 tests.

@shermp
Copy link
Collaborator

shermp commented Jun 5, 2022

Looking nice.

From an ergonomics point of view, is there a way of creating the bit_view without explicitly setting the type? I know C++ can do some magic regarding templates and type inference, but unsure of when/how it can do so.

@kcgen
Copy link
Member Author

kcgen commented Jun 6, 2022

@shermp , yeah, the position and width of the bit_view can give you the largest type needed for that specific bit_view.

So that means, say, for a 16bit register, the first handful of bit_views would use uint8_t and the latter handful would use a uint16_t.

The mixed-sizing made me nervous enough to try to avoid it, and make sure all the bit_views are aliasing /holding the same data-type. (But that's perhaps an unecessary concern!)

Like you're suggesting, ideally there's some introspection that can tell us the size of the parent structure, and figure out that we're sitting inside a union 16-bits in size, then everything could use that. Something like that would get it to the 100% level.

I also like that this approach would prevent use of the wrong type (although the static_asserts guarantee the bit_view won't go out of range).

It's also DRYer: you'd drop in the single data type, and then everything else uses that without any repetition.

@johnnovak
Copy link
Member

The mixed-sizing made me nervous enough to try to avoid it, and make sure all the bit_views are aliasing /holding the same data-type. (But that's perhaps an unecessary concern!)

We're using the bit-views for low-level stuff only, and in that context the register/data widths are fixed and known in advance in 99.99% of the cases. So I very much prefer the current approach of having to specify the width explicitly for each register or data type we're dealing with; that's much more descriptive, reflects actual reality, and is a lot less error prone.

@kcgen
Copy link
Member Author

kcgen commented Jun 6, 2022

If we can lock'em in:

union MyReg {
    uint8_t data; // this is the type we want to learn
    bit_view<...> // about and lock-in for all the peer bit_views
};

I've been scanning the web but can't find how to get the data type from the parent union/class (without macro-rewrite hacks).

Might have to settle for manually repeating it?

@johnnovak
Copy link
Member

johnnovak commented Jun 6, 2022

Nice addition @kcgen, this will make all that low-level bit-twiddling less error-prone and more readable! 👍🏻

Might have to settle for manually repeating it?

If there was a simple way to avoid the manual repetition that would be preferable, of course, but I don't think it's worth complicating the code with lots of template/macro magic just to make that happen somehow.

"bit_view" is a better description for how the class
operates given it's a "view" of the bits held in the the
underlying storage.

This is meant to be similar to "string_view", which is
also a similar "functional wrapper" around the underlying
char* data.

---

The author of the BitField class also describes it like
this:

Q. How do you ensure all the bitfields are usable? Because
   inherently, union will only use the max size member.
   You will not be able to see unique data for other
   members, if one is assigned a value, it will be the
   value for all other members.

A. Remember that this isn't actually using bit-fields, but
   is emulating the effect of them. The "bits" are just a
   "view" into the underlying storage. Changing one is
   intended to be reflected in both.

---

Other changes:

 - Use lower-case types and name similar to 'string_view',
   given there's nothing DOSBox-specific about it. This
   makes it feel more like "standard plumbing", and
   indeed, anyone can take this class into other projects

 - Add compile-time asserts to catch index and width
   issues

 - Make all functionality constexpr (compile-time
   evaluation)

 - Add unit tests for all functions

 - Fix decrement bug caught in unit tests

 - Fix comparison bugs caught in unit tests

 - Drop single-bit template specialization. A bit_view
   with a size of 1 is already known and optimized at
   compile-time now using "constexpr"

 - Remove the bool cast operators in favour of exact
   value operator, which are clearer to understand

 - Add member functions from bitops:
   - clear(): sets  bits to zero
   - flip(): flip bits
   - none(): check if all bits are unset
   - any(): check if at least one bit is set
   - all(): check if all bits are set

 - Add comments

 - License as GPL v2+ (from public domain), given all
   lines have changed. Credit and thanks given to Evan
   Teran (thanks!)
@kcgen
Copy link
Member Author

kcgen commented Jun 6, 2022

It seems GitHub has pushed an update to the macOS runners, however it's trapped my runners in a crash loop:

Current runner version: '2.291.1'
2022-06-06 07:28:10Z: Listening for Jobs

Runner update in progress, do not shutdown runner.
Downloading 2.292.0 runner
Runner update process finished.

An error occurred: Runner package 'https://github.com/actions/runner/releases/download/v2.292.0/actions-runner-osx-x64-2.292.0.tar.gz' failed after 3 download attempts
Runner listener exit with retryable error, re-launch runner in 5 seconds.

Restarting runner...


� Connected to GitHub

Current runner version: '2.291.1'
2022-06-06 07:28:24Z: Listening for Jobs

Runner update in progress, do not shutdown runner.
Downloading 2.292.0 runner
Runner update process finished.

An error occurred: Runner package 'https://github.com/actions/runner/releases/download/v2.292.0/actions-runner-osx-x64-2.292.0.tar.gz' failed after 3 download attempts

Runner listener exit with retryable error, re-launch runner in 5 seconds.

...

Will try to get it sorted tomorrow.

@kcgen
Copy link
Member Author

kcgen commented Jun 6, 2022

Fleet back online 🚢

2022-06-06_06-56

@kcgen
Copy link
Member Author

kcgen commented Jun 6, 2022

@kklobe , @Wengier , @shermp: GitHub recently released native ARM64 builds of their runner, which the previous x64-on-rosetta2 runners couldn't auto-upgrade to. So I had to start with a clean slate.

If you still have ARM-based runners using the x86-binaries, drop me a note and I'll pass the new setup steps (which include the project token).

@kcgen
Copy link
Member Author

kcgen commented Jun 7, 2022

Thanks for the review @shermp and @johnnovak.

From an ergonomics point of view, is there a way of creating the bit_view without explicitly setting the type?

@shermp, your suggestion is now addressed safely and with compile-time asserts; so it's impossible to read or write before or after the view's data value.

Unit tests are passing on all platforms, along with passing all the *SAN tests locally.

Merging.

@kcgen kcgen merged commit 57ee00e into main Jun 7, 2022
@kcgen kcgen deleted the kc/bit_view-1 branch June 7, 2022 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

plumbing Issues related to low-level support functions and classes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants