Discussion: A New Type "Bit" in both C# and VB.NET

[ghost]

Languages such as C# or VB.NET has really done a great job in many areas of types, features……ect. However, sometimes we may be in trouble with "bit-type" designs. (PLEASE IGNORE THE FOLLOWING EXAMPLE'S SYNTAX ERRORS, FOR WE'RE NOT FOCUSING ON THAT).

1) Bit Dealing with："Bit" should be an element type for all numeric types

As far as we see now, for a bit's control. We may use the following symbols such as: &, |, !.
However to apply a deximal or heximal number with another one isn't too sense for us.——Just image if you wanna set a deximal number's certain bit(s) to 0 or 1——Let's see an example:

byte number = 15;     //0000 1111
byte bitAnd=       7;    //00000 0111
byte answer = number & bitAnd;

The given example above is to tell you how to set a certain bit to 0. But that's too hard for us to understand directly. In C# or VB.NET 7.0, we can make it look nicer by the following:

byte number  = 0b0000_1111
byte bitAnd    = 0b0000_0111
byte answer    = number & bitAnd;

However, it should look more beautiful by the following code snippet:

byte number = 15;
number[4] = 0;  //Directly set the 5th bit to 0 (From Left to Right, based on index 0 as the beginning)

Here the number is made up of 8-bit array, and each array is a number whose type is of "Bit". So we can also turn it to this:
bit bitNum = number[5]; //Just like string is made up of char array：We can directly control with the certain bit to some number

2) "Bit" can be also combined together as an array (Different From "BitArray", I'll name it as "BinaryArray" in fact that implements IEnumerable, IConvertible……ect).

We can do this:
bit[] bitNumber = 0b0000_0001; //We can also say a deximal, heximal number……
byte = (byte)bitNumber;
bitNumer = 5;

Suggestions:

1. Any integer from itself to bit[] will be automatically converted.
2. Any binary array from itself to a certain integer type (such as integer, sbyte, byte, char, long, uint……), We ONLY keep the bits from right to left, so this MUST BE AN EXCLIPIT CONVERT:

bit[] bitNumber = 0b1111_1111_1111_1111;
byte bNumber = (byte) bitNumber; //This will ONLY keep "1111_1111".

Of course these are my tough ideas, I know that there must be many questions that I've not covered yet. But this will be a milestone in my mind that .NET will be stronger in bit-controlling in both syntax and functions. And this will be nicer for those in "Bit-Dealing" members such as mobile port transportations, some embedded systems……ect.

[yaakov-h]

We ONLY keep the bits from right to left, so this MUST BE AN EXCLIPIT CONVERT

What if you're on a big-endian system?

[lachbaer]

@yaakov-h
The bit order of multi-byte values is defined in big-endian systems also. Of course you cannot speak of "left-to-right" when talking about the memory representation.

@MaleDong
Have you tried to write your own struct with an idexer and operand overloading that does the same thing? When that type is then heavily used (via NuGet) on several projects, a native .NET and C# integration can be thought of.

[ufcpp]

@MaleDong
https://gist.github.com/ufcpp/64244e996bfad975c2c8e56cad788d03

[ghost]

@lachbaer & @ufcpp ：
Thank you both for suggestions and tips first!
Then what I wanna say is that my tough idea and codes are similar to ufcpp's. However I don't know how to make it like what we see for "int","bool"……. All in all, I don't wanna make it ONLY a wrapper for numberic types but a basic type like int, bool. This may cover CTS, CLS and Language syntax changes. I'm not sure whether it's so easy to implement:)

[ufcpp]

@MaleDong
https://gist.github.com/ufcpp/64244e996bfad975c2c8e56cad788d03#file-16bytes-cs
https://github.com/ufcpp/UfcppSample/tree/master/Demo/2017/BitOperations

It's not so easy but possible.

[ghost]

Hope Microsoft can also join in this project by doing something like "Suger Syntax". Your example is only a simple one, maybe we also have something ignored……@ufcpp

[ufcpp]

@MaleDong
more general version: https://gist.github.com/ufcpp/64244e996bfad975c2c8e56cad788d03#file-genericbits-cs

[fanoI]

Bitfields is the only thing I regret of C/C++ is really more easier to reason of a bitmask as a struct of bit / bits that any field a specific name... yes seeing it as an array of bit would be surely better.
But the CLR has no support for a bit type because well no real machine CLR targets has this... C bitfield struct solve this using packing (they fill the struct with 0x00 until a suitable multiple of 8 is found) in theory this trick could be done in C# too if they wanted.

[ghost]

@fanoI ：Generally speaking. I think C# or any language's development shouldn't ONLY foucs on "suger syntax" to simplify code writing, but improve or enhance real functions as well. C# is based on .NET. Of course I must admit that it's not aimming at hardwares, but with the raise of some .NET-based system (like Cosmos or SharpOS……), and other embedded systems in .NET. Some binary controls should also be supported in C# or VB.NET, which helps .NET on a higher stair.

[fanoI]

Yes if a sort "Bit" type could exist in .Net will be surely better so the example I have posted in the linked issue could become:

struct reg {
         bit bit1;
         bit bit2;
         bit bit3;
         bit bit4;
         bit bit5;
         bit bit6;
         bit bit7;
         bit bit8;
};

This could surely work: it occupies the same space of a byte...

This is another way to represent the same thing instead of a bitfield we have a bit array that could be represented in a byte:

bit[] bitNumber = 0b0000_0001

the problem I see is how is this represented?

bit bitNum = number[5];

what is bitNum? It is in reality a byte with only 1 bit active?

How this type would represent the case in which some fields occupies more than one bit? A bit array would not work as you would need to see the bits contiguous (that is a bit: 2 accepts values from 0 to 3!).
To give you an example how to see any element of a double with your proposal?

https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/IEEE_754_Double_Floating_Point_Format.svg/618px-IEEE_754_Double_Floating_Point_Format.svg.png
(why I cannot link images?)

The proposal of @DerpMcDerp to have a more object oriented solution as the BitField template seems interesting but I'm asking myself how this could be implemented in C#, something similar to Span?
A Span of bit?

I'm a Cosmos developer and as you can imagine when I could at low level you need to do bitmasking in C# with you would prefer to bitfielding having to do this in C# killed a part of myself:

https://github.com/CosmosOS/Cosmos/blob/master/source/Cosmos.HAL/TextScreen.cs#L84

[ghost]

@fanoI ：
What I wanna speak here is that "Bit" is the smallest unit for all kinds of types for numeric types. It can let you get benifits from these points:

1. Direct-Bit controlling by indexing to numeric values.

2. For @gafter 's example (C# Bitfield struct support #465), I admit that they are the same in value with a quite different applications in meanings：A struct-based type can be used for a chain of controls or lamps' lighting or not easily by directly setting the index for certain interfaces of a hardware or an embedded thing(Especially in .NET-based embedded system or other related senarios that maps to a list of things that can be shown through the states of bits). So I have to say that I couldn't agree with gafter 's idea here any more! 👍 But I wanna add one：That's NOT ONLY for BitField type, but keeping all the "Bit" operations as well as "Bit" as the very basic element type.

3. Your example:
   bit bitNum = number[5];
   Yes, this is of no meaning purely on its statement (In fact this means "the 5th position of bitNum is 0 or 1 in bit). However you can image that it will be easy to directly cope with bit from a certain number and do binary setting by indexes. What's more：Some complicated binary algorithms may be completed by "Bit" operators since the beginning of "Bit" unit. And it's far more sensive for us to directly see "Bit"'s constructors of a numeric type, compared with a solo numeric one in deximal type. The same for the floating type as well.

[fanoI]

The ideal thing should be to have the possibility to create types that have particular size in bits than not the canonical one (8, 16, 32, 64).
For example for double example we would need:

1. 1 bit for the sign
2. 11 bits for the exponent
3. 52 bits for the mantissa

The exponent is a integer number between Byte and Short (0 - 4095).

Maybe the two things can be unified a Bit / bit normally has size 1, but with some attribute (or other? Let's try to use ':' as in C) we can indicate its size, being bit a type surely it could exists in a struct so for our double:

struct DoubleView 
{
     bit sign;
     bit exponent: 11;
     bit mantissa: 52;
}

the important thing is that the size of a struct written in this way will 64 bit so I can convert (unsafely) from and to double.

[ghost]

@fanoI ：Yes，yours is right on floating types. What's more：Since "Bit" is the smallest one (You can image the atom is the basic element for all the physical things), we can define more and more customized numeric types in the form of struct to apply for a range of areas more accurately and to help to get involved in mathematics or scientific problems. But operators like +,-,*,/ or overloadings must be implemented by ourselves.

[ufcpp]

I suggest this feature should be realized by Source Generators.

For instance:
https://gist.github.com/ufcpp/f72e7e22c02f88cbaee05ca9367a63cb

[ghost]

@ufcpp ： Yes, to implement the "suger syntax", we should directly put that into CTS, CLS and Source Generators;)

[ufcpp]

revised
https://gist.github.com/ufcpp/f72e7e22c02f88cbaee05ca9367a63cb#file-6-revised-ideal-cs

[fanoI]

Good idea to use an enum as "metadata" it makes the declaration yet more compact!

One interesting example could be to take a perfect valid double (for example 42.42) and making becoming it NaN changing the exponent to all '1' (here new C# binary representation can help to make clear what we are doing) and the mantissa with any value != 0, after we have "poison" it we can print it again as Double and we should get printed "NaN" 👍

[ufcpp]

https://github.com/ufcpp/BitFields

I have implemented "Bit" types.

already implemented:

• Bit1 ～ Bit64 structs
• Analyzer for checking literal convertibility to BitN

planned:

• Bitfileds Generator

[Thaina]

We could do this with just extension indexer #349 #192

We can write namespace for bit. And extend all integer to support indexer for each bit when using that namespace

public static BitExt
{
    public static bool this[this int i,int index]
    {
        get; set; // bit shift operation
    }

    // same for long,short,byte
}

[fanoI]

While in some case could be useful to see a numeric value as "array of bits" this does not solve "elegantly" the case in which you want to see the numeric value as a bit field in which in some cases the fields are more than 1 bit but - at the same time - not one of the predefined numeric values (for example a nibble = 4 bit or 11 bit as in the case of a double exponent part).

I'd say that the class @ufcpp is working should solve this three exigences:

1. Declaring a bitX type as a variable (this seems already working) he is doing a ValueTuple of Bit too!
2. Create a bit field struct (I suggest to have explicit casts to / from numeric values probably this methods should be marked as "unsafe")
3. Create a "View" on numeric value to see it as a bit array

[ufcpp]

The source generator has been implemented: https://github.com/ufcpp/BitFields/blob/master/src/BitFieldsAnalyzer/BitFieldsAnalyzer/BitFieldGeneratorCodeFixProvider.cs
And the preview nupkg has been uploaded to nuget.org: https://www.nuget.org/packages/BitFields/

[fanoI]

Excellent! I'll test it tonight...

Some things I was thinking:

1. The various BitX types are unsigned the name without 'S' is correct in the same vein of Byte, SBitX could exist too? The use will be rare I imagine and sincerely I don't know what valid values a SBit1 could assume (-1 and 0?), a SBit2 could be more useful (-1, 0, +1) one could use it to represent a "trilean" value (false, unknown, true), Messapack "negative fixnum" is another case of Signed Bit (an SBit5)
2. Having at disposal BitX types the more natural way to write a BitField struct would be:

struct DoubleView
{
        Bit11 Fraction;
        Bit52 Fraction;
        Bit1 Sign;
}

the attribute on any field [bitfieldsize] will not be needed anymore... but to get at this perfect result we probably need "code generator" so for now the enum approach + code fix is the best we could have
3. Probably makes sense to have in the generated BitX types conversion operators (implicit or explicit), I've played with your preliminary version yesterday night and it seemed OK to me to do this:

Bit1 Sign = true; // same thing of Bit1 Sign = 1

but it is contrary at how .Net numeric values works there is not explict or implicit cast of a Bool to Int32, you should use the Convert class! Never understood this limitation sincerely...
3. A struct as that DoubleView declared above it has not really a size of 64 bytes right (it is 88 byte if I'm not mistaking)? So I cannot cast it (probably using unsafe code) to a ulong and send it to the wire, probably using [StructLayout(LayoutKind.Explicit, Size=8] would do not what one expects here but probably a cast operator to ulong / the real underlying type (if it does not copies) would suffices for high performance code

What do you think?

[ufcpp]

SBitX
conversion between bool and Bit1

I think these may be useful in some degree, but I don't have enough time. I am satisfied to some extent with the current minimum implementation...

how .Net numeric values works there is not explict or implicit cast of a Bool to Int32

https://gist.github.com/ufcpp/8c6b582e22952e711f1c6ac09e6cff4d

A struct as that DoubleView declared above it has not really a size of 64 bytes

It's 128 bits for the sake of memory alignment. In some case such a struct could be even bigger.

// 24 bytes in sequential layout
// 16 bytes in auto layout
struct X
{
    byte A;
    long B;
    byte C;
}

// 16 bytes
struct Y
{
    byte A;
    byte B;
    long C;
}

I'm not confident but copy cost of the struct may be not ignorable.

[fanoI]

Yes probably is for this that .Net does not permits explicit or implicit casts to / from to avoid to create "strange" bool which underlying value is 2 that in the end it is neither true or false as you have shown in your gist.

Probably the correct thing is to have "safe" methods to do conversion doing unsafe casts would not give the desired results: to have a "real" bitfield CLR support will be needed.
In the generated struct having pack(1) could be beneficial or we will trade struct size for efficiency?

[shaggygi]

Discussion here is not exact, but similar... Proposal: Add a BitManipulation class

[fanoI]

Yes but this idea is more intuitive and simple to use! All the bit manipulations happens in the generated classes code so it is more object oriented too...

I would really need this having to interop with low level in Cosmos I hate to do all that bitmasking it seems "wrong" too in OO language but on the other hand well create little classes by hand to "emulate" bitfields become annoying soon...

[houseofcat]

Would love the inclusion of Bit, for memory saving purposes. It's optimization that computer science guys probably would also really enjoy.

Also is there any talk of supporting bit[] with the larger index sizes of perhaps ulong?

I ran into memory restrictions doing seives (Eratosthenes) and had to create objects with bitshifting around a value types to adjust bits manually.

BitArray and BitVector never worked well enough but did save some space.

[sgf]

request too,
like C/C++ Bit Fields https://docs.microsoft.com/en-us/cpp/c-language/c-bit-fields?view=vs-2019
https://docs.microsoft.com/en-us/cpp/cpp/cpp-bit-fields?view=vs-2019
this is a very useful.
im parse the IP or TCP Header. thats many Bit-Filed in there.
but its hard use C# define that now.
One Way is use shifting operation in Property getter setter.
But the code is hard to read.
but If c# have this feature, then it will become simple.

[psxvoid]

It would be nice to see this feature. However, it seems for me, that it suffices to add the ability to read individual bits of a number using Span (if it's easier to add than Bit). For example, something like this:

uint value = 0xFFFFFFFF;

Span<bool> bits = value.AsBigEndianBitSpan();

and extend BitArray with Span<byte>, Span<ushort>, etc:

BitArray value = new BitArray(14, true);
Span<byte> span = value.AsSpan();
span[0] = (byte) 0x00;

But I'm not completely sure about the performance of such constructs. And it certainly requires more thinking of how to deal with endianness.

[sgf]

It would be nice to see this feature. However, it seems for me, that it suffices to add the ability to read individual bits of a number using Span (if it's easier to add than ). For example, something like this:Bit

uint value = 0xFFFFFFFF;

Span<bool> bits = value.AsBigEndianBitSpan();

and extend with , , etc:BitArray``Span<byte>``Span<ushort>

BitArray value = new BitArray(14, true);
Span<byte> span = value.AsSpan();
span[0] = (byte) 0x00;

But I'm not completely sure about the performance of such constructs. And it certainly requires more thinking of how to deal with endianness.

sizeof(bool) != 1bit

[psxvoid]

Judging by this article size of bool in .NET can vary.

But in assembly/instructions world (and I believe in JITTED code) you can compare individual bits using BT instruction set. Please, see this wiki article. My proposition was not about the current implementation in .NET, it's a proposition which may require redesigning JIT compiler behavior on some platforms as well as introducing/changing some C# keywords and/or methods.

It's possible to achieve the same result by using bitwise operations, unsafe code or span in .NET which are already available (e.g. by packing bools as bits in a byte/uint/etc), but what is proposed here is to introduce some kind of a standard bits-API to the language.

[tannergooding]

That article is describing the size and treatment of bool with regards to the built-in marshalling support (that is, when a bool is passed from managed to native or vice-versa in different contexts).

Types in .NET fundamentally cannot be less than 1-byte. This is also true in C/C++. It would be a massive breaking change in how things work and it's simply not going to happen.

Bit fields are slightly different in that they are a syntax sugar which allows you to more easily pack data into a different type. It still doesn't take up less space than the corresponding backing field, however. That is, uint32_t Field : 1 in C/C++ takes up 1-bit, with then 31-bits of padding. Those 31-bits of padding can be taken up by other sequential bit fields that are also of type uint32_t. However, doing something like sizeof(Field) will report 4 (as in 4 bytes).

---

#465 is a more in depth proposal that talks specifically about bitfields. However, it is also a very niche feature and one that often has multiple tradeoffs in terms of compute complexity.

As the maintainer of https://github.com/dotnet/clangsharp, I am extremely familiar with the complexities of bitfields, and the complexities of getting them "correct" across platforms/architectures (compare https://github.com/dotnet/ClangSharp/blob/main/tests/ClangSharp.PInvokeGenerator.UnitTests/CSharpLatestWindows/StructDeclarationTest.cs#L114-L322 vs https://github.com/dotnet/ClangSharp/blob/main/tests/ClangSharp.PInvokeGenerator.UnitTests/CSharpLatestUnix/StructDeclarationTest.cs#L114-L321 to see how Windows vs Unix differ for the same C struct).

This isn't an "easy" problem to be sure, but it is one with multiple possible solutions and ways to help avoid the current major downsides. A "better" and more sustainable approach might be a source generator which generates the logic for you to ensure it is safe/correct and is something that can be done "now", where-as a language feature needs a champion, general design and agreement from LDM, and then will only be available on new language versions (and only after it bubbles up in priority compared to every other feature people want).

The recent addition of things like unsigned right shift makes the general code you'd have in a property simpler/easier to read as you can have less casting. Coupled with comments or the relevant attributes, it becomes something that is much more sustainable and possible for the relatively few devs who need it.

[TahirAhmadov]

For scenario 1, the #5497 extensions will make it possible with an extension indexed this property.
For scenario 2, as others have noted, you can use BitArray class, or even just use the aforementioned extension property.