Update GDT docs, add user_data_segment function and WRITABLE flag

[64]

Diff should be fairly self-explanatory.

[phil-opp]

Thanks for the pull request! The doc changes look good to me. I'm not sure about the code changes though.

According to the AMD64 manual (https://www.amd.com/system/files/TechDocs/24593.pdf), the content of the ds, es, and ss segment registers are ignored in long mode (section 4.5.3):

In 64-bit mode, the contents of the ES, DS, and SS segment registers are ignored. All fields (base, limit, and attribute) in the hidden portion of the segment registers are ignored.

And according to section 4.8, the read/write flag is ignored as well:

• For code segments (section 4.8.1):

  The readable (R) and accessed (A) attributes in the type field are also ignored.

• For data segments (section 4.8.2):

  The expand-down (E), writable (W), and accessed (A) type-field attributes are ignored.

So adding these flags does not make much sense in my opinion. I'm less sure about the method to create a kernel data segment: data segments still exist in long mode (even if the present field is the only non-ignored part), but the segment registers that reference this data segment are ignored anyway. Or am I missing something?

[64]

For AMD systems, you might be right. However the Intel manual doesn't say anything of the sort. It's hard to know when it's talking about protected mode or long mode sometimes:

• Section 5.4

Segment selectors for code segments that are not readable or for system segments cannot be loaded into data-segment registers (DS, ES, FS, and GS).
Only segment selectors of writable data segments can be loaded into the SS register.

• Section 3.4.5.1

Data segments can be read-only or read/write segments, depending on the
setting of the write-enable bit.
Stack segments are data segments which must be read/write segments. Loading the SS register with a segment selector for a nonwritable data segment generates a general-protection exception (#GP).

I don't see any overriding text for IA-32e mode.

Additionally, in my OS, if I don't set the READ_WRITE flag for the data segment, QEMU / Bochs immediately raise a GPF when I do a load_ss (but you're right that load_ds et al. don't seem to make a difference).

[64]

As further evidence, without the READ_WRITE flag, bochs tells me upon raising the GPF:

00008846180e[CPU0  ] load_seg_reg(SS): not writable data segment

[64]

I've changed the doc comment to mention load_ss instead of load_ds since ds doesn't seem to matter much.

[Darksecond]

@64 I suggest adding a function for user_data_segment along the kernel_data_segment already added here. Since there is also a user_code_segment already.

[phil-opp]

Sorry for taking so long to respond!

I looked through the intel manual and found the following relevant sections:

• Section 5.2.1:

  Code segments continue to exist in 64-bit mode even though, for address calculations, the segment base is treated as zero. Some code-segment (CS) descriptor content (the base address and limit fields) is ignored; the remaining fields function normally (except for the readable bit in the type field).

  (emphasis mine)

  I understand it as if the readable bit is ignored. However, it's much less clear than the AMD manual.

• Section 5.4.1.1:

  In 64-bit mode, the processor does not perform runtime checking on NULL segment selectors. The processor does not cause a #GP fault when an attempt is made to access memory where the referenced segment register has a NULL segment selector.

  This sounds to me as if loading the DS register with a null selector is allowed.

• Section 3.4.4

  Because ES, DS, and SS segment registers are not used in 64-bit mode, their fields (base, limit, and attribute) in segment descriptor registers are ignored.

  The "and attribute" part sounds like the write flag is ignored in data segments as well. This seems to be very similar to the wording in the AMD manual.

So I'm not quite user what this means for us. The documentation for AMD64 is relatively clear due to the figures, but the Intel documentation leaves much room for interpretation. The bochs error you posted doesn't make things much clearer, as it might also be an implementation detail of bochs (e.g. by simply reusing the 32-bit check function for 64-bit mode).

---

As an experiment, I removed all read/write bits from the 64-bit GDT we set up in the bootloader crate, removed the data segment entirely, and set all data segment registers to zero after jumping to long mode. I pushed the result in the zero-segment-registers branch.

The blog_os project still works for me with this modified bootloader, at least in QEMU. (I don't have access to a pre-UEFI machine right now to test it on real hardware). If you want to try it yourself, simply replace the bootloader dependency with the following:

bootloader = { git = "https://github.com/rust-osdev/bootloader.git", branch = "zero-segment-registers", features = ["map_physical_memory"]}

[64]

That's odd... bochs doesn't even complain when I run your version. Will investigate further.

[64]

Actually I guess it makes some sense. It was only complaining when loading an invalid but non-null descriptor. Still, I don't think it should be doing that per the Intel or AMD docs. Do you by any chance know how compatability mode works in long mode? I have a feeling that it involves the GDT and might be relevant to this discussion.

[phil-opp]

Actually I guess it makes some sense. It was only complaining when loading an invalid but non-null descriptor. Still, I don't think it should be doing that per the Intel or AMD docs.

Yeah, that could be. So segment selector 0 seems to be accepted for all data segments in long mode, and I see no reason for creating a data segment in the GDT given that all of its fields are ignored anyway (at least on AMD64).

I just tried to load ds with a data segment in the bootloader, but always caused general protection faults. Not sure what I'm doing wrong… I tried ƀoth 0x0000_9000_0000_0000 and 0x0000_9200_0000_0000, are there any needed bits missing?

Do you by any chance know how compatability mode works in long mode? I have a feeling that it involves the GDT and might be relevant to this discussion.

AFAIK, compatibility mode allows to run 32 bit applications in long mode by switching to a 32-bit GDT segment.

[64]

This seems to suggest that null SS is only valid when not in ring 3.
http://bochs.sourceforge.net/cgi-bin/lxr/source/cpu/segment_ctrl_pro.cc#L43

The plot thickens...

[64]

Same is indicated here: https://www.felixcloutier.com/x86/mov#64-bit-mode-exceptions

[phil-opp]

Interesting! So it seems like we definitely need a data segment for userspace, as @Darksecond suggested. Are there any differences between a kernel data segment and a userspace data segment or should we just add a general data_segment method?

Also, is the read/write flag required for a kernel/user data segment? The links you provided indicate yes. Does it have any effect on code segments (i.e. makes them (non-)readable)?

Either way, we should probably add some documentation that explains

• that data segments are not required in kernel mode (the segment registers can be just set to zero)
• the effect of the read/write flag on code/data segments and whether the official manuals can be trusted

[64]

whether the official manuals can be trusted

Not sure exactly what you'd like me to say for this.

[phil-opp]

Thanks for the update! Looks very good!

Not sure exactly what you'd like me to say for this.

I just thought of how the AMD manual, the Intel manual and the bochs implementation somewhat disagree whether the WRITABLE flag is ignored or not. But I think it's fine to just treat it as non-ignored.

[phil-opp]

Are we sure that the flag is ignored for code segments? The AMD manual says so, but it also says that the flag is ignored for data segments, which seems to be wrong on some platforms. I think we should at least cite the manual as source.

[64]

I think part of the confusion is that the CPU doesn't check certain fields when the segment is used during a memory reference, but does check those fields when loading in a new value to a segment register. See the wording from the AMD manual, 4.8.1:

In Figure 4-20, gray shading indicates the code-segment descriptor fields that are ignored in 64-bit mode when the descriptor is used during a memory reference.

I'm going to read a bit further and get back to you on this, because I doubt there's any straight up mistakes here - just manual ambiguity and bad writing.

[phil-opp]

Ah, that seems to be a reasonable explanation. Thanks!

[phil-opp]

Looks good to me. Let's get this finally merged!

bors r+

[bors]

Build succeeded

• rust-osdev.x86_64

[phil-opp]

Released as version 0.7.2