Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GDT docs, add user_data_segment function and WRITABLE flag #78

Merged
merged 5 commits into from Jul 9, 2019

Conversation

@64
Copy link
Contributor

commented Jun 23, 2019

Diff should be fairly self-explanatory.

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jun 23, 2019

Thanks for the pull request! The doc changes look good to me. I'm not sure about the code changes though.

According to the AMD64 manual (https://www.amd.com/system/files/TechDocs/24593.pdf), the content of the ds, es, and ss segment registers are ignored in long mode (section 4.5.3):

In 64-bit mode, the contents of the ES, DS, and SS segment registers are ignored. All fields (base, limit, and attribute) in the hidden portion of the segment registers are ignored.

And according to section 4.8, the read/write flag is ignored as well:

  • For code segments (section 4.8.1):

    The readable (R) and accessed (A) attributes in the type field are also ignored.

  • For data segments (section 4.8.2):

    The expand-down (E), writable (W), and accessed (A) type-field attributes are ignored.

So adding these flags does not make much sense in my opinion. I'm less sure about the method to create a kernel data segment: data segments still exist in long mode (even if the present field is the only non-ignored part), but the segment registers that reference this data segment are ignored anyway. Or am I missing something?

@64

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2019

For AMD systems, you might be right. However the Intel manual doesn't say anything of the sort. It's hard to know when it's talking about protected mode or long mode sometimes:

  • Section 5.4

Segment selectors for code segments that are not readable or for system segments cannot be loaded into data-segment registers (DS, ES, FS, and GS).
Only segment selectors of writable data segments can be loaded into the SS register.

  • Section 3.4.5.1

Data segments can be read-only or read/write segments, depending on the
setting of the write-enable bit.
Stack segments are data segments which must be read/write segments. Loading the SS register with a segment selector for a nonwritable data segment generates a general-protection exception (#GP).

I don't see any overriding text for IA-32e mode.

Additionally, in my OS, if I don't set the READ_WRITE flag for the data segment, QEMU / Bochs immediately raise a GPF when I do a load_ss (but you're right that load_ds et al. don't seem to make a difference).

@64

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2019

As further evidence, without the READ_WRITE flag, bochs tells me upon raising the GPF:

00008846180e[CPU0  ] load_seg_reg(SS): not writable data segment
@64

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2019

I've changed the doc comment to mention load_ss instead of load_ds since ds doesn't seem to matter much.

@Darksecond

This comment has been minimized.

Copy link

commented Jul 4, 2019

@64 I suggest adding a function for user_data_segment along the kernel_data_segment already added here. Since there is also a user_code_segment already.

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jul 5, 2019

Sorry for taking so long to respond!

I looked through the intel manual and found the following relevant sections:

  • Section 5.2.1:

    Code segments continue to exist in 64-bit mode even though, for address calculations, the segment base is treated as zero. Some code-segment (CS) descriptor content (the base address and limit fields) is ignored; the remaining fields function normally (except for the readable bit in the type field).

    (emphasis mine)

    I understand it as if the readable bit is ignored. However, it's much less clear than the AMD manual.

  • Section 5.4.1.1:

    In 64-bit mode, the processor does not perform runtime checking on NULL segment selectors. The processor does not cause a #GP fault when an attempt is made to access memory where the referenced segment register has a NULL segment selector.

    This sounds to me as if loading the DS register with a null selector is allowed.

  • Section 3.4.4

    Because ES, DS, and SS segment registers are not used in 64-bit mode, their fields (base, limit, and attribute) in segment descriptor registers are ignored.

    The "and attribute" part sounds like the write flag is ignored in data segments as well. This seems to be very similar to the wording in the AMD manual.

So I'm not quite user what this means for us. The documentation for AMD64 is relatively clear due to the figures, but the Intel documentation leaves much room for interpretation. The bochs error you posted doesn't make things much clearer, as it might also be an implementation detail of bochs (e.g. by simply reusing the 32-bit check function for 64-bit mode).


As an experiment, I removed all read/write bits from the 64-bit GDT we set up in the bootloader crate, removed the data segment entirely, and set all data segment registers to zero after jumping to long mode. I pushed the result in the zero-segment-registers branch.

The blog_os project still works for me with this modified bootloader, at least in QEMU. (I don't have access to a pre-UEFI machine right now to test it on real hardware). If you want to try it yourself, simply replace the bootloader dependency with the following:

bootloader = { git = "https://github.com/rust-osdev/bootloader.git", branch = "zero-segment-registers", features = ["map_physical_memory"]}
@64

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

That's odd... bochs doesn't even complain when I run your version. Will investigate further.

@64

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

Actually I guess it makes some sense. It was only complaining when loading an invalid but non-null descriptor. Still, I don't think it should be doing that per the Intel or AMD docs. Do you by any chance know how compatability mode works in long mode? I have a feeling that it involves the GDT and might be relevant to this discussion.

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jul 5, 2019

Actually I guess it makes some sense. It was only complaining when loading an invalid but non-null descriptor. Still, I don't think it should be doing that per the Intel or AMD docs.

Yeah, that could be. So segment selector 0 seems to be accepted for all data segments in long mode, and I see no reason for creating a data segment in the GDT given that all of its fields are ignored anyway (at least on AMD64).

I just tried to load ds with a data segment in the bootloader, but always caused general protection faults. Not sure what I'm doing wrong… I tried ƀoth 0x0000_9000_0000_0000 and 0x0000_9200_0000_0000, are there any needed bits missing?

Do you by any chance know how compatability mode works in long mode? I have a feeling that it involves the GDT and might be relevant to this discussion.

AFAIK, compatibility mode allows to run 32 bit applications in long mode by switching to a 32-bit GDT segment.

@64

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

This seems to suggest that null SS is only valid when not in ring 3.
http://bochs.sourceforge.net/cgi-bin/lxr/source/cpu/segment_ctrl_pro.cc#L43

The plot thickens...

@64

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jul 6, 2019

Interesting! So it seems like we definitely need a data segment for userspace, as @Darksecond suggested. Are there any differences between a kernel data segment and a userspace data segment or should we just add a general data_segment method?

Also, is the read/write flag required for a kernel/user data segment? The links you provided indicate yes. Does it have any effect on code segments (i.e. makes them (non-)readable)?

Either way, we should probably add some documentation that explains

  • that data segments are not required in kernel mode (the segment registers can be just set to zero)
  • the effect of the read/write flag on code/data segments and whether the official manuals can be trusted
@64

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2019

whether the official manuals can be trusted

Not sure exactly what you'd like me to say for this.

@phil-opp phil-opp changed the title Update GDT docs, add kernel_data_segment function and READ_WRITE flag Update GDT docs, add user_data_segment function and WRITABLE flag Jul 9, 2019

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jul 9, 2019

Thanks for the update! Looks very good!

Not sure exactly what you'd like me to say for this.

I just thought of how the AMD manual, the Intel manual and the bochs implementation somewhat disagree whether the WRITABLE flag is ignored or not. But I think it's fine to just treat it as non-ignored.

@@ -125,6 +158,8 @@ pub enum Descriptor {
bitflags! {
/// Flags for a GDT descriptor. Not all flags are valid for all descriptor types.
pub struct DescriptorFlags: u64 {
/// For data segments, this flag sets the segment as writable. Ignored for code segments.

This comment has been minimized.

Copy link
@phil-opp

phil-opp Jul 9, 2019

Member

Are we sure that the flag is ignored for code segments? The AMD manual says so, but it also says that the flag is ignored for data segments, which seems to be wrong on some platforms. I think we should at least cite the manual as source.

This comment has been minimized.

Copy link
@64

64 Jul 9, 2019

Author Contributor

I think part of the confusion is that the CPU doesn't check certain fields when the segment is used during a memory reference, but does check those fields when loading in a new value to a segment register. See the wording from the AMD manual, 4.8.1:

In Figure 4-20, gray shading indicates the code-segment descriptor fields that are ignored in 64-bit mode when the descriptor is used during a memory reference.

I'm going to read a bit further and get back to you on this, because I doubt there's any straight up mistakes here - just manual ambiguity and bad writing.

This comment has been minimized.

Copy link
@phil-opp

phil-opp Jul 9, 2019

Member

Ah, that seems to be a reasonable explanation. Thanks!

64 added some commits Jul 9, 2019

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jul 9, 2019

Looks good to me. Let's get this finally merged!

bors r+

bors bot added a commit that referenced this pull request Jul 9, 2019

Merge #78
78: Update GDT docs, add user_data_segment function and WRITABLE flag r=phil-opp a=64

Diff should be fairly self-explanatory.

Co-authored-by: Matt Taylor <mstaveleytaylor@gmail.com>
@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 9, 2019

Build succeeded

  • rust-osdev.x86_64

@bors bors bot merged commit 20e6c4c into rust-osdev:master Jul 9, 2019

6 of 7 checks passed

continuous-integration/travis-ci/push The Travis CI build could not complete due to an error
Details
bors Build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
rust-osdev.x86_64 Build #20190709.3 succeeded
Details
rust-osdev.x86_64 (Job linux) Job linux succeeded
Details
rust-osdev.x86_64 (Job mac) Job mac succeeded
Details
rust-osdev.x86_64 (Job windows) Job windows succeeded
Details

phil-opp added a commit that referenced this pull request Jul 9, 2019

@phil-opp

This comment has been minimized.

Copy link
Member

commented Jul 9, 2019

Released as version 0.7.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.