PR for x86_64 port merging into loaderarm #430

betahxy · 2021-10-31T10:00:31Z

Summary of this Pull Request (PR)

Add description here.

Intent for your PR

Choose one (Mandatory):

This PR is for a code-review and is intended to get feedback, but not to be pulled yet.
This PR is mature, and ready to be integrated into the repo.

Reviewers (Mandatory):

(Specify @<github.com username(s)> of the reviewers. Ex: @user1, @user2)

Code Quality

As part of this pull request, I've considered the following:

Style:

Comments adhere to the Style Guide (SG)
Spacing adhere's to the SG
Naming adhere's to the SG
All other aspects of the SG are adhered to, or exceptions are justified in this pull request
[ ] I have run the auto formatter on my code before submitting this PR (see doc/auto_formatter.md for instructions)

Code Craftsmanship:

I've made an attempt to remove all redundant code
I've considered ways in which my changes might impact existing code, and cleaned it up
I've formatted the code in an effort to make it easier to read (proper error handling, function use, etc...)
I've commented appropriately where code is tricky
I agree that there is no "throw-away" code, and that code in this PR is of high quality

Testing

I've tested the code using the following test programs (provide list here):

micro_booter
unit_pingpong
unit_schedtests
...(add others here)

- Finish setup tss, gdt, idt

- kernel now begins at 0xffff800000000000

-x86_64 higher VA needs a different mask to calculate PGD's offset -do_div macro in printk doesn't work on x86_64, provide a different implementation

-chal_init works

- fix chal cpu init settings on x86_64

- qemu does not check validity of tss structure in memory, but real machine does check, fix the wrong tss setting in gdt_init - make it pass cap_captbl related assertion in x64

- previous global retype table has a walk[-1] entry when page order is 20, changed it to 22 to walk the correct global page entry

1. setup caps for booter component 2. split CAS into 32-bit and 64-bit version 3. rearrange capability index for booter component

1. keep using 32-bit CAS by default, since other architectures are using the same cos_cas() interface 2. support cos_cas_64(), but it needs to be disscussed before we use it

to kernel space

- .boottext section needs to be aligned to generate correct bootable image - the stack in loader.S needs to be set to .data section, the original .comm will generate a .bss section that could have random address after compiling

- fix the linker script so it can correctly compile a bootable image - add -nmagic flag to $(LD) in the Makefile so $(LD) does not add additional blank information to the bootable image

- enable smp booting into long mode and jump into kmain

cos

gparmer

I'm not done (124/172 files), but I have to run to dinner. Want to make these visible in case I can't come back to it tonight.

Summary: this is amazing. A ton of work.

.gitignore

cos

src/components/cidl/calculate_dependencies.py

src/platform/i386/chal_cpu.h

src/platform/i386/chal_pgtbl.c

src/platform/i386/lapic.c

gparmer

This is fantastic, and I have very very little feedback on things to change.

A higher-order bit: I'm concerned about how much this breaks arm. @msdx321 might have some input.

That said, I think that I'm suggesting very small changes. We likely want more of the tests working if we're all going to move to this branch, which we all want to!!!

src/platform/x86_64/boot_comp.c

tools/init_env.sh

1. set pgtbl/comp cap size to 4 in composer 2. fix memory init bug in __cos_meminfo_populate 3. fix stack pointer alignment bug in cos_asm_stub_indirect stub

- add build_iso.sh and run.sh to help cos finish build and run - soft linked boot_comp.c, share it with both i386 and x86_64 - other minor fixes

gparmer · 2021-11-20T13:34:07Z

If you think that there is discussion left, or you want me to see a comment, you have to "unresolve the conversation". Currently there are no unresolved conversations, so I don't see any comments.

betahxy · 2021-11-20T13:55:40Z

@gparmer
Just added a word size definition, to resolve some places that really need this.

- fix warnings when compiling i386 port

merge loaerarm

gparmer · 2021-11-20T17:03:40Z

Should we just be using __WORDSIZE in userlevel? https://github.com/runtimejs/musl-libc/blob/master/arch/x86_64/bits/user.h#L2

- fix debug_run function

betahxy added 30 commits February 11, 2021 02:53

Initial x86_64 setup for loaderarm branch

088860a

- Finish setup tss, gdt, idt

Map kernel into x64 higher memory zone

3100db0

- kernel now begins at 0xffff800000000000

Add x64 higher VA mask and fix printk bug

ec668cb

-x86_64 higher VA needs a different mask to calculate PGD's offset -do_div macro in printk doesn't work on x86_64, provide a different implementation

Init kernal memory setup

b9ba925

chal_init OK

03bbbf2

-chal_init works

chal_cpu init

7c938ff

- fix chal cpu init settings on x86_64

set to use kernel's own page directory

d5091e0

kernel init paging bug fix

04f7e13

Fix tss setting error and fix to pass cap_captbl assertion

d0f13a7

- qemu does not check validity of tss structure in memory, but real machine does check, fix the wrong tss setting in gdt_init - make it pass cap_captbl related assertion in x64

gdt init bug fix

b2e0dbb

retype table bug fix

0fb29cd

- previous global retype table has a walk[-1] entry when page order is 20, changed it to 22 to walk the correct global page entry

fix gdb not working correctly problem

bedac6f

Acpi init passed initilization temporarily

7011828

fix dump registers problem and some acpi issues

61c1bf6

Initialize the booter component

5c173b2

1. setup caps for booter component 2. split CAS into 32-bit and 64-bit version 3. rearrange capability index for booter component

update CAS operation

e7094f1

1. keep using 32-bit CAS by default, since other architectures are using the same cos_cas() interface 2. support cos_cas_64(), but it needs to be disscussed before we use it

fix compiling 32-bit components problems

a580cfe

elf64 support

6db45cb

update elf_loader API to make it compatible for both elf32 and elf64

bd1082c

support 4-level page table for x86_64

7bc990e

Merge chal code

39623b9

boot elf process passed

47e71de

fix 4 level page table bugs

2e51873

Finish boot_comp init

1bd0036

jmp to user space

155971d

fix interrupt routine bug and tss invalid bug when jmp from user space

23983a8

to kernel space

remove warnings

b078569

remove warnings in i386 arch kernel

7f5865b

fix compiling errors

47e4c15

fix compiling errors

deea822

betahxy added 13 commits October 23, 2021 05:25

fix i386 port pingpong

90107bc

fix pingpong test in x86_64 port

484da5c

improve cos script

d59f58c

improve build scripts

c0aba92

fix i386 bug

cd4111c

- .boottext section needs to be aligned to generate correct bootable image - the stack in loader.S needs to be set to .data section, the original .comm will generate a .bss section that could have random address after compiling

remove unnecessary file

f542dfd

fix x86_64 compiling problems

3ac4f70

- fix the linker script so it can correctly compile a bootable image - add -nmagic flag to $(LD) in the Makefile so $(LD) does not add additional blank information to the bootable image

improve init_env script

7e9aaea

Merge branch 'gwsystems:loaderarm' into loaderarm

54eaeaf

remove unnecessary files

d2f6658

multi-core bootup support in cos script

c650c72

fix smp booting in loader.S

267e596

- enable smp booting into long mode and jump into kmain

fix code format of loader.S

cfc219f

WenyuanShao reviewed Nov 9, 2021

View reviewed changes

cos Show resolved Hide resolved

gparmer reviewed Nov 15, 2021

View reviewed changes

gparmer requested changes Nov 15, 2021

View reviewed changes

src/platform/x86_64/boot_comp.c Outdated Show resolved Hide resolved

tools/init_env.sh Outdated Show resolved Hide resolved

betahxy added 4 commits November 18, 2021 07:12

fix sched.toml tests bugs for x86_64

c1a9ea2

1. set pgtbl/comp cap size to 4 in composer 2. fix memory init bug in __cos_meminfo_populate 3. fix stack pointer alignment bug in cos_asm_stub_indirect stub

fix issues according to PR's suggestions

7bd548c

line up code

8575d0a

fix some issues

ce8bec3

- add build_iso.sh and run.sh to help cos finish build and run - soft linked boot_comp.c, share it with both i386 and x86_64 - other minor fixes

add __WORD_SIZE_64__ and __WORD_SIZE_32__

eac791b

betahxy mentioned this pull request Nov 20, 2021

TODO: Need to improve s_stub.h structure to split different architectures. #431

Open

betahxy added 3 commits November 20, 2021 09:51

fix compiler warnings

5e68126

- fix warnings when compiling i386 port

Merge branch 'loaderarm' into debug_user_level

0342699

merge loaerarm

change captbl and pgtbl size to be 4 on both i386 and x86_64

2b5fb5e

small fix of run.sh

a846806

- fix debug_run function

betahxy closed this Apr 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PR for x86_64 port merging into loaderarm #430

PR for x86_64 port merging into loaderarm #430

betahxy commented Oct 31, 2021

gparmer left a comment

gparmer left a comment

gparmer commented Nov 20, 2021

betahxy commented Nov 20, 2021

gparmer commented Nov 20, 2021

PR for x86_64 port merging into loaderarm #430

PR for x86_64 port merging into loaderarm #430

Conversation

betahxy commented Oct 31, 2021

Summary of this Pull Request (PR)

Intent for your PR

Reviewers (Mandatory):

Code Quality

Testing

gparmer left a comment

Choose a reason for hiding this comment

gparmer left a comment

Choose a reason for hiding this comment

gparmer commented Nov 20, 2021

betahxy commented Nov 20, 2021

gparmer commented Nov 20, 2021