Skip to content

Conversation

@robert-w-gries
Copy link
Contributor

Currently, the Heap and LockedHeap uses the default oom handler, which calls intrinsics::abort(). On the x86_64 architecture, this generates a ud2 instruction, generating an invalid opcode exception.

While using this library in a kernel, allocating too much memory will call the default oom handler and generate an INVALID OPCODE exception that points to the first ud2 instruction in alloc::oom (0x1073fc in my case):

0000000000107ef0 <_ZN5alloc9allocator5Alloc3oom17h5474e96b9ba411a5E>:
  107ef0:	55                   	push   %rbp
  107ef1:	48 89 e5             	mov    %rsp,%rbp
  107ef4:	48 83 ec 10          	sub    $0x10,%rsp
  107ef8:	48 89 7d f8          	mov    %rdi,-0x8(%rbp)
 >107efc:	0f 0b                	ud2    
  107efe:	0f 0b                	ud2    
  107f00:	48 89 75 f0          	mov    %rsi,-0x10(%rbp)
  107f04:	66 2e 0f 1f 84 00 00 	nopw   %cs:0x0(%rax,%rax,1)
  107f0b:	00 00 00 
  107f0e:	66 90                	xchg   %ax,%ax

I find the exception message unhelpful. So instead of generating an exception, we can set a custom oom handler by implementing oom for Heap and LockedHeap. Custom oom handlers will be important later when we want to gracefully handle errors. My PR results in the following message when we run out of memory.

PANIC in /path/to/linked-list-allocator/src/lib.rs at line 180:
    Out of memory

Testing

You can test this PR by adding the following to src/lib.rs of blog_os

let mut stack: alloc::Vec<usize> = vec![0; 100_000_000];

@phil-opp
Copy link
Member

Thanks a lot!

@phil-opp phil-opp merged commit b6cadb4 into rust-osdev:master Dec 15, 2017
@phil-opp
Copy link
Member

phil-opp commented Dec 15, 2017

Published as 0.4.3

@robert-w-gries robert-w-gries deleted the custom_oom_handler branch February 1, 2018 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants