Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ext/node): fix os.freemem #21347

Merged
merged 13 commits into from
Nov 30, 2023
Merged

fix(ext/node): fix os.freemem #21347

merged 13 commits into from
Nov 30, 2023

Conversation

kt3k
Copy link
Member

@kt3k kt3k commented Nov 27, 2023

This PR updates os.freemem().

See #21148 (comment) for details.

closes #21148

Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a test failure to existing test because of this change :(

@kt3k
Copy link
Member Author

kt3k commented Nov 28, 2023

Oh, mem_info.available looks 0 (not implemented) on windows and linux..

pub fn mem_info() -> Option<MemInfo> {

@kt3k
Copy link
Member Author

kt3k commented Nov 28, 2023

It looks like there's no reliable way to calculate MemAvailable equivalent from what we already have. I now think we need to add special op for this, and it calls /proc/meminfo in linux.

@littledivy @mmastrac Do you happen to know a good way to calculate/simulate MemAvailable value only using libc?

@kt3k kt3k marked this pull request as draft November 28, 2023 07:57
@kt3k kt3k marked this pull request as ready for review November 29, 2023 11:02
@littledivy
Copy link
Member

littledivy commented Nov 29, 2023

@kt3k We could implement Deno.systemMemoryInfo().available for Linux systems:

  #[cfg(target_os = "linux")]
  {
    let mut info = std::mem::MaybeUninit::uninit();
    // SAFETY: `info` is a valid pointer to a `libc::sysinfo` struct.
    let res = unsafe { libc::sysinfo(info.as_mut_ptr()) };
    if res == 0 {
      // SAFETY: `sysinfo` initializes the struct.
      let info = unsafe { info.assume_init() };
      let mem_unit = info.mem_unit as u64;
      mem_info.swap_total = info.totalswap * mem_unit;
      mem_info.swap_free = info.freeswap * mem_unit;
      mem_info.total = info.totalram * mem_unit;
      mem_info.free = info.freeram * mem_unit;
      mem_info.buffers = info.bufferram * mem_unit;
    }

    let meminfo = std::fs::read_to_string("/proc/meminfo");
    let line = meminfo.lines().find(|l| l.starts_with("MemAvailable:"));
    if let Some(line) = line {
      let mem = line.split_whitespace().nth(1);
      let mem = mem.and_then(|v| v.parse::<i32>().ok());
      mem_info.available = mem.unwrap_or(0);
    }
}

@kt3k
Copy link
Member Author

kt3k commented Nov 29, 2023

@littledivy

We could implement Deno.systemMemoryInfo().available for Linux systems:

Makes sense. Updated

@@ -211,8 +211,20 @@ pub fn mem_info() -> Option<MemInfo> {
mem_info.swap_free = info.freeswap * mem_unit;
mem_info.total = info.totalram * mem_unit;
mem_info.free = info.freeram * mem_unit;
mem_info.available = mem_info.free;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assigned free to available as a fallback for the case when MemAvailable: is unavailable in /proc/meminfo. This is the same as what libuv (uv_get_free_memory) does on linux https://github.com/libuv/libuv/blob/a5c01d4de3695e9d9da34cfd643b5ff0ba582ea7/src/unix/linux.c#L2064

Copy link
Member

@littledivy littledivy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Can you also update runtime/ops/os/README.md? -

 `mem_info`

 | Target family | Syscall                                                                                                                                       
 | Description |
 | ------------- | ------------------------------------------------------------------------ 
 --------------------------------------------------------------------- | ----------- |
-| Linux         | sysinfo                                                                                                                                       
+| Linux         | sysinfo and `/proc/meminfo`                                                                                                                        
 | -           |
 | Windows       | `sysinfoapi::GlobalMemoryStatusEx`                                                                                                            
 | -           |
 | macOS         | <br> <pre> sysctl([CTL_HW, HW_MEMSIZE]); <br> sysctl([CTL_VM, 
 VM_SWAPUSAGE]); <br> host_statistics64(mach_host_self(), HOST_VM_INFO64) 
 </pre> | -           |

@kt3k
Copy link
Member Author

kt3k commented Nov 30, 2023

Sure!

@kt3k kt3k merged commit 3591ba8 into denoland:main Nov 30, 2023
14 checks passed
@kt3k kt3k deleted the fix-node-os-freemem branch November 30, 2023 13:06
@roj1512
Copy link
Contributor

roj1512 commented Nov 30, 2023

Thank you, @kt3k!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

os.freemem() pollyfill is not compatible with Node.js
4 participants