Reconsider adding FFI to the core

[tianxiadys]

What is the problem this feature will solve?

Call foreign functions

What is the feature you are proposing to solve the problem?

the API is provided directly by node, without the help of addons

What alternatives have you considered?

FFI and FFI-NAPI addons have been chronically under-maintained. Both Deno and Bun already support FFI directly. Java has been testing the FFM API since JDK17. I know node's ffi feature was once rejected, but now is it possible to reconsider

[bnoordhuis]

I don't remember there being any strong objections, it just needs someone to do the (considerable) work.

I've thought about taking it on (because I agree it would be useful) but it's too consuming to do uncompensated.

[fmmoret]

Leaving this as a point of reference re: bun: https://news.ycombinator.com/item?id=32120090

[jasnell]

Strong agreement. There's always been interest in having this but it's a massive task. Big +1 to @bnoordhuis's comment "too consuming to do uncompensated". Would love to see it tho.

[mscdex]

Leaving this as a point of reference re: bun: https://news.ycombinator.com/item?id=32120090

tinycc does not support all of the platforms that node does.

[bengl]

Hey folks. Just registering my intent to work on this.

I have some time this week to start tackling this, which is something I've been interested in for a while. A couple years back, I started at attempt at an FFI. It's not quite as user-friendly as ffi-napi, but I think it could serve as a minimal-implementation core library, which could be built upon in userland. After hacking away at it for the past few weeks, I think it's basically ready to be converted into a PR here. I'll spend this week having a go at that.

One thing to note is that currently it uses dyncall instead of libffi. Unfortunately, dyncall doesn't support all the platforms that Node.js supports, so I think it'll have to be converted to use libffi, which does (AFAICT). Of course, switching it to not use Node-API might be the bigger task 😄.

[tianxiadys]

我对于ffi的设计有一些想法，分享如下
1、首先是加载动态库，以及查找函数，这可以通过如下方式实现

import { loadLibrary } from 'node:ffi'

//这是一个async函数
const lib = await loadLibrary('libc.so')

//这不是async函数，因为它是纯内存操作
//第一个参数是函数名，第二个参数是返回值类型，后续是参数类型
const memcpy = lib.findFunction('memcpy', 'pointer', 'pointer', 'pointer', 'usize')

查找函数的同时，定义函数的参数和返回值类型
类型定义参考deno的定义，如下

ffi类型	js类型	备注
i8	number
i16	number
i32	number
i64	bigint
u8	number
u16	number
u32	number
u64	bigint
usize	bigint（64位系统）/number（32位系统）	出于性能考虑，不同系统应该具有不同的类型
f32	number
f64	number
void	-
buffer	ArrayBuffer/TypedBuffer
pointer	bigint（64位系统）/number（32位系统）	出于性能考虑，不同系统应该具有不同的类型

2、回调函数

import { createFunction } from 'node:ffi'

//这不是async函数，因为它是纯内存操作
//第一个参数是函数，第二个参数是返回值类型，后续是参数类型
const callback = createFunction((a1,a2)=>a1+a2, 'i32', 'i32', 'i32')

3、查询buffer的真实指针，因为很多数据结构内部记录了另一个数据结构（或字符串）的指针

import { getBufferPointer } from 'node:ffi'

//struct{
//  char *name1;
//  char *name2;
//};


const buffer1 = new ArrayBuffer(16)
const text1 = (new TextEncoder()).encode('hello')
const text2 = (new TextEncoder()).encode('world')

//这不是async函数，因为它是纯内存操作
const text1addr = getBufferPointer(text1)
const text2addr = getBufferPointer(text2)


const buffer1view = new DataView(buffer1)
buffer1view.setBigUint64(0, text1addr)
buffer1view.setBigUint64(8, text2addr)

4、通过指针创建ArrayBuffer，这样就可以读取本机内存

import { createNativeArrayBuffer } from 'node:ffi'

//创建一个ArrayBuffer，他的起始地址是0x0001000，具有100字节长度
//确保这个地址区间真实有效，是使用者的任务，ffi不应该对此过多干预（因为ffi本质上就是不安全，这是无法避免的）
const buffer1 = createNativeArrayBuffer(0x0001000, 100)

5、查询当前系统的位宽，这对于数据结构的操作很重要，但是这个参数加到process模块更合适

import { bits } from 'node:process'

if(bits === 32) {
} else if (bits === 64) {
}

有了以上5个操作，我们就可以实现ffi所需要的最小集，其余的部分可以交给npm完成

[masx200]

https://deno.land/manual@v1.31.2/runtime/ffi_api#foreign-function-interface

[Pomax]

Reviving this one: https://github.com/node-ffi/node-ffi was created by a now-former Nodejs employee, and the https://github.com/node-ffi-napi/node-ffi-napi variation actually works with the current LTS, so either hiring the ffi/ffi-napi maintainer(s) and putting them on payroll to spend the time and effort required to get it into the Node codebase as the standard library FFI module should be a no-brainer. (Or, of course, paying them to transfer ownership of the repos to you, and then having someone already on payroll do the integration. Either way, the people who already did the work should get compensated)

The fact that Node's missing such an obvious counterpart to Python's ctypes is just plain weird, and only gets weirder with every new LTS release.

The work has already been done, short of updating it to work with the current Node codebase and then folding it in with the appropriate documentation so it's a new section on https://nodejs.org/api/ called "Foreign function interface" pointing to a new https://nodejs.org/api/ffi.html page

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[Pomax]

It would be lovely if someone from the Node team could look at this.

[richardlau]

This is being worked on in #46905

[Pomax]

That's great! It might be a good idea to add "fixes #46233" to that pull request's first comment too though, so that Github cross-links it to this issue (and automatically closes this issue if the PR ends up landing)