sim/camera: add AVFoundation backend and multi-cam support by PeterBee97 · Pull Request #18745 · apache/nuttx

PeterBee97 · 2026-04-15T14:00:04Z

Summary

Add AVFoundation backend and multi-cam support, enabling macOS users as well as Linux users to use multiple cameras on Simulator.

Impact

Impact on users: Backward compatible
Affected platforms:
- SIM on macOS
- SIM on Linux
Scope:
- Enhances the SIM camera framework for multi-instance support
- Adds a new macOS AVFoundation camera backend
- Fixes mkallsyms host-tool behavior for Objective-C symbols on macOS
Non-affected platforms:
- No impact on other boards or non-SIM targets
- Existing Linux SIM camera behavior remains supported

Testing

Tested on:

Host OS: macOS Tahoe on Apple Silicon, Ubuntu x86_64
Target configurations: sim:nxcamera

acassis

@PeterBee97 WOW! Amazing! Please update the nxcamera Documentation/ to include this new parameter: https://nuttx.apache.org/docs/latest/applications/system/nxcamera/index.html

acassis · 2026-04-15T14:29:43Z

@PeterBee97 please include this picture in the Documentation page, it is funny :-)

PeterBee97 · 2026-04-16T04:23:17Z

@PeterBee97 WOW! Amazing! Please update the nxcamera Documentation/ to include this new parameter: https://nuttx.apache.org/docs/latest/applications/system/nxcamera/index.html

As u wish :)

acassis

@PeterBee97 Amazing! Thank you very much!

simbit18 · 2026-04-17T12:25:12Z

Hi @PeterBee97 please fix

====================================================================================
Configuration/Tool: sim/nxcamera
2026-04-17 09:51:22
------------------------------------------------------------------------------------
  Cleaning...
  Configuring...
  Building NuttX...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  485k  100  485k    0     0  1893k      0 --:--:-- --:--:-- --:--:-- 1893k
sim/posix/sim_host_v4l2.c: In function 'host_video_get_device_path_by_index':
Error: sim/posix/sim_host_v4l2.c:147:15: error: implicit declaration of function 'strlcpy'; did you mean 'strncpy'? [-Werror=implicit-function-declaration]
  147 |           if (strlcpy(devpath, path, devpathlen) >= devpathlen)
      |               ^~~~~~~
      |               strncpy
cc1: all warnings being treated as errors
make[1]: *** [Makefile:408: sim_host_v4l2.o] Error 1
make[1]: Target 'nuttx' not remade because of errors.
make: *** [tools/Unix.mk:568: nuttx] Error 2
make: Target 'all' not remade because of errors.
/github/workspace/sources/nuttx/tools/testbuild.sh: line 386: /github/workspace/sources/nuttx/../nuttx/nuttx.manifest: No such file or directory
  [1/1] Normalize sim/nxcamera
Removing libs/libbuiltin/bin/
Removing libs/libbuiltin/kbin/
Removing libs/libc/kbin/
Removing libs/libnx/bin/
Removing libs/libnx/kbin/
Removing mm/kbin/
/github/workspace/sources/nuttx /github/workspace/sources/nuttx
HEAD detached at pull/18745/merge
nothing to commit, working tree clean
/github/workspace/sources/nuttx
/github/workspace/sources/apps /github/workspace/sources/nuttx
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
/github/workspace/sources/nuttx
Build Attempt 1 of 1
===================================================================================

xiaoxiang781216 · 2026-04-29T13:28:16Z

+      return -EINVAL;
+    }
+
+  if (find_driver(devpath) != NULL)


let's capture_register check dup for you

xiaoxiang781216 · 2026-04-29T13:28:34Z

+      return ret;
+    }
+
+  cmng = find_driver(devpath);


xiaoxiang781216 · 2026-04-29T13:31:31Z

+      return -EEXIST;
+    }
+
+  if (!get_available_registered_data_index(&index))


if you don't want data register twice, why not zero out g_capture_data

it's hard to support mulitple camera instances in the old capture_initialize/imgdata_register/imgsensor_register, that's why capture_register is added. I would suggest you move the related change to new pr to avoid blocking your other change.

xiaoxiang781216 · 2026-04-29T13:37:24Z

@acassis @jerpelea this patch change the common code, before I approve the change, please don't merge the pr.

Allow the SIM camera/V4L2 capture framework to manage multiple imgdata instances with dynamic mounting. This avoids cross-talk between camera streams when multiple devices are used. Signed-off-by: Peter Bee <bijunda@bytedance.com>

Adapt the macOS AVFoundation backend for multi-device discovery, camera index mapping and capture startup compatibility. This enables reliable use of multiple cameras through the SIM camera framework. Signed-off-by: Peter Bee <bijunda@bytedance.com>

/Library/Developer/CommandLineTools/usr/bin/c++filt: error: unknown argument '-[' Signed-off-by: Peter Bee <bijunda@bytedance.com>

Document the nxcamera command usage, including device selection, resolution, and pixel format options. Also mention recent support for multi-instance camera mounting and the macOS AVFoundation backend on the SIM platform. Signed-off-by: Peter Bee <bijunda@bytedance.com>

xiaoxiang781216 · 2026-04-30T04:04:02Z

-    .fmtdescs = g_fmts,
-  }
-};
+static sim_camera_priv_t *g_sim_camera_privs;


delete, don't need

xiaoxiang781216 · 2026-04-30T04:09:11Z

+  count = host_video_get_device_count();
+  if (count <= 0)
+    {
+      count = 1;


why not fail

xiaoxiang781216 · 2026-04-30T04:15:45Z

+  return ret;
+}
+
+static int host_video_count_devices(void)


merge to host_video_get_device_count

xiaoxiang781216 · 2026-04-30T04:17:01Z

+      count++;
+    }
+
+  return ret;


-ENODEV and delete ret

PeterBee97 requested review from Donny9, jerpelea, masayuki2009, pkarashchenko, xiaoxiang781216 and yamt as code owners April 15, 2026 14:00

github-actions Bot added Area: Build system Arch: simulator Issues related to the SIMulator Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. Board: simulator labels Apr 15, 2026

anchao previously approved these changes Apr 15, 2026

View reviewed changes

acassis requested changes Apr 15, 2026

View reviewed changes

simbit18 previously approved these changes Apr 15, 2026

View reviewed changes

PeterBee97 dismissed stale reviews from simbit18 and anchao via 1131750 April 16, 2026 04:21

PeterBee97 force-pushed the avfoundation branch from d0fdb9c to 1131750 Compare April 16, 2026 04:21

PeterBee97 force-pushed the avfoundation branch from 1131750 to f1c856b Compare April 16, 2026 06:23

PeterBee97 requested a review from acassis April 16, 2026 06:26

simbit18 previously approved these changes Apr 16, 2026

View reviewed changes

xiaoxiang781216 reviewed Apr 16, 2026

View reviewed changes

acassis previously approved these changes Apr 16, 2026

View reviewed changes

PeterBee97 dismissed stale reviews from acassis and simbit18 via c2ea6da April 17, 2026 09:16

PeterBee97 force-pushed the avfoundation branch from f1c856b to c2ea6da Compare April 17, 2026 09:16

xiaoxiang781216 reviewed Apr 19, 2026

View reviewed changes

Comment thread tools/mkallsyms.sh

Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated

Comment thread arch/sim/src/sim/posix/sim_host_v4l2.c Outdated

PeterBee97 force-pushed the avfoundation branch 2 times, most recently from 6b24b6b to 6ae23c9 Compare April 20, 2026 02:50

xiaoxiang781216 reviewed Apr 22, 2026

View reviewed changes

PeterBee97 force-pushed the avfoundation branch 2 times, most recently from 18260b8 to bd705d1 Compare April 27, 2026 12:43

acassis previously approved these changes Apr 27, 2026

View reviewed changes

jerpelea previously approved these changes Apr 27, 2026

View reviewed changes

xiaoxiang781216 reviewed Apr 28, 2026

View reviewed changes

Comment thread drivers/video/v4l2_cap.c Outdated

Comment thread drivers/video/v4l2_cap.c

Comment thread drivers/video/v4l2_cap.c Outdated

xiaoxiang781216 reviewed Apr 28, 2026

View reviewed changes

Comment thread arch/sim/src/sim/sim_camera.c Outdated

Comment thread arch/sim/src/sim/sim_camera.c Outdated

Comment thread arch/sim/src/sim/sim_camera.c

PeterBee97 dismissed stale reviews from jerpelea and acassis via a06a9f4 April 29, 2026 03:53

PeterBee97 force-pushed the avfoundation branch 7 times, most recently from 28dda56 to 13efe13 Compare April 29, 2026 09:29

PeterBee97 requested a review from xiaoxiang781216 April 29, 2026 09:33

jerpelea previously approved these changes Apr 29, 2026

View reviewed changes

acassis previously approved these changes Apr 29, 2026

View reviewed changes

xiaoxiang781216 requested changes Apr 29, 2026

View reviewed changes

PeterBee97 dismissed stale reviews from acassis and jerpelea via 408638b April 30, 2026 03:46

PeterBee97 force-pushed the avfoundation branch from 13efe13 to 408638b Compare April 30, 2026 03:46

Peter Bee added 4 commits April 30, 2026 11:51

sim: fix mkallsyms for Objective-C symbols on macOS

a0cb989

/Library/Developer/CommandLineTools/usr/bin/c++filt: error: unknown argument '-[' Signed-off-by: Peter Bee <bijunda@bytedance.com>

PeterBee97 force-pushed the avfoundation branch from 408638b to 60bdeaf Compare April 30, 2026 03:52

xiaoxiang781216 reviewed Apr 30, 2026

View reviewed changes

Conversation

PeterBee97 commented Apr 15, 2026

Summary

Impact

Testing

Uh oh!

acassis left a comment

Choose a reason for hiding this comment

Uh oh!

acassis commented Apr 15, 2026

Uh oh!

PeterBee97 commented Apr 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

acassis left a comment

Choose a reason for hiding this comment

Uh oh!

simbit18 commented Apr 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xiaoxiang781216 commented Apr 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants