Skip to content

[DOC] Add API documentation for Python bindings #69

@RamyaGuru

Description

@RamyaGuru

Report needed documentation

Report needed documentation

The DAQIRI Python bindings (python/daqiri_common_pybind.cpp) expose a comprehensive pybind11 interface — 60+ functions, 17 classes/structs, 20 enums, and 9 module constants — but there is no corresponding Python API documentation. The existing API reference (docs/api-guide.md and docs/daqiri-api.html) covers only the C++ interface with C++ code examples. Users building Python-based applications with DAQIRI have no reference material.

Additionally, the current docs site treats API Reference as a single flat page. As the reference surface grows (C++, Python, YAML configuration), this should be reorganized into a structured section.

Describe the documentation you'd like

Proposed web-docs structure

Restructure the mkdocs.yml nav so that API Reference becomes a section with three sub-pages instead of a single page:

# Current structure:
nav:
  - Getting Started: getting-started.md
  - Configuration: configuration.md        # ← moves under API Reference
  - API Reference: api-guide.md            # ← single page today
  - Tutorials: ...

# Proposed structure:
nav:
  - Getting Started: getting-started.md
  - API Reference:
    - C++ API Reference: api-reference/cpp.md
    - Python API Reference: api-reference/python.md
    - Configuration YAML Reference: api-reference/configuration.md
  - Tutorials: ...

What each tab contains:

  1. C++ API Reference (docs/api-reference/cpp.md)

    • Migrated from the current docs/api-guide.md content
    • Remains the primary reference for the C/C++ free-function API (get_rx_burst, set_udp_header, etc.), types (BurstParams, NetworkConfig, enums), and the Manager interface
    • The standalone docs/daqiri-api.html continues to exist as the interactive HTML version, linked from this page
  2. Python API Reference (docs/api-reference/python.md) — new

    • Documents the daqiri Python module and its full public surface:
      • Initialization: daqiri_init() (which accepts a YAML path, YAML string, dict, or config-like object — more flexible than the C++ API), parse_network_config() variants
      • RX/TX operations: get_rx_burst() (3 overloads: port+queue, port-only, any-port), get_tx_packet_burst(), send_tx_burst()
      • Packet access: get_packet_ptr(), get_segment_packet_ptr(), get_packet_length(), get_num_packets()
      • Header helpers: set_udp_header(), set_ipv4_header(), set_eth_header(), set_header()
      • Buffer management: free_all_packets_and_burst_rx(), free_all_packets_and_burst_tx(), free_rx_burst(), free_tx_burst(), and other free functions
      • RDMA operations: rdma_connect_to_server(), rdma_get_port_queue(), rdma_set_header(), rdma_get_opcode()
      • Socket operations: socket_connect_to_server(), socket_get_port_queue(), socket_get_server_conn_id()
      • Reorder: get_reorder_burst_info(), synchronize_burst_event()
      • File I/O: daqiri_write_raw_to_file(), daqiri_write_pcap_to_file()
      • Utility: get_mac_addr(), format_eth_addr(), print_stats(), shutdown()
      • All 20 enums (Status, ManagerType, Direction, BufferLocation, MemoryKind, StreamType, SocketProtocol, etc.)
      • All 17 config/data classes (NetworkConfig, BurstParams, InterfaceConfig, RxConfig, TxConfig, RDMAConfig, SocketConfig, ReorderConfig, etc.)
      • Module constants (ADV_NETWORK_HEADER_SIZE_BYTES, MAX_NUM_RX_QUEUES, MAX_NUM_TX_QUEUES, MAX_INTERFACES, MAX_NUM_SEGS, etc.)
    • Python code examples for common workflows:
      • Initialization from a YAML file and from a Python dict
      • Basic RX loop (receive, inspect, free)
      • Basic TX loop (allocate, fill UDP, send, free)
      • RDMA client/server connection setup
      • Socket (TCP/UDP) send/receive
      • GPU reorder burst handling with CUDA stream
    • Python-specific notes:
      • How daqiri_init() differs from the C++ version (accepts dicts, config-like objects)
      • Buffer/pointer semantics in Python (pybind11 opaque pointers)
      • GIL considerations for long-running RX/TX loops
      • Build instructions (-DDAQIRI_BUILD_PYTHON=ON and pybind11 dependency)
      • Import pattern (import daqiri or from daqiri import ...)
  3. Configuration YAML Reference (docs/api-reference/configuration.md)

    • Migrated from the current docs/configuration.md — this is already a thorough YAML field-by-field reference
    • Moving it under the API Reference section groups all "look-up-a-parameter" content together, rather than having it as a separate top-level nav item alongside Getting Started and Tutorials
    • No content changes needed for this page beyond the move

Implementation notes:

  • MkDocs Material's tabbed navigation (navigation.tabs) is already enabled in mkdocs.yml, as is pymdownx.tabbed with alternate_style — the sub-page section rendering will work out of the box
  • The existing docs/daqiri-api.html (standalone interactive HTML API page) should be preserved and linked from the C++ API Reference page rather than folded into the nav
  • Create a docs/api-reference/ directory for the three new files
  • Set up redirects or update internal links from api-guide.mdapi-reference/cpp.md and configuration.mdapi-reference/configuration.md to avoid breaking existing bookmarks
  • Update docs/index.html landing page links to point to the new paths
  • Update README.md Documentation table
  • Update CLAUDE.md Documentation section and docs-sync rules

Steps taken to search for needed documentation

  • Reviewed docs/api-guide.md — all examples are C++ only
  • Reviewed docs/daqiri-api.html — standalone HTML reference, C++ only
  • Reviewed docs/configuration.md — comprehensive YAML reference, currently a separate top-level nav item
  • Searched all docs/ files for "python", "pybind", "import daqiri" — no Python API docs found
  • Confirmed bindings exist and are extensive in python/daqiri_common_pybind.cpp (1,052 lines)
  • python/__init__.py re-exports everything from the compiled _daqiri module, confirming this is a user-facing API
  • Verified mkdocs.yml already has pymdownx.tabbed and navigation.tabs enabled — no theme/extension changes needed

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions