ttkmd-2.11.0

Tenstorrent PCIe Driver Version 2.11.0 Release Notes

This release adds a userspace interface for exchanging messages with system-management firmware and forwards Blackhole firmware logs to the kernel log. It also fixes locking and device-removal races and rejects malformed firmware telemetry tables.

---

New Features

System Management Controller Messaging

A new ioctl, TENSTORRENT_IOCTL_SMC_MSG, allows userspace applications to exchange messages with the system management controller through the firmware ARC message queue on Wormhole and Blackhole devices.

The driver owns the firmware queue and serializes userspace messages with messages generated by the driver. Each open file descriptor may have one message outstanding:

• TENSTORRENT_SMC_MSG_POST submits a request.
• TENSTORRENT_SMC_MSG_POLL checks for and returns the response.
• TENSTORRENT_SMC_MSG_ABANDON discards an outstanding request or its eventual response.

POST and POLL are separate, non-blocking operations. POLL returns -EAGAIN while a response is pending and -ESRCH when the file descriptor has no outstanding request. Closing the file descriptor abandons its outstanding request.

The ioctl returns -EOPNOTSUPP when the installed firmware does not provide a usable message queue. Applications are responsible for interpreting the firmware status returned in the first message word; a nonzero firmware status does not cause POLL itself to fail.

Blackhole Firmware Log Forwarding

Blackhole firmware can now forward SMC and DMC log records over PCIe to the kernel log. Firmware writes records to a shared DMA buffer and notifies the driver with an interrupt. The driver validates and sanitizes the records before emitting them with the corresponding kernel log level.

Firmware log forwarding is enabled by default, but requires the device to be behind active IOMMU translation. Without IOMMU translation, the driver disables forwarding and reports the reason in the kernel log. Firmware versions that do not support log forwarding continue to operate without the feature.

Two module parameters control forwarding:

• fw_logging enables or disables Blackhole firmware log forwarding at module load time. The default is 1.
• fw_log_level selects the maximum forwarded level: 1 for errors, 2 for warnings, 3 for informational messages, and 4 for debug messages. The default is 2, so informational and debug records are omitted.

The driver reports malformed records and gaps in firmware log sequence numbers. A corrupted shared-buffer header disables forwarding until the device is reset.

---

Bug Fixes

• TLB mapping deadlock: Fixed a lock-order cycle between the process mmap_lock and the per-file-descriptor mutex. TLB allocation, configuration, export, mapping, release, and reset reclamation now use a dedicated mutex.
• TLB ownership race: Fixed a race in which a TLB window could be freed and assigned to another file descriptor while the original owner was configuring it.
• Character-device removal races: Serialized removal-time file-descriptor cleanup against concurrent open() and close(), and tied driver allocation lifetime to kernel device-model references so an in-progress open() cannot access freed device state.
• Power-state error reporting: TENSTORRENT_IOCTL_SET_POWER_STATE now returns the underlying ARC messaging error instead of replacing it with -EINVAL.
• Firmware telemetry table validation: Wormhole and Blackhole telemetry initialization now rejects entry counts and derived addresses that fall outside the firmware telemetry region. Long valid table scans yield to the scheduler.

---

Diagnostics

On Blackhole, the driver now checks firmware status during probe for a 12V-2x6 power-cable fault. If firmware reports the fault, the kernel log states that the system should be shut down and the cable seating checked. A device in this state may still enumerate over PCIe while Tensix, GDDR, Ethernet, and RISC-V cores remain unavailable.