RZ/V2H DRP-AI Driver

License

Supported Device

RZ/V2H

Overview

This is the DRP-AI device driver in RZ/V2H Linux.
DRP-AI performs AI inferences with high power efficiency.
The DRP-AI device driver provides an interface to easily handle the AI inference execution function of DRP-AI.

Restrictions

None.

Installation

Please see Installation.

1. Functions

The following functions are supported:

DRP-AI Object files*1 can be dynamically allocated at any address.
Get address and size of DRP-AI memory area.
Deploy DRP-AI Object files *1 in memory *2 and set the appropriate cache settings.
Execute AI inference based on the DRP-AI Object files *1 deployed in memory *2.
Wait for AI inference using "select" function or polling.
Stop forcibly during AI inference.
- *1 DRP-AI Object files: Refer to "2 Terminology".
- *2 Reservation area dedicated to DRP-AI.

2. Terminology

Terms	Explanation
BSP	Board support package
DRP	Dynamically Reconfigurable Processor
AI-MAC	Hardware for high-speed processing of 8-bit matrix operations
DRP-AI	AI accelerator consist of DRP and AI-MAC
DRP-AI Object files	Files required to execute DRP-AI generated by DRP-AI Translator i8. - AIMAC descriptor (aimac_desc.bin, aimac_cmd.bin, aimac_param_desc.bin, aimac_param_cmd.bin) - DRP descriptor (drp_desc.bin) - DRP parameter (drp_param.bin) - DRP parameter Information (drp_param_info.txt) - Weight data (weight.dat) - DRP configuration data (drpcfg.mem)
DRP-AI memory area	Dedicated area on Memory Map of RZ/V series (RZ/V2H) Linux Package used by DRP-AI.

3. Module Configuration

3.1 Module Configuration

Figure 3-1 shows the configuration of this module.

3.2 State Transition Diagram

The following Figure 3-2 and Table 3-1 show the state transition of this module.

Table 3-1 State transition table of DRP-AI driver

4. APIs

4.1 Device Node

Table 4-1 shows the device node of this module.

4.2 External Function

This section explains in the following format about the functions this module supplies.

[Overview]

Presents an overview of a function.

[Function Name]

Explains the name of the function.

[Calling format]

Explains the format for calling the function.

[Argument]

Explains the argument(s) of the function.

[Return value]
Explains the return value(s) of the function.

[Error number]

Explains the error number(s) of the function.

[Feature]

Explains the features of the function.

[Remark]

Explains points to be noted when using the function.

Table 4-2 System Calls

Chapter	Function name	Description
4.2.1	open	Initialize the DRP-AI device and driver.
4.2.2	close	End control of DRP-AI.
4.2.3	write	Write data to DRP-AI memory area. Or import drp_param_info.txt into the DRP-AI driver.
4.2.4	read	Read data from DRP-AI memory area.
4.2.5	ioctl(DRPAI_SET_ADRCONV)	Extend the access range of DRP-AI from 32 bits to 40 bits. DRP-AI Object Files can be placed at any address in DDR memory.
4.2.6	ioctl(DRPAI_ASSIGN)	Set the address and size for reading and writing data in DRP-AI memory area.
4.2.7	ioctl(DRPAI_START)	Start AI inference.
4.2.8	ioctl(DRPAI_RESET)	Reset AI inference.
4.2.9	ioctl(DRPAI_GET_STATUS)	Get internal status of DRP-AI.
4.2.10	select / pselect	Waiting for AI inference to complete or an error to occur.
4.2.11	ioctl(DRPAI_ASSIGN_PARAM)	Change the write function mode to load drp_param_info.txt into /dev/drpai0.
4.2.12	ioctl(DRPAI_PREPOST_CROP)	Set a parameter of crop preprocessing operation.
4.2.13	ioctl(DRPAI_PREPOST_INADDR)	Change the input/output address of DRP-AI.
4.2.14	ioctl(DRPAI_GET_DRPAI_AREA)	Get the address and size of DRP-AI memory area.
4.2.15	ioctl(DRPAI_SET_DRP_MAX_FREQ)	Set DRP maximum operating frequency.
4.2.16	ioctl(DRPAI_SET_DRPAI_FREQ)	Set DRP-AI operating frequency.

The flow of the above system calls is described in 4.5.

4.2.1 open

[Overview]

Initialize the DRP-AI device and driver.

[Function Name]

open

[Calling format]

#include <linux/drpai.h>
int open(const char *device_name, O_RDWR);

[Arguments]

	Explanation
device_name	Device name (/dev/drpai0) to open
O_RDWR	Fixed control

[Returns]

	Explanation
File descriptor (a nonnegative integer)	Success
-1	Error

[Error number]

	Explanation
EIO(5)	DRP-AI H/W error. Failed to reset DRP-AI. Try to restart a system.
ENOMEM(12)	Failed to allocate internal memories for DRP-AI driver. Check your memory usage on Linux.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly.

[Feature]

Initializes the DRP-AI hardware and the inside of the driver.
Executing this function will allow you to use the functions of the DRP-AI driver.
Use the returned file descriptor to use other API functions.

[Remark]

Receive the return value of open as an "int" type.
Please open for each AI model.
When open it multiple times, it will be initialized only the first time.

4.2.2 close

[Overview]

End control of DRP-AI

[Function Name]

close

[Calling format]

#include <linux/drpai.h>
int close(int fd);

[Arguments]

	Explanation
fd	File descriptor

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

*1 No error number is returned from DRP-AI driver close operation.
*2 The error number happened in this function is returned from Linux kernel close system call.

[Feature]

End control of the DRP-AI device and driver.

[Remark]

Execute this function the number of times you call the open function.
When this function is executed during AI inference, AI inference is forced to stop. Note that when calling open function multiple times, the only last calling close function terminates AI inference.
Please make sure that all DRP-AI related API and select() / pselect() operation must be finished, before this close function is called.

4.2.3 write

[Overview]

Write data to DRP-AI memory area. Or import drp_param_info.txt into the DRP-AI driver.

[Function Name]

write

[Calling format]

#include <linux/drpai.h>
ssize_t write(int fd, const void *buf, size_t count);

[Arguments]

	Explanation
fd	File descriptor
buf	Write data stock area
count	Write size

[Returns]

	Explanation
Positive value or 0	Success (Written size)
-1	Error

[Error number]

	Explanation
EFAULT(14)	buf is NULL or virtual address translation of the write destination fails.
EINVAL(22)	The argument is invalid. count is 0.
ENOMEM(12)	Failed to get the buffer area inside the DRP-AI driver.
EACCES(13)	Attempted to write more than the size set by ioctl(DRPAI_ASSIGN) function. An attempt was made to write while reading the size set by ioctl(DRPAI_ASSIGN) function. ioctl(DRPAI_ASSIGN) function is not running.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.

[Feature]

This function writes the data (DRP-AI Object files etc.) to the address specified by ioctl(DRPAI_ASSIGN) function.
Import drp_param_info.txt into the DRP-AI driver by ioctl(DRPAI_ASSIGN_PARAM) function.

[Remark]

Execute ioctl(DRPAI_ASSIGN) before executing this function.
This function returns the actual size it wrote. If the sum of the count exceeds the size set previously in the ioctl(DRPAI_ASSIGN) function, the write function size will not write the exceeding part.
If you rewrite DRP-AI Object files during AI inference, it may not work properly.

4.2.4 read

[Overview]

Read data from DRP-AI memory area.

[Function Name]

read

[Calling format]

#include <linux/drpai.h>
ssize_t read(int fd, const void *buf, size_t count);

[Arguments]

	Explanation
fd	File descriptor
buf	Read data stock area
count	Read size

[Returns]

	Explanation
Positive value or 0	Success (Read size)
-1	Error

[Error number]

	Explanation
EFAULT(14)	buf is NULL or virtual address translation of the read destination fails.
EINVAL(22)	The argument is invalid. count is 0.
ENOMEM(12)	Failed to secure the buffer area inside the DRP-AI driver. Attempted to read more than the size set by ioctl(DRPAI_ASSIGN) function.
EACCES(13)	An attempt was made to read while writing the size set by ioctl(DRPAI_ASSIGN) function. ioctl(DRPAI_ASSIGN) function is not running.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.

[Feature]

This function reads the data (DRP-AI output result etc.) from the address specified by ioctl (DRPAI_ASSIGN) function.

[Remark]

Execute ioctl(DRPAI_ASSIGN) before executing this function.
This function returns the actual size it wrote. If the sum of the count exceeds the size set previously in the ioctl(DRPAI_ASSIGN) function, the write function size will not write the exceeding part.
This function is used differently from the read function used for general file operations. This is a dedicated function to read the output of DRP-AI in combination with the ioctl(DRPAI_ASSIGN) function.

4.2.5 ioctl(DRPAI_SET_ADRCONV)

[Overview]

Extend the access range of DRP-AI from 32 bits to 40 bits.
DRP-AI Object Files can be placed at any address in DDR memory.

[Function Name]

ioctl(DRPAI_SET_ADRCONV)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_SET_ADRCONV, drpai_adrconv_t* adrconv);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_SET_ADRCONV	Fixed control
adrconv	Address conversion settings of DRP-AI Object files. Refer to "4.3.2 drpai_adrconv_t".

[Return]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument. adrconv is NULL. adrconv.conv_address is 0x100_0000_0000 or more. The lower 24 bits of adrconv.conv_address and adrconv.org_address are different. adrconv.size is 0.
ENOMEM(12)	adrconv.org_address, adrconv.conv_address and adrconv.size overlap with other settings.
EFAULT(14)	Internal error.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.
EBUSY(16)	AI inference is still ongoing.

[Feature]

This function has two functions.

Extend the access range of DRP-AI from 32 bits to 40 bits.
DRP-AI Object Files can be placed at any address in DDR memory.

This chapter describes them in the following order.
(A)How to place DRP-AI Object files when this function is not used.
(B)How to place DRP-AI Object files when this function is used.

(A)
The starting address of the memory used by DRP-AI is defined in the file Address map definition (*.yaml) file input to the DRP-AI translator i8.
Here, the address map definition (*.yaml) file assumes a configuration with just one element. When translated by the DRP-AI translator i8, the address is determined and listed in addr_map.txt. Among the elements listed in addr_map.txt, weight, drp_config, aimac_param_cmd, aimac_param_desc, aimac_cmd, aimac_desc, drp_param, drp_desc is called DRP-AI Object files.
DRP-AI Object files must be placed in DDR memory using DRP-AI Driver on the actual device according to the address in addr_map.txt.
When using this method, DRP-AI can only handle 32-bit space.
Placement is done using the ioctl(DRPAI_ASSIGN) and write functions of the DRP-AI Driver API.

(B)
When this function is used, the starting address of the memory used by DRP-AI can be changed on the actual device after the DRP-AI translator i8 translation.
The values for the arguments org_address, conv_address and size should be set as shown in the figure below.
Using this method, a 40-bit address can be specified for conv_address.

When the DRP-AI Driver is open, this function is off, no address translation is performed when inference is executed, and the access range is 32 bits.
When mode=DRPAI_ADRCONV_MODE_REPLACE (0), this function is turned on and address translation is enabled during the next inference run.
When mode=DRPAI_ADRCONV_MODE_DISABLE(0xFF), this function is turned off and address translation is disable during the next inference run.
The following figure shows the setting timing of this function.

Multiple address conversion settings can be enabled at the same time.

When mode=DRPAI_ADRCONV_MODE_ADD(1), this function can be called to add a new setting to the setting of the last call to this function.

[Remark]

The range specified by org_address and size, 0xFC00_0000 to 0xFFFFFF_FFFF, cannot be used because it is used by the DRP-AI Driver. Therefore, care should be taken not to use this area during DRP-AI translator i8 translation.
When adding setting using DRPAI_ADRCONV_MODE_ADD, set org_address and conv_address so that they do not overlap with previous settings.

4.2.6 ioctl(DRPAI_ASSIGN)

[Overview]

Set the address and size for reading and writing data in DRP-AI memory area.
In the case of using this function for writing the DRP-AI Object files, the address will be fixed address output by the DRP-AI Translator i8 when the DRP-AI Object files was generated.
Use this function to read AI inference result.

[Function Name]

ioctl(DRPAI_ASSIGN)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_ASSIGN, drpai_data_t* file);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_ASSIGN	DRPAI_ASSIGN : Fixed control
file	Address and size of the DPR-AI Object files. Refer to 4.3.1 about drpai_data_t structure.

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument. file is NULL. file.address is outside the DRP-AI memory area. file.address is not 64-byte aligned. file.size is 0. Failed to assign memory area because desired area already assigned by other thread/process. Set a different memory area from other process/thread.
EACCES(13)	The previous writing or reading has not been completed.
EFAULT(14)	Internal error.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.

[Feature]

Set the address and size for reading and writing data (DRP-AI Object files, DRP-AI output result, etc.) in DRP-AI memory area.
After executing this function, write DRP-AI Object files using the write function, Also, after the AI inference is complete, use the read function to read the DRP-AI output result.
Only one data can be set with this function. Repeat this function and read/write for the number of data you want to read/write.

[Remark]

None.

4.2.7 ioctl(DRPAI_START)

[Overview]

Start AI inference

[Function Name]

ioctl(DRPAI_START)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_START, drpai_data_t* &proc[0]);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_START	Fixed control
proc[DRPAI_INDEX_NUM(10)]	Physical address and size of input/output data, DRP-AI Object files Refer to Table 4-3 for details on the elements of the array.

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument. proc is NULL. The alignment below is not 64 bytes, or is not set within DRP-AI memory area. proc[DRPAI_INDEX_DRP_DESC].address proc[DRPAI_INDEX_DRP_CFG].address proc[DRPAI_INDEX_DRP_PARAM].address proc[DRPAI_INDEX_AIMAC_DESC].address proc[DRPAI_INDEX_WEIGHT].address proc[DRPAI_INDEX_OUTPUT].address proc[DRPAI_INDEX_AIMAC_CMD].address proc[DRPAI_INDEX_AIMAC_PARAM_DESC].address proc[DRPAI_INDEX_AIMAC_PARAM_CMD].address
EBUSY(16)	AI inference is still ongoing.
EFAULT(14)	Internal error.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.
ENOMEM(12)	When executing address conversion using the ioctl(DRPAI_SET_ADRCONV), the conversion source address of the physical address specified by the arguments is no set by the ioctl(DRPAI_SET_ADRCONV).

[Feature]

Start AI inference.
proc[] except proc[DRPAI_INDEX_INPUT] must be placed in the DRP-AI memory area.
proc[DRPAI_INDEX_INPUT] can be specified anywhere it is a continuous physical area.
However, when executing address conversion using the ioctl(DRPAI_SET_ADRCONV), proc[DRPAI_INDEX_INPUT] can only be placed in the area set by ioctl(DRPAI_SET_ADRCONV).
When the ioctl(DRPAI_PREPOST_INADDR) function is used to set the address of the input image data, proc[DRPAI_INDEX_INPUT] will be disabled. The invalidation period is until this driver is terminated using the close function..
Refer to 4.3.1 about drpai_data_t structure.

[Remark]

When executing address conversion using the ioctl(DRPAI_SET_ADRCONV), specify the post-conversion physical address for the address set in the arguments.

4.2.8 ioctl(DRPAI_RESET)

[Overview]

Reset AI inference.

[Function Name]

ioctl(DRPAI_RESET)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_RESET, NULL);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_RESET	Fixed control
NULL	Fixed control

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EIO(5)	DRP-AI H/W error. Failed to reset DRP-AI.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.

[Feature]

AI inference is forced stopped and reset.
DRP-AI Object files deploy in memory is not reset.
When this function is executed, DRPAI_ERRINFO_RESET is stored in drpai_status.err acquired by the ioctl(DRPAI_GET_STATUS) function.

[Remark]

None.

4.2.9 ioctl(DRPAI_GET_STATUS)

[Overview]

Get internal state of DRP-AI.

[Function Name]

ioctl(DRPAI_GET_STATUS)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_GET_STATUS, drpai_status_t* drpai_status);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_GET_STATUS	Fixed control
drpai_status	Internal state of DRP-AI. Refer to 4.3.3 about drpai_status_t structure.

[Returns]

0 : AI inference is complete or before start.
-1 : AI inference is still ongoing or Error.

[Error number]

	Explanation
EINVAL(22)	Invalid argument. drpai_status is NULL.
EBUSY(16)	AI inference is still ongoing.
EFAULT(14)	Internal error.
EIO(5)	DRP-AI H/W error.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.

[Feature]

Gets the internal status of DRP-AI when this function is executed.
Even if the returned value is -1, the internal state of DRP-AI can be acquired.
When the returned value is 0, AI inference is completed.
When the returned value is -1 and the Error number is EBUSY, AI inference is still ongoing.
By using conditional branching based on the return value of this function and Error number, polling can be implemented for waiting the AI inference completion.

[Remark]

None.

4.2.10 select / pselect

[Overview]

Waiting for AI inference to complete or DRP-AI H/W error to occur.

[Function Name]

select / pselect

[Calling format]

#include <linux/drpai.h>
int select(int nfds, fd_set *readfds, NULL, NULL, struct timeval *timeout);
int pselect(int nfds, fd_set *readfds, NULL, NULL, const struct timespec *timeout, const sigset_t *sigmask);

[Arguments]

	Explanation
nfds	Please set fd + 1.
readfds	Set * fdset of FD_SET (int fd, fd_set * fdset).
NULL	Fixed control
NULL	Fixed control
timeout	Set the timeout time*1.
sigmask	Signal mask

[Returns]

	Explanation
Positive value*2.	Success
0	Time out
-1	Error

[Feature]

This function will return by the following factors.

When DRP-AI processing is completed*3
When DRP-AI processing is not completed within the time set in the argument timeout.
When ioctl(DRPAI_RESET) is executed.
Argument is invalid, etc.

Please use pselect, if the Signal mask is needed in the application.

[Remark]

*1 : The structure is different between the select and pselect functions.
*2 :Return value is the number of file descriptors contained in the three returned descriptor sets (that is, the total number of bits that are set in readfds, writefds, exceptfds).
*3 : Use ioctl(DRPAI_GET_STATUS) to check if it completed normally or abnormally.

4.2.11 ioctl(DRPAI_ASSIGN_PARAM)

[Overview]

Change the write function mode to Import drp_param_info.txt into /dev/drpai0.

[Function Name]

ioctl(DRPAI_ASSIGN_PARAM)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_ASSIGN_PARAM, drpai_assign_param_t * assign_param);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_ASSIGN_PARAM	Fixed control
assign_param	Parameter information for pre-processing and post-processing. Refer to 4.3.4 about drpai_assign_param_t structure.

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument. The situations below cause this error. assign_param is NULL. Member variables of assign_param are set as 0.
EACCES(13)	Execution order error. The previous writing or reading has not been completed. Complete previous writing or reading before executing this function.
EFAULT(14)	Internal error.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.

[Feature]

Execute this function to change the mode of the write function.
After executing this function, use the write function to write drp_param_info.txt to /dev/drpai0.

[Remark]

None.

4.2.12 ioctl(DRPAI_PREPOST_CROP)

[Overview]

Set a parameter of crop preprocessing operation.

[Function Name]

ioctl(DRPAI_PREPOST_CROP)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_PREPOST_CROP, drpai_crop_t * crop_param);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_PREPOST_CROP	Fixed control
crop_param	Crop information such as crop position (x, y). Refer to 4.3.5 about drpai_crop_t structure.

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument. This error is occurred by following. crop_param is NULL. crop_param is invalid.
EACCES(13)	Execution order error. The previous writing or reading has not been completed. Complete previous writing or reading before executing this function.
EFAULT(14)	Internal error.
EBUSY(16)	AI inference is still ongoing.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.

[Feature]

Change at runtime the clipping coordinates and size set in "op: crop" in "prepost_.yaml" when executing DRP-AI Translator i8. It also changes the input size of preprocessing option after "op: crop" to the cropping size.
When the cropping coordinates and size are the same as before, there is no need to execute this function.
Before executing this function, write drp_param_info.txt into DRP-AI driver using ioctl(DRPAI_ASSIGN_PARAM) function and write function.
The following lists are notes about "op: crop" in "prepost_.yaml".
- "shape_in" and "shape_out" must be the same value.
- After "op: crop", be sure to define at least one preprocessing operation such as "op: memcopy".
- Only one "op: crop" is available in the preprocessing options.

[Remark]

None.

4.2.13 ioctl(DRPAI_PREPOST_INADDR)

[Overview]

Change the input image address of DRP-AI.

[Function Name]

ioctl(DRPAI_PREPOST_INADDR)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_PREPOST_INADDR, drpai_inout_t * inout_param);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_PREPOST_INADDR	Fixed control
inout_param	Input address of DRP-AI. Refer to 4.3.6 about drpai_inout_t structure.

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument.
EACCES(13)	Execution order error. The previous writing or reading has not been completed. Complete previous writing or reading before executing this function.
EFAULT(14)	Internal error.
EBUSY(16)	AI inference is still ongoing.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.
ENOMEM(12)	When executing address conversion using the ioctl(DRPAI_SET_ADRCONV), the conversion source address of the physical address specified by the arguments is no set by the ioctl(DRPAI_SET_ADRCONV).

[Feature]

This function changes the input image address to be input to the DRP-AI.
Execute this function every time before starting inference using ioctl(DRPAI_START).
When there is multiple input of DRP-AI, change input/output address using this function one by one.
Up to 2 input images are allowed.
Before executing this function, write drp_param_info.txt to /dev/drpai0 using ioctl(DRPAI_ASSIGN_PARAM) and write only once.
The input image address can be specified anywhere it is a continuous physical area.
However, when executing address conversion using ioctl(DRPAI_SET_ADRCONV), the input image address can only be placed in the area set by ioctl(DRPAI_SET_ADRCONV).
When executing address conversion using ioctl(DRPAI_SET_ADRCONV), be sure to call ioctl(DRPAI_SET_ADRCONV) before calling this function.

[Remark]

None.

4.2.14 ioctl(DRPAI_GET_DRPAI_AREA)

[Overview]

Get the address and size of DRP-AI memory area.

[Function Name]

ioctl(DRPAI_GET_DRPAI_AREA)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_GET_DRPAI_AREA, drpai_data_t* drpai_data);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_GET_DRPAI_AREA	Fixed control
drpai_data	The address and size of DRP-AI memory area is returned. Refer to 4.3.1 about drpai_data_t structure.

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument. drpai_data is NULL.
EFAULT(14)	Internal error.
ERESTART(85)	Failed to get a read/write permission. Try this system call again. There is sample code in 4.5.2 Multi process / thread.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.

[Feature]

Gets the address and size of DRP-AI memory area.

[Remark]

None.

4.2.15 ioctl(DRPAI_SET_DRP_MAX_FREQ)

[Overview]

Set DRP maximum operating frequency.

[Function Name]

ioctl(DRPAI_SET_DRP_MAX_FREQ)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_SET_DRP_MAX_FREQ, uint32_t *mindiv);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_SET_DRP_MAX_FREQ	Fixed control
mindiv	2 to 127 2 : 420MHz (Default) 3 : 315MHz ... 127 : 9.84MHz Calculation Formula : 1260MHz /(mindiv+1)

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument.(Other than 2 to 127)
EFAULT(14)	Internal error.
EBUSY(16)	AI inference is still ongoing.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.

[Feature]

Set DRP maximum operating frequency.
Leave this driver open to keep the frequency setting.

[Remark]

None.

4.2.16 ioctl(DRPAI_SET_DRPAI_FREQ)

[Overview]

Set DRP-AI operating frequency.

[Function Name]

iocto(DRPAI_SET_DRPAI_FREQ)

[Calling format]

#include <linux/drpai.h>
int ioctl(int fd, DRPAI_SET_DRPAI_FREQ, uint32_t *divfix);

[Arguments]

	Explanation
fd	File descriptor
DRPAI_SET_DRPAI_FREQ	Fixed control
divfix	1 to 127 1, 2 : 1GHz (Default) 3 : 630MHz ... 127 : 10MHz Calculation Formula(When divfix=3 or more.) : 1260MHz /(divfix-1)

[Returns]

	Explanation
0	Success
-1	Error

[Error number]

	Explanation
EINVAL(22)	Invalid argument.(Other than 1 to 127)
EFAULT(14)	Internal error.
EBUSY(16)	AI inference is still ongoing.
ETIMEDOUT(110)	Exclusive control of DRP-AI for multi process/threads failed. To solve this error, check your process/thread that use DRP-AI works properly. Or in most cases, DRP-AI H/W may not be working properly, so try to restart a system.

[Feature]

Set DRP-AI operating frequency.
Leave this driver open to keep the frequency setting.

[Remark]

None.

4.3 Structure

4.3.1 drpai_data_t

Type	Member name	Overview
uint64_t	address	Physical address*
uint64_t	size	Data size*

*When using this structure for ioctl(DRPAI_ASSIGN), "address" must be 64-byte aligned and target object located within the DRP-AI memory area.

4.3.2 drpai_adrconv_t

Type	Member name	Overview
uint32_t	org_address	Address of before conversion. - Valid only when mode != DRPAI_ADRCONV_MODE_DISABLE(0xFF)
uint64_t	conv_address	Address of after conversion. - Valid only when mode !=DRPAI_ADRCONV_MODE_DISABLE(0xFF) - 40bit valid - The lower 24 bits should be the same as conv_address.
uint32_t	size	Conversion Size. - Valid only when mode !=DRPAI_ADRCONV_MODE_DISABLE(0xFF)
uint8_t	mode	DRPAI_ADRCONV_MODE_REPLACE(0) : Enable address conversion, clear all conversion table that is set in previous and add new conversion table. DRPAI_ADRCONV_MODE_ADD(1) : Enable address conversion and add new conversion table. (Old table is not cleared.) DRPAI_ADRCONV_MODE_DISABLE(0xFF) : Clear all conversion table and disable address conversion.

4.3.3 drpai_status_t

Type	Member name	Overview
uint32_t	status	DRP-AI status Refer to Table 4-4 for the stored values.
uint32_t	err	Latest DRP-AI processing error information. Refer to Table 4-5 for the stored values. It is valid from the time when the completion notification is sent by the select function or the status member variable is IDLE until the next ioctl (DRPAI_START) or ioctl (DRPAI_RESET) is executed.
uint32_t	reserved[DRPAI_RESERVED_NUM(10)]	Reserved area

4.3.4 drpai_assign_param_t

Type	Member name	Overview
uint32_t	info_size	Size of drp_param_info.txt.
drpai_data_t	obj	Address and size of drp_param.bin Refer to 4.3.1 for the stored values.

4.3.5 drpai_crop_t

Be sure to set the value within the range from the lower limit value to the upper limit value.

Type	Member name	Overview
uint16_t	img_owidth	Width of cropped image (pixel) MAX : Width of the input image MIN : 1
uint16_t	img_oheight	Height of cropped image (pixel) MAX : Height of the input image MIN : 1
uint16_t	pos_x	x origin of cropped image on original image (pixel) MAX : Width of the input image - 1 MIN : 0
uint16_t	pos_y	y origin of cropped image on original image (pixel) MAX : Height of the input image - 1 MIN : 0
drpai_data_t	obj	Address and size of drp_param.bin Refer to 4.3.1 for the stored values.

4.3.6 drpai_inout_t

Type	Member name	Overview
char	name[DRPAI_MAX_NODE_NAME]	"name" attribute of DRP-AI input in prepost_.yaml example : input_to_pre: - name: "camera_data"1 format: "YUY2" order: "HWC" shape: [480, 640, 2] type: "uint8"
drpai_data_t	data	DRP-AI input address and size. This area must be contiguous memory in physical memory.
drpai_data_t	obj	Address and size of drp_param.bin Refer to 4.3.1 for the stored values.

4.4 Global Variables and Constants

4.4.1 Global Variables

There are no global variables for this module.

4.4.2 Global Constants

Table 4-3 to Table 4-5 shows the global constants used by DRP-AI driver.

Table 4-3 Array element of argument proc of ioctl(DRPAI_START) function

Global Constant Name	Value	Remark
DRPAI_INDEX_INPUT	0	Input image data
DRPAI_INDEX_DRP_DESC	1	DRP descriptor
DRPAI_INDEX_DRP_CFG	2	DRP configuration data
DRPAI_INDEX_DRP_PARAM	3	DRP parameter
DRPAI_INDEX_AIMAC_DESC	4	AIMAC descriptor (aimac_desc.bin)
DRPAI_INDEX_WEIGHT	5	Weight data
DRPAI_INDEX_OUTPUT	6	Output data of DRP-AI
DRPAI_INDEX_AIMAC_CMD	7	AIMAC descriptor (aimac_cmd.bin)
DRPAI_INDEX_AIMAC_PARAM_DESC	8	AIMAC descriptor (aimac_param_desc.bin)
DRPAI_INDEX_AIMAC_PARAM_CMD	9	AIMAC descriptor (aimac_param_cmd.bin)

Table 4-4 DRP-AI status

Global Constant Name	Value	Remark
DRPAI_STATUS_INIT	0	DRP-AI driver initialization completed
DRPAI_STATUS_IDLE	1	AI inference can be started
DRPAI_STATUS_RUN	2	AI inference processing in progress

Table 4-5 DRP-AI error information

Global Constant Name	Value	Remark
DRPAI_ERRINFO_SUCCESS	0	No abnormality.
DRPAI_ERRINFO_DRP_ERR	-1	An error occurred in DRP This error occurs when the following files are invalid.* drp_desc.bin, drp_cfg.mem, drp_param.bin
DRPAI_ERRINFO_AIMAC_ERR	-2	An error occurred in AI-MAC This error occurs when the following files are invalid.* aimac_desc.bin
DRPAI_ERRINFO_RESET	-3	DRP-AI has been reset.

* If an error occurs, check the following three points.

Check if the Address map definition (addrmap_in_*.yaml) set in the DRP-AI Translator i8 and the physical address of the file placed in the memory are the same.
Check if the memory where the file is placed is within the DRP-AI memory area.
Make sure that the file placed in the memory matches the file generated by the DRP-AI Translator i8.

4.5 Operation

4.5.1 Single process / thread

Below is a flowchart of a DRP-AI driver basic usage example.

Figure 4-1 DRP-AI control

Figure 4-2 Deploy DRP-AI Object files in memory

Figure 4-3 Read AI inference results from memory

Figure 4-4 Deploy drp_param_info.txt into "/dev/drpai0"

4.5.2 Multi process / thread

Figure 4-5 is a flowchart of a DRP-AI driver at multi process / threads (2 process / threads) example.
The DRP-A driver can be called from multiple processes and threads. However, one open corresponds to one AI model. Also, AI inference cannot be run at the same time.

This is a sample code when ERESTART occurs.

#include <errno.h>
errno = 0;
ret = <DRP-AI Driver API function>;
if((-1 == ret) && (ERESTART == errno))
{
    do
    {
        ret = 0;
        errno = 0;
        ret = <DRP-AI Driver API function>;
    } while((-1 == ret) && (ERESTART == errno))
}

5. Kernel Settings

5.1 Target Files

5.2 Kernel configuration

To enable this module, set as shown in Figure 5-1 in the kernel configuration.

5.3 Device Tree

Device tree settings are shown as follows.

5.3.1 DRP-AI part of r9a09g057.dtsi

        drpai0: drpai@16800000 {
            compatible = "renesas,rzv2h-drpai";
            reg = <0 0x17000000 0 0x1000000>,
                  <0 0x16800000 0 0x400000>;
            interrupts = <GIC_SPI 914 IRQ_TYPE_LEVEL_HIGH>,
                         <GIC_SPI 915 IRQ_TYPE_LEVEL_HIGH>,
                         <GIC_SPI 916 IRQ_TYPE_LEVEL_HIGH>,
                         <GIC_SPI 917 IRQ_TYPE_LEVEL_HIGH>;
            resets = <&cpg R9A09G057_DRPAI_ARESETN>;
            status = "disabled";
        };

5.3.2 DRP-AI part of r9a09g057h4-evk.dts

        drp_reserved: DRP-AI@240000000 {
            reusable;
            reg = <0x2 0x40000000 0x0 0x20000000>;
        };

        &drpai0 {
            memory-region = <&drp_reserved>;
            status = "okay";
        };

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
img		img
installation		installation
meta-rz-features/meta-rz-drpai		meta-rz-features/meta-rz-drpai
LICENSE		LICENSE
README.md		README.md

License

renesas-rz/rzv2h_drp-ai_driver

Folders and files

Latest commit

History

Repository files navigation

RZ/V2H DRP-AI Driver

License

Supported Device

Overview

Related Documents

Restrictions

Installation

Contents

1. Functions

2. Terminology

3. Module Configuration

3.1 Module Configuration

3.2 State Transition Diagram

4. APIs

4.1 Device Node

4.2 External Function

[Overview]

[Function Name]

[Calling format]

[Argument]

[Error number]

[Feature]

[Remark]

4.2.1 open

[Overview]

[Function Name]

[Calling format]

[Arguments]

[Returns]

[Error number]

[Feature]

[Remark]

4.2.2 close

[Overview]

[Function Name]

[Calling format]

[Arguments]

[Returns]

[Error number]

[Feature]

[Remark]

4.2.3 write

[Overview]

[Function Name]

[Calling format]

[Arguments]

[Returns]

[Error number]

[Feature]

[Remark]

4.2.4 read

[Overview]

[Function Name]

[Calling format]

[Arguments]

[Returns]

[Error number]

[Feature]

[Remark]

4.2.5 ioctl(DRPAI_SET_ADRCONV)

[Overview]

[Function Name]

[Calling format]

[Arguments]

[Return]

[Error number]

[Feature]

[Remark]

4.2.6 ioctl(DRPAI_ASSIGN)

[Overview]

[Function Name]

[Calling format]

[Arguments]

[Returns]