Performance Tradeoffs

These results were obtained for the data collected during our hardware experiments. Different computation configurations can be selected by varying the parameters in the config file. We analyze four configurations:

• Default: nphi = 360 , ntheta = 180, sub_sample_channel_data = False.
• Low Res: nphi = 180 , ntheta = 90, sub_sample_channel_data = False.
• Sub-sample: nphi = 360 , ntheta = 180, sub_sample_channel_data = True.
• Low-and-Sub: nphi = 180 , ntheta = 90, sub_sample_channel_data = True.

For results using default configuration, please refer here

Accuracy comparison between different configuration

Groundtruth displacement was obtained using Optitrack motion capture cameras and estimated displacement using Intel T265 Tracking camera.

 

Experiment details	Default	Low Res	Sub-sample	Low-and-Sub
Total Samples	343	343	343	343
Rejected Samples	32	96	31	96
Percentage of Rejected Samples	9.32	27.9	9.0	27.9
Average profile variance	0.73	0.80	0.74	0.80

The performance of Low Res and Low-and-Sub configurations can potentially improve by modifying other configuration parameters like peak_radius. Additional results for one of the configurations are show below.

Results of 2D robot displacement using Low-and-Sub configuration

 

Runtime performance for different configurations

The runtime comparison for the WSR Toolbox to calculate AOA profile, using different configurations using a workstation and laptop (baseline) and the on-board computers (UP Squared Board, Intel NUC) are as follows:

Average Computation Time (sec)

Configuration	Processed Packet count	Workstation	Standard Laptop	UP-Squared board	Intel NUC8i3CYSN
Default	872	0.44	0.94	6.57	6.1
Low-Res	872	0.14	0.23	1.65	1.16
Sub-sample	436	0.23	0.47	3.20	2.28
Low-and-Sub	436	0.07	0.12	0.85	0.6

Average Memory Utilization (GB)

Configuration	Processed Packet count	Workstation	Standard Laptop	UP-Squared board	Intel NUC8i3CYSN
Default	872	4.81	3.89	3.82	3.82
Low-Res	872	2.35	1.43	1.35	1.35
Sub-sample	436	3.93	3.01	2.94	3.01
Low-and-Sub	436	2.12	1.21	1.13	1.2

Platform CPU Details:

1. Workstation ( 64 cores, AMD Ryzen Threadripper Pro 3995WX )
2. Standard Laptop ( 8 cores, Intel Core i7 10th Gen)
3. UP-Squared board ( 4 cores, Intel Pentium N4200)
4. Intel NUC8i3CYSN (2 cores, Intel core i3 8th Gen)

Note:

1. The runtime is calculated using all the available cores and not other process running on the system.
2. The runtime on Intel NUC for the Default configuration was found to be varying within +- 1.5 sec of the reported value.
3. The baseline systems are only used for offboard computation, while the onboard SBCs can also collect CSI data.

Results of 3D robot displacement using Low-and-Sub configuration

Coming soon!!