Artifact available at:

https://github.com/auditoryeye/auditoryeye_artifact

Paper reference:

@inproceedings{tu2023auditory,
  title={Auditory Eyesight: Demystifying $\mu$s-Precision Keystroke Tracking Attacks on Unconstrained Keyboard Inputs},
  author={Tu, Yazhou and Shan, Liqun and Hossen, Md Imran and Rampazzi, Sara and Butler, Kevin and Hei, Xiali}
  booktitle={Proceedings of the 2023 32nd USENIX Security Symposium, Anaheim, CA, USA},
  year={2023}
}

Data, Benchmark Results, and Software of Auditory Eyesight

Overview:

There is a lack of an existing reference study of acoustic side-channel keystroke attacks with publicly available datasets. To address this issue, we publish the dataset, benchmark results, and the software of Auditory Eyesight to reproduce the results.

This artifact is suitable for various research purposes. The dataset can be used to benchmark different acoustic-channel keyboard attack methods in the future. For example, future works can investigate integrating additional signal processing or other extracted features to improve the attack performance.

We organize the contents of the artifact in test cases. There are 11 folders in the repository. Each folder contains the data, benchmark results, and code. In each folder, there is a main.m file.

After downloading, the artifact can be evaluated after navigating to one of the folders and opening the main.m file in Matlab. Users can follow the Step-by-Step Instructions to start the Parallel Pool and then use the code.

Table of Contents

Supported Environments
Instructions to Start the Parallel Pool
Step-by-Step Instructions to Use the Code in the First Test
Experiment Section 4
Experiment User Study
Experiment Additional Test Case
User Typing Illustration

The results of each of the steps are illustrated. Users can try to introduce additional signal processing and feature extraction (such as FFT, MFCC) in the future to improve the results. The data and code can be accessed from the following links:

Links, Required minimum disk size, and Matlab computation time report (with dual E5-2683 V3 CPUs and 32-GB RAM):

Experiment	Quick Project Download Link	Project Main File	Disk Size	Computation time	Dataset
E1Apple	LinkE1 or Link	main.m	1.77 GB	255.688 s	Data Apple
E2Razor	LinkE2 or Link	main.m	1.76 GB	260.267 s	Data Razor
E3User01	LinkE3 or Link	main.m	8.61 GB	1190.288 s	Data User
E4Angle01	LinkE4 or Link	main.m	1.76 GB	103.066 s	Data A01
E5Angle02	LinkE5 or Link	main.m	1.75 GB	116.119 s	Data A02
E6Angle03	LinkE6 or Link	main.m	1.75 GB	188.849 s	Data A03
E7Angle04	LinkE7 or Link	main.m	1.75 GB	118.643 s	Data A04
E8Angle05	LinkE8 or Link	main.m	1.75 GB	116.372 s	Data A05
E9Distance1m	LinkE9 or Link	main.m	601 MB	59.129 s	Data D01
E10Distance2m	LinkE10 or Link	main.m	632 MB	57.629 s	Data D02
E11NLOSLaptop	LinkE11 or Link	main.m	608 MB	125.927 s	NLOS laptop

Description:

Downloading

Github may limit the file size and number downloaded per day. The zip file might also be truncated by Github because of this limit.

Here is a quick, reliable download link

Users may also use tools such as download-directory, GitZip, DownGit, to download the specific directory.

Additional Comment

The performance report is generated by Screenshots and Matlab Performance Reports

Supported Environments

1. The supported environment is Matlab (2020b or newer), which is compatible with Windows, Linux, and Mac. The system compatibility document can be found at:

https://www.mathworks.com/support/requirements/previous-releases.html

2. We utilized the parallel computing toolbox on Matlab to speed up the processing. We recommend using at least 32-GB RAM and 12 CPU Cores to support parallel computing. The detailed requirements of the Matlab parallel computing toolbox are available at: https://www.mathworks.com/support/requirements/parallel-computing-toolbox.html

3. We used 32-GB RAM and 2x E5-2683 v3 CPUs. The system is Windows 10 Pro (version 21H2). We attached a screenshot of the system specification.

4. The code does not rely on GPU.

---

Instructions to Start the Parallel Pool

Since our implementation utilized parallel computation, it is required to start the Matlab parallel pool to run the code.

1. The first step is to click the icon in the bottom-left corner:

2. The second step is to click Start Parallel Pool:

3. Alternatively, the Parallel Pool can be directly started by the command parpool:

There is a default parallel pool called local. It usually works without further configuration. If necessary, users may select pool size and cluster following this documentation:

https://www.mathworks.com/help/parallel-computing/run-code-on-parallel-pools.html

---

Step-by-Step Instructions to Use the Code in the First Test

1. Navigate to the 01_proofofconcept_multiround_apple_keys folder:

We can see that recording1_keys_interpolated folder does not contain computation results before running the code.

2. Evaluate the command run('YZProcessing04_tdoa.m');:

3. We can see that the results are written to the txt files in recording1_keys_interpolated folder:

4. Evaluate the command run('YZProcessing05_statistics_remoutlier.m');:

5. We can see the illustrated I-Round keystroke localization results:

6. Evaluate the command run('YZProcessing05_statistics_remoutlier.m');:

We can see the illustrated mean and standard deviation statistics of I-Round results:

7. Evaluate the command run('Statistics201_figure_2d_1round.m');:

We can see the 2D illustration of I-Round localization results:

8. Evaluate the command run('YZProcessing06_2ndround.m');:

We can see that the results are written to the txt files in recording1_keys_interpolated folder:

9. Evaluate the command run('YZProcessing07_2ndroundstatistics.m');:

We can see the illustrated B-Round keystroke localization results:

10. Evaluate the command run('Statistics101_round2.m');:

We can see the illustrated mean and standard deviation statistics of B-Round results:

11. Evaluate the command run('Statistics201_figure_2d_2round.m');:

We can see the 2D illustration of B-Round localization results:

12. Evaluate the command run('YZProcessing08_3rdround.m');:

We can see that the results are written to the txt files in recording1_keys_interpolated folder:

13. Evaluate the command run('YZProcessing09_3rdroundstatistics.m');:

We can see the illustrated T-Round keystroke localization results:

14. Evaluate the command run('Statistics101_round3.m');:

We can see the illustrated mean and standard deviation statistics of T-Round results:

15. Evaluate the command run('YZProcessing10_4thround.m');:

We can see that the results are written to the txt files in recording1_keys_interpolated folder:

16. Evaluate the command run('YZProcessing11_4throundstatistics.m');:

We can see the illustrated C-Round keystroke localization results:

17. Similaryly, evaluate the commands run('YZProcessing12_5thround.m'); and run('YZProcessing13_5throundstatistics.m');

18. Evaluate the command run('Statistics101_round5.m');:

We can see the illustrated mean and standard deviation statistics of Final-Round results:

19. Evaluate the command run('Statistics201_figure_2d_5round.m');:

We can see the 2D illustration of Final-Round localization results:

20. Evaluate the command run('Statistics_accuracy_calculation.m');:

We can observe accuracies:

The detailed results are written into the keys_tdoa4_results.txt and keys_tdoa4_results_accuracy.txt files.

---

Dataset:

Data Apple
Data Razor
Data User
Data A01
Data A02
Data A03
Data A04
Data A05
Data D01
Data D02
NLOS laptop

---

Section 4

Required minimum disk space: 1.77 GB. Reference computation time with dual E5-2683 V3 CPUs and 32-GB RAM): 255.688 s

Data Apple

Ground Truth of 598 Keystrokes

Code

Localization Results of 598 Keystrokes

Ground Truth / Recovery of 598 Keystrokes

---

Data Razor

Ground Truth of 595 Keystrokes

Code

Localization Results of 595 Keystrokes

Ground Truth / Recovery of 595 Keystrokes

---

User Study

Data

Ground Truth of 2909 Keystrokes

Code

Localization Results of 2909 Keystrokes

Ground Truth / Recovery of 2909 Keystrokes

---

Additional test cases

Angle Test Case 1

Data

Ground Truth of 595 Keystrokes

Code

Localization Results of 595 Keystrokes

Ground Truth / Recovery of 595 Keystrokes

---

Angle Test Case 2

Data

Ground Truth of 594 Keystrokes

Code

Localization Results of 594 Keystrokes

Ground Truth / Recovery of 594 Keystrokes

---

Angle Test Case 3

Data

Ground Truth of 594 Keystrokes

Code

Localization Results of 594 Keystrokes

Ground Truth / Recovery of 594 Keystrokes

---

Angle Test Case 4

Data

Ground Truth of 594 Keystrokes

Code

Localization Results of 594 Keystrokes

Ground Truth / Recovery of 594 Keystrokes

---

Angle Test Case 5

Data

Ground Truth of 594 Keystrokes

Code

Localization Results of 594 Keystrokes

Ground Truth / Recovery of 594 Keystrokes

---

Long distance keystroke localization results

1m

Data

Ground Truth

Code

Localization Results

Results with 1m attack distance (Apple Magic keyboard)

---

2m

Data

Ground Truth

Code

Localization Results of 625 Keystrokes

Results with 2m attack distance (Apple Magic keyboard)

---

Non-line-of-sight (NLOS) laptop

Data

Ground Truth of 601 Keystrokes

Code

Localization Results of 601 Keystrokes

Ground Truth / Recovery of 601 Keystrokes

User Study Illustration

Typing addresses

Typing GPS coordinates

Typing dates

Typing passwords

Typing real-world texts (with punctuation, numbers, capital letters, typos)

Long distance keystroke localization results

Results with 1m attack distance (Apple Magic keyboard)

Results with 2m attack distance (Apple Magic keyboard)

Software

Test

Multi-Round Code

Benchmark Results

User364

Data

Ground Truth of 2910 Keystrokes

Ground Truth / Recovery of 2910 Keystrokes

Benchmark Results

User597

Data

Ground Truth of 2402 Keystrokes

Ground Truth / Recovery of 2402 Keystrokes