Thermal Finger Swipe Pressure Detection
This work was completed for my Clarkson University undergraduate Honors Thesis.
data/user/video.mov contains the original videos for each user
data/user/segments/video.mov contains the video segments for each user
data/user/frames/material/pressure/image.jpg contains raw grayscale frames
data/user/swipe_frames/material/pressure/image.jpg contains processed frames
- ffmpeg was used to crop the videos in time to avoid dynamic rescaling occurring at the start of videos. The syntax for performing this with minimal information loss due to compression is as follows:
> ffmpeg -i input.mov -ss start/fps -t (end-start)/fps -vf "crop=256:256:xstart:ystart" -c:v libx264 -crf 17 output.mov
- segment.py is called manually for each unsegmented video with the correct X0 and Y0 parameters to set the bounding boxes and capture swipe paths. Several swipes were removed after inspection due to flat-field-correction dropping frames. Bounding boxes and removed swipes are listed in bounding_boxes_and_removed_swipes.csv.
- extract_frames.py extracts a specified number of the heuristically-determined "most relevant" frames from each video segment.
- classification.ipynb is run on a Jupyter Notebook server using the GPU in Docker.
Running project in Jupyter Notebook on GPU
The current directory should contain both all Github code and a folder with the swipe data. First ensure that the code is up-to-date with the Github repo:
> git pull
Then rebuild the docker file "thesis" with the updated current directory:
> docker build -t thesis .
Run the Docker file with GPU backend, leaving port 8888 open for our Jupyter Notebook server and open up this instance in BASH:
> sudo docker run --runtime=nvidia -it -p 8888:8888 thesis:latest bash
Run the Jupyter Notebook server in the Docker container:
$ jupyter notebook --ip 0.0.0.0 --no-browser --allow-root
Access Jupyter Notebook from your host machine (outside the Docker container) by navigating to
http://localhost:8888/tree. The required session token should be listed in the terminal where you started the Jupyter Notebook server inside the Docker container.
To get a second Docker shell
List all currently running Docker containers
> docker container ls
Using the hash value listed next to thesis:latest, open up a shell in BASH:
> docker exec -t <hash> bash
You should see a new shell spawn.