diff --git a/_toc.yml b/_toc.yml index 48d61e77f..829e5d7aa 100644 --- a/_toc.yml +++ b/_toc.yml @@ -9,7 +9,7 @@ parts: - file: docs/installation - file: docs/recipes/installTips - file: docs/docker -- caption: User Guides +- caption: Main User Guides chapters: - file: docs/standardDeepLabCut_UserGuide - file: docs/maDLC_UserGuide @@ -36,6 +36,7 @@ parts: - file: docs/recipes/MegaDetectorDLCLive - caption: Cookbook chapters: + - file: docs/tutorial - file: docs/convert_maDLC - file: docs/recipes/io - file: docs/recipes/nn @@ -47,6 +48,13 @@ parts: - file: docs/recipes/flip_and_rotate - file: docs/recipes/pose_cfg_file_breakdown - file: docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook +- caption: Beginner's Guide to DeepLabCut in the GUI + chapters: + - file: docs/beginners-guide + - file: docs/manage-project + - file: docs/labelling + - file: docs/Training-Evaluation + - file: docs/video-analysis - caption: DeepLabCut Benchmark chapters: - file: docs/benchmark diff --git a/docs/Training-Evaluation.md b/docs/Training-Evaluation.md new file mode 100644 index 000000000..52c4f9b2a --- /dev/null +++ b/docs/Training-Evaluation.md @@ -0,0 +1,50 @@ +# Training and Evaluation in the GUI + +## Your Training Dataset + +Before training your model, the first step is to assemble your training dataset. + +- **Create Training Dataset:** Move to the corresponding tab and click **`Create Training Dataset`**. For starters, the default settings will do just fine. While there are more powerful models and data augmentations you might want to consider, you can trust that for most projects the defaults are an ideal place to start. + +> 💡 **Note:** This guide assumes you have a GPU on your local machine. If you're CPU-bound and finding training challenging, consider using Google Colab. Our [Colab Guide](https://colab.research.google.com/github/DeepLabCut/DeepLabCut/blob/master/examples/COLAB/COLAB_YOURDATA_TrainNetwork_VideoAnalysis.ipynb) can help you get started! + +## Kickstarting the Training Process + +With your training dataset ready, it's time to train your model. + +- **Navigate to Train Network:** Head over to the **`Train Network`** tab. +- **Set Training Parameters:** Here, you'll specify: + - **`Display iterations/epochs`:** To specify how often the training progress will be visually updated. Note that our TensorFlow models are "iterations" while PyTorch is epochs. + - **`Maximum Iterations/epochs`:** Decide how many iterations to run. For TensorFlow models for a quick demo, 10K is great. For PyTorch models, 200 epochs is fine! + - **`Number of Snapshots to keep`:** Choose how many snapshots of the model you want to keep, **`Save iterations`:** and at what iteration intervals they should be saved. +- **Launch Training:** Click on **`Train Network`** to begin. + +You can keep an eye on the training progress via your terminal window. This will give you a real-time update on how your model is learning (added bonus of the PyTorch model is it also shows you evaluation metrics after each epoch!). + +![DeepLabCut Training in Terminal with TF](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779598041-DC8UJA2NXJXG65ZWJH1O/training-terminal.png?format=500w) + +## Evaluate the Network + +After training, it's time to see how well your model performs. + +### Steps to Evaluate the Network + +1. Find and click on the **`Evaluate Network`** tab. +2. **Choose Evaluation Options:** + - **Plot Predictions:** Select this to visualize the model's predictions, similar to standard DeepLabCut (DLC) evaluations. + - **Compare Bodyparts:** Opt to compare all the bodyparts for a comprehensive evaluation. +3. Click the **`Evaluate Network`** button, located on the right side of the main window. + +>💡 Tip: If you wish to evaluate all saved snapshots, go to the configuration file and change the `snapshotindex` parameter to `all`. + + +### Understanding the Evaluation Results + +- **Performance Metrics:** DLC will assess the latest snapshot of your model, generating a `.CSV` file with performance metrics. This file is stored in the **`evaluate network`** folder within your project. + + +![Combined Evaluation Results in DeepLabCut](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779617667-0RLTM9DVRALN9YIKSHJZ/combined-evaluation-results.png?format=750w)) +- **Visual Feedback:** Additionally, DLC creates subfolders containing your frames overlaid with both the labeled bodyparts and the model's predictions, allowing you to visually gauge the network's performance. + +![Evaluation Example in DeepLabCut](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779623162-BFDAW37B9TO94EGME2O5/check-labels.png?format=500w)) + diff --git a/docs/beginners-guide.md b/docs/beginners-guide.md new file mode 100644 index 000000000..1fccb78ea --- /dev/null +++ b/docs/beginners-guide.md @@ -0,0 +1,99 @@ +# Using DeepLabCut + +## Installation + +Before you begin, make sure that DeepLabCut is installed on your system. For detailed installation instructions, refer to the [Installation Guide](https://deeplabcut.github.io/DeepLabCut/docs/installation.html). + +## Starting DeepLabCut +>### 🔔 Reminder: How to Open a Terminal +> +> - **Windows:** +> - Use the Start menu to search for 'Anaconda Command Prompt' if you are using Miniconda, or 'Command Prompt' if not. +> +> - **Linux:** +> - Press `Ctrl + Alt + T` to open a new terminal. +> +> - **Mac:** +> - Press `Cmd + Space` and search for 'Terminal'. + +### Activating DeepLabCut Environment + +If you have installed DeepLabCut via conda, activate the environment with the following command: + +```bash +conda activate DEEPLABCUT +``` +>**⚠️ Attention macOS M1 Users:** +>
+> +>🍏 If you are on a macOS with an M chipset, please use the DEEPLABCUT_M1 conda file and then activate: +>```bash +>conda activate DEEPLABCUT_M1 + +## Launching the DeepLabCut GUI +In the terminal, enter: +```bash +python -m deeplabcut +``` +This will open the DeepLabCut App (note, the default is dark mode, but you can click "appearance" to change: + +![DeepLabCut GUI Screenshot](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779625875-5UHPC367I293CBSP8CT6/GUI-screenshot.png?format=500w) + +> 💡 **Note:** For a visual guide on navigating through the DeepLabCut GUI, check out our [YouTube tutorial](https://www.youtube.com/watch?v=tr3npnXWoD4). + +## Starting a New Project + +### Navigating the GUI on Initial Launch + +When you first launch the GUI, you'll find three primary main options: + +1. **Create New Project:** Geared towards new initiatives. A good choice if you're here to start something new. +2. **Load Project:** Use this to resume your on-hold or past work. +3. **Model Zoo:** Best suited for those who want to explore Model Zoo. + +#### Commencing a Your Work: + +- For a first-time or new user, please click on **`Start New Project`**. + +## 🐾 Steps to Start a New Project + +1. **Launch New Project:** + - When you start a new project, you'll be presented with an empty project window. In DLC3+ you will see a new option "Engine". + - We recommend using the PyTorch Engine: + + ![DeepLabCut Engine](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717780414978-17LOVBUJ8JR102QVSFDY/Screen+Shot+2024-06-07+at+7.13.14+PM.png?format=1500w)) + +2. **Filling in Project Details:** + - **Naming Your Project:** + - Give a specific, well-defined name to your project. + + > **💡 Tip:** Avoid empty spaces in your project name. + + - **Naming the Experimenter:** + - Fill in the name of the experimenter. This part of the data remains immutable. + +3. **Determine Project Location:** + - By default, your project will be located on the **Desktop**. + - To pick a different home, modify the path as needed. + +4. **Multi-Animal or Single-Animal Project:** + - Tick the 'Multi-Animal' option in the menu, but only if that's the mode of the project. + - Choose the 'Number of Cameras' as per your experiment. + +5. **Adding Videos:** + - First, click on **`Browse Videos`** button on the right side of the window, to search for the video contents. + - Once the media selection tool opens, navigate and select the folder with your videos. + + > **💡 Tip:** DeepLabCut supports **`.mp4`**, **`.avi`**, and **`.mov`** files. + - A list will be created with all the videos inside this folder. + - Unselect the videos you wish to remove from the project. + +6. **Create your project:** + - Click on **`Create`** button on the bottom, right side of the main window. + - A new folder named after your project's name will be created in the location you chose above. + + +### 📽 Video Tutorial: Setting Up Your Project in DeepLabCut + +![DeepLabCut Create Project GIF](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779616437-30U5RFYV0OY6ACGDG7F4/create-project.gif?format=500w) + diff --git a/docs/labelling.md b/docs/labelling.md new file mode 100644 index 000000000..d3180bfa2 --- /dev/null +++ b/docs/labelling.md @@ -0,0 +1,67 @@ +# Labeling GUI + +## Selecting Frames to Label + +In DeepLabCut, choosing the right frames for labeling is a key step. The trick is always to select the MOST DIVERSE data you can that your model will see. That means good lighting, bad lighting, anything you want to throw at it. So, first, pick a range of diverse videos! Then, we will help you pick frames. You've got two easy ways to do this: + +1. **Let DeepLabCut Choose:** DeepLabCut can extract frames automatically for you. It's got two neat ways to do that: + - **Uniform:** This is like taking a snapshot at regular time intervals. + - **K-means clustering:** This one applies k-means and picks images from different clusters. This is typically better, as it gives you a variety of actions and poses. Note, as it is a clustering tool, it will mis rare events, so ideally run this step, then perhaps consider running the mannual GUI to get some rare frames! You can do both within DLC. + +2. **Pick Frames Yourself:** Just like flipping through a photo album, you can go through your video and pick the frames that catch your eye - this is great for finding rare frames. Choose the **`manual`** extraction method. + +#### Here's how to get started: + +- **Step 1:** Click on **`automatic`** in the frame selection area. +- **Step 2:** Choose **`k-means`** for some variety. +- **Step 3:** Hit the **`Extract Frames`** button, usually found at the bottom right corner. + +By default, DeepLabCut will grab 20 frames from each of your videos and put them into sub-folders, per video, under **labeled-data** in your project. Now, you're all set to start labeling! + +## Labeling Your First Set of Frames in DeepLabCut + +Alright, you've got your extracted frames ready. Now comes the labeling! + +### Entering the Label Frames Area + +- **Click on `Label Frames`:** This takes you straight to where your frames are, sorted in the **labeled-data** folder, each video in its own sub-folder. +- **Open a Folder:** Click on the first one to start, and then click **`open`**. + +### The napari DeepLabCut Labeler + +- **Plugin Window Opens:** As soon as you click **`open`**, the napari DeepLabCut plugin window appears, your main stage for labeling. +- **Tutorial Popup:** A quick tutorial window shows up. It's a brief guide, so give it a look to understand the basics. + +![Labeling Frames in DeepLabCut using Napari Interface](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779602092-LVR2TI6OADSHEYRCGS6F/labeling-napari.png?format=500w)) + +### Labeling Setup + +- **Frames on Display:** Your frames are lined up in the middle, with a slider below to shuffle through them. +- **Tools and Keypoints:** To the bottom right, you find a list of bodyparts from your configuration. On the top left, all your labeling tools are ready. + +### The Labeling Process + +- **Start with `Add points`:** Click this to begin placing keypoints on your first frame. If you can't see a bodypart, just move to the next one. +- **Navigate Through Frames:** Use the slider to go from one frame to the next after you're done labeling. +- **Save Progress:** Remember to save your work as you go with **`Command and S`** (or **`Ctrl and S`** on Windows). + +> 💡 **Note:** For a detailed walkthrough on using the Napari labeling GUI, have a look at the [DeepLabCut Napari Guide](https://deeplabcut.github.io/DeepLabCut/docs/napari_GUI.html). Additionally, you can watch our instructional [YouTube video](https://www.youtube.com/watch?v=hsA9IB5r73E) for more insights and tips. + + +### Completing the Set + +Work through all the frames in the first folder. Then, proceed to the next, continuing this way until each folder in your **labeled-data** directory is done. + +## Checking Your Labels + +After you've labeled all your frames, it's important to ensure they're accurate. + +### How to Check Your Labels + +- **Return to the Main Window:** Once you're done with labeling, head back to DeepLabCut's main window, and click on **`Check Labels`**. +- **Review the Labeled Folders:** The system will have created new folders for each labeled set inside your labeled-data folder. These folders contain your original frames overlaid with the keypoints you've added. + +![Checking Labels in DeepLabCut](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779615252-6BNW661XB2ULH85RTAD3/evaluation-example.png?format=500w) + +Take the time to go through each folder. Accurate labels are key. If there are mistakes, the model might learn incorrectly and mislabel your videos later on. It's all about setting the right foundation for accurate analysis. + diff --git a/docs/manage-project.md b/docs/manage-project.md new file mode 100644 index 000000000..e2dc1d89f --- /dev/null +++ b/docs/manage-project.md @@ -0,0 +1,43 @@ +# Managing Your DeepLabCut Project + +## Editing the Configuration File + +After creating your DeepLabCut project, you'll return to the main GUI window, where you'll start managing your project from the **Project Management Tab**. + +### Accessing the Configuration File + +- **Locate the Configuration File:** At the top of the main window, you'll find the file path to the configuration file. +- **Edit the File:** Click on **`Edit config.yaml`**. This action allows you to: + - Define the bodyparts you wish to track. + - Outline the skeleton structure. + +A **`Configuration Editor`** window will open, displaying all the configuration details. You'll need to modify some of these settings to align with your research requirements. + +### Steps to Edit the Configuration + +#### 1. Defining Bodyparts + +- **Locate the Bodyparts Section:** In the Configuration Editor, find the **`bodyparts`** category. +- **Modify the List:** Click on the arrow next to **`bodyparts`** to expand the list. Here, you can: + - Update the list with the names of the bodyparts relevant to your study. + - Add more entries by right-clicking on a row number and selecting **`Insert`**. + + +![Editing Bodyparts in DeepLabCut's Config File](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779624617-CIVZCM23U69NYK9BO3GY/bodyparts.png?format=500w) + + +#### 2. Defining the Skeleton + +- **Navigate to the Skeleton Section:** Scroll down to the **`skeleton`** category. +- **Adjust the Skeleton List:** Click on the arrow to expand this section. You can then: + - Update the pairs of bodyparts to define the skeleton structure of your model. + +![Defining the Skeleton Structure in Config File](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779598505-HQNECHIKSQ6XL033JX8M/skeleton.png?format=500w)) + +> 💡 **Tip:** If you're new to DeepLabCut, spend some time visualizing how the chosen bodyparts can be connected effectively to form a coherent skeleton. + +### Saving Your Changes + +- **Save the Configuration:** Once you're satisfied with the modifications, click **`Save`**. This will store your changes and return you to the main GUI window. + + diff --git a/docs/video-analysis.md b/docs/video-analysis.md new file mode 100644 index 000000000..3b4a489b9 --- /dev/null +++ b/docs/video-analysis.md @@ -0,0 +1,30 @@ +# Video Analysis with DeepLabCut + +After training and evaluating your model, the next step is to apply it to your videos. + +## How to Analyze Videos + +1. **Navigate to the 'Analyze Videos' Tab:** Begin applying your trained model to video data here. +2. **Select Your Video Format and Files:** + - **Choose Video Format:** Pick the format of your video (`.mp4`, `.avi`, or `.mov`). + - **Select Videos:** Click **`Select Videos`** to find and open your video file. +3. **Start Analysis:** Click **`Analyze`**. The analysis time depends on video length and resolution. Track progress in the terminal or Anaconda prompt. + +### Reviewing Analysis Results + +- **Find Results in Your Project Folder:** After analysis, go to your project's video folder. +- **Analysis Files:** Look also for a `.metapickle`, an `.h5`, and possibly a `.csv` file for detailed analysis data. +- **Review the Plot Poses Subfolder:** This contains visual outputs of the video analysis. + +![Plot poses](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1717779600836-YOWM5T2MBY0JN1LB537B/plot-poses.png?format=500w) + +## Creating a Labeled Video + +1. **Go to 'Create Labeled Video' Tab:** The previously analyzed video should be selected. +2. If not already selected, choose your video. +3. Click **`Create Videos`**. + +### Viewing the Labeled Video + +- Your labeled video will be in your video folder, named after the original video plus model details and 'labeled'. +- Watch the video to assess the model's labeling accuracy.