Skip to content

Commit

Permalink
1323 btb
Browse files Browse the repository at this point in the history
  • Loading branch information
bhjelstrom committed Apr 26, 2023
1 parent b3884ac commit c3d43d6
Showing 1 changed file with 17 additions and 3 deletions.
20 changes: 17 additions & 3 deletions docs/building_a_pipeline.rst
@@ -1,8 +1,20 @@
Getting Started with Neural Networks
==============================================================

.. _downloads page: https://limelightvision.io/pages/downloads

With Limelight's neural network pipelines, once-impossible computer vision challenges are now trivial. Learning-based vision already plays an enormous role in bleeding-edge robots and self-driving vehicles, so we are
excited to bring this technology to FIRST students.
excited to bring this technology to FIRST students.

Check out 2023 World Champion 1323's use of Limelight's Neural Network pipeline:


.. raw:: html

<iframe width="100%" height="500px" src="https://www.youtube.com/embed/hz3MolcRe2M?start=463" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>


Download pretrained neural networks from our `downloads page`_.

In FRC, teams have always wanted to track game pieces on the field during the autonomous and teleoperated periods.
Using Limelight's "Neural Detector" pipeline, teams are able to track pieces just like any other target with zero tuning.
Expand All @@ -14,7 +26,7 @@ A Limelight pointed inside a robot could run a classifier trained to determine o
Neural Detector and Classifier networks require the addition of a Google Coral USB accelerator. The Google Coral Accelerator is an ASIC (application specific integrated circuit)
that is purpose-built for neural network inference. You can think of the term "inference" as "execution" or "running data through the neural network and producing an output".

If you are interested in building a deeper understanding, we recommend starting with this video from 3blue1brown:
If you are interested in building a deeper understanding of machine learning, we recommend starting with this video from 3blue1brown:
https://www.youtube.com/watch?v=aircAruvnKk

Programmers can learn more in a hands-on fashion with the following book:
Expand All @@ -28,13 +40,15 @@ Neural Detector Pipeline

To get started, ensure your Google Coral is plugged into the USB-A port on your Limelight.

Change "Pipeline Type" to "Neural Detector" to start running inference on the built-in test model. Game-specific models should arrive mid-season.
Change "Pipeline Type" to "Neural Detector" to start running inference on the built-in test model. Download pretrained neural networks from our `downloads page`_, and upload it to begin tracking game pieces.

Change the "confidence threshold" slider to adjust the required confidence for a successful detection. All results are posted over JSON, but we recommend using the built-in sorting interface
to optimize for a single target which will be represented by networktables values "tx," "ty," "ta," and "tclass."

Change the crop window to easily ignore objects outside of the desired detection zone.



----------


Expand Down

0 comments on commit c3d43d6

Please sign in to comment.