OctoPNP

OctoPNP is an OctoPrint plugin for camera assisted pick 'n place applications in 3D-Printers.

Introduction

OctoPNP is an extension that allows Octoprint to control printers and similar devices with additional hardware for handling of SMD-parts. It currently requires the following hardware extensions:

• A Tray consisting of a grid of boxes to store SMD parts in a defined position
• A head camera to locate the exact part position on the tray
• A (second) bed camera to precisely align the parts during the placing operation
• A vacuum nozzle to grip parts

Installation

Prerequirements

To achieve higher compatibility and modularity, OctoPNP doesn't acces the cameras directly. Every time an image is required, OctoPNP executes a user defined script which must be adapted for every installation according to the deployed camera setup. OctoPNP expects a set of correctly cropped and rotated images after executing the script. Filenames and path for images and script must be set in the settings dialog. The calibration feature supports direct camera access via http requests to generate a live view e.g. via mjpg_streamer.

OpenCV on embedded hardware (Raspberry Pi)

OctoPNP requires numpyand OpenCV, both are listed in the requirements.txt and should be automatically resolved via pip during installation. However, on many embedded systems, including the RasPi, opencv-python is not available. To solve this:

• Remove (or comment) the opencv-python line in requirements.txt
• Manually install OpenCV on your system (official tutorial). This should build OpenCV packages for Python 2 and 3 if both versions are installed on the system.
• If OctoPrint is installed in a virtual environment (as recommended), at least in some cases the OpenCV library must be linked (or copied) into the venv like this:

ln -s /usr/local/lib/python3.6/dist-packages/cv2/python-3.6/cv2.cpython-36m-arm-linux-gnueabihf.so ~/OctoPrint/venv3/lib/python3.6/site-packages/

Installing the package

The plugin itself can be installed as any regular python package: pip install https://github.com/platsch/OctoPNP/archive/master.zip

Make sure you use the same Python environment that you installed OctoPrint under, otherwise the plugin won't be able to satisfy its dependencies. Further information can be found in the Octoprint documentation OctoPNP requires numpy and opencv2. Try installing those packages by hand if automatic dependency resolving failes.

Data Format

The information for the PNP system is integrated into normal gcode files as a commented XML flavor somewhere in the gcode. OctoPNP extracts the required information automatically everytime a gcode file is loaded in Octoprint. Example files are included in the utils folder. The XML structure is still under development and will probably change during the next months. The comment-semicolon causes Octoprint to ignore the SMD-data.

;normal printing commands
G1 X10 F3000
G1 Y10 F3000
G1 Z5 F3000

;pick part nr 2 and place it to the destination given in the XML description
M361 P2

;description of the actual parts
;<object name="simple_lines">
;<part id="1" name="ATTINY">
;  <position box="1"/>
;  <size height="1"/>
;  <shape>
;    <point x="-2.6" y="-3.4"/>
;    <point x="-2.6" y="3.4"/>
;    <point x="2.6" y="3.4"/>
;    <point x="2.6" y="-3.4"/>
;  </shape>
;  <pads>
;    <pad x1="-2.155" y1="-4.254" x2="-1.655" y2="-2.054"/>
;    <pad x1="-0.895" y1="-4.254" x2="-0.395" y2="-2.054"/>
;    <pad x1="0.375" y1="-4.254" x2="0.875" y2="-2.054"/>
;    <pad x1="1.645" y1="-4.254" x2="2.145" y2="-2.054"/>
;    <pad x1="-2.155" y1="2.054" x2="-1.655" y2="4.254"/>
;    <pad x1="-0.885" y1="2.054" x2="-0.385" y2="4.254"/>
;    <pad x1="0.385" y1="2.054" x2="0.885" y2="4.254"/>
;    <pad x1="1.655" y1="2.054" x2="2.155" y2="4.254"/>
;  </pads>
;  <destination x="100" y="90" z="2.75" orientation="90"/>
;</part>
;
;<part id="2" name="RES_1206">
;  <position box="2"/>
;  <size height="1"/>
;  <shape>
;    <point x="-1.6891" y="-0.8763"/>
;    <point x="-1.6891" y="0.8763"/>
;    <point x="1.6891" y="0.8763"/>
;    <point x="1.6891" y="-0.8763"/>
;  </shape>
;  <pads>
;    <pad x1="0.622" y1="-0.9015" x2="2.222" y2="0.9015"/>
;    <pad x1="-2.222" y1="-0.9015" x2="-0.622" y2="0.9015"/>
;  </pads>
;  <destination x="108.304" y="90.674" z="2.75" orientation="0"/>
;</part>
;</object>

Configuration

Good configuration and calibration of the printer is absolutely crucial to successfully use multiple extruders and cameras.

Tray

The tray-position is set in relation to the primary extruder (usually the plastic extruder). To find the position, move the primary extruder to the bottom left corner of the tray and note the position. A negative Z-offset can be used if the tray is lower than the printbed.

Extruders / Nozzles

The minimal setup typically requires 3 nozzles / tools:

• The plastic extruder to print the object (primary extruder)
• An Extruder for liquids to print the conductive wires
• A vacuum nozzle to grip SMD-parts

Offsets between tools can be handled by the slicer, by OctoPNP or by the printer firmware. Having the firmware manage all offsets is generally highly encouraged to avoid frequent G-code generation, since for most setups the offset has to be re-calibrated at least after every refill of the extruder. The offset for the liquid extruder must be handled by the slicer or by the firmware, OctoPNP is not aware of this extruder. The offset for the vacuum nozzle must be handled by OctoPNP or by the firmware. OctoPNP provides a calibration tool to quickly correct the firmware offsets.

For toolchanger setups all offsets should generally be measured in relation to a fixed point at the toolhead. This way, once a tool is mounted, the bed camera and tray positions are automatically in the correct frame. If the head camera is mounted to the toolhead instead of being an individual tool, all images should probably be taken with no tool attached (extruder T-1).

For setups with all tools mounted to a single carriage offsets should generally be measured in relation to the first extruder (T0).

Cameras

Position

The camera position is relative to the primary nozzle for the head camera (this camera is mounted somewhere next to the extruder at the X-axis and follows the printheads movements). The bed camera is mounted to the printers frame, the camera position is absolute and again relative to the primary extruder.

• Focus distance defines the printbead Z-position for the optimal focus point.
• Focus axis sets the name of the axis to move the camera. Default is Z for a camera mounted to the printbed. RepRapFirmware supports ABCUV and W to address additional axis. Leave blank if the camera is entirely static and the printhead cannot be moved with respect to the camera in the vertical dimension.
• The px/mm value is used by the image processing to measure offsets and only required for the bed camera, since the head camera utilizes the known box size for on demand calibration.
• The binarization threshold is used by the image processing to seperate object and background. The value depends on the camera and illumination, currently there is no good calibration method.
• Image path and grab script path denote the locations of script and preprocessed images as described in the Prerequirements section.
• http-path can be used if the camera provides an http interface (e.g. with mjpg_streamer). This is only used to generate a live view during calibration, not for actual pick and place operations. Leave empty to use the script base method for calibration. Note that for technical reasons only still-images can be used, mjpg streams or other video formats are not supported!

Image acquisition

The cameras folder contains simple scripts to fetch single images from

• any camera which can be accessed with an http get command (http)
• Basler industrial cameras supporting the pylon framework (pylon, tested with Pilot and Ace models)

Calibration wizard

The calibration "wizard" is an experimental set of tools to quickly achieve several calibration tasks with high precision.

• Calibration of extruders is done by printing a defined pattern (e.g. a cross) on the printbed. To calibrate an offset, the head camera moves to the printed pattern (position given by user) and the user uses the arrows keys to correct possible offsets by using the camera image superimposed by cross hairs. Offsets between extruders can be directly saved to the eeprom if a repetier firmware is used.
• The tray position can be calibrated similar by moving the camera to the corners and correcting possible offsets with manual key control.
• The bed camera can be calibrated against any calibrated extruder that is able to move over the camera. While some extruders contain sticky, liquid content, it might be a good idea to cover the camera with some glass.
• With a calibrated bed camera any extruder that is able to move over the camera, including the vacuum nozzle, can be calibrated.

Development / Code Analyzer

The included static code analyzer scripts might help to find potential errors and give suggstions for possible better solutions.

Preparing for Windows

Installing shellcheck:

1. See shellcheck

Installing pylint:

1. Follow this instruction to install python and pip
2. You may add the python*/Scripts folder to the Path ENV
3. pip install pylint

Installing clang-tidy:

1. See clang-tidy

Enable git hook

For automatic code analysis before every commit, add this hook to your local git installation by symlinking it:

ln -s ../../.ci/pre-push-hook .git/hooks/pre-commit