0.3.0

This release adds GPU acceleration for client side image processing and adds controls to track cells in live cell movies. Add track=true to any URL to display the tracking controls and add division labels to any project. Add download=true to a project URL to display a download button instead of a submit button. Downloading with track=true in the URL will download a .trk file and otherwise will download an .npz file.

There is also a new API route to add channels to an existing project. Posting to /api/raw/{projectID} with an attached .npz will add the array in the .npz as an additional channel for that project. Make sure that the attached array is 4-dimensional with axes order ZYXC and that there is only one channel in the attached array.

The release also improves the development environment in the repository by adding pre-commit hooks, updating the Python and node versions tested, and updating the Elastic Beanstalk support to Amazon Linux 2.

Wrap GPU.js kernel functions in templates @tddough98 (#299)

Our minified production code replaces expression like if (x) { y } to x && y, breaking how GPU.js compiled functions into a kernel. By wrapping the functions in ticks, they won't be minified and all the syntax is preserved for GPU.js when it compiles the function at runtime.

These issues on the GPU.js repo detail this:
https://github.com/gpujs/gpu.js/issues/152
gpujs/gpu.js#584

Fix canvas states for changing cursor @tddough98 (#298)

After changing the canvas in #294 to be easier to mock, the states changes slightly but were not updated in the Canvas component, so the cursor does not change to a grabbing hand as before. This updates the state names in the component.

Fixes for tracking before release @tddough98 (#297)

This adds in a couple of fixes, mostly for tracking jobs, including

• enabling uploading .trk from the homepage
• removing an extra FrameSlider above the image controls
• hiding the Frame label when FrameSlider is hidden
• fixing the colors provided to the Division components to be provided in hex format instead of a list of [r, g, b]

Fix GPU.js handling of WebGL @tddough98 (#296)

With the most recent pull request #294, I noticed that Firefox and Chrome handled the new canvas as expected, but Safari was not behaving correctly. It seems that that Safari only received WebGL2 support in Safari 15 released a few months ago with macOS 15: https://caniuse.com/webgl2.

I fixed the initialization of the GPU-accelerated contexts to check which context is available in the browser and set one or the other up as needed. I also had issues with the WebGL canvases clearing their data after being drawn, causing it to appear once and then to appear blank in later renders. To fix this, I drew the WebGL kernel output onto a separate canvas as soon as it is created, and this other canvas with a standard 2d context does not get cleared when drawn. This strategy works just as well for WebGL2, WebGL and canvas based kernels, so I implemented it as the standard approach regardless of WebGL support

Add tracking and QC available on label.deepcell.org to main branch @tddough98 (#282)

The deployed app on label.deepcell.org is ahead of the main branch as it contains tracking and quality control features.

Code to support tracking includes

• actions in label.py like add_daughter, remove_daughter, and replace_with_parent,
• tests for these actions
• Tracking UI components in visualizer/src/Controls/Tracking
• trackMachine.js to send events to edit the lineage
• toolMachine.js to switch between trackMachine to edit the lineage and segmentMachine to edit the segmentation

Code to support quality control includes

• QC components in visualizer/src/Controls/QualityControl
• qualityControlMachine to make judgments on projects

Other small improvements include

• adding ETags to routes to avoid unnecessarily resending data,
• cleaning label arguments for actions to be valid integers between 0 and a new label

Add prettier pre commit hook @tddough98 (#290)

I added our prettier setup for formatting the javascript side of the project to the pre-commit configuration and action. I also updated the configuration to be closer to prettier's recommended style, only overriding to use single quotes instead of double quotes as we do for the Python formatting as well. Specifically, I removed "semi": true, from .prettierrc because that's already the prettier default and I removed "arrowParens": "avoid", after reading prettier's reasoning here.

I ran the updated prettier config on all files, which is why there are so many changed files. The only changes of note are in .prettierrc and .pre-commit-config.yaml.

Add pre-commit hooks to automatically fix Python code @tddough98 (#289)

Here we enforce autoformatting of Python code with black.

There are three contenders for python autoformatters: autopep8, yapf, and black. Of these three, black is the most popular with over 23k stars on github. A number of high profile libraries are now using black, including Django, PyTest, tox, pandas, sqlalchemy, and virtualenv.

The main selling point of black is that it is very opinionated so that black makes all the decisions about code formatting. The only configuration I'm aware of is max line length (default is 88), and whether to replace single quotes with double quotes (default is to replace with double quotes). I left both of these on their defaults.

I've also added an action to run black on pull requests to enforce its usage. The action checks that the code is consistent with black, and if not, it adds a commit to the PR to autoformat the code.

Add testing for python 3.9 and 3.10 @tddough98 (#288)

In #287, we drop support for Python 3.6, but did not test for the newer Python releases 3.9 and 3.10. Here we add them to our test matrix to make sure our codebase works for these versions for when we'll need to switch to them down the road.

However, pytest-pep8 is incompatible with Python 3.10. Python 3.10 requires at least pytest 6.2.4 per this issue and pytest-pep8 is incompatible with pytest 6, as seen in this error message

Direct construction of Pep8Item has been deprecated, please use Pep8Item.from_parent.
See https://docs.pytest.org/en/stable/deprecations.html#node-construction-changed-to-node-from-parent for more details.

As pytest-pep8 is not an actively mantained project and the last release was in 2014, I switched to flake8 and pytest-flake8. Flake8 checks for pep8 as well as pyflakes and circular complexity, keeping the style checks and adding some error checks.

In summary, this PR contains:

• additions to tests.yaml to test Python 3.9 and 3.10
• switches out pep8 checks for flake8 checks
• fixes issues highlighted by flake8

Migrate Elastic Beanstalk to Python 3.8 running on Amazon Linux 2 @tddough98 (#287)

Our Elastic Beanstalk instances have been running on a deprecated platform Python 3.6 running on 64bit Amazon Linux. Here, we migrate to a supported platform Python 3.8 running on Amazon Linux 2, updating the EB configuration to work with the new platform.

This PR also drops support for Python 3.6, which reaches its end of life and will stop receiving security updates in 2 weeks on December 23 2021. I bumped all our dependencies and removed any references to Python 3.6. I did not add Python 3.9 and 3.10 to our testing workflow as there are only Python 3.7 and 3.8 platforms available on Elastic Beanstalk at the moment.

Add instructions on .env to README @tddough98 (#285)

Issue #284 had issues with setting environment variables. Adding some instructions to the readme on how to use the example .env.example will point users to the resources to configure their envvars correctly.

Add route to add channels to an existing project @tddough98 (#283)

Here we add an API route to add additional channels to a project. This supports displaying predictions from Impartial which creates probability maps from scribbles created in DeepCell Label, so these maps need to be added later in the project lifecycle. By showing the Impartial results in the same project, a user can compare their scribbles to Impartial predictions and update the scribbles to repeat the process.

Create an npz with the new channel with shape (frames, height, width, 1) and post the channel to the /api/raw/{projectId} route with the npz as an attached file.
For example, with curl this looks like

curl --location --request POST 'label.deepcell.org/api/raw/EXAMPLE-ID' --form 'file=@"newChannel.npz"'

Areas for improvement could include loading channels from more filetypes, specifying a flexible dimension order, and adding multiple channels with one request.

🚀 Features

Accelerate image processing on the client with GPU.js @tddough98 (#294)

Here, we improve the image processing and display for the DeepCell Label client with GPU acceleration. I revamped existing ____Canvascomponents to use a kernel compiled with GPU.js instead of processing image data on the CPU, resulting in large speedups for image processing, especially for larger images. For a 4.5 megapixel image, changing the range of a channel now take around 6ms to complete the render compared to about 80ms in our existing deployment.

Each Canvas is now explicitly composed into a single canvas, instead of implicitly composed with a stack of canvases in the same position on the page. This address issue #274.

I've also addressed #273 by sending some additional ADD_LAYER and REMOVE_LAYER events to parent actors to ensure that the relevant hooks update when the event occurs.

I removed routes and logic from the backend that are now exclusively handled by the client including

• creating a label image from the label array with a colormap
• creating outlines from the label array

🐛 Bug Fixes

Put a version ceiling on coveralls, 3.3.0 has a bug preventing builds. @willgraf (#277)

What

• Pin the coveralls version to be < 3.3.0.

Why

• coveralls released version 3.3.0 yesterday which is causing builds to break. This change prevents the new version from being installed.

🧰 Maintenance

Reduce Docker image with slim base image and multi-stage builds \& small code quality improvements. @willgraf (#275)

What

• Remove unused imports from various backend files.
• Fix exporters_test.py, use with app.app_context() in the tests directly, remove test fixtures.
• Add .dockerignore
• Update Dockerfile to use python:3.7-slim-bullseye and multi-stage builds for a smaller final image.
• Install and use gunicorn as the production server inside the Dockerfile
• Update README to format the code blocks as bash (resolves markdown warning).
• Upgrade Python dependencies to latest patch releases and allow future patches to be used.

Why

• Incremental improvements to code quality.
• Build a better Dockerfile to serve as the backend. This should make migration from EB to ECS (or others) easier.

Upgrade to react-router v6 @tddough98 (#276)

React Router released a new major version 6 which has more flexible hooks and is more composable. It replaces the component with which avoids bugs due to route ordering and lets us place the / route before the /project route, which @willgraf requested in a recent review.

Put a version ceiling on coveralls, 3.3.0 has a bug preventing builds. @willgraf (#277)

What

• Pin the coveralls version to be < 3.3.0.

Why

• coveralls released version 3.3.0 yesterday which is causing builds to break. This change prevents the new version from being installed.