Repository navigation
Releases: wildintelproject/wildintel-uploader
Release list
Dev build (0.0.0-dev+25659d9)
Development build — not for production use.
Built from commit 25659d9b8856632aba11d4110e8cd39d6deb4dcd on branch development.
Added
-
Import deployment: organize a folder of camera-trap images into the local collections folder
(<Documents>/wildintel-uploader/collections/<research project>/<R000N>/<deployment id>), in a wizard of four
stages: images → origin and dates → check details → prepare and save.- Validate: corrupted images, shooting order vs. filename sequence, folder structure, same camera,
required EXIF fields and duplicate images — each one optional, and each one can be required to continue. - Origin: the research project and location, picked from the collections folder, added by hand or
taken from Trapper. - Details: the deployment's id from its revision (
R0003-DONA_01), the period with its timezone and
Camtrap DP's own fields, validated in the browser and in the backend.
It says when that deployment already exists in the collections folder, and, when an earlier revision of it
was kept, offers to fill the form in from it (everything but the start and end dates). - Postvalidation: the naming and the dates against wildintel-tools' rules, and statistical checks
(number of images, of sequences and their length) against the previous revisions of the location. - Preprocessing: renaming, resizing to a width and XMP metadata (as wildintel-tools did), listed before
the import;preprocessing.jsonrecords what was done to each image, with its hashes (source_hash
of the original,hashof the image kept,final_hashof the file with its metadata). - The camera model and id are read with ExifTool (Pillow without it) — the executables ship with their
own copy; the collection's
<COLLECTION>_FileTimestampLog.csvis kept up to date. - After a good import, a button opens the folder in the file explorer.
- Validate: corrupted images, shooting order vs. filename sequence, folder structure, same camera,
-
Import session: a folder with one subfolder per deployment. The revision, the checks and the
preprocessing are common; the location, the dates and the rest of the details are asked for each
deployment, with a filterable list for big sessions, "next incomplete", and filling the details from the
previous revision of the same deployment (one or all).- If the session folder has a
<COLLECTION>_FileTimestampLog.csv(the file wildintel-tools asks for beside
a collection's deployments), the start and end of each deployment it has a row for are taken from it
instead of from the images' EXIF, and the Details step says so. Seeexamples/session_example_timestamp_log.
- If the session folder has a
-
The timezone and the summer-time ("ignore DST") setting belong to the location, never to a deployment: the details
no longer ask for them, nor does any request carry them. They are said, in bold, in a sentence before the details
(the location, its GPS coordinates, its timezone and whether it ignores summer time), and the images are read and
the deployment registered in the ones the location has kept. New locations ask for both; a location without a
timezone is reported instead of guessing one (/api/deployment-import/locations/updatestill changes them). -
The revision of a deployment (and of a session) starts as the next one expected — one after the highest kept for the
location (/api/deployment-import/next-revision) — and says so; it can be changed. -
Once a deployment or a session is imported, an Upload to Trapper button goes to the upload page with the research
project and the collection already chosen. Pressing the title in the navbar goes back to the welcome screen. -
Upload deployment to Trapper: pick a collection kept locally and upload its deployments — creating the
location and the deployment in Trapper if they are missing, packing the images in zip + yaml parts and
sending them. A dry run says what would be done changing nothing, and only generate the files writes the
zips, the yamls and the collection's deployments csv to leave them for later.
A Test connection button checks, changing nothing, that the account reaches the research project, its locations
(and which of the deployments' own would be created) and the uploader's login, each one reported on its own.
The collection's yaml names the research project's classification project (picked under the deployments, or the only
one it has; typed by pk when generating without an account) — not the research project's own pk.
The package names the deployment in lower case, as Trapper keeps it, and declares the timezone and summer-time
setting of the location it goes to (Trapper refuses a package that disagrees with it). -
Import info from Epicollect5: in When was it collected?, the period, camera, site and notes of the deployment are filled in from the entry of the Epicollect5 form (one row per deployment) of that location and revision, through the mapping of Settings › Epicollect5, whose When group keeps the day and the time apart and maps the revision; Test connection tries the project and client typed there.
-
Epicollect5 settings: a switch turns the use of Epicollect5 on or off (off by default), and another sends what an entry answers in inputs with no mapping to the comments of the deployment. When was it collected? says when Epicollect5 has an entry for the location and revision.
-
Utils (menu): Repair local deployments and Sync local collections are kept under one button, apart from the import and upload tasks.
-
Sync local collections: pick a research project and one of its classification projects in Trapper, and the collections
folder gets what it lacks — the research project (research_project.json) and itslocations.json, the collections
starting with R (each with itscollection.jsonand<collection>_FileTimestampLog.csv) and each deployment's
deployment.json, a deployment going to the collection its id starts with. What is already there is kept as it is, nothing is
changed in Trapper and no image is downloaded unless asked for (Also download the images from Trapper: through the classification project's media, four at a time, the ones already there kept), so nopreprocessing.jsonis made. Each deployment also gets animages.jsonmade
from what Trapper holds for its images (names, Trapper's pk, when they were taken, observation types, species and tags — not the
size, the pixels nor the hashes), marked"source": "trapper". -
Every deployment kept locally gets an
images.jsonbeside itsdeployment.json: the number of images, the first and the
last, and for each one its name (and the original, when renamed), when it was taken — the camera's wall clock, the instant
and its epoch seconds — the camera, the size and the pixels, and the hashes, for statistics without opening the images again.
A deployment copied without preprocessing has the names, the EXIF wall-clock time, the size and the pixels. -
Validating and preprocessing the images use several CPUs at once (Settings › General › Workers; by default the
CPUs the machine has, up to 4). -
Settings: General (images folder, log, update check), Trapper (account and test), Validation and
Postvalidation (each check an entry with a quick switch, and a gear only where it has settings),
Preprocessing (steps, width, authorship, license, dates) and Config — several settings files to switch
between. -
Tables of checks have "all" boxes to run, or require, every check at once.
-
A command line (
wildintel-uploader) sharing the settings with the web app. -
Documentation: a web and a command-line user manual, a developer manual, features and about pages, with screenshots
generated byuv run wucli docs screenshots(Playwright over the built frontend and a fake backend). -
tools/make_example_deployment.pymakes a deployment of any number of sample images (3000 by default), each with the
camera's make, model and serial number and its capture date in the EXIF.
With--faultyit comes with problems for every validation: corrupted and empty images, duplicates, shots out of order, missing
EXIF, a second camera and a subfolder. -
Reports: every validation, postvalidation and preprocessing writes a report of what was checked and done, image by image
(<images folder>/reports/*.json). Each step shows it — the checks with how many images passed and failed, and the images that
failed with the check and why — and it downloads as a CSV (a row per image and check) or a JSON. A Reports task lists the
reports to look at again or delete.
Changed
- Scan only counts the files and the images of the folder; it no longer opens them. Their dates and camera are read when the
deployment details open (/api/deployment-import/guess-details), so scanning thousands of images is instant. - The validation step is a dashboard: when it was last run, the valid images, the images with issues and the tests that failed, a table of tests with
a badge and the number of failed images each, and — opened from one — the images that failed it as thumbnails, with search, filter by error, sort, a
grid or a list, pages and a view of everything the report says of an image. A report's images are served as thumbnails
(/api/reports/{id}/image). - Preprocessing is a step of its own (
/api/deployment-import/preprocess), run like the validations, with the same dashboard: the steps to do, a status for each, the images processed and skipped, and the skipped ones as thumbnails. The deployment is left pending — it can be preprocessed again — until the Import step consolidates it (/api/deployment-import/consolidate): itsdeployment.jsonis written and it is sealed. - Repair local deployments (new task): a deployment is valid if its
seal.jsonstill matches it; for the others the metadata files (deployment.json,preprocessing.json,images.json,seal.json) are written again from the fo...