MVI Validator is an accuracy validator for Maximo Visual Inspection.
-
Install Python Confirm that you can execute
python3andpip3commands in terminal window.$ python3 --version Python 3.11.0 $ pip3 --version pip 22.3 from pip (python 3.11)
-
Install
pip install 'git+https://github.com/IBM/mvi-validator.git' -
Confirm that you can execute
mvi-validatorcommand in terminal window.$ mvi-validator --version 0.0.13 # If you see "Command not found", use `python -m mvi-validator` instead of `mvi-validator` $ python -m mvi-validator --version 0.0.13
-
MVI > left menu > dataset > select the test dataset you created > Click the right top export button
> Save the zip file on your PC

-
Expand the zip file
- Example) if you expand in the directory
test_ball_bearing, The dirctory structure looks like below.$ tree . | head . └── test_ball_bearing ├── 00196b51-d6e7-4372-81a6-f99f15541520.jpg ├── 00196b51-d6e7-4372-81a6-f99f15541520.xml ├── 002dc8fb-1806-4f73-9df8-4e93210e08f7.jpg ├── 002dc8fb-1806-4f73-9df8-4e93210e08f7.xml ├── 004b9976-3b36-4002-861b-d692c7db43dd.jpg ├── 004b9976-3b36-4002-861b-d692c7db43dd.xml ├── 005d5394-64ea-4d1f-a353-fac0fb5ebcb4.jpg ├── 005d5394-64ea-4d1f-a353-fac0fb5ebcb4.xml
- Example) if you expand in the directory
-
MVI > left menu > Model > select the model you want to verify > deploy
-
MVI > left menu > deployed model > click
copybutton in API endpoint section -
Open your terminal > execute
mvi-validator deployed-model detection --api [API endpoint URL] [The directory of test dataset]-
Example) If APIURL is
https://mvi.com/api/dlapis/bb44e214-e208-4e6a-a88b-d9ab173023da, directory istest_ball_bearingthen$ mvi-validator deployed-model detection --api https://mvi.com/api/dlapis/bb44e214-e208-4e6a-a88b-d9ab173023da test_ball_bearing
-
The result will be shown in markdown table
# Summary | num_images | num_gt_bbox | num_pd_bbox | total_tp | total_fp | total_fn | precision | recall | f-measure | mAP | model_id | |-------------:|--------------:|--------------:|-----------:|-----------:|-----------:|------------:|---------:|------------:|---------:|:-------------------------------------| | 28 | 27 | 9 | 8 | 1 | 19 | 1 | 0.185185 | 0.444444 | 0.888889 | bb44e214-e208-4e6a-a88b-d9ab173023da |
-
If you use
--format csvoption withmvi-validatorcommand, it generates CSV instead of markdown table.
-
- Install with notebook option
$ pip3 install -e '.[notebook]' - Start jupyter notebook
$ jupyter notebook
Open Issue here .
Takahide Nogayama
MIT
