docs: simple index + add docs on test / docker (#8181)

- do not copy README as index anymore (too confusing)
- have a specific sleek index.md instead
- publish doc on makefile targets for docker
- add a doc on writing and running tests

fixes:
- #8104

docker/README.md

@@ -1,17 +1,27 @@
-# Product Opener on Docker
+# Reference Docker / Makefile commands
 
-This directory contains `docker-compose` overrides for running Product Opener on [Docker](https://docker.com).
+<!--
+NOTE: this file is copied to ref-docker-commands.md at documentation build time
+-->
+
+See also [Docker best practice at Open Food Facts](https://openfoodfacts.github.io/openfoodfacts-infrastructure/docker/)
+
+The docker/ directory contains `docker-compose` overrides for running Product Opener on [Docker](https://docker.com).
 The main docker-compose file [`docker-compose.yml`](../docker-compose.yml) is located in the root of the repository.
 
 The step-by-step guide to setup the Product Opener using Docker is available on [dev environment quick start guide](./docs/dev/how-to-quick-start-guide.md).
 
-## Docker Compose
+## Makefile targets
+
+Makefile targets are handy for beginners to start the project and for some usual tasks.
 
-### Makefile commands
+It's better though, as you progress, if you understand how things work and be able to use targeted docker-compose commands.
+
+See also [targets to run tests](../docs/dev/how-to-write-and-run-tests.md#running-tests)
 
 | Command                   | Description                                                                            | Notes                                                         |
 | ------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
-| `make dev`                | Setup a fresh dev environment.                                                         | Run only once, then use the `up`, `down`, `restart` commands. |
+| `make dev`                | Setup a fresh dev environment.                                                        | Run only once, then use the `up`, `down`, `restart` commands. |
 | `make up`                 | Start containers.                                                                      |                                                               |
 | `make down`               | Stop containers and keep the volumes.                                                  | Products and users data will be kept.                         |
 | `make hdown`              | Stop containers and delete the volumes (hard down).                                    | Products and users data will be lost !                        |
@@ -26,8 +36,3 @@ The step-by-step guide to setup the Product Opener using Docker is available on
 | `make import_sample_data` | Load sample data (~100 products) into the MongoDB database.                            |                                                               |
 | `make import_prod_data`   | Load latest prod data (~2M products, 1.7GB) into the MongoDB database.                 | Takes up to 10m. Not recommended for dev setups !             |
 
-## Kubernetes
-
-**WARNING**: The template might not be up to date and needs work
-
-The `productopener` directory contains a <a href="https://helm.sh">Helm</a> template, so that you can set up a new ProductOpener instance on <a href="https://kubernetes.io">Kubernetes</a>. Note that the deployments will create a `PersistentVolumeClaim` (PVC) with `ReadWriteMany` access mode, because the nginx container(s) and Apache container(s) will need to access the volume at the same time. This mode is not supported by every storage plugin. See [access modes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes) for more information.

docs/api/index.md

@@ -1,4 +1,4 @@
-# Open Food Facts API Documentation
+# Introduction to Open Food Facts API documentation
 
 Everything you need to know about Open Food Facts API.
 

docs/api/intro-robotoff.md

@@ -1,4 +1,4 @@
-# The Robotoff Project
+# Introduction to the Robotoff Project
 
 The Robotoff project is intended to complete missing product information by prompting users to confirm predictions inferred by Artificial Intelligence algorithms. These algorithms are calculated based on "insights", which are facts about a product that have been extracted or deduced from the product pictures, ingredients, categories, labels, etc.
 

docs/dev/how-to-write-and-run-tests.md

@@ -0,0 +1,100 @@
+# How to write and run tests
+
+If you are a developer you are really encouraged to write tests as you fix bug or develop new features.
+
+Having a test is also a good way to debug a particular piece of code.
+
+We would really love to see our test coverage augment.
+
+If you are new to tests, please read:
+- [something about test pyramid](https://automationstepbystep.com/2020/05/02/what-is-a-test-pyramid/) to understand importance of unit tests and integration tests
+- [perldoc on test](https://perldoc.perl.org/Test)
+- [Test::More module doc](https://perldoc.perl.org/Test::More)
+
+
+## Helpers
+
+We have some helpers for tests.
+
+See mainly:
+* [Test.pm](https://openfoodfacts.github.io/openfoodfacts-server/dev/ref-perl-pod/ProductOpener/Test.html) (notably `init_expected_results` and `compare_to_expected_results`)
+* [TestAPI.pm](https://openfoodfacts.github.io/openfoodfacts-server/dev/ref-perl-pod/ProductOpener/APITest.html)
+
+and other modules with Test in their name !
+
+
+## Unit and Integration tests
+
+Unit tests are located in `tests/unit/`.
+
+Integration tests are in `tests/integration/`.
+
+Most integration tests issue queries to an open food facts
+
+## Integration with docker-compose
+
+Using Makefile targets, tests are run 
+* with a specific `COMPOSE_PROJECT_NAME° to avoid crashing your development data while running tests (as the project name [changes container, network and volumes names](https://docs.docker.com/compose/environment-variables/envvars/#compose_project_name))
+* with a specific expose port for Mongodb, to avoid clashes with dev instance.
+
+## Writing tests
+
+You can read other tests to understand how we write them (inspire yourself from recently created tests).
+
+One effective way is to create a list of tests each represented by a hashmap with inputs and expected outputs and run them in a loop. Add an `id` and/or a `desc` (description) and use it as last argument to check functions (like `ok`, `is`, …) to easily see tests running and identify failing tests.
+
+If your outputs are consequent, you might use json files (one per tests), stored on disk. See tests using `init_expected_results` (and see below to refresh those files).
+
+
+## Running tests
+
+The best way to run all test is to run:
+
+```bash
+make tests
+```
+
+To run a single test you can use:
+
+* for unit test:
+   ```bash
+   make unit-test test="filename.t"
+   ```
+* for integration test:
+   ```bash
+   make int-test test="filename.t"
+   ```
+
+If you made change that impact stored expected results, you can use:
+
+* to re-generate all expected results:
+  ```bash
+  make update_tests_results
+  ```
+* or to generate expected results for a single test
+  (here for integration test, `unit-test` otherwise)
+  ```bash
+  make int-test="filename.t --update-expected results"
+  ```
+
+If you re-generate test results, be sure to look carefully that the changes your commit are expected changes.
+
+
+## Debugging with tests
+
+Launching a test is a very effective way to understand what's going on in the code using the debugger.
+
+This is done calling the test with `perl -d`.
+You can also use "args" argument with make target:
+
+```bash
+make test-unit test="my-test.t" args="-d"
+```
+
+Most of the time, you will have to use the next command "n" four times, before landing in you test, where you can easily set a breakpoint with `b <line-number>`.
+
+Read [perldoc about debugger](https://perldoc.perl.org/perldebug) to learn. more.
+
+
+> :pencil: Note: With this explanation, in integration tests that issue requests to the server, you won't be able to run the debugger inside the server code, only in the test.
+

docs/dev/index.md

@@ -1,4 +1,4 @@
-# Product Opener - Developer Documentation
+# Introduction to Product Opener developer documentation
 
 This documentation is for developers who wants to understand technical aspects of Product Opener.
 

docs/dev/ref-docker-commands.md

@@ -0,0 +1,7 @@
+# Reference Docker / Makefile commands
+
+<!--
+DO NOT MODIFY this file it is overwritten by the above fine at compile time
+-->
+
+See [README in `docker` folder](../../docker/README.md)

docs/dev/ref-perl.md

@@ -3,4 +3,4 @@
 The documentation in Plain Old Format (aka POD) for perl module
 is compiled from in file documentation.
 
-[See the Perl reference documentation](./ref-perl-pod/)
+[See the Perl reference documentation](ref-perl-pod.md)

docs/index.md

@@ -0,0 +1,13 @@
+# Product Opener (Open Food Facts web server) documentation
+
+Welcome to the documentation of **Product Opener**, the web server at the heart of **[Open Food Facts](https://world.openfoodfacts.org/)** project. 
+
+It also powers the sibling **Open \[[Beauty](https://world.openbeautyfacts.org/)|[Pet Food](https://world.openpetfoodfacts.org/)|[Products](https://world.openproductsfacts.org/)\] Facts** projects
+
+* If you want to use the API or are interested in the data, look at [API documentation](api/index.md)
+
+* If you are a developer, look at [Developer documentation](dev/index.md)
+
+The repository for the project is at https://github.com/openfoodfacts/openfoodfacts-server/
+
+Do not hesitate to contribute to this documentation, this is highly appreciated.

scripts/build_mkdocs.sh

@@ -12,8 +12,9 @@ EOF
 # get group id to use it in the docker
 GID=$(id -g)
 
-# copy README.md as the index but change links starting with ./docs/ to ./
-sed -e 's|(\./docs/|(./|g' README.md > docs/index.md
+# copy docker/README.md as ref-docker-commands.md
+cp docs/dev/ref-docker-commands.md{,.original}
+sed -e 's|(\.\./docs/dev|(|g' docker/README.md > docs/dev/ref-docker-commands.md
 
 # we use minidocks capability to add entrypoint to install some pip package
 # we use also it's capability to change user and group id to avoid permissions problems
@@ -23,4 +24,5 @@ docker run --rm \
   -v $(pwd):/app -w /app \
   minidocks/mkdocs build
 # cleanup
-rm docs/index.md $PIP_INSTALL
+rm $PIP_INSTALL
+mv docs/dev/ref-docker-commands.md{.original,}