[DO NOT MERGE] Update pydantic to v2

[tcompa]

This is a proof-of-principle, to see how hard it would be to move to Pydantic v2 (ref #592).
Proceeding in this direction would require several different discussions and tests - TBD.

[tcompa]

                               Summary of Failures                               
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃               ┃               ┃  Function     ┃              ┃               ┃
┃  File         ┃  Function     ┃  Line         ┃  Error Line  ┃  Error        ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│  tests/task…  │  test_args_…  │  160          │  199         │  KeyError     │
│  tests/task…  │  test_workf…  │  581          │  652         │  AssertionE…  │
└───────────────┴───────────────┴───────────────┴──────────────┴───────────────┘
Results (190.14s):
         2 failed
       122 passed
         1 xpassed
       499 warnings

[tcompa]

With the current code of this branch, the following example

from typing import Optional

def tmp(
        a: int,
        b: Optional[int],
        c: int = 1,
        d: Optional[int] = None,
) -> None:
    """
    Short docstring for function

    Long docstring for function

    Arguments:
        a: Description of a
        b: Description of b
        c: Description of c
        d: Description of d
    """
    pass

prints

{
  "additionalProperties": false,
  "properties": {
    "a": {
      "title": "A",
      "type": "integer",
      "description": "Description of a"
    },
    "b": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "title": "B",
      "description": "Description of b"
    },
    "c": {
      "default": 1,
      "title": "C",
      "type": "integer",
      "description": "Description of c"
    },
    "d": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "D",
      "description": "Description of d"
    }
  },
  "required": [
    "a",
    "b"
  ],
  "type": "object"
}

---

If we subclass the generator (see https://docs.pydantic.dev/latest/concepts/json_schema/#customizing-the-json-schema-generation-process, and https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema.nullable_schema) as in

class MyGenerateJsonSchema(GenerateJsonSchema):
    def nullable_schema(self, schema):
        """Generates a JSON schema that matches a schema that allows null values.

        Args:
            schema: The core schema.

        Returns:
            The generated JSON schema.
        """
        null_schema = {'type': 'null'}
        inner_json_schema = self.generate_inner(schema['schema'])

        if inner_json_schema == null_schema:
            return null_schema
        else:
            return inner_json_schema

then the output is

{
  "additionalProperties": false,
  "properties": {
    "a": {
      "title": "A",
      "type": "integer",
      "description": "Description of a"
    },
    "b": {
      "title": "B",
      "type": "integer",
      "description": "Description of b"
    },
    "c": {
      "default": 1,
      "title": "C",
      "type": "integer",
      "description": "Description of c"
    },
    "d": {
      "default": null,
      "title": "D",
      "type": "integer",
      "description": "Description of d"
    }
  },
  "required": [
    "a",
    "b"
  ],
  "type": "object"
}

---

Note that this is meant to be an exploration of possible directions it's not clear whether it would be a robust solution to e.g. fractal-analytics-platform/fractal-web#479. The schema for e, for instance, looks wrong.