Multiple fixes for 3.19 (#756)

.github/stale.yml

@@ -0,0 +1,18 @@
+# Number of days of inactivity before an issue becomes stale
+daysUntilStale: 90
+# Number of days of inactivity before a stale issue is closed
+daysUntilClose: 30
+# Issues with these labels will never be considered stale
+exemptLabels:
+  - pinned
+  - security
+  - bug
+# Label to use when marking an issue as stale
+staleLabel: wontfix
+# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Thank you
+  for your contributions.
+# Comment to post when closing a stale issue. Set to `false` to disable
+closeComment: false

.github/workflows/main.yml

@@ -45,6 +45,7 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
       - name: Upgrade PIP
+        continue-on-error: true
         run: pip install --user --upgrade pip
       - name: Install project and test cli
         run: |
@@ -143,6 +144,7 @@ jobs:
           python-version: ${{ matrix.python-version }}
       - name: Install Pip
         run: pip install --user --upgrade pip
+        continue-on-error: true
       - name: Install project
         run: pip install --use-deprecated=legacy-resolver .[test]
       - name: run tests
@@ -162,6 +164,7 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
       - name: Install Pip
+        continue-on-error: true
         run: pip install --user --upgrade pip
       - name: Install project
         run: pip install --use-deprecated=legacy-resolver .[test]

.gitpod.yml

@@ -0,0 +1,7 @@
+---
+tasks:
+  - init: >
+      pyenv install 3.9.12 &&
+      pyenv install 3.10.4 &&
+      pyenv global 3.10.4 &&
+      make install

Makefile

@@ -14,6 +14,7 @@ test_examples:
 	cd example/;pwd;python full_example.py
 	cd example/;pwd;python compat.py
 	cd example/app;pwd;python app.py
+	cd example/apply_default_on_none;pwd;python app.py
 	cd example/dunder;pwd;python app.py
 	cd example/format;pwd;python app.py
 	cd example/app_with_dotenv;pwd;python app.py
@@ -37,6 +38,9 @@ test_examples:
 	cd example/pytest_example/app;pwd;python app.py
 	cd example/pytest_example/app;pwd;pytest tests/
 	cd example/pytest_example/flask;pwd;pytest tests
+	cd example/python_loader;pwd;python app.py
+	cd example/python_loader_with_hooks;pwd;python app.py
+	cd example/module_impersonation;pwd;python main.py
 	cd example/validators/with_python/;pwd;python app.py
 	cd example/validators/with_toml/;pwd;PYTHONPATH=. dynaconf -i config.settings validate
 	cd example/toml_with_secrets/;pwd;python program.py
@@ -119,7 +123,13 @@ test_examples:
 	cd example/issues/494_using_pathlib;pwd;python app.py
 	cd example/issues/519_underscore_in_name;pwd;ATC_BLE__device_id=42 EXPECTED_VALUE=42 python app.py
 	cd example/issues/519_underscore_in_name;pwd;ATC_BLE__DEVICE_ID=42 EXPECTED_VALUE=42 python app.py
-	cd example/issues/734_validate_only_current_env;pwd;python app.py
+	cd example/issues/685_disable_dotted_lookup;pwd;python app.py
+	cd example/issues/709_yaml_merge_with_env;pwd;python app.py
+	cd example/issues/718_dynaconf_dotted_lookup;pwd;python app.py
+	cd example/issues/720_load_dotenv;pwd;python src/app/app.py
+	cd example/issues/722_docs_example;pwd;python app.py
+	cd example/issues/729_use_default_when_setting_is_blank;pwd;python app.py
+	cd example/issues/741_envvars_ignored;pwd;sh recreate.sh
 
 test_vault:
 	# @cd example/vault;pwd;python write.py

docs/configuration.md

@@ -488,4 +488,21 @@ Options: safe_load, unsafe_load, full_load
 
 ---
 
+### **dotted_lookup**
 
+By default dynaconf reads `.` as separator when setting and reading values.
+this feature can be disabled by setting `dotted_lookup=False`
+
+- type: bool
+- default: True
+
+!!! tip
+    This can also be set per file using `dynaconf_dotted_lookup: false` in the top level of the file and works for setting values. For reading you can pass `setting.get("key.other", dotted_lookup=False)`
+
+Options: True, False
+
+
+### **apply_default_on_none**
+
+YAML reads empty values as `None` in this case if you want to have `Validator` defaults applied
+to the `None` values you must set `apply_default_on_none` on `Dynaconf` class or specific on `Validator` instance.

docs/envvars.md

@@ -32,6 +32,17 @@ export DYNACONF_STRING_NUM="'76'"                            # str: "76"
 export DYNACONF_PERSON__IS_ADMIN=true                        # bool: True (nested)
 ```
 
+!!! warning
+    When exporting datastructures sych as `dict` and `list` you have to use one of:  
+    ```
+    export DYNACONF_TOML_DICT={key="value"}
+    export DYNACONF_TOML_LIST=["item"]
+    export DYNACONF_JSON_DICT='@json {"key": "value"}'
+    export DYNACONF_JSON_LIST='@json ["item"]'
+    ```
+    Those 2 ways are the only ways for dynaconf to load `dicts` and `lists` from envvars.
+
+
 with the above it is now possible to read the settings from your `program.py` using.
 
 ```python

docs/merging.md

@@ -368,7 +368,10 @@ DATABASES = {
 }
 ```
 
-> **IMPORTANT** lower case keys are respected only on *nix systems, unfortunately Windows environment variables are case insensitive and Python reads it as all upper cases, that means that if you are running on Windows the dictionary can have only upper case keys.
+!!! warning
+    lower case keys are respected only on *nix systems, unfortunately Windows environment variables are case insensitive and Python reads it as all upper cases, that means that if you are running on Windows the dictionary can have only upper case keys.
+    **Also** only first level keys are case insensitive, which means `DYNACONF_FOO__BAR=1` is different than `DYNACONF_FOO__bar=1`  
+    **in other words**, except by the first level key, you must follow strictly the case of the variable key.
 
 Now if you want to add a new item to `ARGS` key:
 
@@ -527,9 +530,6 @@ settings.DATABASE == {'host': 'server.com', 'user': 'dev_user', 'password': 1234
 
 The **dynaconf_merge** and **@merge** functionalities works only for the first level keys, it will not merge subdicts or nested lists (yet).
 
-For deeper nested objects use [dunder merge](/envvars/#nested-keys-in-dictionaries-via-environment-variables).
-
-
 
 ## More examples
 

docs/settings_files.md

@@ -11,7 +11,7 @@ mixed settings formats across your application.
 ## Supported formats
 
 - **.toml** - Default and **recommended** file format.
-- **.yaml|.yml** - Recommended for Django applications.
+- **.yaml|.yml** - Recommended for Django applications. (see [yaml caveats](#yamlcaveats))
 - **.json** - Useful to reuse existing or exported settings.
 - **.ini** - Useful to reuse legacy settings.
 - **.py** - **Not Recommended** but supported for backwards compatibility.
@@ -26,6 +26,10 @@ mixed settings formats across your application.
     You can create your custom loader and read any data source.
     read more on [extending dynaconf](/advanced/)
 
+!!! warning
+    To use the `.ini` or `.properties` file format you need to install extra dependency
+    `pip install configobj` or `pip install dynaconf[ini]`
+
 ## Reading settings from files
 
 On files by default dynaconf loads all the existing keys and sections
@@ -217,3 +221,33 @@ You can for example name it `[testing]` or `[anything]`
 !!! tip
     It is also possible to switch environments programmatically passing
     `env="development"` to `Dynaconf` class on instantiation.
+
+
+### YAML Caveats
+
+#### Nonetypes
+
+Yaml parser used by dynaconf (ruamel.yaml) reads undefined values as `None` so
+
+```yaml
+key:
+key: ~
+key: null
+``` 
+
+All those 3 keys will be parsed as python's `None` object.
+
+When using a validator to set a default value for those values you might want to use one of:
+
+```py
+Validator("key", default="thing", apply_default_on_none=True)
+```
+
+This way dynaconf will consider the default value even if setting is `None` on yaml.
+
+or on yaml you can set the value to `@empty`
+
+```yaml
+key: "@empty"
+```
+>> **NEW** in 3.1.9

docs/validation.md

@@ -56,6 +56,39 @@ settings = Dynaconf(
 The above will raise `dynaconf.validators.ValidationError("AGE must be lte=30 but it is 35 in env DEVELOPMENT")` and `dynaconf.validators.ValidationError("PROJECT must be eq='hello_world' but it is 'This is not hello_world' in env PRODUCTION")`
 
 
+## Lazy validation
+
+Instead of passing `validators=` argument to `Dynaconf` class you can register validators after the instance is created.
+
+```python
+settings = Dynaconf(...)
+
+custom_msg = "You cannot set {name} to {value} in env {env}"
+settings.validators.register(
+    Validator("MYSQL_HOST", eq="development.com", env="DEVELOPMENT"),
+    Validator("MYSQL_HOST", ne="development.com", env="PRODUCTION"),
+    Validator("VERSION", ne=1, messages={"operations": custom_msg}),
+    Validator("BLABLABLA", must_exist=True),
+)
+```
+
+Having the list of validators registered you can call one of:
+
+### Validate and raise on the first error:
+
+```python
+settings.validators.validate()
+```
+
+### Validate and accumulate errors, raise only after all validators are evaluated.
+
+```python
+settings.validators.validate_all()
+```
+
+The raised `ValidationError` will have an attribute `details` holding information about each
+error raised.
+
 ### Providing default or computed values
 
 
@@ -70,6 +103,10 @@ Validator("FOO", default="A default value for foo")
 Then if not able to load the values from files or environment this default value will be set for that key.
 
 
+!!! warning
+    YAML reads empty keys as `None` and in that case defaults are not applied, if you want to change it
+    set `apply_default_on_none=True` either globally to `Dynaconf` class or individually on a `Validator`.
+
 #### Computed values
 
 Sometimes you need some values to be computed by calling functions, just pass a callable to the `default` argument.