Skip to content

Commit

Permalink
Fix ScalarFloat inference
Browse files Browse the repository at this point in the history
  • Loading branch information
jacebrowning committed Feb 23, 2020
1 parent 7adb41e commit 018721c
Show file tree
Hide file tree
Showing 18 changed files with 176 additions and 131 deletions.
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ $(MKDOCS_INDEX): docs/requirements.txt mkdocs.yml docs/*.md

docs/requirements.txt: poetry.lock
@ poetry run pip freeze -qqq | grep mkdocs > $@
@ poetry run pip freeze -qqq | grep Pygments >> $@

.PHONY: mkdocs-serve
mkdocs-serve: mkdocs
Expand Down
2 changes: 2 additions & 0 deletions datafiles/converters/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from typing import Any, Dict, Optional, Union

import log
from ruamel.yaml.scalarfloat import ScalarFloat

from ..utils import cached
from ._bases import Converter
Expand All @@ -23,6 +24,7 @@ def register(cls: Union[type, str], converter: type):

register(Integer.TYPE, Integer)
register(Float.TYPE, Float)
register(ScalarFloat, Float)
register(Boolean.TYPE, Boolean)
register(String.TYPE, String)

Expand Down
7 changes: 6 additions & 1 deletion datafiles/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,12 @@ def all(self) -> Iterator[HasDatafile]:
if path.is_absolute() or self.model.Meta.datafile_pattern.startswith('./'):
pattern = str(path.resolve())
else:
root = Path(inspect.getfile(self.model)).parent
try:
root = Path(inspect.getfile(self.model)).parent
except TypeError:
level = log.DEBUG if '__main__' in str(self.model) else log.WARNING
log.log(level, f'Unable to determine module for {self.model}')
root = Path.cwd()
pattern = str(root / self.model.Meta.datafile_pattern)

splatted = pattern.format(self=Splats())
Expand Down
2 changes: 1 addition & 1 deletion datafiles/mapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def path(self) -> Optional[Path]:
cls = self._instance.__class__
try:
root = Path(inspect.getfile(cls)).parent
except TypeError: # pragma: no cover
except TypeError:
level = log.DEBUG if '__main__' in str(cls) else log.WARNING
log.log(level, f'Unable to determine module for {cls}')
root = Path.cwd()
Expand Down
18 changes: 6 additions & 12 deletions docs/api/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ A model is created by either extending the `Model` class or using the `datafile(

Given this example dataclass:

```
#!python"
```python
from dataclasses import dataclass

@dataclass
Expand All @@ -19,8 +18,7 @@ class Item:

Synchronization is enabled by adding the `@datafile(<pattern>)` decorator:

```
#!python hl_lines="5"
```python hl_lines="5"
from dataclasses import dataclass

from datafiles import datafile
Expand All @@ -35,8 +33,7 @@ class Item:

or by replacing the `@dataclass` decorator entirely:

```
#!python hl_lines="3"
```python hl_lines="3"
from datafiles import datafile

@datafile("items/{self.name}.yml")
Expand Down Expand Up @@ -79,8 +76,7 @@ The following options can be passed to the `@datafile()` decorator:

For example:

```
#!python hl_lines="3 9"
```python hl_lines="3 9"
from datafiles import datafile

@datafile("items/{self.name}.yml", manual=True, defaults=True)
Expand All @@ -98,8 +94,7 @@ class Config:

Alternatively, any of the above options can be configured through code by setting `datafile_<option>` in a `Meta` class:

```
#!python hl_lines="9 10 11 12 13 14"
```python hl_lines="9 10 11 12 13 14"
from datafiles import datafile, converters

@datafile("items/{self.name}.yml")
Expand All @@ -120,8 +115,7 @@ class Item:

Finally, a datafile can explicitly extend `datafiles.Model` and set the pattern in the `Meta` class:

```
#!python hl_lines="11 12 13 14 15"
```python hl_lines="11 12 13 14 15"
from dataclasses import dataclass

from datafiles import Model, converters
Expand Down
3 changes: 1 addition & 2 deletions docs/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,7 @@ Additional examples can be found in this [Jupyter Notebook](https://github.com/j
Additional formats are supported through a registration system.
Either extend the `datafiles.formats.Formatter` base class or map one of the existing formatter classes:

```
#!python hl_lines="4"
```python hl_lines="4"
from datafile import datafile, formats
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
mkdocs==1.0.4
Pygments==2.5.2
30 changes: 14 additions & 16 deletions docs/types/containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ The `List` annotation can be used to define a homogeneous collection of any othe
from typing import List, Optional
```

| Type Annotation | Python Value | YAML Content |
| --- | --- | --- |
| `foobar: List[int]` | `foobar = []` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`-` |
| `foobar: List[int]` | `foobar = [1.23]` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`- 1` |
| `foobar: List[int]` | `foobar = None` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`-` |
| `foobar: Optional[List[int]]` | `foobar = None` | `foobar: ` |
| Type Annotation | Python Value | YAML Content |
| ----------------------------- | ----------------- | ------------------------------------------ |
| `foobar: List[int]` | `foobar = []` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`-` |
| `foobar: List[int]` | `foobar = [1.23]` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`- 1` |
| `foobar: List[int]` | `foobar = None` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`-` |
| `foobar: Optional[List[int]]` | `foobar = None` | `foobar:` |

More examples can be found in this [Jupyter Notebook](https://github.com/jacebrowning/datafiles/blob/develop/notebooks/patched_containers.ipynb).

Expand All @@ -27,21 +27,20 @@ The `Dict` annotation can be used to define a loose mapping of multiple types.
from typing import Dict, Optional
```

| Type Annotation | Python Value | YAML Content |
| --- | --- | --- |
| `foobar: Dict[str, int]` | `foobar = {}` | `foobar: {}` |
| `foobar: Dict[str, int]` | `foobar = {'a': 42}` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`a: 42` |
| `foobar: Dict[str, int]` | `foobar = None` | `foobar: {}` |
| `foobar: Optional[Dict[str, int]]` | `foobar = None` | `foobar: ` |
| Type Annotation | Python Value | YAML Content |
| ---------------------------------- | -------------------- | -------------------------------------------- |
| `foobar: Dict[str, int]` | `foobar = {}` | `foobar: {}` |
| `foobar: Dict[str, int]` | `foobar = {'a': 42}` | `foobar:`<br>&nbsp;&nbsp;&nbsp;&nbsp;`a: 42` |
| `foobar: Dict[str, int]` | `foobar = None` | `foobar: {}` |
| `foobar: Optional[Dict[str, int]]` | `foobar = None` | `foobar:` |

_Schema enforcement is not available with the `Dict` annotation._

## Dataclasses

Other dataclasses can serve as the annotation for an attribute to create nested structure:

```
#!python hl_lines="14"
```python hl_lines="14"
from dataclasses import dataclass

from datafiles import datafile
Expand Down Expand Up @@ -74,8 +73,7 @@ bar:
For convenience, `@datafile` can also be used in place of `@dataclass`:

```
#!python hl_lines="4"
```python hl_lines="4"
from datafiles import datafile
Expand Down
23 changes: 10 additions & 13 deletions docs/types/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,17 @@ Additional types are supported though custom type as annotations.

Custom types can be saved and loaded by extending one of the included converter classes:

| Class | Description |
| --- | --- |
| `converters.Converter` | Base class for all converters. |
| `converters.Boolean` | Converts to `bool` before serialization. |
| `converters.Integer` | Converts to `int` before serialization. |
| `converters.Float` | Converts to `float` before serialization. |
| `converters.String` | Converts to `str` before serialization. |
| Class | Description |
| ---------------------- | ----------------------------------------- |
| `converters.Converter` | Base class for all converters. |
| `converters.Boolean` | Converts to `bool` before serialization. |
| `converters.Integer` | Converts to `int` before serialization. |
| `converters.Float` | Converts to `float` before serialization. |
| `converters.String` | Converts to `str` before serialization. |

For example, here is a custom converter that ensures floating point numbers are always rounded to two decimal places:

```
#!python hl_lines="7"
```python hl_lines="7"
from datafiles import converters


Expand Down Expand Up @@ -58,8 +57,7 @@ that can be loaded as follows:

It's also possible to extend an existing class in order to have instances inherit the functionality of that class. For example, here is a custom converter based on the `datetime` class that serializes using the ISO format:

```
#!python hl_lines="9 14"
```python hl_lines="9 14"
from datetime import datetime

from datafiles import converters, datafile
Expand Down Expand Up @@ -109,8 +107,7 @@ datetime.datetime(2019, 1, 30, 23, 17, 45)

Finally, if you'd rather not have to modify your own classes (or don't have control over the source of a class), you can also register a custom converter for any class:

```
#!python hl_lines="9 14"
```python hl_lines="9 14"
from datetime import datetime

from datafiles import converters, datafile
Expand Down
37 changes: 18 additions & 19 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,23 @@ edit_uri: https://github.com/jacebrowning/datafiles/edit/develop/docs
theme: readthedocs

markdown_extensions:
- codehilite:
linenums: False
- codehilite

nav:
- Overview: index.md
- API:
- Model: api/model.md
- Manager: api/manager.md
- Mapper: api/mapper.md
- Supported Types:
- Builtins: types/builtins.md
- Extensions: types/extensions.md
- Containers: types/containers.md
- Custom Types: types/custom.md
- File Formats: formats.md
- Utilities: utilities.md
- Settings: settings.md
- About:
- Release Notes: about/changelog.md
- Contributing: about/contributing.md
- License: about/license.md
- Overview: index.md
- API:
- Model: api/model.md
- Manager: api/manager.md
- Mapper: api/mapper.md
- Supported Types:
- Builtins: types/builtins.md
- Extensions: types/extensions.md
- Containers: types/containers.md
- Custom Types: types/custom.md
- File Formats: formats.md
- Utilities: utilities.md
- Settings: settings.md
- About:
- Release Notes: about/changelog.md
- Contributing: about/contributing.md
- License: about/license.md
49 changes: 26 additions & 23 deletions notebooks/file_inference.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -99,12 +99,12 @@
"output_type": "stream",
"text": [
"names:\n",
"- Alice\n",
"- Bob\n",
" - Alice\n",
" - Bob\n",
"numbers:\n",
"- 1\n",
"- 2\n",
"- 3\n"
" - 1\n",
" - 2\n",
" - 3\n"
]
}
],
Expand Down Expand Up @@ -134,12 +134,12 @@
"name": "stdout",
"output_type": "stream",
"text": [
"dist: xenial\n",
"\n",
"language: python\n",
"matrix:\n",
" include:\n",
" - python: 3.7\n",
" dist: xenial\n",
" sudo: true\n",
"python:\n",
" - 3.7\n",
" - 3.8\n",
"\n",
"cache:\n",
" pip: true\n",
Expand Down Expand Up @@ -229,27 +229,30 @@
"name": "stdout",
"output_type": "stream",
"text": [
"dist: xenial\n",
"\n",
"language: python\n",
"matrix:\n",
" include: []\n",
"python:\n",
" - 3.7\n",
" - 3.8\n",
"cache:\n",
" pip: true\n",
" directories: []\n",
"before_install:\n",
"- curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py\n",
" | python\n",
"- source $HOME/.poetry/env\n",
"- make doctor\n",
" - curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py\n",
" | python\n",
" - source $HOME/.poetry/env\n",
" - make doctor\n",
"install:\n",
"- make install\n",
"- 'echo 123 # <= This line is new'\n",
" - make install\n",
" - 'echo 123 # <= This line is new'\n",
"script:\n",
"- make test-repeat\n",
"- make check\n",
"- make mkdocs\n",
" - make test-repeat\n",
" - make check\n",
" - make mkdocs\n",
"after_success:\n",
"- pip install coveralls\n",
"- coveralls\n",
" - pip install coveralls\n",
" - coveralls\n",
"notifications:\n",
" email:\n",
" on_success: never\n",
Expand Down
4 changes: 2 additions & 2 deletions notebooks/format_options.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -224,8 +224,8 @@
"my_dict:\n",
" value: 0\n",
"my_list:\n",
"- value: 1\n",
"- value: 2\n",
" - value: 1\n",
" - value: 2\n",
"my_bool: true\n",
"my_float: 1.23\n",
"my_int: 42\n",
Expand Down
Loading

0 comments on commit 018721c

Please sign in to comment.