Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make error handling part of the public interface #25

Merged
merged 2 commits into from
Jul 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
release = "1.0.0"


extensions = ["sphinx.ext.autodoc"]
extensions = ["sphinx.ext.autodoc", "sphinx.ext.doctest"]
language = "python"
html_theme = "sphinx_rtd_theme"
40 changes: 40 additions & 0 deletions docs/source/error_handling.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
Error Handling
==============

Reading of ecl files will throw :py:class:`ecl_data_io.EclParsingError`
when the given file does not contain valid ecl data:

.. doctest::

>>> from ecl_data_io import read, write, EclParsingError, EclWriteError
>>> from io import StringIO
>>>
>>> file_contents = StringIO("Not valid ecl content")
>>> try:
... read(file_contents)
... except EclParsingError as e:
... print(e)
Expected "'" before keyword, got N at 1

Similarly, write will produce :py:class:`ecl_data_io.EclWriteError`
when the given data is not suitable for writing.

.. doctest::

>>> try:
... write("my_file.egrid", [("FILEHEAD", ["a"*100])])
... except EclWriteError as e:
... print(e)
Could not convert numpy type <U100...


For file and stream operations, the underlying exceptions from open(), read(), and
write() are passed through:

.. doctest::

>>> try:
... read("does_not_exist/my_file.egrid", [])
... except OSError as e:
... print(e)
[Errno 2] No such file or directory...
72 changes: 38 additions & 34 deletions docs/source/example_usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,52 +12,61 @@ read
The read function will open a given file and give you a list of tuples
of the keywords and arrays.

... testsetup::

>>> import ecl_data_io as eclio
>>>
>>> eclio.write(
... "my_grid.egrid",
... [
... ("FILEHEAD", []),
... ("GRIDHEAD", []),
... ("COORD", []),
... ("ZCORN", []),
... ("ACTNUM", []),
... ("MAPAXES", []),
... ],
... fileformat=eclio.Format.FORMATTED,
... )

>>> import ecl_data_io as eclio
>>> for kw, arr in eclio.read("my_grid.egrid"):
... print(kw)
"FILEHEAD"
"GRIDHEAD"
"COORD"
"ZCORN"
"ACTNUM"
"MAPAXES"
... print(kw.strip())
FILEHEAD
GRIDHEAD
COORD
ZCORN
ACTNUM
MAPAXES

write
-----

The :meth:`ecl_data_io.write` function will write such files
from lists of keywords, array tuples:

>>> import ecl_data_io as eclio
>>> eclio.write("my_grid.egrid", ["FILEHEAD": [...], "GRIDHEAD": [10,10,10]])
>>> eclio.write("my_grid.egrid", [("FILEHEAD", [1]), ("GRIDHEAD", [10,10,10])])

The default format is is binary (unformatted), but it is possible to
read and write ascii (formatted) aswell:


>>> import ecl_data_io as eclio
>>> eclio.write(
>>> "my_grid.egrid",
>>> {"FILEHEAD": [...], "GRIDHEAD": [10,10,10]},
>>> fileformat=eclio.Format.FORMATTED
>>> )
... "my_grid.fegrid",
... {"FILEHEAD": [1], "GRIDHEAD": [10,10,10]},
... fileformat=eclio.Format.FORMATTED
... )

lazy reading
------------

It is possible to read through the file without loading all arrays into
memory, ie. lazily:

import ecl_data_io as eclio

>>> for item in eclio.lazy_read("my_grid.egrid"):
>>> print(item.read_keyword())
"FILEHEAD"
"GRIDHEAD"
"COORD"
"ZCORN"
"ACTNUM"
"MAPAXES"
>>> for item in eclio.lazy_read("my_grid.fegrid"):
... print(item.read_keyword())
FILEHEAD
GRIDHEAD


Note that :meth:`ecl_data_io.lazy_read` in the above example is a generator of array
Expand All @@ -71,18 +80,15 @@ For better control, one can pass the opened file:
... generator = eclio.lazy_read(f)
... item = next(generator)
... print(item.read_keyword())

"FILEHEAD"
FILEHEAD

Writing MESS
------------

The special MESS types keyword can be written as follows:


>>> from ecl_data_io.types import MESS
>>> from ecl_data_io import write, MESS
>>> write("output.EGRID", [("MESSHEAD", MESS)])
>>> eclio.write("output.EGRID", [("MESSHEAD", eclio.MESS)])

array types
-----------
Expand Down Expand Up @@ -110,12 +116,10 @@ Say you want to update the first keyword name `"OLD_NAME"`, change the array
to `new_array` and the name to `NEW_NAME`, then that can be done
with the following:

>>> import ecl_data_io as eclio
>>>
>>> new_array = ...
>>> new_array = [2]
>>>
>>> with open("my_grid.egrid", "br+") as f: # Open with read and write
... for entry in eclio.lazy_read(f):
... if entry.read_keyword() == "OLD_NAME":
... entry.update(keyword="NEW_NAME", array=new_array)
... if entry.read_keyword() == "FILEHEAD":
... entry.update(keyword="FILEHEAD", array=new_array)
... break
32 changes: 25 additions & 7 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ ecl-data-io: Low level IO for ecl files

example_usage
the_file_format
error_handling
developer_guide
api_doc

Expand All @@ -25,15 +26,32 @@ Quick Start Guide
Using the library
-----------------

... testsetup::

>>> import ecl_data_io as eclio
>>>
>>> eclio.write(
... "my_grid.egrid",
... [
... ("FILEHEAD", []),
... ("GRIDHEAD", []),
... ("COORD", []),
... ("ZCORN", []),
... ("ACTNUM", []),
... ("MAPAXES", []),
... ],
... fileformat=eclio.Format.FORMATTED,
... )

>>> import ecl_data_io as eclio
>>> for kw, arr in eclio.read("my_grid.egrid"):
... print(kw)
"FILEHEAD"
"GRIDHEAD"
"COORD"
"ZCORN"
"ACTNUM"
"MAPAXES"
... print(kw.strip())
FILEHEAD
GRIDHEAD
COORD
ZCORN
ACTNUM
MAPAXES


For more see :ref:`example-usage`.
Expand Down
11 changes: 10 additions & 1 deletion src/ecl_data_io/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import ecl_data_io.version

from .errors import EclParsingError, EclWriteError
from .format import Format
from .read import lazy_read, read
from .types import MESS
Expand All @@ -10,4 +11,12 @@

__version__ = ecl_data_io.version.version

__all__ = ["read", "lazy_read", "write", "Format", "MESS"]
__all__ = [
"read",
"lazy_read",
"write",
"Format",
"MESS",
"EclParsingError",
"EclWriteError",
]
11 changes: 7 additions & 4 deletions src/ecl_data_io/_unformatted/write.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,9 @@ def write_str_list(stream, str_list, ecl_type):
return
max_len = max(len(s) for s in str_list)
if max_len > 99:
raise ValueError("Ecl files does not support strings of length > 99")
raise EclWriteError("Ecl files does not support strings of length > 99")
if max_len > str_size:
raise ValueError(
raise EclWriteError(
f"Inconsistent type size, have {str_size} type but longest string is {max_len}"
)
str_list = [s.ljust(str_size) for s in str_list]
Expand All @@ -79,7 +79,7 @@ def write_str_list(stream, str_list, ecl_type):
s.encode("ascii") if isinstance(s, str) else s for s in str_list
]
except UnicodeEncodeError as e:
raise ValueError(
raise EclWriteError(
"Cannot write non-ascii strings to unformatted ecl files"
) from e

Expand All @@ -100,7 +100,10 @@ def write_str_list(stream, str_list, ecl_type):

def write_array_like(stream, keyword, array_like):
array = np.asarray(array_like)
ecl_type = ecl_types.from_np_dtype(array)
try:
ecl_type = ecl_types.from_np_dtype(array)
except ValueError as e:
raise EclWriteError(f"{e}") from e
if ecl_type == b"MESS":
write_array_header(stream, keyword, ecl_type, 0)
array = np.array([])
Expand Down
4 changes: 2 additions & 2 deletions src/ecl_data_io/errors.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
class EclParsingError(Exception):
class EclParsingError(ValueError):
"""
Indicates an error occurred during reading of an ecl file.
"""

pass


class EclWriteError(Exception):
class EclWriteError(ValueError):
"""
Indicates an error occurred during writing of an ecl file.
"""
Expand Down
11 changes: 11 additions & 0 deletions src/ecl_data_io/read.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ def lazy_read(filelike, fileformat=None):
:param fileformat: Either ecl_data_io.Format.FORMATTED for ascii
format, ecl_data_io.Format.UNFORMATTED for binary formatted files
or None for guess.

:raises ecl_data_io.EclParsingError: If the file is not a valid
ecl file.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ecl always referring to Eclipse? If so, why don't we write the full name?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ecl is the name of the group that made eclipse. Its a convention now, perhaps something to change, but that is perhaps a bigger issue than this PR as ecl is in the name of the package.


.. note::
If given a file to be open (as opposed to a stream), the errors
(various `IOError` s) associated with the default behavior of the
built-in `open()` function may be raised.

When given a stream, the exceptions associated with the stream will
pass through.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to have a test for error handling? It would be easier to visualise what you mean by the condition file opposed to stream.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added some doctests for this

"""
if fileformat is None:
fileformat = guess_format(filelike)
Expand Down
11 changes: 11 additions & 0 deletions src/ecl_data_io/write.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,17 @@ def write(filelike, contents, fileformat=Format.UNFORMATTED):
will be converted according to ecl_data_io.types.to_np_type
:param fileformat: Either ecl_data_io.Format.FORMATTED for ascii
format or ecl_data_io.Format.UNFORMATTED for binary format.

:raises ecl_data_io.EclWriteError: If the given contents cannot be
written to an ecl file.

.. note::
If given a file to be open (as opposed to a stream), the errors
(various `IOError` s) associated with the default behavior of the
built-in `open()` function may be raised.

When given a stream, the exceptions associated with the stream will
pass through.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also added doctest for this.

"""
stream, didopen = get_stream(filelike, fileformat, mode="w")

Expand Down
4 changes: 3 additions & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ envlist =
[testenv]
deps =
-rdev-requirements.txt
commands = python -m pytest tests
commands = python -m pytest

[testenv:style]
deps = pre-commit
Expand All @@ -23,6 +23,8 @@ commands =
addopts =
-ra
--durations=5
--doctest-glob="*.rst"
--doctest-modules

[gh-actions]
python =
Expand Down