Skip to content

Better From DatafusionError trait for PyErr #17745

@mesejo

Description

@mesejo

Is your feature request related to a problem or challenge?

The Python errors are badly mangled in datafusion-python, see apache/datafusion-python#1226.

Basically, when going from Python -> Rust -> Python, the original Python stack trace is lost. We could use DataFusionError::External to store the original PyErr, as demonstrated in this POC, but I think the right place for this transformation is the From trait.

Describe the solution you'd like

A solution would be to change the From trait defined in pyarrow.rs to

use pyo3::exceptions::{PyException, PyNotImplementedError};
use pyo3::prelude::PyErr;
use pyo3::types::{PyAnyMethods, PyList};
use pyo3::{Bound, FromPyObject, IntoPyObject, PyAny, PyObject, PyResult, Python};

use crate::{DataFusionError, ScalarValue};

impl From<DataFusionError> for PyErr {
    fn from(err: DataFusionError) -> PyErr {

        match err {
            DataFusionError::External(boxed) => match boxed.downcast::<PyErr>() {
                Ok(py_err) => *py_err,
                Err(original_boxed) => PyException::new_err(original_boxed.to_string()),
            },
            DataFusionError::NotImplemented(message) => PyNotImplementedError::new_err(message),
            _ => PyException::new_err(err.to_string()),
        }
    }
}

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions