New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide comparison between PyO3 and rust-cpython #55

Closed
ssokolow opened this Issue Jul 23, 2017 · 9 comments

Comments

Projects
None yet
2 participants
@ssokolow

ssokolow commented Jul 23, 2017

When I come to the PyO3 frontpage, having never seen PyO3 before, my first questions are "how is this different from rust-cpython?" and "Why would I want to use this instead?"

So far, all I've been able to determine is:

  1. Judging by #5 and my experience using rust-cpython on stable, this adds a dependency on nightly Rust.
  2. This uses attributes rather than macros
  3. I don't remember the list of ToPyObject impls being this long in rust-cpython.

It'd be a good idea to add a blurb to the README explaining:

  1. What motivated the fork (ie. how PyO3's goals differ from rust-cpython's)
  2. What PyO3 does better
  3. What PyO3 does worse (ie. successfully compiling on stable)
@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Jul 23, 2017

at this point, base concepts, compared to rust-cpython, are totally different.

  1. All objects are owned by pyo3 library and all apis available with references. in rust-cpython, you own
    python objects. here is example of PyList api:

rust-cpython:

impl PyList {

   fn new(py: Python) -> PyList {...}

   fn get_item(&self, py: Python, index: isize) -> PyObject {...}
}

pyo3:

impl PyList {

   fn new(py: Python) -> &PyList {...}

   fn get_item(&self, index: isize) -> &PyObject {...}
}

Because pyo3 allows only references to python object, all refs uses Gil lifetime. So Python object
is not required, and it is safe to have fn py<'p>(&'p self) -> Python<'p> {} function.

@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Jul 23, 2017

rust-cpython custom class

  1. rust-cpython uses whole new language based on macros
  2. it is very hard to extend py_class! macros, for example async/await support.
  3. generated functions are used for access to struct attributes.
  4. To drop PyObject GIL is required.

pyo3 custom class

  1. use proc_macro and specialization for class implementation (nightly is required)
  2. pyo3 does not modify rust struct. it is possible to define needed traits and make rust type compatible with python class without using #[py::class] macro.
  3. class customization is based on specialization and traits and associated types. separate trait is defined for each type
    of customization (i.e. PyMappingProtocol, PyNumberProtocol). macro is used for simplicity, but it is possible to make rust type compatible with specific customization without using proc macros.
  4. pyo3 does not inc ref for borrowed ptrs, PyObject does not need GIL to drop.
  5. pyo3 at least 15% faster.
@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Jul 23, 2017

motivation

  1. seems rust-cpython is not maintained anymore. I tried to communicate with author, didn't get any response. I submitted multiple PR, none of them had any reaction.
  2. I don't like class implementation based on macros. first of all, this is new language within rust, second it is very hard to extend and modify.
  3. pyo3 is just rust. proc macros are used for automatically implement extra traits and wrapping code, so it possible to do the same manually.

downsides:

  1. pyo3 depends on nightly rust (proc_macro, specialization, const_fn).
  2. proc_macro error reporting is bad.
@ssokolow

This comment has been minimized.

ssokolow commented Jul 23, 2017

In that case, definitely something I'll be switching over to once it's possible to resolve #5.

@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Jul 23, 2017

I used rust-cpython, and had occasional segfaults. never got any response from author.
I use pyo3 in production without this segfault.

@ssokolow

This comment has been minimized.

ssokolow commented Jul 23, 2017

In that case, once I've finished migrating as much of my project to the Rust side as possible, if I encounter segfaults and can't resolve them, maybe I'll drop to something lower-level like Snaek.

...or just port everything back to pure Python. Avoiding dependency on nightly Rust is non-negotiable.

@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Jul 24, 2017

I hope proc_macro will be stabilized by the end of year.

@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Jul 26, 2017

Just landed pretty big change.

rust-cpython requires Python parameter for PyErr, so error handling ergonomics is pretty bad.
It is not possible to use ? with rust errors.

pyo3 on other hand does not require Python for PyErr creation, it is required only if you want to set exception to python interpreter with PyErr::restore() method. so it is possible to use std::convert::From<Err> for PyErr trait and ? is supported automatically.

for example:

use pyo3::*;

fn parse_int(s: String) -> PyResult<usize> {
    Ok(s.parse::<usize>()?)
}

The code snippet above will raise ValueError in Python if String::parse() return an error.

@fafhrd91

This comment has been minimized.

Contributor

fafhrd91 commented Aug 12, 2017

closing, there is link in readme

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment