Why Python complains when called method of pybind11 type_cast-ed class that derives from C std::ve-CodePudding

Using pybind11 I wrap a C lib that I cannot modify. An issue came from a class that derives from std::vector. (Notes: This my first pybind11 project. I am not 'fluent' in Python. I was looking for solution on the web but without success.)

Intro. Instance of E carries error data. E could be instantiated only by Es - a collector. Es gets enlarged with new instances of E by method(s) like addFront(...) while returning back from failed method(s) (i.e unwinding the call stack).

Minimalistic source code:

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;
using namespace pybind11::literals;

// Classes
enum class ID { unknown, wrongParam };

class E {
public:
    ID GetID() const { return id; }

protected:
    E( ID _id ) { id = _id; };
    ID  id;
    friend class Es;
};

class Es : public std::vector< E > {
public:
    Es() {}
    Es( ID _id ) { push_back( E( _id ) ); }

    Es& addFront( ID _id ) {
        insert( begin(), E( _id ) );    // Base class methods!
        return *this;
    }
};

Since derived from std::vector, as I learned, a type_caster for Es should be applied so it can be used as a list on Python side:

namespace pybind11 { namespace detail {
    template <> struct type_caster< Es > : list_caster< Es, E > {};
}} // namespace pybind11::detail

The pybind11 part is:

void Bind( py::module_& m ) {
    py::enum_< ID >( m, "ID" )
        .value( "unknown", ID::unknown )
        .value( "wrongParam", ID::wrongParam );

    py::class_< E >( m, "E" )
        .def( "GetID", &E::GetID );

    py::class_< Es >( m, "Es" )
        .def( py::init<>() )
        .def( py::init< ID >(), "id"_a )
        .def( "addFront", &Es::addFront );
}

When this Python code is executed:

from AWB import ID, E, Es
es = Es( ID.wrongParam )
es.addFront( ID.unknown )

python complains:

E:\Projects\AWB>py
Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from AWB import ID, E Es
>>> es = Es( ID.wrongParam )
>>> es.addFront( ID.unknown )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: addFront(): incompatible function arguments. The following argument types are supported:
    1. (self: List[AWB.E], arg0: AWB.ID) -> List[AWB.E]

Invoked with: <AWB.Es object at 0x000000000260CEB0>, <ID.unknown: 0>
>>>

Q: What I am doing wrong?

Q: Why arg0: AWB.ID is incompatible with ID.unknown?

Q: Maybe the type casting should be more precise?

Well, in real world, I don't expect the Es will enlarge on Pyhon side. Mostly, I would need to export the collection (in human readble manner) what the Es has collected so far (on the C side). But since I am writting a test case - I need to be sure it works.

Q: Even if it is possible, to use addFront, on Python side, to add an item into C std::vector... Would it be automatically visible in the Python's list?

CodePudding user response：

Since you want to use C member functions specific to Es, I think you shouldn't try to use type-casting in this case. If you would type-cast Es, that means your Python type will be a copied list of E objects but it wouldn't have methods like addFront - you'll have append etc.

What you can do is to wrap your type as an opaque type, and export the methods you need. This example is from the pybind11 documentation:

py::class_<std::vector<int>>(m, "IntVector")
    .def(py::init<>())
    .def("clear", &std::vector<int>::clear)
    .def("pop_back", &std::vector<int>::pop_back)
    .def("__len__", [](const std::vector<int> &v) { return v.size(); })
    .def("__iter__", [](std::vector<int> &v) {
       return py::make_iterator(v.begin(), v.end());
    }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
    // ....

CodePudding user response：

According to unddoch answer, I slightly reworked the code. Here are the changes that might be of help for others.

First, replace the type_caster block of code with:

PYBIND11_MAKE_OPAQUE(Es);

and if binding instructions for py::class_<Es>(m, "Es") are enlarged with:

    .def("__len__", [](const Es& v) {return v.size();})

the Python sequence executes as:

>>> from AWB import ID, E, Es
>>> es = Es(ID.wrongParam)
>>> es = es.addFront(ID.unknown)
>>> len(es)
2