Managing Logs/warnings In Python Extensions

TL;DR version: What do you use for configurable (and preferably captured) logging inside your C++ bits in a Python project? Details follow. Say you have a a few compiled .so module

Solution 1:

I'm a big believer in having as much work happen in Python as possible, leaving only the work that has to happen in C in C. So I like #2 better than #1, but you are right, it clutters up all your function signatures.

I'd create a module-level object to handle the logging, sort of like a callback. The Python code could create the object any way it likes, then assign it to the module object. The C code can simply use the global object to do its logging:

# Python:

import my_compiled_module

def log_it(level, msg):
    print "%s: Oh, look: %s" % (level, msg)

my_compiled_module.logger = log_it

# C

staticvoidlog_it(unsignedint level, char * msg){
    PyObject * args = Py_BuildValue("(Is)", level, msg);
    PyObject_Call(log_it, args, NULL);
    Py_DECREF(args);
}

Now you can simply call the C log_it function throughout your code, and not worry about how the Python code gets it done. Of course, your Python log_it function would be richer than this one, and it would let you get all of your logging integrated into one Python logger.

Solution 2:

Thanks for the information guys. I found the PyObject_CallFunction easier to use.

// C++ code with logger passed as 2nd argumentstatic PyObject *lap_auction_assign(PyObject *self, PyObject *args){
  PyObject *cost_dict;  PyObject *pyLogger;
 /* the O! parses for a Python object (listObj) checked to be of type PyList_Type */if( !PyArg_ParseTuple( args, "O!O", &PyDict_Type, &cost_dict, &pyLogger)) 
    returnNULL;
/*
....... then call the logger
*/char astring[2048];

sprintf( astring, "%d out of %d rows un-assigned", no_new_list, num_rows );
PyObject_CallFunction( pyLogger, "s", astring );

/* python */# where logging is the python logging module and lap_auction is your extension
cost_dict = {}
tmp_assign_dict = lap_auction.assign( cost_dict, logging.info )