iRODS

News / 11 July 2016

Writing a Rule Engine Plugin (Python example)

One of the major features coming in iRODS 4.2 is the pluggable rule engine framework. This enables plugins to be written (in C++) that enable the development of rules in languages other than the native iRODS rule language. The Python rule engine plugin that will ship with iRODS 4.2 is the first example of such a plugin. This blog post walks through the source for the Python rule engine plugin, detailing the functions and behaviors that need to be implemented by any rule engine plugin. This is intended not only to introduce the Python rule engine plugin, but also to guide the development of future rule engine plugins for other languages.

Required operations

Any rule engine plugin needs to implement the following six functions to comply with the iRODS 4.2 pluggable rule engine framework: start, stop, rule_exists, exec_rule, exec_rule_text, and exec_rule_expression. Each of these functions has a return type of irods::error. (The following code can be copied verbatim into your implementation of a rule engine plugin.)

extern "C"
irods::pluggable_rule_engine*
plugin_factory(const std::string& _inst_name, const std::string& _context) {
    irods::pluggable_rule_engine* re = new irods::pluggable_rule_engine( _inst_name , _context);

     re->add_operation&(
             "start",
             std::function( start ) );

     re->add_operation(
             "stop",
             std::function( stop ) );

     re->add_operation(
             "rule_exists",
             std::function( rule_exists ) );

     re->add_operation(
             "exec_rule",
             std::function( exec_rule ) );

     re->add_operation(
             "exec_rule_text",
             std::function( exec_rule_text ) );

     re->add_operation(
             "exec_rule_expression",
             std::function( exec_rule_expression ) );

    return re;
}

These functions are used as follows:

start - Called when the rule engine is initialized. start needs to handle any setup that is needed to use this rule engine: loading libraries, initializing interpreters, initializing global variables, etc.
stop - Called when the rule engine terminates. If any teardown needs to happen, it should happen within stop.
rule_exists - Called by the dispatcher whenever a rule is called. This function needs to check if a rule with the given name is implemented in this particular rule engine, and if so, set its _return parameter to true.
exec_rule - Called whenever a rule is triggered from within the rule engine. This is where rules are “normally” executed, that is, not from irule or delay/remote execution. This function needs to convert the input parameters (which arrive as a list of boost::anys) into a format that is usable in the language the rule engine is written for, execute the rule given those input parameters, and return output parameters (again as a list of boost::anys).
exec_rule_text - Called when a rule is executed using irule. Needs to parse the input rule string and turn it into a function that can be called with the given input and output parameters.
exec_rule_expression - Called when a block of code is run from the delay or remote execution queues. Likely to be similar to exec_rule_text, although the input code is formatted differently and so needs to be handled separately.

`start`

The start function is called when the rule engine is initialized. start needs to handle any setup that is needed to use this rule engine: loading libraries, initializing interpreters, initializing global variables, etc.

irods::error
start(irods::default_re_ctx&, const std::string&) {
    // Load the libpython2.7 shared object
    dlopen("libpython2.7.so", RTLD_LAZY | RTLD_GLOBAL); // https://mail.python.org/pipermail/new-bugs-announce/2008-November/003322.html
    // Initialize the Python interpreter
    Py_InitializeEx(0);

    try {
        // Adds /etc/irods to the system path
        bp::object main_module = bp::import("__main__");
        bp::object main_namespace = main_module.attr("__dict__");
        bp::exec("import sys\n"
             "sys.path.append('/etc/irods')",
             main_namespace);

        // Defines the variables defined in PYTHON_GLOBALS (a std::map of std::string to 
        //     std::string) in the Python core namespace
        bp::object core_module = bp::import("core");
        bp::object core_namespace = core_module.attr("__dict__");
        for (const auto& it : PYTHON_GLOBALS) {
            core_namespace[it.first] = it.second;
        }

        initplugin_wrappers();
        StringFromPythonUnicode::register_converter();

    } catch (const bp::error_already_set&) {
        const std::string formatted_python_exception = extract_python_exception();
        LOG("caught python exception\n", formatted_python_exception);
        std::string err_msg = std::string("irods_rule_engine_plugin_python::") + __PRETTY_FUNCTION__ + " Caught Python exception.\n" + formatted_python_exception;
        return ERROR(-1, err_msg);
    }

    return SUCCESS();
}