Writing a Rule Engine Plugin (Python example)

One of the major features coming in iRODS 4.2 is the pluggable rule engine framework. This enables plugins to be written (in C++) that enable the development of rules in languages other than the native iRODS rule language. The Python rule engine plugin that will ship with iRODS 4.2 is the first example of such a plugin. This blog post walks through the source for the Python rule engine plugin, detailing the functions and behaviors that need to be implemented by any rule engine plugin. This is intended not only to introduce the Python rule engine plugin, but also to guide the development of future rule engine plugins for other languages.

Required operations

Any rule engine plugin needs to implement the following six functions to comply with the iRODS 4.2 pluggable rule engine framework: start, stop, rule_exists, exec_rule, exec_rule_text, and exec_rule_expression. Each of these functions has a return type of irods::error. (The following code can be copied verbatim into your implementation of a rule engine plugin.)

These functions are used as follows:

  • start – Called when the rule engine is initialized. start needs to handle any setup that is needed to use this rule engine: loading libraries, initializing interpreters, initializing global variables, etc.
  • stop – Called when the rule engine terminates. If any teardown needs to happen, it should happen within stop.
  • rule_exists – Called by the dispatcher whenever a rule is called. This function needs to check if a rule with the given name is implemented in this particular rule engine, and if so, set its _return parameter to true.
  • exec_rule – Called whenever a rule is triggered from within the rule engine. This is where rules are “normally” executed, that is, not from irule or delay/remote execution. This function needs to convert the input parameters (which arrive as a list of boost::anys) into a format that is usable in the language the rule engine is written for, execute the rule given those input parameters, and return output parameters (again as a list of boost::anys).
  • exec_rule_text – Called when a rule is executed using irule. Needs to parse the input rule string and turn it into a function that can be called with the given input and output parameters.
  • exec_rule_expression – Called when a block of code is run from the delay or remote execution queues. Likely to be similar to exec_rule_text, although the input code is formatted differently and so needs to be handled separately.

start

The start function is called when the rule engine is initialized. start needs to handle any setup that is needed to use this rule engine: loading libraries, initializing interpreters, initializing global variables, etc.

In the Python rule engine plugin, it:

  • Loads the libpython2.7 shared object
  • Initializes the Python interpreter
  • Adds /etc/irods to the system path
  • Defines the variables defined in PYTHON_GLOBALS (a std::map of std::string to std::string) in the Python core namespace

stop

The stop function is called when the rule engine terminates. If any teardown needs to happen, it should happen within stop.

In the Python rule engine plugin, it:

  • Does nothing; there’s nothing to tear down, so we simply return SUCCESS()

rule_exists

The rule_exists function is called by the dispatcher whenever a rule is called. This function needs to check if a rule with the given name is implemented in this particular rule engine, and if so, set its _return parameter to true.

In the Python rule engine plugin, it:

  • Imports the Python core namespace
  • Calls PyObject_HasAttrString to see if there is a function with the name rule_name defined in that namespace, and returns true if so, false if not.

exec_rule

The exec_rule function is called whenever a rule is triggered from within the rule engine. This is where rules are “normally” executed, that is, not from irule or delay/remote execution.

In the Python rule engine plugin, it:

  • Imports the Python core namespace
  • Imports the function rule_name from the Python core namespace as rule_function
  • Calls serialize_parameter_list_of_boost_anys_to_python_list on the input parameters to convert them from a C++ list of boost::anys to a Python list
  • Calls the Python function rule_function with 2 parameters: The input parameters as a Python list, and the rule engine callback
  • Converts the parameter list from a Python list back to a list of boost::anys so they can be used as output parameters elsewhere

exec_rule_text

The exec_rule_text function is called when a rule is executed using irule.

In the Python rule engine plugin, it:

  • Extracts the rule input parameters and output parameter from rule_arguments_cpp
  • Converts the input parameters and output parameter to a Python dict
  • Defines the variables defined in PYTHON_GLOBALS in the Python core namespace
  • Formats the rule_text so that it is valid Python code
  • Extracts the rule_name from the formatted rule text
  • Calls the Python function rule_function with 2 parameters: The input parameters as a Python dict, and the rule engine callback

exec_rule_expression

The exec_rule_expression function is called when a block of code is run from the delay or remote execution queues.

In the Python rule engine plugin, it:

  • Extracts the rule input parameters and output parameter from rule_arguments_cpp
  • Converts the input parameters and output parameter to a Python dict
  • Defines the variables defined in PYTHON_GLOBALS in the Python core namespace
  • Formats the rule_text so that it is a valid Python function
  • Calls the Python function rule_function with 2 parameters: The input parameters as a Python dict, and the rule engine callback

The RuleCallWrapper struct

The RuleCallWrapper struct wraps a call to a rule in the Python Rule Engine plugin, and does the heavy lifting in the exec_rule, exec_rule_text, and exec_rule_expression functions.

It:

  • Extracts the rule’s parameter list from Python and converts it to a list of std::maps using convert_python_iterable_to_list_of_maps(rule_args_python)
  • “Deserializes” input data types into iRODS msParam types – This code does this for genQueryInp and genQueryOut. This deserialization step is necessary for any input parameters that cannot be represented as strings. Here, a Python dictionary is parsed to fill objects of genQueryInp and genQueryOut types.
  • Calls the effect_handler with the rule_name and the deserialized input parameters, returning an irods::error object
  • Checks the returned irods::error object to see if an exception needs to be thrown
  • Populates the returned object with the irods::error code, the irods::error status, and the serialized parameter values
  • Returns this object to Python

Next:

Previous: