At the iRODS Workshop at Supercomputing 2017 a couple weeks ago in Denver, we demonstrated three new capabilities:
The ingest demo was covered in Monday's post. The storage tiering demo was covered in yesterday's post. This post will be about demonstrating the recently released Audit (AMQP) Rule Engine Plugin.
The design goal for this demo was to tell a story about a particular data object within iRODS as it moved through a standard data object lifecycle. This might include ingestion, annotation, sharing, etc.
For compliance and provenance use cases, this feature is very powerful and provides an unprecedented ability to make assertions about an entire infrastructure and its internal interactions.
https://asciinema.org/a/150261
When the demo begins, the full stack and message flow has already been configured and established. iRODS is running, along with ApacheMQ, Logstash, and Elasticsearch.
We create and put a file (named science.txt), modify its metadata, show the metadata, then get the data object and remove it from iRODS. During this sequence, we query the elasticsearch database to monitor the activity associated with the data object through its lifecycle.
The code powering the demo can be found here: https://github.com/irods/irods_sc17/tree/master/audit