Overview and Training Materials
Data to Compute
In many existing enterprise and research deployments, HPC clusters are separated from long term data storage technologies. When data needs to be moved to HPC and back again, the Data to Compute design pattern can leverage metadata-driven workflows and automate the execution of an organization's data management policy.
Compute to Data
When data is stored in specific locations due to a requirement for specialized software or hardware or only because it is too big and expensive to move, compute requests can be routed to the appropriate location automatically. This metadata-driven computation design pattern could serve as a bridge until the time services are more fully containerized.
As data matures and reaches a broader community, data management policy must also evolve to meet these additional requirements. iRODS virtualizes the stages of the data lifecycle through policy evolution.
iRODS provides a layer of abstraction which integrates with your pre-existing infrastructure. This flexibility allows your infrastructure to continue to change over time.
From Prototype to Production
Deploying iRODS requires making decisions about how quickly and how deeply to integrate with existing systems. The flexibility of iRODS allows for a dynamic approach that supports building confidence and trust in the software.
Tighter integration and automation can lead to better performance and stronger assertions about what has happened to your data throughout its lifecycle.
With metadata being a central part of how iRODS fosters best practices in workflows and provenance, it is also important to encourage good metadata curation. Metadata templates afford iRODS a friendly UI for specifying requirements, validation, and standardization.
Multipart Data Objects: Transfer
An effort to improve reliability and predictability of large file transfers in iRODS, multipart data objects will improve transport speeds (parallel and/or multisource), allow for cache-free object storage plugins, and provide natural support for reliable restarts.
User Group Meetings
Communication with the iRODS Community
Common Citations (iRODS 4.x)
Hao Xu, Ben Keller, Antoine de Torcy, Jason Coposky (2016) QueryArrow: Bidirectional Integration of Multiple Metadata Sources. 8th iRODS User Group Meeting, University of North Carolina at Chapel Hill. June 2016. (PDF
Reagan W. Moore, Hao Xu, Mike Conway, Arcot Rajasekar, Jon Crabtree, Helen Tibbo (2016) Trustworthy Policies for Distributed Repositories. 133pp. (publisher
Hao Xu, Jason Coposky, Ben Keller, Terrell Russell (2015) Pluggable Rule Engine Architecture. 7th iRODS User Group Meeting, University of North Carolina at Chapel Hill. June 2015. (PDF
Hao Xu, Jason Coposky, Dan Bedard, Jewel H. Ward, Terrell Russell, Arcot Rajasekar, Reagan Moore, Ben Keller, Zoey Greer (2015) A Method for the Systematic Generation of Audit Logs in a Digital Preservation Environment and Its Experimental Implementation In a Production Ready System. 12th International Conference on Digital Preservation, University of North Carolina at Chapel Hill. November 2-6, 2015. (PDF
) (direct link
Terrell Russell, Jason Coposky, Harry Johnson, Ray Idaszak, Charles Schmitt (2013) iRODS Composable Resources. 5th iRODS User Group Meeting, University of North Carolina at Chapel Hill. June 2013. (PDF
Reagan Moore, Arcot Rajasekar, Hao Xu (2015) DataNet Federation Consortium Preservation Policy Toolkit. 12th International Conference on Digital Preservation, University of North Carolina at Chapel Hill. November 2-6, 2015. (PDF
) (direct link
Arcot Rajasekar, Terrell Russell, Jason Coposky, Antoine de Torcy, Hao Xu, Michael Wan, Reagan W. Moore, Wayne Schroeder, Sheau-Yen Chen, Mike Conway, Jewel H. Ward (2015) The integrated Rule-Oriented Data System (iRODS 4.0) Microservice Workbook. 248pp. (PDF
Papers and White Papers
Policy-Based Data Management: The Future of Reproducible, Data-Driven Research
Dave Fellinger, DataDirect Networks; Reagan W. Moore and Hao Xu, University of North Carolina at Chapel Hill
American Laboratory. March 18, 2015
Data Intensive processing with iRODS and the middleware CiGri for the Whisper project
Xavier Briand, ISTerre, & Bruno Bzeznik, Universite Joseph Fourier
Control Your Data
RENCI and iRODS Consortium
Using an Integrated Rule-Oriented Data System (iRODS) with Isilon Scale Out NAS (external link, emc.com)
Principles of Archival of Digital Assets
John Burns, Archive Analytics
Concepts in Distributed Data Management, or DICE History of the DICE Group
Moore, et al.; UNC, UCSD
Composable Resources Paper, iRODS User Group 2013: March 2013 (279kB, PDF)
Russell, et al.; RENCI
Hardening iRODS for an Initial Enterprise Release (E-iRODS), iRODS User Group 2012: March 2012 (poster)
Russell, et al.; RENCI
Towards a Theory of Digital Preservation, IJDC, Issue 1, Vol 3, June 2008
Reagan W. Moore
Media & more ...