Settling into the two month sprint to SC19 in Denver...
NFSRODS is nearing a v0.9.0 release that includes support for NFSv4 ACLs. We've added some additional caching for speed, and groups are behaving as expected. There are a couple more edge cases regarding temporary files and permissions that we're still chasing down, but this will be the first release we can recommend for public consumption.
This month saw the release of two versions of the iRODS Automated Ingest client, fixing two path-related bugs affecting scanning and syncing from S3 into iRODS. Please update via pip to v0.3.7.
We have designed and prototyped a parallel transfer engine that uses the new libraries included in 4.2.x and provides a clean interface for requesting data movement. Eventually, the plan is to refactor all existing iRODS calls to use this parallel code to reduce the number of codepaths and increase throughput.
We have also prototyped a new C++-based REST API for iRODS and are looking to implement a simple S3-compatible API above that. If you are interested in this work, please reach out.
Work on our parallel filesystem connectors has picked back up and has seen the splitting of the Lustre connector into a listener (connects to and receives updates from an external system) and an aggregator (does some rollup of changes before sending them to iRODS). This allows for different types of listeners to be paired with different systems' changelog formats. A BeeGFS listener has been developed to demonstrate the new model. We would like to encourage standardization of the changelog format through a Changelog Working Group and will announce more formal plans in the coming months.
The Metadata Templates Working Group has made significant progress in the last month on interoperability planning. We hope to publish a block diagram soon and demonstrate end-to-end running code. Please join our monthly calls if you want to know more about validation and verification of the metadata in your iRODS Zone.
Work continues on our Dockerized CI infrastructure and the intermediate replica state. We hope to be able to release 4.2.7 from our new infrastructure and sunset our existing Jenkins build and test server. Release of the cacheless S3 resource is awaiting this effort as well.
New Development Work
Active Development Work
Python iRODS Client (PRC)
Cacheless and Detached S3
iRODS Capability - Automated Ingest
Multipart Transfer, v5 API
Continuous Integration (CI)