News / 22 May 2024

iRODS Development Update: May 2024


Hello reader,

iRODS 4.3.2 was released earlier this month and the iRODS User Group Meeting is happening next week. May is turning out to be a pretty big month!

Development on iRODS has slowed a bit due to conference preparations. We're working hard to make sure this year's UGM is the best it can be.

For the server, work to add support for Ubuntu 24.04 has begun. Being able to focus on adding new operating systems is a clear sign that things are headed in the right direction.

Automated Ingest received a few updates to its dependencies and the tests are now passing. Aside from that, we're considering the idea of making backwards incompatible changes to the interface. If you have ideas, questions, and/or concerns about that, please reach out to us.

The Globus Connector now has an automated testing framework. The plugin was updated to report its name to the iRODS server for ips. It now sets the O_CREAT flag on file uploads. Lastly, support for file modification time preservation and rename functionality have been implemented.

That wraps up this month's development update. See you at UGM!

May Technology Working Group

  • Imaging Working Group

    • initial effort is an OMERO integration
    • working meeting
      • Thur, June 20, 2024, 10am ET

  • Authentication Working Group

    • building a flexible authentication mechanism
    • now incorporated into iRODS 4.3.1
    • working meeting
      • Tues, June 25, 2024, 10am ET

  • S3 Working Group

    • have released C++ S3 API
    • working meeting
      • Fri, June 7, 2024, 3pm ET

  • Metadata Templates Working Group

    • draft paper for circulation among group
    • working on microservices
    • working meeting
      • Tues, June 18, 2024, 10am ET

  • TRiRODS

    • Wed, August 14, 2024, 4pm ET
    • speaker, TBD

  • 4.3.3

    • 41 bugs / 112 open, 0 closed

  • 4.3.4

    • 70 bugs / 242 open, 0 closed

  • 4.3 Backlog

    • 127 bugs / 290 open, 0 closed

  • 5.0.0

    • 6 bugs / 42 open, 2 closed

  • 5.0 Backlog

    • 0 bugs / 39 open, 0 closed

  • New Development Work

    • no updates

  • Active Development Work

  • Background Items

    • Docker-based Development Environment

      • updated to support libstdc++

    • Docker-based Testing Environment

      • support for podman contributed by Marco Roeland
        • https://github.com/irods/irods_testing_environment/pull/180

    • iRODS Kubernetes Testing Environment

      • v9 of the build and test infrastructure
      • can stand up iRODS Providers and Consumers
      • can run core tests
      • running in RENCI K8s cluster, but could run on any cluster

    • iRODS Documentation

      • added policy for reporting security vulnerabilities

    • iRODS HTTP API

      • released 0.3.0

    • iRODS S3 API

      • released 0.2.0
        • removed use of C++20 coroutines
        • implemented multipart uploads
          • using 'store and forward', at least for now
        • implemented DeleteObjects and GetObjectTagging
        • added config validation, connection pooling, and more

    • NetCDF microservices

      • released for 4.3.1
      • considering archiving this project in favor of other solutions
        • possible existing Python libraries are 'enough'

    • iRODS GenQuery2

      • packages for 4.2 and 4.3 can be built using the iRODS Development Environment
        • use the plugin builder images

    • iRODS Command Line Interface (CLI)

      • added support for many basic operations
        • ls, mv, tree, mkdir, get, put, cd, pwd, touch, etc
      • presented at UGM 2023

    • NFSRODS

      • released 2.2.0

    • Zone Management Tool

      • now compatible with iRODS 4.3.1
      • added support for remembering rows per page
        • /servers, /resources, /users, /groups
      • added ability to view ticket information
        • ticket creation not supported yet
      • added ability to change where the delay server is run
        • retrieving where the delay server runs is not supported yet

    • iRODS Password Booth

      • CherryPy web application, uses PRC to let users modify their own password

    • Metalnx

      • working towards removing the required local database

    • R client (rirods)

      • released 0.1.1
      • submitted and accepted to CRAN
      • new website - https://irods4r.org
      • presented at UGM 2023

    • Policy Composition

      • merged PR submitted for 4.3.0 compatibility

    • Publishing Capability

      • needs more README
      • awaiting CI

    • Logical Locking

      • read-locks to be implemented

    • New RPC API framework

      • leverages design from Authentication Working Group
      • supports synchronous and asynchronous operations
      • refactored into parallel_collection_operation base class
        • supports pre/post operations and an object operation

    • Hard Links Rule Engine Plugin

      • awaiting more use cases before release

    • AWS S3 Lambda

      • future release
        • could brute-force confirm checksums to detect renames
        • would persist metadata
      • could use atomic database operations to increase batch size > 1

    • Parallel Filesystem Integration

      • iRODS API plugin
      • have split BeeGFS and Lustre implementations into aggregator and listener
      • testing against known configurations

    • CockroachDB Database Plugin

    • Mungefs

      • removed dependency on externals, can be built as a standalone project

    • Jargon

      • preparing release of 4.3.3.0
      • added option to test framework that allows users to install other versions of iRODS 4.3
      • updated FilePermissionEnum to support new permission levels
      • added server version detection methods for iRODS 4.3
      • StartupPack now sends the correct value for

  • Discussion

    • Automated Ingest Capability
      • is anyone concerned with us making changes to the interface?
Kory Draughn