iBridges configuration

The iRODS GUI works with iRODS instances of versions 4.2.11, 4.2.12 and 4.3.0.

Here, we show how to configure the application to work your specific iRODS instance. Please note, that some features (tab views in the GUI) depend on certain server settings. You will find those dependencies at the end of the document.

System dependencies

Python: you will need Python 3.9 or higher
pip: to install package dependencies you will need pip3, version 22.2.2 or higher

Installation

Install from git

git clone git@github.com:chStaiger/iBridges-Gui.git
cd iBridges-Gui
python -m pip install -r requirements.txt

You can already start the application with

./iBridges.py 
#or
python iBridges.py

Please see below how to configure the software to connect to your iRODS instance.

Install from zip/release

unzip iBridges-Gui.zip
cd iBridges-Gui
python -m pip install -r requirements.txt

You can already start the application with

./iBridges.py 
#or
python iBridges.py

Please see below how to configure the software to connect to your iRODS instance.

Download and install binaries

The installation of binaries or executables will follow soon.

Configuration

iRODS environment.json

Please create a directory ~/.irods
Your iRODS admin will provide an irods_environment.json file, its contents, or instructions on how to create it. Place that file into the .irods directory/folder.
- Linux: /home/\<username\>/.irods/irods_environment.json
- Mac: /Users/\<username\>/.irods/irods_environment.json
- Windows: C:\\\\....\\\<username\>\\.irods\\irods_environment.json
Here is an example that can be created with the iinit iCommands on Linux:

{
    "irods_host": "server.fqdn.nl", 
    "irods_port": 1247, 
    "irods_user_name": "username", 
    "irods_zone_name": "myZone", 
    "irods_default_resource": "myResc" 
}

iBridges config.json

iBridges will create its own configuration file in ~/.ibridges/ibridges_config.json containing the name of the last iRODS environment file used. This ibridges_config.json file can be updated to control other aspects of iBridges. For example:

{
    "last_ienv": "irods_environment.json",
    "davrods_server": "https://server.fqdn.nl",
    "eln_token": "<api key>",
    "amber_token": "<api key>"
    "ui_tabs": [
        "tabUpDownload",
        "tabELNData",
        "tabAmberWorkflow"
        "tabDataBundle",
        "tabCreateTicket"
    ],
    "force_transfers": false,
    "check_free_space": false
}

Options:

davrods_server: the address of a webdav server, used to create entries in ElabJournal
eln_token, amber_token: API keys for tabELNData or tabAmberWorkflow respectively
ui_tabs: configure which tabs are shown (Browser and Info tabs always are)
- tabUpDownload: a two-pane upload/download tab
- tabELNData: for the Electronic Lab Notebook, eLabJournal
- tabAmberWorkflow: up and download data to and from Amberscript directly from iRODS.
- tabDataBundle: (un)bundle datasets from/to four supported formats
- tabCreateTicket: create iRODS tickets for anonymous access
force_transfers: always write data to iRODS
check_free_space: checks the metadata free_space of resources and only shows resources with a free_space larger than 0.

The order of the list ui_tabs defines in which order the tabs will be presented in the GUI. By default the first tab is always the iRODS Browser and the last tab is always the Info tab; see here. These two tabs will always be loaded even when no tabs are configured.

The check_free_space and force_transfers options are REQUIRED and if not provided will be added automatically. The default values are

"check_free_space": true,
"force_transfers": false

However, if the iRODS instance does not provide information about free space through the resource’s metadata free_space, no resources will be shown and no uploads will be allowed.

To enable and allow data transfers in case free_space is not set, set the options to:

"check_free_space": false,
"force_transfers": true

This setting makes top-level resources which are not annotated with their free space visible in the drop-down menus allowing selection of them. In addition, it sets the force flag for uploads overriding resource overflow protection.

The logs for both GUI and CLI clients can be found in ~/.ibridges/iBridges.log.

Usage

export PYTHON_IRODSCLIENT_DEFAULT_XML=QUASI_XML
python iBridges.py

env PYTHON_IRODSCLIENT_DEFAULT_XML=QUASI_XML ./iBridges.py

iRODS server dependencies

Storage capacity

To protect the iRODS resources from overflowing you should install an event hook on the iRODS servers that fill the resources’ RESC_FREE_SPACE attribute in the iCAT. These can be either catalog or resource servers. Please add the following to the /etc/irods/core.re or another rule engine file:

Example storage capacity rule

######################################################
# Storage capacity policies.
# Update the metadata field free_space of the resource
# when data is moved there or deleted from it.
#
# Author: Christine Staiger (2021)
#######################################################

acPostProcForParallelTransferReceived(*leaf_resource) {
    msiWriteRodsLog("LOGGING: acPostProcForParallelTransferReceived", *Status);
    msi_update_unixfilesystem_resource_free_space(*leaf_resource);
}

acPostProcForDataCopyReceived(*leaf_resource) {
    msiWriteRodsLog("LOGGING: acPostProcForDataCopyReceived", *Status);
    msi_update_unixfilesystem_resource_free_space(*leaf_resource);
}

# for iput
acPostProcForPut {
    msi_update_unixfilesystem_resource_free_space($KVPairs.rescName);
}

# for storage update after irmtrash
acPostProcForDelete {
    msi_update_unixfilesystem_resource_free_space($KVPairs.rescName);
}

For very busy systems, updating this value for every upload or delete can be prevented by commenting out or removing the last two stanzas if performance is being hampered.

For more complex resource hierarchies, the top of the resource tree (the root node) will usually not be updated with the free space values, but if it is (the sum of all leaf nodes is asssumed), the value in any leaf nodes will be ignored. If the root node has no free space value, the sum of the leaf nodes will be used instead. If none of the resource nodes are annotated, an error will occur. This feature can be overridden by annotating the root node’s free space value with an arbitrarily large value. Please note, that this action disables the built-in protection offered by this client.

Resource status

In the tab Info users can view the status of the resources. This metadata can be set by admins:

iadmin modresc <resource name> status <String>

Data (un)bundling

iRODS 4.2.x currently has no support for compressed structured files outside the iCommand ibun. Therefore, without custom microservices installed on a given iRODS server, only uncompressed TAR files are supported.

TAR file format

The ibun help gives the example to use the -C option to change into the directory containing the potential contents of the TAR file. The bundling done with iBridges assumes this same format and creates TAR files likewise. For example bundling the contents of the collection /testZone/home/user/testColl containing:

/testZone/home/user/testColl/file1.ext
/testZone/home/user/testColl/file2.ext
/testZone/home/user/testColl/file3.ext

stores only the three data objects:

file1.ext
file2.ext
file3.ext

into /testZone/home/user/testColl.tar. Unbundling this data object recreates the /testZone/home/user/testColl collection if it does not exist and deposits the data objects into it. If there are already data objects or collections existing there, an error will result and the bundle will not be extracted.

(Un)bundle options

iBridges has one option for (un)bundling data: Force operations. If the option is checked, two types of operations will be forced: one is to overwrite a bundle/collection that already exists, and the other is to remove the bundle files or collection contents without first sending them to the bin. If this behavior is undesirable, DO NOT USE THIS FORCE OPTION.

It is recommended that any kind of destructive actions be done in a separate step.