Skip to content
Snippets Groups Projects
s3_file_storage.rst 3.55 KiB

S3 file storage

To share files S3 (Simple Storage Service) is used. In this tutorial we use MinIO client to work with it.

You can get access, browse and download files using your browser, using this link:

ATHENA S3 Space

The LOGIN and PASSWORD for read access are available for all members (but can't be published at open places like this site). Please ask on #software-helpdesk Slack channel or email one of the SWG members.

One can access S3 directly using TFile like this

TFile* file = TFile::Open("s3https://dtn01.sdcc.bnl.gov:9000/eictest/ATHENA/<path>.root","AUTH=<login>:<password>");

Or set environment variables S3_ACCESS_KEY and S3_SECRET_KEY instead of using AUTH and putting login and password to code;

export S3_ACCESS_KEY=<login>
export S3_SECRET_KEY=<password>

XRootD access

One can utilize XRootD server to get access to files without credentials using:

root://sci-xrootd.jlab.org//osgpool/eic/ATHENA/<path to file>
std::string fileName = "RECO/JETS/crossDivNrgCrab/DIS_NC_Q2gt10_crossDivNrgCrab_25mRad_18x275_v1.0001.root";
TFile* file = TFile::Open("root://sci-xrootd.jlab.org//osgpool/eic/ATHENA/" + fileName);
// ...

(!) Currently (July 2021) only RECO is on xrootd (since we only have a 2.5 TB limit there)

Installation

Installing MinIO client:

wget https://dl.min.io/client/mc/release/linux-amd64/mc
mkdir -p $HOME/bin && mv mc $HOME/bin/mc && chmod +x  $HOME/bin/mc
alias mc="$HOME/bin/mc"
mc --autocompletion

> You may name the command diffirently if mc aka midnight commander which also uses mc is installed