HELP: Amazon S3 or S3-compatible cloud object store TYPE: OBJECT SYNTAX: S3(PROVIDER(),BUCKET/CONTAINER='str',PREFIX='str',TIMEOUT=num)
Use Amazon S3 or S3-compatible cloud object store for archive read.
The archive is read from the specified S3 bucket. The version to read can be selected by specifying a version number (0 = latest, negative = relative to latest, positive = absolute version ID).
On z/OS this store does not use FLAM's libcurl-based HTTP client, which is not available there, but the z/OS Client Web Enablement Toolkit. That toolkit speaks HTTP/1.1 at most, so the requests are serialised and neither the prefetch nor the write queue can help; and outside 31-bit z/OS UNIX it needs a helper program in a second address space. A cloud archive on z/OS is therefore markedly slower than the same archive on an open platform, and raising DATBUF -- fewer, larger objects -- is the one setting with real effect. The STORE section "Storage Backends and How FLAM Reaches Them" explains the whole arrangement in one place.
BUCKET specifies the S3 bucket name (required). ACCESSKEY and SECRETKEY provide the AWS credentials (both required, kept secret). SESSIONTOKEN provides the AWS STS session token for temporary credentials obtained via AssumeRole, AWS SSO, or IAM Roles for EC2/ECS; it must be combined with matching ACCESSKEY and SECRETKEY (kept secret). REGION selects the AWS region (default: us-east-1). ENDPOINT overrides the endpoint URL for S3-compatible stores such as MinIO or Ceph. PREFIX adds a key prefix to all objects written in the bucket. PATHSTYLE switches from virtual-hosted-style to path-style URLs (required for some S3-compatible stores). PORT overrides the default HTTPS/HTTP port (0 = default).
SECRETKEY and SESSIONTOKEN are marked sensitive (CLPFLG_PWD) and are never echoed or logged. To avoid exposing credentials in shell history or process listings, read sensitive parameters from a file using the CLEP file-input syntax:
flcl xcmp store.s3(BUCKET=mybucket ACCESSKEY=AKIAIOSFODNN7EXAMPLE SECRETKEY=f'~/.fl5cred/s3_secret.txt') input.dat
The notation f'<path>' reads the parameter value from the named file. Store credential files in a dedicated hidden directory with strict permissions:
mkdir -m 700 ~/.fl5cred echo -n "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" > ~/.fl5cred/s3_secret.txt chmod 600 ~/.fl5cred/s3_secret.txt
Alternatively, export the credentials as environment variables before invoking flcl (see CS3 Environment Variable Fallbacks below). This is the recommended approach for non-interactive use (scripts, CI, container environments) as the values remain outside the command line entirely.
All connection parameters can also be supplied via environment variables. The CLEP parameter takes precedence when it is set to a non-empty value; the environment variable is consulted only when the parameter is absent or empty:
BUCKET -- FL_S3_BUCKET ACCESSKEY -- AWS_ACCESS_KEY_ID SECRETKEY -- AWS_SECRET_ACCESS_KEY SESSIONTOKEN -- AWS_SESSION_TOKEN REGION -- AWS_DEFAULT_REGION ENDPOINT -- AWS_ENDPOINT_URL PREFIX -- FL_S3_PREFIX PATHSTYLE -- FL_S3_PATH_STYLE (any non-empty value enables path-style)
STRING: BUCKET/CONTAINER='str' - S3 bucket name [required; env: FL_S3_BUCKET]STRING: PREFIX='str' - Object key prefix within the bucket [default: none; env: FL_S3_PREFIX]NUMBER: TIMEOUT=num - Seconds a request may make no progress: connect, or a transfer below 1 KB/s (0 = default) [30]