Below are a few common examples for the use of FLCL. This examples can be loaded in FLCC from the use case dialog.
Display information for a file on Windows command shell.
flcl info get.file=test.zip
Show all supported CCSIDs on z/OS using a JCL batch job.
//FLCLINFO EXEC PGM=FLCL,REGION=0M,PARM='INFO GET.CCSIDS' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=*
Read a FLAMFILE and write it as record-oriented dataset on z/OS. This is the same as FLAM DECO of FLAM4. If you use FLAMFILE/FLAMIN and FLAMOUT as DD name, then the FILE specifications are not required. The DD name FLAMIN is the default for input files on z/OS. If FLAMOUT or FLAMFILE (only if it is a FLAMFILE) not allocated the default DSN (original name plus extension) is used if no file specification given. The default on other platforms would be the standard input or output stream, so you can use this to pipe the data streams.
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='CONV=DD:PARM MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //INPUT DD DSN=&SYSUID..TEST.ADC,DISP=SHR //OUTPUT DD SYSOUT=* //PARM DD * READ.FLAM4( FILE='DD:INPUT' ) WRITE.RECORD( FILE='DD:OUTPUT' ) /*
Read a member from a FLAMFILE and write it as record oriented dataset on z/OS with character conversion from CP1252 to IBM1141 (same as the job above but with dynamic allocation).
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='CONV=DD:PARM MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * &1140; READ.FLAM4( FILE='~.TEST.ADC' MEMBER=MEMBERNAME CCSID=1252 ) WRITE.RECORD( FILE=STREAM CCSID=IBM-1141 ) /*
Be aware that tilde (abbreviation for <SYSUID>
) is a character with
different code points in EBCDIC. The environment variable LANG or the
corresponding system variables must be defined to the CCSID used to
build the in line control statements so that CLP can interpret this
correctly else you can define the CCSID used for this CLP string like
in the example above.
Read a record-oriented dataset on z/OS and write it to two targets at the same time.
The password for both instances is specified in a file (which is written
inline for better understanding of the example). There is no parameter
file assigned to the CONV command. In this case the default DD name
FLAMPAR
are used.
The next example shows how you can reconstruct the stored host dataset from the ZIP archive.
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='CONV MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //FLAMIN DD DSN=&SYSUID..TEST.DAT,DISP=SHR //FLAMFILE DD DSN=&SYSUID..TEST.ADC,DISP=NEW //FLAMOUT DD DSN=&SYSUID..TEST.ZIP,DISP=NEW //PASSFILE DD * e'Password' /* //PARM DD * READ.RECORD() WRITE.FLAM(pass=f'DD:PASSFILE') WRITE.RECORD(encr.pgp(pass=f'DD:PASSFILE') archive.zip()) /*
The password is defined to use EBCDIC encoding as binary interpretation.
With an a
you can ensure ASCII but normally we recommend to use x
for an hexadecimal string.
Read the PGP encrypted host dataset from the ZIP archive and reconstruct it with the original name and DCB parameter.
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='CONV MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //FLAMIN DD DSN=&SYSUID..TEST.ZIP,DISP=SHR //PASSFILE DD * e'Password' /* //PARM DD * READ.RECORD(decode decr.pgp(pass=f'DD:PASSFILE')) WRITE.RECORD() /*
Read a text file on windows and write EBCDIC records to a FLAMFILE for a German z/OS.
flcl conv "read.text(file=test.txt) write.flam(file=test.adc ccsid=1141)"
Read a remote text file from a windows system (its a GZIP stream in the home directory) and write it on z/OS to a record oriented dataset in EBCDIC(1047).
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='CONV=DD:PARM MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * READ.TEXT( FILE=ssh://<cuser>@example.com/test.gz CCSID=1252 ) WRITE.RECORD( FILE=<SYSUID>.OUTPUT.DAT CCSID=1047 ) /*
Write a record oriented host dataset as text in UTF-8 to a PGP file (signed and encrypted), which will be stored as ASCII-armor encoded member in a remote ZIP archive on a UNIX system and convert the member name to a path name.
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='CONV=DD:PARM MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //STDENV DD * LANG=de_DE.IBM-1141 HOME=/u/hugo USER=hugo ENVID=T /* //PARM DD * READ.RECORD( FILE=~.TEST.XMIT(MYTEXT) CCSID=1141 ) WRITE.TEXT( FILE=ssh://<cuser>@example.com/out.zip CCSID=UTF-8 encrypt.pgp(userid='receiver.name' signid='my.name' armor()) archive.zip(member='[member].txt') ) /*
If the input output name mapping with square brackets used, then the
CCSID for the correct interpretation of the CLP string must be defined
(same for tilde). The @
in the URL is also a character with different
code points. In this case FLAM supports also the ampersand as separation
between the user and the server, to prevent the requirement to know the
CCSID used for correct interpretation. In the example above the DD name
STDENV are used to define the most important environment variables
required to run FLAM commands.
Read all XML files from ZIP archive on UNIX in UTF-8 and convert it to a FLAM archive for z/OS in EBCDIC. XML in a FLAMFILE is record-oriented and pretty printed by default.
flcl conv "read.xml(file=test.zip/?*.xml) write.flam(file=test.adc member=<CUSER>.XMLPDS([base]) ccsid=1141)"
Compare the content of a BZIP file with the content of a FLAMFILE text record by text record with suppression of trailing whitespace on a UNIX system.
flcl diff "read.text(file=test.bz ccsid=Latin-1 SUPTWS) compare.text(file=test.adc ccsid=1141 SUPTWS)"
Write and read a FLAMFILE on UNIX with piping (DECO is default).
cat test.txt | flcl flam comp | flcl flam
OpenPGP key management and encryption using the PGP key ring implementation of libfkm5 on a UNIX system (PGPRING is the default function of libfkm5 library on non IBM platforms. For example, on z/OS, PGPCCA is the default function and with z/OSv2r2 no additional FKM5 parameters are required. This means that the same will work on z/OS with ICSF except that SETENV as first step is not required).
flcl setenv FL_DEFAULT_PGPRNGPARA="pub='~/.pgp/pubrng.pgp',sec='~/.pgp/secrng.pgp',pass=f'~/.pgp/keyring.pwd'"
flcl key "gen.pgp(user='<cuser>')"
flcl key "exp.pgp(user='<cuser>' file=~/mykey.pgp)"
flcl key "imp.pgp(file=~/max.pgp kidver=12B738)"
flcl conv "read.binary(file='message.txt') write.binary(file='message.pgp' encrypt.pgp(userid['Max','<cuser>'] signid='<cuser>'))"
flcl conv "read.binary(file='message.pgp' deco) write.binary(file='message.out')"
Use SSH in interactive mode to read a remote ZIP file from a Windows server,
convert the character set and write all members (/?*
) as remote gzip files to
a Unix server (not in interactive mode (URL used)) and delete all members from
the ZIP archive (remove) and the empty archive itself. The interactive mode
(enabled by hostkeycheck=ask
) allows FLCL to require user input instead of
failing, e.g. if the SSH host is unknown and must be granted trust manually.
flcl conv "read.text(net.ssh(hostkeycheck=ask, user='<cuser>', host='winserver') file='test.zip/?*' remove ccsid=1251) write.text(file=ssh://<cuser>@example.com/[name].gz ccsid='UTF-8' comp.gzip())"
Convert a FB dataset with 2 columns to a CSV text file without a headline. The first column is a whitespace-padded name in EBCDIC (local character set) with up to 32 bytes. The second column is a 4 byte packed BCD number with the age of the person. The string should be collapsed (remove leading, trailing and repeated whitespace) for the CSV file, which will be written in the local character encoding (EBCDIC).
//FLCLCONV EXEC PGM=FLCL,REGION=0M,PARM='XCNV=DD:PARM MAXCC=-2' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * INPUT(SAV.FIL(FIO.REC(NAME='DD:FBINP') FMT.TAB(FORMAT=FIX ROW='DD:FBROW'))) OUTPUT(SAV.FIL(FMT.TAB(FORMAT=CSV DEFAULTS(NOHDLN)) FIO.REC(NAME='DD:CSVOUT'))) /* //FBROW DD * NAME='my.fb.record' COLUMN( NAME='name' TYPE.STRING(CHRSET(WHITESPACE=COLLAPSE)) MAXLEN=32 ) COLUMN( NAME='age' TYPE.INTEGER(FORMAT.BCD(TYPE=PACKED)) MAXLEN=4 ) /*
Binary copy of a file per SSH to a remote system, with checksum calculation over the local original file as pre-process and over the copied remote file as post process and a final post-processing which compares the check sums. This could be used for example to realize a receipt for a data transfer.
flcl xcnv "input(save.file(fio.blk(file=wrtorg.bin pre(command='sha1sum [copy]' stdout=wrtorg.local.sha1)))) output(save.file( fio.blk(file=ssh://<USER>:password@example.com/wrtorg.bin post(command='sha1sum [copy]' stdout=wrtorg.remote.sha1)) post(command='diff wrtorg.local.sha1 wrtorg.remote.sha1' stdout=log)))"
Binary copy of a file per SSH to an IBM mainframe system with a post-processing which copies the file in a MVS data set and removes the copied file from USS on success.
flcl xcnv "input(save.file(fio.blk(file=wrtorg.bin))) output( save.file( fio.blk(file=ssh://<USER>:password@example.com/wrtorg.bin postpro(command='cp -B [copy] "//TEST.XMIT([base])"') postpro(command='rm [copy]' on=success))))"
Sometimes it is required to copy some directories from USS and/or a remote
system to one (or more) PDS(E) on MVS. The following example is a job step that
is part of FLAM5 license generation. This step copies assembler macros from
three USS directories (IBM1140) to a PDSE on MVS which is required to assemble
the license module. The library is newly created every time (RENEW (removes the
DSN and re-allocates the library with DISP=NEW
)). No preceding IEFBR14 step
is required to delete the output dataset first. The example does something
similar to what OGETX does, but with FLAM the input files could also be in
UTF-8 on another system, if you use a URL and/or members in a ZIP archive
and/or PGP encrypted.
//COPYMAC EXEC PGM=FLCL,PARM='CONV=DD:PARM MAXCC=-2',REGION=0M //STEPLIB DD DSN=LIMES.FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * READ.TEXT(FILE='/u/<cuser>/git/FL5/HASMMAC/*.mac' FILE='/u/<cuser>/git/FL5/HASMMAC/ZOS/*.mac' FILE='/u/<cuser>/git/FL5/HASMMAC/COLUMBUS/*.mac' CCSID=1140) WRITE.RECORD(FILE='<SYSUID>.LICORDER.HASMMAC([base])' FALLOC(ORGA=LIB RECF=FB RECL=80 SPACE(PRIMARY=1 SECONDARY=1) RENEW) CCSID=1140) MESSAGE(MINIMAL) /*
This example is also used in our build, test and deploy automation and use the SYSOUT allocation of a mail writer on z/OS to send an email if anything is ready with the converted log as attached GZIP file.
//FLCLMAIL EXEC PGM=FLCL,REGION=0M,PARM='CONV MAXCC=-2' //STEPLIB DD DSN=&SYSUID..AUTH.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //FLAMPAR DD * read.record(file='<SYSUID>.SCLM.FITSTOUT.*' ccsid=1141) write.text(method=unix ccsid='UTF-8' file='[copy].gz' FALLOC(SYSOUT(FORMAT.MAIL( FROM='<SYSNAME>@zos.example.com' TO='mail@example.com'))) COMPRESS.GZIP()) /*
By default the CLASS='A'
and the WRITER='CSSMTP'
are used. Additional,
it is possible to redirect the request with a remote workstation
specification and a destination user ID to another LPAR.
To encrypt a backup with OpenSSL-ENC.
flcl xcnv "input(save.file(fio.blk(file='wrtorg.bin' frcblk))) output(save.file( cnv.edc(KDF=PBKDF2 ALGO=AES MODE=CBC KEYLEN=KL256 PASS=a'123456789012345678901234567890') fio.blk(file='wrtorg.bin.enc')))"
To decrypt an OpenSSL-ENC backup
flcl xcnv "input(save.file(fio.blk(file='wrtorg.bin.enc' frcblk) cnv.edc(KDF=PBKDF2 ALGO=AES MODE=CBC KEYLEN=KL256 PASS=a'123456789012345678901234567890'))) output(save.file(fio.blk(file='wrtorg.bin')))"
Find datasets containing a certain pattern.
//FIND EXEC PGM=FLCL,PARM='CONV MAXCC=-2',REGION=0M //STEPLIB DD DSN=LIMES.FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //FLAMPAR DD * READ.TEXT(FILE='HLQ.**' SUPTWS CCSID=DEFAULT CHRMODE=SUBSTITUTE REGEXP(PATTERN='^find$' NOCASE IGNREC)) WRITE.TEXT(FILE=DUMMY CCSID=LOCAL CHRMODE=SUBSTITUTE) DIR(LINK ALIAS HIDDEN RECURSIVE ARCHIVE) MESSAGE(MATCH,SOURCE) /*
This example searches for the record with content find
in all datasets
(PS/PO/VSAM) under the high level qualifier HLQ
(not case sensitive).
A log message with the match count and the filename of each match is
written to the log. The datasets can be record-oriented, with length
fields in front of each record or with delimiter after each record. The
datasets can be encoded, encrypted, compressed and/or in clear form. The
search term is specified as regular expression. The output with the records
found are written to the trash, but can also be written to SYSPRINT
(FILE=STREAM
) or to any other dataset (FILE='HLG.MY.FINDS'
).
FLAM5 archives can be used for a multitude of tasks. Below are simple examples that demonstrate the life cycle of such an archive, including archive creation, adding new versions, managing keys and old versions, copying segments or extracting the data.
flcl archive "comp(read.binary(file='file.bin') to.new(store.folder(name='archive-folder' overwrite) flam(encrypt.pwd(data(pw='password')))))"
flcl archive "comp(read.input(sav.file(fio.blk(name='file.bin'))) to.old(store.folder(name='archive-folder') flam(decrypt.pwd(password='password'))))"
flcl archive "comp(read.text(file='newfile.txt') to.modify(store.folder(name='archive-folder') flam(decrypt.pwd(password='password'))))"
flcl archive "list(store.folder(name='archive-folder') decrypt.pwd(password='password'))"
flcl archive "verify(store.folder(path='archive-folder(1)') flam(decrypt.pwd(password='password')))"
flcl archive "deco(from(store.folder(name='archive-folder') flam(decrypt.pwd(password='password') member='file.bin')) write.binary(file='file.bin'))"
flcl archive "deco(from(store.folder(NAME='archive-folder(-1)') flam(decrypt.pwd(password='password'))) write.binary(file='directory/[FILENAME]'))"
KEYMODE=REPLACE
).
This will create a new archive version with the same content as the
previous version, but with internal session keys re-encrypted with the
new password. All older versions are deleted. Otherwise, data from
older versions would still be accessible using the old password (or
potential PGP keys). On success, the archive is pruned, reducing the
size of the archive by the amount of data that has been deleted.flcl archive "rekey(store.folder(name='archive-folder' delete=ALL-OLD-VERSIONS prune) keymode=REPLACE import.pwd(password='password') export.pwd(data(pw='123456')))"
KEYMODE=ADD
is the default
and therefore does not need to be specified (put as comment below). If
you do not want to add the new encrypted keyset but replace it, you
need to specify KEYMODE=REPLACE
like in the example above.flcl archive "rekey(store.folder(name='archive-folder') #keymode=ADD# import.pwd(password='123456') export.pwd(data(pw='password')))"
flcl archive "copy(from(store.folder(name='archive-folder(0)') flam(decrypt.pwd(password='password'))) to.duplicate(store.file(name='archive.file.fl5' overwrite)))"
file.bin
is no longer acessible in the
archive. Otherwise, the previous version would still contain file.bin
.
The DELVSN command could also be used to explicitly delete the version
afterwards. The PRUNE command actually removes all previously deleted
data by re-organizing the archive. This can be done in a single step or
with multiple commands:flcl archive "delete(store.file(name='archive.file.fl5' delete=PREVIOUS-VERSION prune) flam(member='*.bin' decrypt.pwd(password='123456')))"
is equivalant to the following 3 commands:
flcl archive "delete(store.file(name='archive.file.fl5') flam(member='*.bin' decrypt.pwd(password='123456')))"
flcl archive "delvsn(store.file(name='archive.file.fl5' delete=PREVIOUS-VERSION) decrypt.pwd(password='password'))"
flcl archive "prune(store.file(name='archive.file.fl5'))"
flcl archive "deco(from(store.file(name='archive.file.fl5') flam(member='*.txt' decrypt.pwd(password='password'))) write.text())"
password
as the second encrypted keyset above
and now we want to remove it again so that only the first password
123456
remains valid. Since we are doing this with the password
password
, we are removing our own access, which must be forced with
FORCE=OWN
. With the index specification of -1, all encrypted keysets
with indices greater than 1 are deleted.flcl archive "delkey(store.file(name='archive.file.fl5' delete=ALL-OLD-VERSIONS prune) decrypt.pwd(password='password') index=-1 force=OWN)"
123456
works. Of
course you can also delete the file and the directory on the disc,
bypassing the credentials check, but if you use FLAM you must have
access to the keys and the associated rights.flcl archive "delarc(store.file(name='archive.file.fl5') decrypt.pwd(password='123456'))"
flcl archive "delarc(store.file(name='archive-folder') decrypt.pwd(password='password'))"
The second example for the use of FLAM5 archives illustrates the actual
core business, in which we index the first 2 bytes of text files and
then select all records beginning with Ab
and store them as a subset.
If you do not use a text file but a VSAM-KSDS, for example, its indexing
is automatically transferred to the archive and you can update, insert
and delete records accordingly. Since not everyone has a mainframe with
VSAM, we use a simple text file for demonstration, but if you use a
complex row specification, you can index every column and thus perform
complex accesses to the archive. Only a minimum number of segments is
used for this, which saves CPU cycles and increases security. For the
sake of simplicity, we only use data encryption (PROTECT=DATA) in this
example. We don't define the FKM5 parameter so that the commands do not
become too complex.
read.text()
, internally a TXTRST matrix is used,
where the first column (RECORD) contains the record and the second
column (REST) contains the rest, consisting of trailing whitespace
(SUBTWS) and the delimiter. We now want to index the record column,
for which we need to specify a corresponding row specification.ARCHIVE.COMP( READ.TEXT( FILE='*.txt' SUPTWS CHRMODE=SUBSTITUTE ) TO.NEW( STORE.FOLDER( PATH='ssh://user@server/archive-folder' OVERWRITE ) FLAM( ENCRYPT.PGP( PROTECT=DATA DATA( ID='max@mustermann.de' ) SIGNID='me@my.de' ) ROW( NAME='TXTRST' COLUMN( NAME='RECORD' INDEX( METHOD=BF2 DATLEN=2 SIGLEN=SL8 ) ) ) ) ) )
s
for all
records beginning with Ab
, where we have defined case sensitive
comparisons (the default) in the indexing above. The result is
specified to be a subset. If you don't define the subset parameter,
all members containing a record beginning with Ab
are transfered to
the output. Decryption with PGP takes place via the key id, which
means that no further parameters are required if the FKM5 standard is
used. ARCHIVE.DECO( FROM( STORE.FOLDER( PATH='ssh://user@server/archive-folder' ) FLAM( MEMBER='s*' MATCH.DATASET( FORMAT=CSV ROW( NAME='TXTRST' COLUMN( NAME='RECORD' ) ) RECORD='Ab*' ) SUBSET ) ) WRITE.TEXT( FILE='ab_erg.txt' ) )
ARCHIVE.LIST( STORE.FOLDER( PATH='ssh://user@server/archive-folder' ) ROWLIST='rowlist.txt' )
ARCHIVE.FILTER( ROWLIST='rowlist.txt' DATASET( FORMAT=CSV ROW( NAME='TXTRST' COLUMN( NAME='RECORD' ) ) RECORD='Ab*' ) OUTPUT='filter.txt' )
ARCHIVE.COPY( FROM( STORE.FOLDER( PATH='ssh://user@server/archive-folder' ) FLAM( MEMBER='s*' MATCH[='filter.txt'] SUBSET ) ) TO.DUPLICATE( STORE.FILE( NAME='archive.file.fl5' ) ) )
Ab
at the beginning from the
subset archive, which leads to the same result as above. ARCHIVE.DECO( FROM( STORE.FILE( NAME='archive.file.fl5' ) FLAM( MATCH.DATASET( FORMAT=CSV ROW( NAME='TXTRST' COLUMN( NAME='RECORD' ) ) RECORD='Ab*' ) SUBSET ) ) WRITE.TEXT( FILE='ab_erg.txt' ) )
Of course, this procedure only makes sense if access to the clear data
is cryptographically restricted to the owner and the archive operator
only has permission to search for suitable compressed and encrypted
segments. Since SAFTY=DATA
was specified for archive creation at the
beginning of this example, only the data segments are encrypted. The
directory containing the table of contents and the member information is
unprotected, which means that anyone can use the filter to compile the
subset, but only max@mustermann.de can read the clear records via the
DECO command.
dataset.txt:
FORMAT=CSV ROW( NAME='TXTRST' COLUMN( NAME='RECORD' ) ) RECORD='Ab*'
Use in FILTER and DECO command:
ARCHIVE.FILTER( ROWLIST='rowlist.txt' DATASET='dataset.txt' OUTPUT='filter.txt' )
ARCHIVE.COPY(#as above#)
ARCHIVE.DECO( FROM( STORE.FILE( NAME='archive.file.fl5' ) FLAM( MATCH.DATASET='dataset.txt' SUBSET ) ) WRITE.TEXT( FILE='ab_erg.txt' ) )
This example only hints at the possibilities that FLAM makes available
via its archives. To update, insert and delete records, you have to
split your data into tables and can then index some or all columns. For
indexed columns, you can define columns containing unique values as
primary keys. If you have such a dataset, you can then use the UPDATE
and DELETE commands to change or delete individual records, which
usually only involves unpacking and repacking a segment in the archive.
With COMP(... TO.MODIFY())
you can also perform mass insertions,
updates and deletes on such an indexed archive.
When searching, you can work with wildcards. Each indexed column
reduces the number of segments that need to be touched. So, if you
do not index at all in the example above or search for all records that
begin with O
('O*') and there is a wildcard in the index, then all
segments match and the whole archive is transferred as a subset and
then all records that begin with an O
are output as the result.
This means that in this case everything has to be calculated,
which means a lot of computational effort. Proper indexing reduces
this accordingly.