Starting with version 5.1.21, custom commands can be executed before and/or after conversions. These pre or post commands can be executed per file and/or per conversion task. Pre and post commands can be executed on the reading and the writing side of a conversion.
Per-file commands are executed either before the correspond file is opened or after it is closed. Commands are executed on the machine where the file is read from / written to. So, when writing a file via SSH, associated per-file commands are also executed on the remote host.
Executing commands per file is only supported with the FIO objects of the XCNV command at INPUT and OUTPUT. The READ and WRITE overlays of the CONV object currently only support global pre and post commands.
Per-file commands can access the filename / path of the associated file through the square bracket syntax known from filenames (see the section INPUT TO OUTPUT NAME MAPPING for details).
At write the file post-processing are only done by default in the final close of the I/O component. This results at directory walk in only one post-processing for the final file. This is required for example if you will write all the files to one archive or append anything to one file. If you process n input files to n output files it could be useful to enforce the post processing per file. Since version 5.1.24 of FLAM the switch 'PERFILE' could be activate for this.
Below you can find an example using XCNV with pre and post commands:
flcl xcnv
"input(sav.fil(fio.blk(file=ssh://user@in.ssh.server/test.txt
prepro(command='touch [name]')
postpro(command='rm [name]' #ccsid='ISO8859-15'#
stderr=stdout stdout=log on=success)
)))
output(sav.fil(fio.blk(file='ssh://user@out.ssh.server/test.gz' comp.gzip()
postpro(command='sha1sum [name]' stdout='ssh://user@in.ssh.server/test.sum/&UTF-8' on=success)
postpro(command='rm [name] on=failure)
)))"
In the input portion, a pre-processing step makes sure that the input file exists on the remote host. Output written to stdout or stderr by the command is ignored. Only in case of an error, the first bytes of stderr output are written to the error trace. The CCSID parameter can be set to the charset used on the remote system. This converts any command output from this CCSID to the local character set. For commands executed via SSH, this defaults to UTF-8 with automatic fallback to Latin-1. For local commands, the local character set is used by default. In the example above, the correct CCSID for USS on z/OS is included as comment.
The post-processing step deletes the input file, but only if the conversion succeeds. This command's standard output (stdout) and standard error (stderr), if any, is written as STDOUT to the log. The default CCSID (UTF-8/Latin-1) normally fit, but if the Euro-Symbol in the data, ISO8859-15 would be the better choice.
In the output portion, there are two post processing steps, one is executed on successful conversion, the other when an error occurs. On success, a SHA1 checksum of the written gzip file is calculated and the output is written (in UTF-8) to a file on the SSH host of the input file for verification purposes. The command output is converted to UTF-8 (from 'UTF-8' with 'Latin-1' fallback because no CCSID is specified.
On error, it is attempted to delete the possibly partially written file.
We recommend to use SAF checks to restrict how this feature can be used because it can execute arbitrary shell commands with the rights of the current user.