This appendix describes the handling of the FLUC subsystem in IBM z/OS environment.
FLUC-sub is an interface to FLAM (z/OS), the compression and encryption utility, and may only be used in conjunction with a license for FLAM utility and FLUC subsystem.
Further information on FLAM can be found at the beginning of this manual.
FLUC (FLAM Universal Converter) is part of the FLAM family and converts, compresses and encrypts whole files. While FLAM focuses on FLAMFILES as archive/exchange file, FLUC focuses on processing any file regardless of its origin (original z/OS files, PC files, FLAM, GZIP, OpenPGP, XML, ZIP, ...). Data can be consciously converted and prepared for / by a target system. The data is read/written as z/OS files, USS files or can be sent directly, e.g. via SSH.
FLAM and FLUC contain interfaces (APIs) that can be used by custom applications. This allows flexible adaptation to existing tasks that cannot be handled by the utility alone.
While a reprogramming of the interface is certainly no problem, an embedding into existing programs is always associated with effort.
FLUC-sub is a subsystem of the z/OS operating system and is activated by the subsystem interface of the command language (JCL).
All calls of an application program to file operations (e.g. opening, closing a file, reading and writing sentences) are received and processed by FLUC-sub.
Changes are usually not necessary in the calling programs, the calling program does not notice any difference to conventional file processing. If necessary, DCB specifications must be specified in the DD statement or utilities (such as IEBGENER, SORT, special assembler routines). Without DCB information, the subsystem makes this data available when DISP=SHR.
If the application program opens its file, FLUC-sub opens the file named in the parameters. If a data set is written, compressed, encrypted or converted, FLUC-sub converts it into this file. If a data set is read, FLUC-sub takes it from the file and transfers it to the application program decrypted, decompressed and converted. Closing the original file also causes the named file to be closed.
With the parameter inputs, the entire variety of FLUC is available to the application program, i.e. an existing application program (in Cobol, PL/1, Assembler, C, Java) can use FLUC-sub to convert, compress, encrypt each data record during writing and store the data generated in this way directly in the same system, transport it to the target computer using an SSH connection. The file can then be a completely different type, completely independent of the one defined in the program, such as a FLAMFILE, GZIP, OpenPGP,ZIP, ... Likewise, the data can be formatted as required in the program, regardless of the origin of the data (ASCII, EBCDIC, XML, UTF8/16/32, ..., PC format, ...). And here too, the file to be read by FLUC is completely independent of the file defined in the program.
The actual processing does not take place in the address space of the application program but in a subtask of the server (started task). Thus the address space is relieved and only small resources are used by the caller. A subtask is used for each file. The number of subtasks thus determines the number of files to be processed simultaneously.
FLUC-sub is a subsystem of the z/OS operating system and is activated by the subsystem interface of the command language (JCL). The name of the subsystem is FLUC.
//ddname DD SUBSYS=(FLUC,['subsys-parameter',]'fluc-parameter')
possibly with
// DISP=SHR
This instructs the file with ddname to be processed with the FLUC subsystem. A file name is arbitrarily selectable and has no meaning for the processing and can also be omitted (then JES generates a name). This file name is transferred into a FLAMFILE when writing, so flexible use is possible.
FLUC parameters have to be given to the process, as you are used to with FLCL in the utility. Parameters for controlling the subsystem are only recognized, if they are specified first.
Faulty parameters are rejected by FLAM as JCL errors, so the job does not start at all. The JES check of the DD command remains unchanged.
If DISP=SHR is specified in the DD command, FLUC-sub inserts the DCB
specifications of a file to be read into the JCL, if no specifications
have been made and the file name can be recognized in the parameters
(READ.xxxx(FILE=filename)).
The number of files to be processed "simultaneously" by the subsystem depends on the number of generated subtasks of the server, otherwise FLUC does not limit it.
With this technique, almost any file can be processed sequentially in read or write mode, which otherwise could not be processed in the application program (see the example collection).
If DCB attributes are specified for ddname in the DD command, FLUC interprets them (in addition to the application program) as values for the original data, otherwise the specifications are defined in the program.
Before a batch job is started by JES, all JCL specifications are checked. JES passes the parameters for the subsystem; these are checked by FLUC-sub. If an error occurs, the DD statement is rejected as a JCL error. In addition, a message is displayed in the JCL list (see Example Collection; Messages). The job is therefore only started if there are no errors and is otherwise terminated as usual.
The connection to the FLUC subsystem is established by calling the OPEN command in the program. For Open Input only read, decrypted if necessary, decompressed and converted, for Open Output only write, converted if necessary, compressed, encrypted. With OPEN, the file type, file organization, record and block length are taken from the caller's program (or the DCB specifications in the JCL) and used to process the data records.
At the open time, the FLUC parameters specified in the JCL are also active and the processing routines are set accordingly.
If an error occurs, the error routine remains active in the calling program and is not overlaid by the subsystem. The error return codes returned by the subsystem correspond to those of the file system (see also: Manual DFSMS/MVS Macro Instructions for Data Sets) for VSAM error codes, that is, the error routines do not have to be changed. For FLUC errors and subsystem errors, additional messages are output using WTO, Routcde=11 using FLUC-sub (see Messages; Example Collection). They are documented both in the system log and in the JESYSMSG list.
With CLOSE on the file, the actually processed file is also closed by FLUC-sub. A new OPEN call is allowed.
FLUC-sub requires only small resources in the address space of the user. A possible error or termination of a program, file access or similar has no influence on subsystem accesses of other processes and users.
The "actual" processing and thus the utilization of resources takes place in a server (started task FLAMSTC). Sub-tasks are generated in the server at startup. Each OPEN on a file is assigned to such a task. The number of these tasks thus determines the maximum number of files to be processed in parallel. With a CLOSE on the file, the corresponding task is released again. The maximum possible number of tasks is ultimately limited by the size of the memory area required.
Errors that are detected in the server are reported to the application program as errors in the I/O system, and processing may not be continued. The associated sub-task is restarted in the server if necessary and is available for the next OPEN call. System errors (such as a classic addressing error 0C4) in the calling program are recognized, the file to be processed in the server is closed, and the server sub-task is terminated. The server sub-task is again available to a new Open.
FLAM/FLUC must be installed both as a utility and as a subsystem on the current computer, the associated started task (the server) must be active.
Data records can be written or read. The accesses must be logical, i.e. in sequential order. Access via key or pointer is not possible !
If no file attributes are defined in the application program (such as record or block size), these specifications can be defined in the DD statement (see example collection). For example, utilities such as IEBGENER or SORT require these specifications. It is also possible to set the desired format using the FORMAT parameter specification. With DISP=SHR in the DD command, the file attributes recognized in the FLUC are set accordingly.
When reading, the record buffers must match the data to be read. If the buffer is too small, the subsystem reports an error (RC=44 (VSAM)).
Following restrictions apply:
Programming languages with a Runtime System (like COBOL, C, ...) do not allow VSAM files via FLUC subsystem as input/output. Unlike in the FLAM subsystem, FLUC-sub can only support assembler programs that do not (so accurately) check whether a VSAM file really exists.
Parameters for FLUC are transferred in the DD statement with the SUBSYS specification:
SUBSYS=(FLUC,['subsys-parameter',]'FLUC-parameter1,FLUC-parameter2,...')
or also
//...SUBSYS=(FLUC,['subsys-parameter',] // 'FLUC-parameter1', // 'FLUC-parameter2',...')
SUBSYS-parameter is used to control the subsystem itself and must be specified before the FLUC parameters.
Often the bytes allowed for parameters are not sufficient. With the specification
//ddname DD DSN=filename, // SUBSYS=(FLUC,'PARMDD=ddname2')
the ddname of a parameter file (DSORG=PS,RECFM=V/VB/F/FB) is specified, as is now also common in the EXEC PGM statement. E.G.
//ddname2 DD *
parameter, ...
parameter, ...
/*
Subsequent parameters are ignored.
The FLUC parameters correspond to those of the FLUC Utilities (see also: FLCL manual).
FLUC-sub only displays a message at the console in case of an error (via WTO ,ROUTCDE=11). It is also logged in the JCL log.
The letter after the message number specifies the time at which the error was detected:
I - during initializationC - when analyzing the JCLO - when opening a fileD - during the I/O processE - when closing a fileApart from the initialization of the subsystem, errors only affect the specified file of the respective job, i.e. the subsystem remains active for the other files and users.
FLM0500I |
INITIALIZATION OF SUBSYSTEM FLUC COMPLETED The FLUC subsystem has been initialized correctly and is now available. Response: None |
|---|---|
FLM0501I |
SUBSYSTEM FLUC NOT INSTALLED ON THIS SYSTEM The subsystem cannot be initialized. It has not been installed on the computer. Response: Define and start the started task in SYS1.PROCLIB. |
FLM0502I |
SUBSYSTEM FLUC ALREADY INITIALIZED The subsystem is already defined and active, a restart was rejected. The already active subsystem remains active. Response: FLUC-sub need not be restarted. |
FLM0503I |
NO MEMORY RECEIVED FOR INITIALIZATION The operating system cannot make any memory available for initialization. The subsystem is not active. Response: Increase REGION specification, analysis may be required (system dump). |
FLM0504I |
SUBSYSTEM COULD NOT FREE SYSTEM MEMORY The memory made available by the operating system cannot be freed. The subsystem has nevertheless been initialized and is now available. Response: An analysis is necessary (system dump). |
FLM0505I |
SUBSYSTEM FLUC FUNCTION MODULE NOT FOUND The work area received from the operating system could not be released. However, the subsystem has been initialized and is ready. Response: Set the subsystem modules in a library of the master identification (SYS1.) and link them with the LINKLIB. |
FLM0510I |
INTERNAL ERROR. RC = no An internal error has been detected. The subsystem has not been initialized. Response: Please inform your sales partner. |
FLM0515I |
INITIALIZATION OF SUBSYSTEM FLUC FAILED The subsystem cannot be initialized due to an error. The error situation was documented in a previous message. |
FLM0519I |
FLUC SUBSYSTEM IS NOT LICENSED Your license does not allow using the subsystem. Response: Please inform your sales partner. |
FLM0520C |
PARAMETER ERROR: ... An invalid parameter was entered in the DD statement of the JCL for FLUC-sub. Response: Correct parameters and restart job. |
FLM0523C |
NO MEMORY RECEIVED FOR JCL-CONVERTION A subsystem module was not allocated any memory by the operating system. Response: An analysis is necessary (system dump). |
FLM0530A |
PARAMETER ERROR: ... An invalid parameter was entered in the DD statement of the JCL for FLAM-sub. The invalid parameter is displayed. Response: Correct parameters and restart job. |
FLM0531A |
NO MEMORY RECEIVED FOR ALLOCATION A subsystem module was not allocated any memory by the operating system. Response: Increase REGION specification, analysis may be required (system dump). |
FLM0540O |
OPEN ERROR. DDNAME=ddname. RC= no FLUC/VSAM An error was detected for the file assigned with ddname in the JCL at open time. FLUC return codes are decimal, VSAM errors are decimal (see also: manual DFSMS/MVS Macro Instructions for Data Sets). This message is followed by the IEC141I 013-C0 message of the data management system, where the DD name used by the program is output. For VSAM accesses, the message IEC161 is output if necessary. The FLUC error codes correspond to the utility. Here are the most important ones: 40 = Error when reading the 1st record of a file Reaction: |
FLM0541O |
OPEN ERROR. MESSAGE FROM FLAM-SERVER: The message of the server about the Open error is logged. |
FLM0552D |
I/O ERROR. DDNAME = ddname. RC = no (FLUC)/RPLERRCD file-name An error was detected by the FLUC subsystem while reading or writing in the specified file ddname. FLUC return codes are specified in decimal format and VSAM errors RPLERRCD in hexadecimal format. All FLUC return codes are described in the FLCL manual. You can find a list of VSAM return codes in the manual DFSMS/MVS Macro Instructions for Data Sets). The message is followed by IEC020I 001-Message of the DMS. Some VSAM return codes: |
FLM0553D |
NO MEMORY RECEIVED FOR I/O OPERATIONS A subsystem module was not allocated any memory by the operating system. Consequently, no other information is available. Response: Specify a higher value for REGION in the EXEC statement, an analysis may be required (system dump). |
FLM0570E |
CLOSE ERROR. DDNAME = ddname. RC = no file-name An error was detected when an attempt was made to close the specified file. Since records may still have to be written when closing a file, see message FLUC552D. |
The subsystem is administered via a Started Task (Server). The name of the Started Task is freely selectable, the name in the delivery is FLAMSTC. It should be stored in the file SYS1.PROCLIB or correspondingly.
When starting the Started Task from the Console
S FLAMSTC,REUSASID=YES
both the server and the FLUC subsystem are started.
P FLAMSTC
terminates the server and the subsystem.
If the command
COM='S FLAMSTC,REUSASID=YES'
is inserted into the member COMMNDxx in SYS1.PARMLIB, FLUC-sub is automatically started at the IPL of the operating system and does not have to be entered extra at the console.
F FLAMSTC,LIST,VERSION
displays the names and versions of the modules involved in the console:
FLZS111I FLZ Controller Version 1.0, Date 20180504 16.02 started FLUC517I FLUCSUB V1.00 05/04/18-16.02 FLUC517I FLUCS07 V1.00 05/04/18-16.03 FLUC517I FLUCS16 V1.00 05/04/18-16.03 FLUC517I FLUCS17 V1.00 05/04/18-16.03 FLUC517I FLUCS38 V1.00 05/04/18-16.03 FLUC517I FLUCS39 V1.00 05/04/18-16.03
F FLAMSTC,LIST,TASKS
The list of generated tasks is displayed, as well as their status (active/inactive), the last function and the job name:
FLZS116I Number of server tasks: 10 Index TCB address active request area last function JOB ID 1 008FC320 NO 20C1E160 00000008 2 008C7528 NO 211B2160 00000000 3 008C7378 NO 21746160 00000000 4 008C7048 NO 21CD9160 00000000 5 008BEE88 NO 2226D160 00000000 6 008BEB58 NO 22800160 00000000 7 008BE9A8 NO 22D94160 00000000 8 008BE678 NO 23327160 00000000 9 008BE4C8 NO 238BB160 00000000 10 008BE198 NO 23E4E160 00000000
F FLAMSTC,REMOVE,TASK,##
releases the task with index number ## (from the list of the command LIST, TASKS, see above) again. If a task is blocked for any reason, it is terminated and restarted.
If it is necessary to start or stop the subsystem independently from the server, the following commands are available:
F FLAMSTC,FLUC,START zum Starten des Subsystems F FLAMSTC,FLUC,STOP zum Beenden des Subsystems
After deactivating FLUC-sub, modules of a newer FLUC-sub version could be
installed, which could then be activated (after a LLA REFRESH to the library)
by a new F FLAMSTC,FLUC,START.
The subsystem has only a limited ability to output error messages. As a rule, the application program only receives corresponding return codes, and of course it depends on the program how it reacts. Since the parameter entries are identical to those of the utility, we recommend that you first test the specifications with the utility:
//stepname EXEC PGM=FLCL,PARM='CONV=DD:PARAMS'
You receive detailed messages on the process.
Files are always read and written in the address space of the server (sub)tasks. The rights (RACF) of the caller apply.
For reasons of simplification, we write all parameters in the examples to a file and assign them to the subsystem:
//stepname EXEC PGM=FLCL,PARM='CONV=DD:PARAMS'
and e.g.
//PARAMS DD *
parameter, ...
parameter, ...
/*
In the examples only the parameter input is described.
When reading and writing via a subsystem, there are basically two different cases:
a) At the open time of the file, the program has specified the block and block length in the control block (e.g. in the FD section for COBOL).
Then the SUBSYS specification in the DD statement is sufficient:
//ddname DD SUBSYS=(FLUC,'PARMDD=PARAMS')
Since the "correct" values are specified in the calling program, the data records are passed/received by the subsystem according to these specifications. If the specifications with the file to be processed are not correct, error codes are set and a message is output (I/O error). Here the "correct" values can be set by the FORMAT specification, the read data records are then converted into the format string according to the statement.
b) The calling program has not defined block lengths. The format is also missing.
In the "normal case" (without subsystem) this information would be taken from the file catalog and entered into the control block (IEBGENER, SORT and many utility programs work in this way).
Additional DCB attributes can be specified in the DD statement:
//ddname DD SUBSYS=(FLUC,PARMDD='PARAMS'), // DCB=(RECFM=tt,LRECL=nnn,BLKSIZE=mmm),
If you do not know the record length, we recommend that you specify the following
DCB=(RECFM=VB,LRECL=32756)
if possible.
When reading and specifying DISP=SHR, the known attributes of the file are set correctly by the subsystem. If, for example, a FLAMFILE is to be read, the attributes of the compressed file are specified.
//ddname DD SUBSYS=(FLUC,'READ.RECORD(FILE=datei)'), // DISP=SHR
For unknown file attributes (such as files in USS, Unix, Windows, ...), the
DCB=(RECFM=VB,LRECL=32756,BLKSIZE=32760)
is set.
It must be possible to recognize the file name in the parameter specification of the subsystem (PARMDD= does not work there). If necessary, a separate parameter file can be specified with the READ command (READ=file name), which contains the file name to be read (see also: FLCL manual).
Finally, the attributes can also be specified by the FORMAT specification (FORMAT.RECORD(RECL= ....)). See: FLCL manual.
In the JESYSMSG listing you can recognize the subsystem activity by the system messages:
IEF237I FLUC ALLOCATED TO ddname . . IEF285I data.set.name SUBSYSTEM
If no file name is specified in the DD statement, the operating system generates a name. This name is visible in the message IEF285I.
In FLUC the concept of sentence has been extended. The sentence format, length, blocking of a file catalogued in z/OS no longer determines what a "sentence" is. This "physical format" can be used to interpret a "logical" one (see also: FLCL manual).
| BIN |
|
| CHAR |
|
| FLAM |
|
| RECORD |
|
| TEXT |
|
The file USER.EXAMPLE.INPUT is a file cataloged in the z/OS system. This file must be read via FLUC-sub.
Parameter:
READ.RECORD(FILE=USER.EXAMPLE.INPUT)
as a minimum. For this purpose several conversion, decompression, decryption and formatting instructions could be specified.
The file USER.EXAMPLE.INPUT could be a normal sequential PS file with the same record format, length as defined in the program, but also of the type FLAMFILE, GZIP, ZIP, ..., whose original files in the compressed files have these (max.) record lengths.
Instead of READ.RECORD you can also specify READ.TEXT, READ.BIN, READ.FLAM, etc. (see: FLCL manual).
If blocks longer than defined in the program (or specified in the DD statement) are read via FLUC, FLUC reports a block length error.
If the cataloged file is one with print control characters (e.g. FBA), a special feature applies:
By default, FLUC removes the control character from the record, so that for an FBA file with a cataloged record length of 133 bytes, only 132 are passed (as with FLCL utility). If the control character is to be retained, the following must be entered without further FORMAT specification
READ.RECORD(FILE=filename PRNCONTROL=RETAIN)
or
FORMAT.RECORD(RECF=FBA,LRECL=..., ...)
must be specified !
The file USER.EXAMPLE.OUTPUT is a file of the z/OS system. It has to be written via FLUC-sub. If the file is not cataloged, it is created again.
Parameter:
WRITE.RECORD(FILE=USER.EXAMPLE.OUTPUT)
as a minimum. In addition, several conversion, compression, encryption and formatting instructions could be specified.
The file /srv/ftp/input/sample.txt is stored in the USS under the UserID
of the caller. It is a text file (i.e. it contains printable characters
with delimiters).
Parameter (reading):
READ.TEXT(FILE='/srv/ftp/input/sample.txt')
Parameter (writing):
WRITE.TEXT(FILE='/srv/ftp/output/sample.txt')
The file /srv/ftp/input/beispiel.bin can be read binary via the SSH protocol
in units of 1024 bytes. It is located on a foreign server, the access is
protected with a password.
Parameter:
READ.BIN(FILE='/srv/ftp/input/beispiel.bin' RECL=1024 NET.SSH(HOST=tcp/ip-addresse USER=user PASSWORD=password HOSTKEY=ACCEPT))
The special feature here is the cutting of the binary data by FLUC in units of 1024 bytes (parameter RECL). The application program must be able to process this length, if necessary RECFM=VB,LRECL=1028 must be specified in the DD statement. Of course, other record lengths, as required by the program, can also be specified here.
Without RECL specification, blocks of 64 KB are transferred!
All records are to be written to a password encrypted OpenPGP file in the USS (Zip archive).
Parameter:
WRITE.TEXT(FILE='/tmp/pgpfile.zip') METHOD=UNIX CCSID='ISO8859-1' ENCRYPT.PGP(PASSWORD=password) ARCHIVE.ZIP()
Below two sample job steps using the IEBGENER for a better total understanding:
//STEPWRIT EXEC PGM=IEBGENER
//STEPLIB DD DSN=FLAM.AUTH.LOAD,DISP=SHR
//SYSOUT DD SYSOUT=*
//SYSDUMP DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD *
Test Data Set
with two records
/*
//SYSUT2 DD SUBSYS=(FLUC,'PARMDD=FLUCDD'),
// DCB=(DSORG=PS,RECFM=FB,LRECL=80)
//FLUCDD DD *
FORMAT.RECORD(CCSID=IBM-1141)
WRITE.TEXT(FILE='flucFS/physical/gz/test1.txt.gz'
NET.SSH(HOST='192.168.80.3' USER='falk')
METHOD=UNIX SUPTWS CCSID='UTF-8' COMPRESS.GZIP())
/*
//STEPREAD EXEC PGM=IEBGENER
//STEPLIB DD DSN=FLAM.AUTH.LOAD,DISP=SHR
//SYSOUT DD SYSOUT=*
//SYSDUMP DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD SUBSYS=(FLUC,'PARMDD=FLUCDD'),DISP=SHR,
// DCB=(DSORG=PS,RECFM=FB,LRECL=80,BLKSIZE=32000)
//SYSUT2 DD SYSOUT=*
//FLUCDD DD *
READ.TEXT(FILE='flucFS/physical/gz/test1.txt.gz'
NET.SSH(HOST='192.168.80.3' USER='falk'))
FORMAT.RECORD(CCSID=1141 recf=FBA recl=79 padchar='?')
/*