... print CLE_DOCTYP_COVER = FLAM4 Manual :doctype: book = FLAM4 Manual limes datentechnik(R) gmbh v5.1.28-33135, 2024-03-08: release. The new FLAM4 utility conforms to version 4 or older using the new FLUC components. It supports the FLCL FLAM command as a self-contained program. The FLAM command is the default command. It must not be provided on command line. The new FLAM4 utility is backwards-compatible to the old one for Unix, Windows and Mainframes, except: * Member selection * Output name conversion * And the protocol written All differences in detail can be find in the <>. This documents contains the complete old FLAM4 documentation in the new HTML format. Since version 5.1.21 of FLAM, the old PDF documentation is no longer supported. Beside the FLAM command, only the INFO command is supported. If you need more functionality please use the FLCL(R) utility. Copyright (C) limes datentechnik (R) gmbh + All rights reserved ... print CLE_DOCTYP_CHAPTER == Preface [preface] == Preface The new FLAM4 utility and the corresponding subprogram FLM4UP replace the old FLAM4 utilities and FLAMUPs on Windows and Unix. On z/OS the old FLAM utility and FLAMUP still exist. The new load modules FLAM4 and FLM4UP are additional entries available with FLAM version 5.1. This book will also replace the old FLAM4 documentation for Windows, Unix and Mainframes. The new FLAM command supports the FLAM syntax known form Windows, Unix and z/OS plus few extensions (UNICODE, SSH). To migrate from the old FLAM utility to the new one, the differences (completion codes, log output, input output name mapping, dafault values, etc.) are documented in the <> below. For the use of the old utilities and subprograms with this documentation the differences are written to each parameter below. The new FLAM4 utility based like the FLCL utility on the CLE/P library. The CLE/P library was developed by limes datentechnik and released as open source under the ZLIB license. CLE/P is a compiler to provide a platform independent command line interface for each kind of batch processing environment. The CLE/P library provides a lot of features including the <> used below. For example: We use this library to automatically create this document as part of our build process; If we add a parameter to the CLE/P tables and build the FL5 project this manual will be regenerated as well in order to be always up to date. This manual can by generated with the built-in functions GENDOCU or HTMLDOC provided by CLE/P library by calling the command below: ----------------------------------------------------------------------- flam4 gendocu flm4book.txt flam4 htmldoc flm4book.html ----------------------------------------------------------------------- To generate HTML documentation yourself instead of using the pre-built documentation, a service provider DLL must be available. The text file can be fed to Asciidoctor to build a PDF and/or an HTML5 document. ... print CLE_DOCTYP_CHAPTER === Trademarks === Trademarks Below, you can find all trademarks or registered trademarks of limes datentechnik (R) gmbh. These trademarked terms are marked with the appropriate symbol ((R) or (TM)), indicating registered or common law trademarks owned by limes datentechnik (R) gmbh at the time this information was published. The following terms are trademarks of limes datentechnik (R) gmbh in Germany, other countries, or both: * LIMES(R) - Short company name of the owner of this document * limes datentechnik(R) - Company name of the owner of this document * FLCL(R) - Frankenstein Limes Command Line * FLCC(R) - Frankenstein Limes Control Center * FLAM(R) - Frankenstein Limes Access Method * FLUC(R) - Frankenstein Limes Universal Converter * FLIES(R) - Frankenstein Limes Integrated Extended Security * FKMS(R) - Frankenstein Key Management System * FKME(R) - Frankenstein Key Management Extension/Exit * FLAMFILE(R) - A file based on FLAM syntax * CLE/P(R) - Command line Executer/Parser ... print CLE_DOCTYP_CHAPTER === License Terms === License Terms The license terms for all software components owned by limes datentechnik gmbh(R) can be found in LICENSE(.txt) in the installation folder doc/txt or in the partitioned dataset HLQ.FLAM.DOCTXT. Additional license information for external libraries used by this software can be found in ABOUT(.txt). The replacement of external libraries (LGPL) is described following the changelog in the file README(.txt). All license terms of the external libraries are included as files (LICxxxxx(.txt)) in doc/txt folder or in HLQ.FLAM.DOCTXT partitioned dataset. All old FLAM components till version 4 are completely owned by limes datentechnik gmbh (R). Only on Linux systems the glibc is used in unmodified form as prerequisite (part of the operating system, not part of the distribution) based on the LGPL license. The old FLAM4 components don't provide additional about and version commands or functions. The service provider interfaces (exits, extensions) of FLAM are owned completely by limes datantechnik gmbh(R) and provided to our users for individual extension to this software. The driver is conforming to this license terms but LIMES(R) is not responsible for such individual implementations which are not part of our distributen. Our own implementations against these SPIs (e.g. FAVECAV, PKCS#11, IBM-CCA/ICSF) are in sync with the stated license terms here. A lot of this information is generated in the package build process and can be requested over the corresponding built-in commands of our utilities or the corresponding functions of the APIs. To request the current valid state, please use the command or functions below (there are more APIs with analog functions available): ------------------------------------------------------------------------ COMMAND API 1 API 2 ... API n > flam4 license fcblic() FCRLIC() ... fliconv_license() > flam4 about fcbabo() FCRABO() ... fliconv_about() > flam4 version fcbvsn() FCRVSN() ... fliconv_version() ------------------------------------------------------------------------ The about and version information are also added as appendix to the manual's provided by limes datentechnik gmbh(R). The version information contains version, release and revision number of each component followed by the current build number. The versioning of the different components is independent of each other. The build number must be the same for each component and must correspond to the build number of the installation package. If the build number is not identical for all compiled and linked components, then the installation is wrong. ... print CLE_DOCTYP_CHAPTER === About Limes === About Limes Since 1985, limes datentechnik(R) gmbh has been successfully producing quality software that simplifies data exchange across platforms and across borders (formatting, converting), secures data archiving and transmitting (encrypting, signature) while reducing storage use (compressing), and, finally, allows accessing data without depending on particular platforms or data formats (access method). .What do we offer? * Professional management of compressed and encrypted data including strong access control * Secure data storage (archive) and data transfer * Protection of data privacy, integrity, and completeness * Cutting-edge security technologies for handling of sensitive data being outsourced, securing clouds etc. * Meeting safety requirements such as "need-to-know", PCIDSS, etc. * Easy data exchange across borders, platforms, and formats (with regard to internationalization moves, when migrating, etc.) * Saving of time and money through automation options and reduced resource utilization (CPU, memory, external storage) .How do we achieve it? * Converting, compressing, encrypting, and signing of data * Reading and writing methods for various data and file formats * Transforming data into a neutral internal format (FLAMFILE(R)) that allows searching and positioning within encrypted and compressed data * Storage of data in various formats (file, cluster, VSAM, database) * Separating data compression and encryption (client) from data storage (server) * Neutral access to individual data records, data elements, or bytes via various APIs * Connecting solutions to any cryptographic infrastructure with access control via key-management including periodic or adhoc re-keying * Unrestricted extendibility and adaptability through customer-APIs for Cobol/PL1, c/C++ and Java (other programming languages on request) * Transparent integration as I/O subsystem, user space file system, ISP, driver or by incorporating as a subroutine * Scripting, backup, restart and stream capability for switching to automated operation * Programming in ANSI-C and assembler utilizing hardware accelerators and special instructions of the various platforms .What do we stand for? * Beside decades of experience with cross-platform and cross-border data exchange, limes datentechnik(R) stands for protection of your investment and for lasting reliability due to the uncompromising downwards compatibility of its products and the friendly customer care provided as part of its support services. * limes datentechnik(R) not only offers solutions for bulk data handling in the mainframe environment, it also supports all current platforms and systems for migrating and long-term archiving. * For more than 30 years limes datentechnik(R), located in Bad Homburg, Germany, has been providing resilient, error-free, high-quality software meeting professional standards - guaranteed 'Made in Germany'. ... print CLE_DOCTYP_CHAPTER === FLAM4 Overview [[flam4.overview]] === FLAM4 Overview FLAM4 is a software product for vertical and lossless data compression, encryption/decryption for structured data. FLAM4 supports different data formats as well as data exchange between several operating systems including necessary character conversions. FLAM4 is also used for long-term archiving of data. The archived data, which are compressed and encrypted, can be accessed directly, while the read data records are decompressed and decrypted. According to this approach other stored, compressed and encrypted data of the same file are not involved and remain compressed and encrypted. This handling makes FLAM4 100% PCI DSS compliant. ... print CLE_DOCTYP_CHAPTER ==== Introduction to FLAM4 ==== Introduction to FLAM4 FLAM4 is a software product for data compression typically used in applications in banking, wholesale and retailing, industry and public administration. FLAM4 is best suited for record-oriented data. FLAM4 compresses the standardized data formats as used in banking with a typical compression ratio of 4:1. For lists of material the compression ratio can be as high as 95% (20:1). Although FLAM was not specifically developed for banking applications, it is now accepted as the optional standard in data compression within electronic funds transfer. FLAM4 is used because of its flexibility, data conversion across platforms and the proven short turn around time. With each new FLAM implementation, new benefits arise for each user without additional costs. It is in the interest of each user to ask for support of FLAM by hardware manufacturers and third party software houses, and to support the installation of FLAM at sites of a business partner with whom electronic data is exchanged. That is the special benefit of FLAM regarding cost effectiveness. FLAM4 uses the algorithm of the Frankenstein-Limes-Access-Method for structure oriented data compression. This method has been patented in Germany, Europe and USA beginning with 19.7.1985. FLAM4 works without pre analysis of the data and without additional tables. This ensures that decompression can be always performed based on the syntax of the compressed file (FLAMFILE) and the FLAM program. This also ensures upward compatibility that allows for long time archiving. FLAM4 does not need any additional specifications about the data to be compressed. The compression method is invariant to file, record, and field formats. However, the compression effect depends on the actual data. Structural distortion mostly leads to worse compression results. ... print CLE_DOCTYP_CHAPTER ==== Principles of FLAM4 ==== Principles of FLAM4 The design principles of FLAM: .Transparency An online data storage media files compressed with FLAM4 can be used in connection with both sequential and index sequential access methods without additional intermediate conversions. The same transparency applies for data exchange (file transfer) in a heterogeneous network between computers with different hardware and different operating systems. .Portability Formatting of the compressed files can be controlled in a way that all requirements are met for maximum memory usage and for portability on any type of transmission lines with any file transfer product. This is especially true for punched card formats (80 columns) and for FTAM formats. Compressed records can be created in both fixed and variable format. .Convertibility FLAM4 is even able to generate compressed files in a printable format. This allows to convert the compressed file from EBCDIC to ASCII and vice versa at any time. At the other hand, code conversions can be performed in combination with compression or decompression. .Compatibility Optionally FLAM4 can convert file and record formats. This allows FLAM to solve compatibility problems between heterogeneous systems or version dependent file management systems. Restrictions regarding record format (fix), duplicate key, etc. are neutralized using FLAM as an access method. .System independence A FLAMFILE can be used on all computer systems where FLAM is available. The FLAMFILE is the base for the access method FLAM without sacrificing the different system specific access methods of the particular file management system. .Continuity A FLAMFILE can be converted during decompression into any file or record format as specified by the user. This guarantees continuity. An archived FLAMFILE can always be processed (especially decompressed) on any system. This insures independency from a particular operating system. However, it must be made sure, that the storage medium can be read by the hardware (e.g. tape unit). Also the FLAMFILE should not be converted into a system dependent format of any given archiving product. .Strict data security FLAM4 encrypts data and seals the compressed files using checksums for better protection. The FLAMFILE has internal synchronisation points, which allow to restart decompression after an erroneous data block caused by physical defects. Requirements for revision and controlling are fully supported. .Interface subdivision FLAM4 provides a variety of interfaces derived from a real file management system with index sequential access. FLAM can be executed as a subprogram under control of an application. User exits within FLAM4 allow pre- and postprocessing of uncompressed records as well as FLAMFILE records. .Standardisation FLAM4 is an optional compression standard for different applications used in German banking, within the framework of Banking Communication Standard (BCS), TARGET2, SEPA, and others. ... print CLE_DOCTYP_CHAPTER ==== Advantages of FLAM4 ==== Advantages of FLAM4 Below is a list of advantages separated between data in transit and data at rest (archiving). .Data transmission * Cost reduction by volume reduction (e.g., packet switching) * Faster transmission by virtualisation * Implicit acceleration of other transmissions * Change to slower physical lines possible * Lower access and operation costs * Less transmission faults due to physically slower transmission * Solution for technological bottlenecks * Increase of potential transmission frequency * Reduction of network node and buffer workload * More efficient reaction possible to line breakdowns, transmission faults, or operating mistakes * FLAMFILE in parking position saves space, allows immediate restart of transmission and can be archived * Compatibility of the FLAMFILE in heterogeneous networks * Portability of the FLAMFILE due to format options * Convertibility of the FLAMFILE for printable data by Pre- and post character conversion possible * Conversion of record and file formats possible (Utility) * Transparency of FLAMFILE to other applications * More remote controlling possible due to volume reduction * More data exchange via line possible due to volume reduction * More swapping to emergency computer centres possible due to volume reduction * Automated remote archiving possible * Automated remote restore possible * Better data revision due to automated procedures * More data integrity due to check sum technique * More data security due to FLAM typical data encryption * Higher efficiency of additional cryptographic methods .Data storage * Reduction of data storage on all media * Fewer requirements for physical space * Less multi volume files (Disc, Tape, Floppy) * Fewer requirements for power, air conditioning, protection * Less fixed capital required * Less overhead in archive and more continuity * Less I/O, less work load for channels * Probably fewer controllers, I/O ports, buffers * Acceleration of batch copy processes and of backup-/restart processes, thus more resources for automation of computer centre * Shorter processing times and shifts * Additional data protection due to FLAM access * Integrated protection against manipulation due to FLAM syntax * Process typical data encryption * Effective protection of logical deleted data * Innovative (combined) access methods for index sequential and logically blocked data in heterogeneous environments. ... print CLE_DOCTYP_CHAPTER === Supported Systems === Supported Systems FLAM4 is available for a variety of operating systems. For example: ------------------------------------------------------------------------ Platform: Operating System: x86 Linux, Windows, Solaris SPARC Solaris pSeries AIX, Linux zSeries z/OS, VSE, z/Linux BS2000 OSD ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER == Command Line Parser [[clep.main]] == 1. Command Line Parser The COMMAND LINE PROCESSOR (CLE/P(R)) is a user application programming interface for several platforms including Windows, Unix and z/OS that simplifies and standardizes parameter string processing. The CLE/P(R) provides two separate functions: * COMMAND LINE PARSER (CLP) - Compiles a parameter string from a command to a data structure * COMMAND LINE EXECUTOR (CLE) - Supporting execution of different commands with parameter strings The COMMAND LINE EXECUTOR provides a list of commands and some built-in functions, please use <>, <> and <> to get extensive information about these capabilities. Additionally the built-in function <> can generate a complete or a part of the user manual. To read the parameter of a command, a compiler (CLP) is applied. To see the regular expressions (lexemes) and the corresponding grammar, please use the built-in functions <> and <>. The return/condition/exit codes of the executable and the reason codes of the different commands can be reviewed with the built-in function <>. See sections <> and, if available, <> for the meaning of the used return and reason codes. The CLE uses an owner management in order to separate the settings for different clients and a property management for each command. If problems occur, a trace might be activated. For each command execution the owner and environment variables can be defined in the configuration file. To see the current configuration use the built-in function <>. The default trace file is 'stdout'. If the trace is activated before a trace file is defined, the trace will be printed on the screen. To manage the tracing use built-in function <>. Last but not least it is possible to view the license, version and other information about the program. This is the RELEASE build version of the FLAMCLEP. ... print CLE_DOCTYP_CHAPTER === Command Line Considerations === 1.1. Command Line Considerations The command line parser supports abbreviation of keywords depending on the unique first characters. This is only useful if you work in dialog. If the command is specified in a persistent way (e.g. in a script) writing the complete keyword is recommended, to ensure that the script must not be changed to work with a later version of the program. For example: Version X supports only one keyword starting with V (VERSION). Version X+1, supports the keywords VERSION and VERIFY. If you use version X with VER=4711 anything is fine but with with version X+1 it will fail, because one more character is required (VERS=4711) for the keyword to be unique. The same is valid for the required string support. The CLP supports strings without enclosing. These strings depend on the keywords used in a name space. If new key words are added in future version, the string could cause errors and must be enclosed. Concerning this it is recommended to enclose strings if the command is defined in a script. For example: ------------------------------------------------------------------------ Version X: STRING=my/data/Name NUMBER=42 Version X+1 supports the new keyword NAME, this requires: STRING='my/data/Name' NUMBER=42 NAME='Hugo' ------------------------------------------------------------------------ The last item which could result in a damaged command syntax after a program update is the use of implicit array definitions after an assignment, because the end of such an array might change, for example if an array of numbers defined an a switch is the end. In a new version such a switch could be a number with a default value, if only the keyword is defined. In this case the defined number referenced by this keyword is then part of the number array and the parameter is lost. Concerning this it is recommended to use enclosed array definitions. For example: ------------------------------------------------------------------------ instead of: FLOATARRAY=1,2,3,PI use: FLOATARRAY[1,2,3,PI] ------------------------------------------------------------------------ All these capabilities are implemented to be compatible with several existing command lines syntaxes and to limit the effort to port old commands or to simplify the interactive use. But in certain cases, the command interpretation could fail, if not the unique/enclosed variant of keywords, strings and arrays are used. ... print CLE_DOCTYP_USEDENVAR === Used Environment Variables [[clep.main.usedenv]] === 1.2. Used Environment Variables Environment variables can be managed via the operating system or via the FLAM4 configuration. The latter can be managed with the built-in functions <>, <> and <>. To list the current state of all usable environment variables the built-in function <> can be used. For help messages for each usable environment variable please use <> There are also some platform-specific mechanisms that can be used to set environment variables: ..Linux, Windows, USS and other Unix-like OS Environment variables are read from the file `.stdenv` in the current working directory or the user's home directory. The file in the home directory is only considered if the file does not exist in the working directory or cannot be read. ..Mainframe systems `DD:STDENV` or the data set name `&SYSUID..STDENV` can alternatively be used to set environment variables. The `STDENV` file allows to define the `FLAM4_CONFIG_FILE`, `FLAM4_DEFAULT_OWNER_ID`, `LANG` and other environment variables in JCL. Example with a z/OS DD name allocation: ----------------------------------------------------------------------- //STDENV DD * FLAM4_CONFIG_FILE=GLOBAL.FLAM4.CONFIG LANG=de_DE.IBM-1141 HOME=/u/hugo USER=hugo ENVID=T /* ----------------------------------------------------------------------- The definition of the LANG variable outside of the program configuration file is recommended, so that the system character set is defined in each situation. The program configuration filename is used as is. No placeholder substitutions (e.g. ``) are possible. It might be useful to have a dedicated environment per user on mainframes. In such a case, it makes sense to define the environment in a dedicated file for each user. ----------------------------------------------------------------------- //STDENV DD DSN=USER.ENVIR(&SYSUID.), DSP=SHR ----------------------------------------------------------------------- Since z/OSv2r1 you can also use exported JCL symbols like environment variables. The exported JCL symbols have lower priority than the same environment variable. The service CEEGTJS is called dynamically and the language environment must be in the STEPLIB concatenation to use it. Additionally, all z/OS system symbols can be used in the string replacement (<&LDAY>). The environment variables have the highest priority, followed by the exported JCL symbols. System symbols have the lowest priority. If the variable name is not found, then no replacement is done and the angle brackets are kept in place. ------------------------------------------------------------------------ //E0 EXPORT SYMLIST=* //S1 SET BLKSIZE=27886 //S3 SET LRECL=128 ------------------------------------------------------------------------ ...Default record format For z/OS, the default record format and record length for non-existent output files is `recfm=VB, lrecl=516`. An attempt is made to open an existing file with `recfm=*` for overwriting (DISP=OLD). If this fails, another attempt is made with `recfm=VB, lrecl=516` to write a new file (DISP=NEW). The environment variable `CLP_DEFAULT_DCB` can be used to change this default file mode string. See "Keyword Parameters for File Mode" for fopen() in _z/OS XL C/C++ Runtime Library Reference_ for more information and for the correct syntax. Beside `recfm` with a valid record format (not + or *) and `lrecl`, the `space` parameter can be set. All other file mode parameters are managed by CLP. ..Property file vs. environment variables Beside all the environment variables managed by CLE it is possible to set all properties as environment variables to override the hard coded default values with CLP. If a property is defined as environment variable and in a property file, then the value in the property file overrides the setting in the environment. The environment variable name for each property is built by the rules below: * convert all characters to upper case * replace all dots (`.`) by underline (`_`) To get a list and help for all properties please use the built-in function `GENPROP` to generate property files. The properties can be defined per owner, per program and globally. The owner-specific definition overrides the program specific definition and the program-specific definition overrides the global definition. Examples: ----------------------------------------------------------------------- CONV_READ_TEXT_ENL2LF=OFF #in general the 0x15 to 0x25 conversion is off# HUGO_FLCL_CONV_READ_TEXT_ENL2LF=ON # for owner 'hugo' the conversion is on# ----------------------------------------------------------------------- The value string behind the sign (including the comment) will be used as supplement for the command line processor. Aliases are not supported in this case. You can only define properties for the main argument. If a string must be enclosed with apostrophe, please don't use double quotation marks because these are used in addition if a new property file is built based on the environment settings. ----------------------------------------------------------------------- FLCL_ICNV_FROM='IBM-1141' # this is the best solution FLCL_ICNV_TO=UTF-8 # "UTF-8" could result in errors ----------------------------------------------------------------------- See <> for the current property file content. ..Usable environment variables Below you can find the list of static environment variables. * LANG (&FLANGLC_&FLANGCC.&FLANGCS) - The default language and CCSID/encoding string for this platform (Format: de_DE.UTF-8 or de_DE.IBM-1141) * HOME - To determine the home directory on UNIX/WIN * USER - To determine the current user id on UNIX/WIN * CUSER - To determine the current user in upper case * Cuser - To determine the current user in title case * cuser - To determine the current user in lower case * SYSUID - To determine the current user id on z/OS * ENVID (&FLENVID) - The current environment qualifier for key label templates (default value is 'T' or the first valid letter of &SYSCLONE on z/OS) * OWNERID - Used for current owner if not already defined (used in key label templates (can also be used in file names)) * HOSTNAME - Current host name * MAILADDR - The email address used as sender for outgoing emails (only relevant on z/OS with SYSOUT format set to MAIL) * CLE_QUIET - Disables the normal log output of the command line executer (must set to YES or ON) ** YES - Yes, do it (same as ON) ** ON - Activate it (same as YES) ** NO - No, don't do it (same as OFF) ** OFF - Deactivate it (same as NO) * CLE_SILENT - Disables log and errors messages of the command line executer (must set to YES or ON) ** YES - Yes, do it (same as ON) ** ON - Activate it (same as YES) ** NO - No, don't do it (same as OFF) ** OFF - Deactivate it (same as NO) * CLP_NOW - The current point in time used for predefined constants (0tYYYY/MM/DD.HH:MM:SS) * CLP_STRING_CCSID - CCSID used for interpretation of critical punctuation character on EBCDIC systems (default is taken from LANG) * CLP_DEFAULT_DCB - The default record format and record length in C file mode format (default is 'recfm=VB, lrecl=516' only for z/OS) * CLP_SYMTAB_STATISTICS - Print symbol table statistics to STDERR stream at close (must set to YES or ON) ** YES - Yes, do it (same as ON) ** ON - Activate it (same as YES) ** NO - No, don't do it (same as OFF) ** OFF - Deactivate it (same as NO) * CLP_MALLOC_STATISTICS - Print memory allocation statistics to STDERR stream at close (must set to YES or ON) ** YES - Yes, do it (same as ON) ** ON - Activate it (same as YES) ** NO - No, don't do it (same as OFF) ** OFF - Deactivate it (same as NO) * FL_PLATFORM - Platform definition (WIN/UNX/MAC/ZOS/USS/BS2[.B/.L]) for emulation at read or write operations ** WIN - Choose Windows as platform (CRLF) ** UNX - Choose Unix derivative as platform (LF) ** ZOS - Choose ZOS as platform (NL) ** USS - Choose USS of ZOS as platform (NL) ** VSE - Choose VSE as platform (NL) ** BS2 - Choose BS2000-OSD as platform (NL) ** MAC - Choose MAC as platform (CR) * FL_DEFAULT_ASCII_CCSID - Default CCSID for ASCII character sets (if ASCII machine and LANG, then default is taken from LANG else default is 'ISO-8859-1')) * FL_DEFAULT_EBCDIC_CCSID - Default CCSID for EBCDIC character sets (if EBCDIC machine and LANG, then default is taken from LANG else default is 'IBM-1047')) * FL_PGP_PREF_SYMALG_AES128 (&FLPA128) - Invert order of preferred symmetric algorithms from AES256/192/128 to AES128/192/256 in PGP certificates ** YES - Yes, do it (same as ON) ** ON - Activate it (same as YES) ** NO - No, don't do it (same as OFF) ** OFF - Deactivate it (same as NO) * FL_ALLOW_LOW_ENTROPY - If defined and no source of entropy is available, the random number generate is seeded with a low entropy which may result in weak encryption keys * FL_DEFAULT_CHAR_MODE (&FLCHARM) - Default error handling if character not available in target character set [STOP] ** STOP - Stop at the first non-convertible character ** IGNORE - Ignore non-convertible characters ** SUBSTITUTE - Substitute non-convertible characters ** IDENTITY - Copy non-convertible characters (supported only for single byte conversions) ** TRANSLIT - Activate ICONV transliteration and ignore the rest (MODE=IGNORE SYSTAB=ICONV) * FL_DEFAULT_PGPCCAPARA - Default PGP IBM-CCA (ICSF) parameter (normally not required, only profile and pin (if applicable) or trust store name) * FL_DEFAULT_PGPP11PARA - Default PGP PKCS#11 parameter (LIB=/usr/lib/libsofthsm2,PIN=f'pkcs11.pin') * FL_DEFAULT_PGPRNGPARA - Default PGP key ring parameter (pub='pubrng.pgp',sec='secrng.pgp',pass=f'keyring.pwd') * FL_DEFAULT_CCALIB - Default path/name of CCA library (not required for ICSF) * FL_DEFAULT_CCAPKDS - Default path/name of CCA public key data set * FL_DEFAULT_P11LIB - Default path/name of PKCS#11 library * FL_DEFAULT_P11PIN - Default PIN/passphrase for PKCS#11 token (it is not recommended to use this if authentication is required) * FL_DEFAULT_P11SLOT - Default PKCS#11 slot number * FL_DEFAULT_RNGPUBFIL - Default public key file for key ring implementation * FL_DEFAULT_RNGSECFIL - Default secret key file for key ring implementation * FL_DEFAULT_RNGPASSWD - Default password for key ring implementation (it is not recommended to use this if authentication is required) * FL_INFO_BLOCK_SIZE - Set a custom block size for the INFO command (useful mainly for PGP files) * FL_USAGE_SYSLOG_IDENT (&FLOGIDT) - Activate usage logging to SYSLOG on each platform (in addition to the usage log stream on z/OS) * FL_DEACTIVATE_HW_CRYPTO_ACCELERATION - Deactivate hardware acceleration for cryptographic operations (enforce use of software implementation) * FL_PRIMARY_SPACE_TRK (&FLPRISP) - Default primary space in tracks (only relevant on mainframes, default is 600) * FL_SECONDARY_SPACE_TRK (&FLSECSP) - Default secondary space in tracks (only relevant on mainframes, default is 1200) * FL_DIRECTORY_BLOCKS (&FLDIRBL) - Default directory blocks (only relevant on mainframes, default is 45) * FL_ZEDC_SUPPORT (&FLZEDC) - Controls use of RCF1951 (GZIP) compression in FLAM4 ADC modules (ON, OFF, AUTO(default), threshold (nnn) in percent) * FL_SSH_HOME_DIR - Directory containing SSH key files and known_hosts file (/home/user/.ssh) * FL_GZ_EXPANSION_FACTOR (&FLGZEXP) - Expansion factor for GZIP decompression (default is 5) * FL_GZ_COMPRESSION_FACTOR (&FLGZCMP) - Compression factor for GZIP compression (default is 2) * FL_BZ_EXPANSION_FACTOR (&FLBZEXP) - Expansion factor for BZIP2 decompression (default is 6) * FL_BZ_COMPRESSION_FACTOR (&FLBZCMP) - Compression factor for BZIP2 compression (default is 2) * FL_XZ_EXPANSION_FACTOR (&FLXZEXP) - Expansion factor for XZ decompression (default is 10) * FL_XZ_COMPRESSION_FACTOR (&FLXZCMP) - Compression factor for XZ compression (default is 2) * FL_ZS_EXPANSION_FACTOR (&FLZSEXP) - Expansion factor for ZSTD decompression (default is 10) * FL_ZS_COMPRESSION_FACTOR (&FLSZCMP) - Compression factor for ZSTD compression (default is 2) * FL_KEY_EXPIRE_WARN_DAYS (&FLKEYWD) - Amount of days to warn before a key will expire (default is 90 (3 months)) * FL_RETURN_IF_LOG_WARNING (&FLRLOGW) - If set to OFF, the completion code 0 is returned instead of 1 if a relevant warning is written to the log ** YES - Yes, do it (same as ON) ** ON - Activate it (same as YES) ** NO - No, don't do it (same as OFF) ** OFF - Deactivate it (same as NO) * FL_FAVE_LIBRARY - Set default library for anti virus scanning (default is 'libfave') * FL_FAVE_FUNCTION - Set default function for anti virus scanning (default is 'CLAMAVC') * FL_FAVE_PARAMETER - Set default parameter for anti virus scanning (default is 'clamav.daemon:3310') * FL_CLAMAV_DAEMON - Set default communication parameter for ClamAV client (default is 'clamav.daemon:3310:10') * FL_AVSCAN_READ - If defined, anti virus scanning when reading is activated (corresponding default values must be defined) * FL_AVSCAN_WRITE - If defined, anti virus scanning when writing is activated (corresponding default values must be defined) * FL_AVSCAN_COMPRESS - If defined, anti virus scanning is done on a compressed GZIP data stream (must be supported by service provider) * FL_SYSOUT_MAIL_CLASS (&FLSOMC) - Default class for SYSOUT allocation if FORMAT.MAIL() is used (only on z/OS) * FL_SYSOUT_MAIL_WRITER (&FLSOMW) - Default program for SYSOUT allocation if FORMAT.MAIL() is used (only on z/OS) * FL_LITERAL_CACHE_SIZE (&FLITCS) - Size of the hash table used (by power of 2) for the literal cache (8>=x<=20 (256 to 1.048.576 entries)) * FL_RECORD_FORMAT_MAPPING (&FLRECFM) - If this variable is not set to DATATYPE (is the default), records with length field (RECF=VAR) are used ** DATATYPE - Binary data as binary (RECF=BIN), Text as text (RECF=TXT) * FL_IBM_UNICODE_SERVICE (&FLIBMUS) - Define use of z/OS Unicode Services for simple character conversions to lower performance [AUTO] ** AUTO - Use it, if behavior the same and faster ** ON - Activate it (with round trip conversion) ** OFF - Deactivate it (no use of Unicode Services) * FLAM4MF - Enable Microfocus Enterprise Server support for FLAM and FLUC record interfaces (ccsid/yes/no) * FLAM4MF_TRACEFILE - Enable a trace file for the MF-EDZ stubs supporting the record interface (Windows or Linux filename) * FLAM4MF_STATIC_SYSVAR - Static system variables are read from filename into the environment (filename in EBCDIC or ASCII, Syntax: 'SYMBOL=value', also use 'DD:SYSVAR') * FLAM4MF_DYNAMIC_SYSVAR - Dynamic system variables are read from filename into the environment (in conjunction with the JCL User Exit of MF-EDZ to add dynamic variables) * FL4SEPA_SIGFILE_PRIMARY_SPACE_TRK - Primary space of signature file for migration (if not defined the required space is determined from the used space of the old signature file) * FL4SEPA_SIGFILE_SECONDARY_SPACE_TRK - Secondary space of signature file for migration (if not defined the required space is determined from the used space of the old signature file)) * _EDC_ZERO_RECLEN (&FLZRECL) - Used for handling of empty records (see dedicated <> below) on z/OS Beside the static environment variables there a lot of dynamic formed environment variables supported. * FLAM4_CONFIG_FILE - the configuration filename (default is '$HOME/.flam4.config' on UNIX/WIN or '&SYSUID..FLAM4.CONFIG' on mainframes) * FLAM4_DEFAULT_OWNER_ID - the default owner ID (default is 'limes') * owner_FLAM4_command_PROPERTY_FILENAME - To override default property file names * path_argument - to override the hard coded default property value ..FLAM specific handling of environment variables On platforms (mainly mainframe systems) where system symbols are available, some of the environment variables can be pre-defined over corresponding system variables. See install.txt or use built-in function <> (in round brackets the corresponding system symbols are listed). Trailing whitespace is removed from variable values before being used. A system symbol must start with `&` or an alphabetic character followed by alphanumeric characters. ...Considerations for the LANG variable The environment variable LANG is also used on EBCDIC systems to interpret several syntax characters correctly, which depend on the codepage. See chapter <> for how the CCSID can be defined. It is important to set this variable to the correct value on EBCDIC systems. If this environment variable is not set, files and literals are interpreted in IBM-1047 on z/OS and USS. If you use another CCSID for entry, you must define the environment variable LANG to enable correct parsing for strings that contain special characters like `~`. We recommend to define the corresponding system variables (&FLANGLC, &FLANGCC and &FLANGCS) via `SYS1.PARMLIB(IEASYMxx)` on z/OS (see also install.txt). These three system symbols are used to build the environment variable LANG based on the string below: ------------------------------------------------------------------------ "&FLANGLC_&FLANGCC.&FLANGCS" ------------------------------------------------------------------------ For example with the definitions below ------------------------------------------------------------------------ SYMDEF(&FLANGLC='de') SYMDEF(&FLANGCC='DE') SYMDEF(&FLANGCS='IBM1141') ------------------------------------------------------------------------ the environment variable below is defined. ------------------------------------------------------------------------ LANG=de_DE.IBM1141 ------------------------------------------------------------------------ The language code (de) is important for correctly selecting the default CCSID if an ASCII code page was detected. ...Considerations for the ENVID variable The `ENVID` variable is normally one character to separate test and production environments. On z/OS the default is simply the first letter of system symbol `&SYSCLONE` or if not defined (on all platforms) 'T' for test. ...Considerations for the FLAM4MF variable family (only Microfocus Enterprise Server) The variable FLAM4MF and (if needed) FLAM4MF_TRACEFILE should be set in the environment of the Microfocus Enterprise Server instance where the FLAM or FLUC record interfaces are used. To define static and dynamic system variables, the FLAM4MF_STATIC/DYNAMIC_SYSVAR environment variable can be used to read the system variables to the environment. The second file was mainly introduce for the MF-EDZ JCL user exit, which can be used to provide the dynamic symbols like JOBNAME as keyword=value pair as the dynamic file. All the files read to build the environment undergo automatic ASCII and EBCDIC detection. A semicolon can be used to mark the end of a keyword-value pair (an inline FB80 encoding in JCL on EDZ does not contain new line characters). To undefine an environment variable, there must be no characters behind the equal sign. To replace a variable with nothing upon use, the environment variable must be set to one or more whitespace character(s). Trailing whitespace is removed from the value string, so a value which contains only whitespace characters will result in an empty string upon substitution. For Micro Focus EDZ environments, the DD name SYSVAR is supported to define system variables. The function used to read the `keyword=value` pairs automatically detects if EBCDIC is used (character > 0x80) and converts the lines automatically to the local character set. ...Other used system symbols on z/OS The LE-less part of FLAM can only use system symbols. Below is a list of available variables: *&FLREUCEE - Activate reuse of language environment in FLAM ZEDC support *&FLOGQL1 - First qualifier to specify the FLAM usage log stream name *&FLOGQL2 - Second qualifier to specify the FLAM usage log stream name *&FLOGQL3 - Third qualifier to specify the FLAM usage log stream name See install.txt for z/OS for more information. ...Other ways to define environment variables On Unix platforms the '/etc/profile' or .bashrc or command line can be used to export environment variables. On windows under system control global or user specific environment variables can be defined. On z/OS additional the DD:CEEOPTS can be used to define environment variables, see example for ZEDC usage below: ------------------------------------------------------------------------ //CEEOPTS DD * ENVAR("_HZC_COMPRESSION_METHOD=software, _HZC_DEFLATE_THRESHOLD=1, _HZC_INFLATE_THRESHOLD=1") /* ------------------------------------------------------------------------ Additional with global relevance the environment variables can also be defined over the PARMLIB member CEEPRMxx on z/OS, see examples for all 3 environments below: ------------------------------------------------------------------------ CEECOPT(ALL31(ON), ENVAR('TZ=EST5EDT') ) CEEDOPT(ALL31(ON), ENVAR('TZ=EST5EDT') ) CELQDOPT(ALL31(ON), ENVAR('TZ=EST5EDT') ) ------------------------------------------------------------------------ from 'Specifying environment variables' of IBM z/OS documentation. ... print CLE_DOCTYP_CHAPTER === Environment Variable Mapping === 1.3. Environment Variable Mapping Each value enclosed with angle brackets (`<>`) are replaced with the corresponding environment variable (``). This feature is available at each starting point of a lexeme and inside of a normal string. If the environment variable not defined the replacements below are still possible: * - Current user id in upper case * - Current user id (case sensitive) * - Current user id in upper case == * - Current user id in title case * - Current user id in lower case * - The current host name of the system * - The environment ID (default is T for test) * - Replaces with the users data directory, if this not available the replacements below are done: ** On UNIX with /home/ ** On USS with /u/ ** On ZOS with Additional few abbreviation are supported for different kind of strings. * '~' for in file strings * '!' for in key labels * '~' for in key labels * '^' for in key labels All these abbreviation character have different code points in EBCDIC code pages and can only be used if the CCSID known be CLP, see chapter <> how the CCSID can be defined for CLP. ... print CLE_DOCTYP_CHAPTER === Filename Mapping [[clep.main.filemap]] === 1.4. Filename Mapping All filenames used by CLE/P(R) are additionally mapped based on the rules below: * The tilde character (`~`) is replaced with the string `` * DD names on mainframes must be prefixed with "DD:" * Data set names on mainframes are always fully qualified * Path names on mainframes must contain at least one slash (`/`) * Data set names on USS must start with `//` ** Fully qualified names with HLQ must be enclosed in apostrophes (`//''`) ** If apostrophes are not used the SYSUID is prefixed as HLQ * Normal file names on other platforms could be relative NOTE: If a requested environment variable is not defined, the replacement is done with the empty string. This can result in unexpected behavior. To use a "<" or "~" as a part of a filename the character must be specified twice. Beside this rules the replacement technologies of your shell can be used, but on some platforms $HOME, $USER or something similar might not be available, for such cases the possibilities above are implemented. This file name mapping is provided by the library CLEPUTL and should also be used for file names managed by the commands supported with this program. ... print CLE_DOCTYP_CHAPTER === Key Label Mapping === 1.5. Key Label Mapping The key label name mapping works like the file name mapping, but you can additional use `^` to replace it with the OWNERID (`^=`), `!` to replace it with the environment identifier (`!=`) and `~` is only replaced with login user id. If the environment ID not defined then `T` for test is used by default. ... print CLE_DOCTYP_CHAPTER === Special EBCDIC Codepage Support [[CLEP.MAIN.EBCDIC]] === 1.6. Special EBCDIC Codepage Support To interpret commands correctly the punctuation characters below are on different code points depending on the EBCDIC CCSID used to enter these values. ------------------------------------------------------------------------ CRITICAL PUNCTUATION CHARACTERS: ! $ # @ [ \\ ] ^ ` { | } ~ ------------------------------------------------------------------------ These critical characters are interpreted normally dependent on the environment variable LANG. If the environment variable LANG is not defined then the default CCSID IBM1047 (Open Systems Latin-1) is used. Below is the current list of supported CCSIDs on EBCDIC systems. ------------------------------------------------------------------------ SUPPORTED EBCDIC CODE PAGES FOR COMMAND ENTRY: "IBM-1140","IBM-1141","IBM-1142","IBM-1143", "IBM-1144","IBM-1145","IBM-1146","IBM-1147", "IBM-1148","IBM-1149","IBM-1153","IBM-1154", "IBM-1156","IBM-1122","IBM-1047","IBM-924", "IBM-500","IBM-273","IBM-037","IBM-875","IBM-424", "IBM-277","IBM-278","IBM-280","IBM-284","IBM-285", "IBM-297","IBM-871","IBM-870","IBM-1025","IBM-1112", "IBM-1157" ------------------------------------------------------------------------ You can define the code page explicitly (LANG=de_DE.IBM-1141) or only the language code (LANG=de_DE, LANG=C). If only the language code defined then the CCSID is derived from the language code (DE=IBM-1141, US=IBM-1140, C=IBM-1047, ...). If possible, these critical characters are also converted for print outs. At output it is not possible to convert anything correctly, because some strings for print out are coming from other sources (like system messages and others). Only all known literals are converted, for unknown variables such a conversion is not possible and CLE/P(R) expects that such strings are encoded in the correct system code page, but there is no guaranty for this. To be independent of the environment escaping of the CLP strings is possible for the critical punctuation characters on EBCDIC systems. See list below: ------------------------------------------------------------------------ ! = &EXC; - Exclamation mark $ = &DLR; - Dollar sign # = &HSH; - Hashtag (number sign) @ = &ATS; - At sign [ = &SBO; - Square bracket open \ = &BSL; - Backslash ] = &SBC; - Square bracket close ^ = &CRT; - Caret (circumflex) ` = &GRV; - Grave accent { = &CBO; - Curly bracket open | = &VBR; - Vertical bar } = &CBC; - Curly bracket close ~ = &TLD; - Tilde ------------------------------------------------------------------------ A escape sequence starts with the ampersand (&) followed be 3 character (not case sensitive) and is terminated with semicolon (;). With an X and 2 hexadecimal digit the definition of a binary byte value are possible. If such a sequence in the string required then the ampersand must be typed twice (&&). To mark a part or the of whole string in a certain CCSID the escape sequence must contain 1 to 6 digits. Few samples can be found below: ------------------------------------------------------------------------ &1047; all in 1047 &0; reset to system code page &1140; no in 1140 ------------------------------------------------------------------------ At the beginning the system code page is active except the environment variable CLP_STRING_CCSID is defined. The defined CCSID is valid till the next CCSID definition. An unsupported CCSID (0) can be used to restore the interpretation to the system code page. NOTE: This is no character set conversion, only the UNICODE code points 0-127 which are on different code points in the different EBCDIC code pages are replaced. All other higher code points (e.g. German Umlauts ('ä','ü','ö')) are not touched. The partial CCSID conversion are mainly useful for application programming interfaces. At compile time the CCSID for literals must be defined. This CCSID could be differ from the system CCSID (local character set) of variable parameter. In such a case a application can mark the literal part in the CCSID used for literals at compile time, and the variable part could be conform to the CCSID defined over the LANG variable. See the C example below: ------------------------------------------------------------------------ snprintf(acCmd,sizeof(acCmd),"&1047;get.file='&0;%s&1047'",pcFilename); ------------------------------------------------------------------------ C-Code are normally in 1047 and if no literal conversion defined then the literals also in 1047. The file name is a parameter in local character set. In Cobol the default code page for literals is 1140. To define the CCSID for CLP strings from outside, the environment variable CLP_STRING_CCSID can be defined. This is useful if an application cannot recompile with the correct code page for the literals. On ASCII/UTF-8 platforms a miss-interpretation of punctuation characters smaller than 128 is not possible. On such platforms the LANG variable is not used for command interpretation or printouts. The escape and CCSID sequences are supported but have no effect on these platforms. This helps to develop platform independent code for CLP strings. ... print CLE_DOCTYP_BUILTIN == Built-in Functions [[CLEP.BUILTIN.FUNCTIONS]] == 2. Built-in Functions Numerous built-in functions are available to obtain extensive help about syntax, errors, commands and functions of a program with help messages, manual pages and documentation. Other built-in functions can be used for owner, property, environment and trace management. All built-in functions have the purpose to simplify handling of powerful and sometimes complex commands. [[CLEP.BUILTIN.SYNTAX]] === 2.1. Function 'SYNTAX' .Synopsis ----------------------------------------------------------------------- HELP: Provides the syntax for each command PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 SYNTAX [command[.path] [DEPTH1 | ... | DEPTH9 | ALL]] ----------------------------------------------------------------------- .Description The syntax function shows how the arguments can be set on the command line. Depending on depth the syntax is printed in one line or in structured form. A structured print out means that you have a question or exclamation mark before the argument. * question mark - optional parameter * exclamation mark - required parameter The *path* is a dotted keyword list in accordance with the syntax of the corresponding command. ------------------------------------------------------------------------ command.para1.para2.para3 ------------------------------------------------------------------------ .Options * DEPTH1 / DEPTH2 / ... / DEPTH9 ** Specifies the depth of the output. Default is DEPTH1. If the depth is greater than 1 the structured print out is used. * ALL ** Shows complete syntax for the specified path, not just DEPTH1. Structured print out is used. .Examples ----------------------------------------------------------------------- :> FLAM4 SYNTAX :> FLAM4 SYNTAX command :> FLAM4 SYNTAX command ALL :> FLAM4 SYNTAX command.para :> FLAM4 SYNTAX command.para ALL ----------------------------------------------------------------------- [[CLEP.BUILTIN.HELP]] === 2.2. Function 'HELP' .Synopsis ----------------------------------------------------------------------- HELP: Provides quick help for arguments PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 HELP [command[.path] [DEPTH1 | ... | DEPTH9 | ALL]] [MAN] ----------------------------------------------------------------------- .Description The help function can provide extensive information about each parameter which is adjustable on the command line. With the help functionality you get a structured list of arguments: * keyword - the keyword for the argument * (TYPE: ...) - the type of this argument * help message - the short help text * (PROPERTY: [...]) - the current hard coded property value (optional) The keyword can be abbreviated, but it must be unique for this list. The required letters are upper case if case sensitive mode is used. With the case sensitive mode an additional line is required to mark the appropriate letter. The *path* is a dotted keyword list in accordance with the syntax of the corresponding command. ------------------------------------------------------------------------ command.para1.para2.para3 ------------------------------------------------------------------------ Help can also be used to show the corresponding detailed description of the manual pages with keyword *MAN*. .Options * DEPTH1 / DEPTH2 / ... / DEPTH9 ** Specifies the depth of the output. Default is DEPTH1. * ALL ** Shows complete help below specified path, not just DEPTH1 and includes all aliases defined for a certain argument. * MAN ** Shows additionally the detailed description for this program, command or parameter and the next level of arguments including the aliases. .Examples ----------------------------------------------------------------------- :> FLAM4 HELP :> FLAM4 HELP MAN :> FLAM4 HELP command :> FLAM4 HELP command MAN :> FLAM4 HELP command ALL :> FLAM4 HELP command.para :> FLAM4 HELP command.para MAN :> FLAM4 HELP command.para ALL ----------------------------------------------------------------------- [[CLEP.BUILTIN.MANPAGE]] === 2.3. Function 'MANPAGE' .Synopsis ----------------------------------------------------------------------- HELP: Provides manual pages (detailed help) PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 MANPAGE [function | command[.path][=filename]] | [filename] ----------------------------------------------------------------------- .Description Shows the manpages of the program, all commands and built-in functions and if available for each argument up to selections/constant definitions. This function prints the corresponding section from the user manual on the screen (doctype: *book*). If a filename is given then the doctype *manpage* of ASCIIDOC is written to the file. Do not use space characters between the key, assignment character (=) and filename. The files in ASCIIDOC format can be used to generate manpages with the tool http://www.methods.co.nz/asciidoc/[ASCIIDOC]. .Examples ----------------------------------------------------------------------- :> FLAM4 MANPAGE :> FLAM4 MANPAGE filename :> FLAM4 MANPAGE command :> FLAM4 MANPAGE command=filename :> FLAM4 MANPAGE function :> FLAM4 MANPAGE function=filename :> FLAM4 MANPAGE command.para :> FLAM4 MANPAGE command.para=filename ----------------------------------------------------------------------- [[CLEP.BUILTIN.GENDOCU]] === 2.4. Function 'GENDOCU' .Synopsis ----------------------------------------------------------------------- HELP: Generates auxiliary documentation (ASCIIDOC text format) PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 GENDOCU [command[.path]=]filename [NONBR] [SHORT] ----------------------------------------------------------------------- .Description The complete user manual is generated when only the filename is given. If a command, or command path, is specified the corresponding section of the user manual is written to a file. The generated files are in ASCIIDOC format, to be readable as normal text document on all platforms and easily convertible into HTML, PDF or other formats with http://www.methods.co.nz/asciidoc/[ASCIIDOC]. The example below generates the complete manual and convert it in a second step to a HTML5 document. ------------------------------------------------------------------------ :> flam4 gendocu book.txt :> asciidoc -b html5 -f asciidoc-html5.conf -a toc2 -a toclevels=4 -o book.html book.txt ------------------------------------------------------------------------ The numbering will vary in places where the complete manual uses a bullet list for arguments and constants, when no detailed description is available and some of these arguments are required for the path. The numbering can be disabled by setting the optional keyword NONBR. This is useful if numbering is generated by a post processing utility (for example 'pdflatex'). To create a short manual without the CLP parameter description the optional keyword SHORT can be used. This is useful for example, if you deliver a text file and no dictionary can be used. In opposite to the built-in function <> at text file generation the corresponding path string is written to the synopsis but index chapter and index terms are not generated anymore. Do not use space characters between the key, assignment character (=) and filename. .Examples ----------------------------------------------------------------------- :> FLAM4 GENDOCU flam4.manual.txt NONBR :> FLAM4 GENDOCU flam4.manual.txt SHORT :> FLAM4 GENDOCU command=command.manual.txt :> FLAM4 GENDOCU command.argument=argument.manual.txt ----------------------------------------------------------------------- [[CLEP.BUILTIN.HTMLDOC]] === 2.5. Function 'HTMLDOC' .Synopsis ----------------------------------------------------------------------- HELP: Generates the complete HTML documentation into a folder PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 HTMLDOC [path] [NUMBERS] [TYPES] [SHORT] ----------------------------------------------------------------------- .Description This built-in function generates the complete user manual in HTML format into a folder. The folder can be defined as optional parameter. If no path given, then the current directory is used as default. With the optional keyword `NUMBERS` the numbering of headlines can be activated. The keyword `TYPE` add the type information to each generated headline. The keyword `SHORT` will result in a short version of the document, not containing the parameter of the commands and other CLP strings. In opposite to the built-in function <> at HTML generation no path is written to the synopsis but index terms are generated by default. The built-in function can only be used if a call back function for HTML documentation provided. If not a error will be issued if HTMLDOC is executed. The callback function is called for each prepared page and the unique pointer to the original manual string can be used to determine if this page already created and if so then only a link to this existing page is required. This interface allows to generated more flexible (with dictionary, index, search) and redundancy free HTML documentation. To understand the service provider interface (open, close and callback function) for HTML generation see the DOXYGEN documentation of FLAMCLE/CLP.h .Examples ----------------------------------------------------------------------- :> FLAM4 HTMLDOC :> FLAM4 HTMLDOC . SHORT :> FLAM4 HTMLDOC /user/home/docu/flclbook :> FLAM4 HTMLDOC NUMBERS SHORT :> FLAM4 HTMLDOC /user/home/docu/flclbook SHORT ----------------------------------------------------------------------- [[CLEP.BUILTIN.GENPROP]] === 2.6. Function 'GENPROP' .Synopsis ----------------------------------------------------------------------- HELP: Generates a property file PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 GENPROP [command=]filename ----------------------------------------------------------------------- .Description A property file is generated for all commands if only a filename is provided. If a command is specified, a property file for this specific command is created. Property files can be used to define default settings for commands. For each parameter, a property entry is written to a text file. The text file contains comments to help in editing the properties. A property file can be activated with the built-in function SETPROP. Property files will be activated by their owner. A property file overrides hard coded properties and properties defined over environment variables. A specific property file overrides the general property file. At time of generation all current default settings are written to the property file. We recommend to work only with command specific property files, because the built-in function CHGPROP can be used to generate, update and activate a property file for a certain command. Do not use space characters between the key, assignment character (=) and filename. To optimize the program performance, one can remove all unnecessary properties. See <> for the current property file content. .Examples ----------------------------------------------------------------------- :> FLAM4 GENPROP owner.general.properties :> FLAM4 GENPROP command=owner.command.properties ----------------------------------------------------------------------- [[CLEP.BUILTIN.SETPROP]] === 2.7. Function 'SETPROP' .Synopsis ----------------------------------------------------------------------- HELP: Activate a property file PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 SETPROP [command=]filename ----------------------------------------------------------------------- .Description The filename is set as current global property file when no command is specified. The file name is set as current local property file when a command is given. The property file can be generated with GENPROP and will be activated for the current owner. After generation you can activate the property file, but there will be no effect until you change the settings inside the property file, because the current default settings are written. The filename can contain replacement/mapping rules (/). Do not use space characters between the key, assignment character (=) and filename. We recommend to work only with command specific property files, because the built-in function CHGPROP can be used to generate, update and activate a property file for a certain command. See <> for the current property file content. .Examples ----------------------------------------------------------------------- :> FLAM4 SETPROP general.properties :> FLAM4 SETPROP command=command.properties ----------------------------------------------------------------------- [[CLEP.BUILTIN.CHGPROP]] === 2.8. Function 'CHGPROP' .Synopsis ----------------------------------------------------------------------- HELP: Change a property value in the currently active property file PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 CHGPROP command [path[=value]]* ----------------------------------------------------------------------- .Description The built-in function updates property values in the currently active (see <>) property file for a dedicated command. If no property file is activated, a property file is generated (see <>), updated with the provided property list and activated (see <>). The first argument of CHGPROP is the command name. All other arguments are key-value pairs of properties that should be updated. The key is internally prefixed with the root (owner.program.command.). The values are enclosed in double quotation marks. Multiple properties of a command can be updated at once by separating them with a space character (please do not use spaces between key, assignment character (=) and the value). This set of properties is parsed by the CLP. If no property is provided, the property file for the command is either parsed or generated (if it does not exist) and then activated without updating properties. This behavior can be used to generate and activate command specific property files for the different commands. The built-in function reads and parses the currently active property file. Then it parses the provided property list and writes the updated property file back to disk. The currently active property file could be a global, not command specific property file. In such a case, a new command specific property file is generated and activated. Properties from the global property file are copied into the newly generated one. If no property file is defined (see <>) that corresponds to the owner, program and command, a property file is generated. The filename formats below are used to read or write property files from/to the home directory: On non-mainframe systems (WINDOWS, UNIX, MAC): ------------------------------------------------------------------------ ".owner.program.command.properties" ------------------------------------------------------------------------ On mainframe systems (ZOS, VSE, BS200, ...): ------------------------------------------------------------------------ ".OWNER.PROGRAM.COMMAND.PROPS" ------------------------------------------------------------------------ If the default filenames and/or path are not sufficient, you can change the filename of each property file by setting environment variables using the following naming convention: ------------------------------------------------------------------------ OWNER_PROGRAM_COMMAND_PROPERTY_FILENAME ------------------------------------------------------------------------ If no home directory is defined, then the current working directory is used as default directory to save the property files. On mainframes SYSUID is used for the first level qualifier, to represent the "home directory". Property files are managed per owner: This means that updates are only done for the current owner. To delete a property you can simply specify the path without the root and with out a sign. See <> for the current property file content. .Examples ----------------------------------------------------------------------- :> FLAM4 CHGPROP command overlay.object.argument=value :> FLAM4 CHGPROP command overlay.object.argument :> FLAM4 CHGPROP command object.argument=value1 argument=value2 ----------------------------------------------------------------------- [[CLEP.BUILTIN.DELPROP]] === 2.9. Function 'DELPROP' .Synopsis ----------------------------------------------------------------------- HELP: Remove a property file from configuration PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 DELPROP [command] ----------------------------------------------------------------------- .Description Without a command the current global property file is deleted from the configuration data. A command specific property file can be deleted from the configuration data when the command name is given. Property files are managed per owner, this means that delete is only done for the current owner. We recommend to work only with command specific property files. See <> for the current property file content. .Examples ----------------------------------------------------------------------- :> FLAM4 DELPROP :> FLAM4 DELPROP command ----------------------------------------------------------------------- [[CLEP.BUILTIN.GETPROP]] === 2.10. Function 'GETPROP' .Synopsis ----------------------------------------------------------------------- HELP: Show current properties PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 GETPROP [command[.path] [DEPTH1 | ... | DEPTH9 | DEPALL | DEFALL]] ----------------------------------------------------------------------- .Description All or certain properties can be shown for a dedicated path. The *path* is a dotted keyword list according to the syntax of the corresponding command. ------------------------------------------------------------------------ command.para1.para2.para3 ------------------------------------------------------------------------ A help message for a certain parameter will also show the assigned property value. See <> for the current property file content. .Options * `DEPTH1 / DEPTH2 / ... / DEPTH9` ** Specifies the depth of the output. Default is DEPTH1. * `DEPALL` ** Shows all properties to the specified path, not just DEPTH1. * `DEFALL` ** Shows all defined properties to the specified path. .Examples ----------------------------------------------------------------------- :> FLAM4 GETPROP :> FLAM4 GETPROP command :> FLAM4 GETPROP command DEFALL :> FLAM4 GETPROP command.para :> FLAM4 GETPROP command.para DEPALL ----------------------------------------------------------------------- [[CLEP.BUILTIN.SETOWNER]] === 2.11. Function 'SETOWNER' .Synopsis ----------------------------------------------------------------------- HELP: Defines the current owner PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 SETOWNER name ----------------------------------------------------------------------- .Description The current owner id (prefix or separation) can be changed. The owner concept is very powerful to separate different property settings. You can run the same command with different properties simply by switching the owner. The owner is a general solution to separate all settings and can be used for environment, client and other purposes. If not already defined, the owner is written to the environment table under the key word OWNERID. This environment variable can for example used as replacement in file names (~/.dat) or to refer a signature key. .Examples ----------------------------------------------------------------------- :> FLAM4 SETOWNER com.company ----------------------------------------------------------------------- [[CLEP.BUILTIN.GETOWNER]] === 2.12. Function 'GETOWNER' .Synopsis ----------------------------------------------------------------------- HELP: Show current owner setting PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 GETOWNER ----------------------------------------------------------------------- .Description The current owner id (prefix for separation) will be shown. The owner concept allows to manage different configurations as a logical separation of what is done with this id. .Examples ----------------------------------------------------------------------- :> FLAM4 GETOWNER ----------------------------------------------------------------------- [[CLEP.BUILTIN.SETENV]] === 2.13. Function 'SETENV' .Synopsis ----------------------------------------------------------------------- HELP: Set an environment variable (defined in the config file) PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 SETENV variable=value ----------------------------------------------------------------------- .Description This function allows to set environment variables within the configuration file for a certain owner. All defined environment variables are placed in the current environment table of the process before a command is executed. Do not use space characters between the key, assignment character (=) and value. *Attention:* The environment variables will be temporary overwritten .Examples ----------------------------------------------------------------------- :> FLAM4 SETENV LANG=DE_DE.IBM-1141 :> FLAM4 SETENV LANG=DE_DE.UTF-8 :> FLAM4 SETENV ENVID=T :> FLAM4 SETENV ENVID=P ----------------------------------------------------------------------- [[CLEP.BUILTIN.GETENV]] === 2.14. Function 'GETENV' .Synopsis ----------------------------------------------------------------------- HELP: Show the environment variables (defined in the config file) PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 GETENV ----------------------------------------------------------------------- .Description This function lists all environment variables which are set within the configuration file for a certain owner. All shown environment variables are placed in the environment table of the process. *Attention:* The environment variables will be temporary overwritten .Examples ----------------------------------------------------------------------- :> FLAM4 GETENV ----------------------------------------------------------------------- [[CLEP.BUILTIN.DELENV]] === 2.15. Function 'DELENV' .Synopsis ----------------------------------------------------------------------- HELP: Delete an environment variable (defined in the config file) PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 DELENV variable ----------------------------------------------------------------------- .Description This function will delete environment variables from the configuration file for a certain owner. .Examples ----------------------------------------------------------------------- :> FLAM4 DELENV LANG ----------------------------------------------------------------------- [[CLEP.BUILTIN.LSTENV]] === 2.16. Function 'LSTENV' .Synopsis ----------------------------------------------------------------------- HELP: List status of all possible usable environment variables PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 LSTENV ----------------------------------------------------------------------- .Description This function will list the status of all possible useable environment variables. .Examples ----------------------------------------------------------------------- :> FLAM4 LSTENV ----------------------------------------------------------------------- [[CLEP.BUILTIN.HLPENV]] === 2.17. Function 'HLPENV' .Synopsis ----------------------------------------------------------------------- HELP: List help message for all possible usable environment variables PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 HLPENV ----------------------------------------------------------------------- .Description This function will print the help message for all possible usable environment variables. .Examples ----------------------------------------------------------------------- :> FLAM4 LSTENV ----------------------------------------------------------------------- [[CLEP.BUILTIN.TRACE]] === 2.18. Function 'TRACE' .Synopsis ----------------------------------------------------------------------- HELP: Manage trace capabilities PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 TRACE ON | OFF | FILE=filename ----------------------------------------------------------------------- .Description Tracing for command line parser and commands can be enabled or disabled. A trace is only required to diagnose errors. In such a case a trace file must be defined first, then the trace can be activated, the erroneous command executed and the trace file sent to the support team. If no trace file is defined, stdout/stderr is used as default. To prevent tracing on the screen, please define a trace file first before tracing is switched on. Do not use space characters between the key, assignment character (=) and filename. .Examples ----------------------------------------------------------------------- :> FLAM4 TRACE FILE=filename :> FLAM4 TRACE ON :> FLAM4 TRACE OFF ----------------------------------------------------------------------- [[CLEP.BUILTIN.CONFIG]] === 2.19. Function 'CONFIG' .Synopsis ----------------------------------------------------------------------- HELP: Shows or clear all the current configuration settings PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 CONFIG [CLEAR] ----------------------------------------------------------------------- .Description The actual configuration list with owner id, property file names trace setting, environment variables and more is shown. With the option CLEAR the configuration data including the corresponding file will be deleted. The configuration file name can be defined by the environment variable: ------------------------------------------------------------------------ FLAM4_CONFIG_FILE=config.file.name ------------------------------------------------------------------------ This can be used to work with a global configuration file. If this location is not defined, the configuration file is stored in the home directory with the name: ------------------------------------------------------------------------ .flam4.config ------------------------------------------------------------------------ If no home directory is defined for the current user (this case is very improbable), the working directory is used. To read the configuration file when the environment variable above is not defined, CLE first looks in the working and then in the home directory. This makes it possible to prefer a property file, by copying it to the current working directory. On mainframes the data set name 'SYSUID.FLAM4.CONFIG' is used as default. SYSUID as high level qualifier is a kind of replacement for the current home directory in other operating systems in this way. .Examples ----------------------------------------------------------------------- :> FLAM4 CONFIG :> FLAM4 CONFIG CLEAR ----------------------------------------------------------------------- [[CLEP.BUILTIN.GRAMMAR]] === 2.20. Function 'GRAMMAR' .Synopsis ----------------------------------------------------------------------- HELP: Shows the grammar for commands and properties PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 GRAMMAR ----------------------------------------------------------------------- .Description The grammar used for interpreting the command line and property files is shown. This is helpful for understanding how the commands and property files are interpreted by the command line processor. See <> for an explanation and the output of this command. .Examples ----------------------------------------------------------------------- :> FLAM4 GRAMMAR ----------------------------------------------------------------------- [[CLEP.BUILTIN.LEXEMES]] === 2.21. Function 'LEXEMES' .Synopsis ----------------------------------------------------------------------- HELP: Shows the regular expressions accepted in a command PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 LEXEMES ----------------------------------------------------------------------- .Description The regular expression used for the tokens of the command syntax will be shown. This could be helpful in understanding how the commands and property files are interpreted by the command line processor. Moreover it will be shown how to define a number, a string, a keyword and other values inside a command. See <> for an explanation and the output of this command. .Examples ----------------------------------------------------------------------- :> FLAM4 LEXEMES ----------------------------------------------------------------------- [[CLEP.BUILTIN.LICENSE]] === 2.22. Function 'LICENSE' .Synopsis ----------------------------------------------------------------------- HELP: List license information for the program PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 LICENSE ----------------------------------------------------------------------- .Description The license string of the software product is shown. This string is provided to the command line executer (CLE) by the calling program and should reflect the current license text for this program. .Examples ----------------------------------------------------------------------- :> FLAM4 LICENSE ----------------------------------------------------------------------- [[CLEP.BUILTIN.VERSION]] === 2.23. Function 'VERSION' .Synopsis ----------------------------------------------------------------------- HELP: List version information for the program PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 VERSION ----------------------------------------------------------------------- .Description The version of the software product is shown. This string is provided to the command line executer (CLE) by the calling program and should reflect the current version information for this program. See <> for the output of this command. .Examples ----------------------------------------------------------------------- :> FLAM4 VERSION ----------------------------------------------------------------------- [[CLEP.BUILTIN.ABOUT]] === 2.24. Function 'ABOUT' .Synopsis ----------------------------------------------------------------------- HELP: Show information about the program PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 ABOUT ----------------------------------------------------------------------- .Description Shows information about the software product like version, copyright, external libraries, license and other attributes. This string is provided to the command line executer (CLE) by the calling program and should reflect the current about information for this program (flam4). See <> for the output of this command. .Examples ----------------------------------------------------------------------- :> FLAM4 ABOUT ----------------------------------------------------------------------- [[CLEP.BUILTIN.ERRORS]] === 2.25. Function 'ERRORS' .Synopsis ----------------------------------------------------------------------- HELP: Show information about return and reason codes of the program PATH: limes.flam4 TYPE: BUILT-IN FUNCTION SYNTAX: > flam4 ERRORS ----------------------------------------------------------------------- .Description This built-in function prints explanations for the return/condition/exit codes of the executable and, if available, the reason codes of executed commands. The command line executer (CLE) initialized the CLP structure. Calls the command line processor (CLP). Calls the mapping function, which converts the CLP structure in a corresponding command structure. Then it runs the command and as last step it calls the finish function with the command structure, to close all files and free all the allocated memory. If one of these steps fails, then a corresponding return (condition/exit) code is given back from the executable. Only the command execution can result in a error (return code 8) or in a warning (return code 4). The command it self must return a reason code for such a warning or error. This means that the return (condition/exit) code can be used to control the batch processing. The reason code gives more information about the error or warning. See sections <> and <> for an explanation and the output of this command. .Examples ----------------------------------------------------------------------- :> FLAM4 ERRORS ----------------------------------------------------------------------- ... print CLE_DOCTYP_PROGRAM == FLAM4 Utility == 3. FLAM4 Utility Frankenstein Limes Access Method (FLAM) 4 is a batch utility based on the Frankenstein Limes Universal Converter (FLUC(R), a FL5 component) to provide the old famous command line interface which was used up to FLAM version 4. It replaces the original FLAM4 utility. The new FLAM4 utility can be used like the old one on UNIX and Windows. On z/OS, the old FLAM utility is still supported and the new FLAM4 utility can be used if one of the new features is needed. The FLAM4 utility behaves like the FLAM command of the FLCL and can be used synonymously. This batch utility supports the following commands: * FLAM - default command for FLAM4 command line if no command specified * INFO - to determine several information The FLAM4 utility can be used in scripts to automate batch processing. The command interpretation is platform independent and optimized for different shells like WINDOWS(R) command prompt, UNIX bash or z/OS JCL. FLAM as a utility can be started on job control level via a job control command. Parameters define the mode of operation. Depending on the operating system, parameters may be supplied as a part of the command or may be entered via screen. Parameters can also be read from a parameter file. This file can be assigned to the process using job control commands or command parameters. === Compatibility with Version 4.3 To be compatible with the older flam4 utility on Windows and Unix the default record format can be adjusted by setting the properties. --------------------------------------------------------- flam4 CHGPROP flam outrecformat=stream inrecformat=stream --------------------------------------------------------- ... print CLE_DOCTYP_CHAPTER === Overview [[flam4.main.overview]] === 3.1. Overview This command line interface is an implementation of the FLAM algorithms and access method up to version 4. Its purpose is to compress and decompress files in a format that allows easy exchange of data with numerous computer types from a wide range of manufacturers. Moreover, it offers character conversion and the option of encrypting compressed or uncompressed data. Its heterogeneous compatibility also renders FLAM particularly suitable for long-term archiving of data, since the reproducibility of this data is no longer tied to the life cycle of a specific system. In addition, FLAM enables files to be converted in almost any way, i.e. to files with a different organization, a different record format or a different character set. The architecture of the software is layer-oriented, i.e. it is made up of a series of hierarchically arranged functional groups which are completely distinct from one another and which communicate via defined interfaces. Since these interfaces are open, it is up to the user to choose the extent to which he wishes FLAM to be integrated in his applications. FLAM can be invoked either by means of a command or by an application program via the corresponding interface. Although FLAM is capable of compressing any type of file, it was originally designed for files with a largely standardized record structure and character-coded data. It is most efficient at compressing such files. The above requirements are normally satisfied if the processing application programs are written in a high-level programming language. This chapter attempts to assist the user in deciding when, and if so how, FLAM can be used effectively and with the optimum benefit. The flam command may contain the necessary information either in the form of parameters in the command line itself or as FLAM parameters in a parameter file, which is referred to by the parameter parfile=parameter file. The FLAM command is described in a own chapter below and discusses these parameters in greater detail. Unless otherwise explicitly mentioned, all the information in this chapter relating to the use of parameters in the command line applies equally to the parameters in the parameter file. ... print CLE_DOCTYP_CHAPTER === Mode of Operation [[flam4.main.mode.of.operation]] === 3.2. Mode of Operation When a file is being compressed with FLAM, one or more original files are read record by record, and a compressed file (optionally encrypted), called FLAMFILE, is created. When a file is decompressed using FLAM, the compressed file is read sequentially and one or more decompressed files are created. The compression process consists of a series of recurring cycles. Each cycle generally entails reading a fixed number of records in the original file; these records are then buffered in the matrix buffer, compressed and output in the form of a compressed block. The number of records is between 1 and 255 or 4095, depending on the compression mode used, and is specified when the compression procedure is invoked. A compressed block containing correspondingly fewer records is generated when the end of the file is reached or if the matrix buffer overflows. The compressed blocks created by FLAM are written into a sequential FLAMFILE. By the record-oriented reading, the file's structural information is preserved, which is of great importance for reproducing the original file in computing environments with data management systems. Depending on the compression mode, each record in the FLAMFILE is given either a binary checksum or a check character, which is used to verify the integrity of the data when the file is decompressed. The FLAMFILE can be saved or exchanged with another computer. When the file is decompressed, it may have a different organization, a different record format or a different record size from the original file, depending on the settings entered by the user or on the default settings. .The FLAMFILE Independently from the Frankenstein-Limes compression method FLAM establishes a concept for file conversion that satisfies compatibility requirements as much as possible. The compressed file created by FLAM is a logical image of the records of the original file. This is the basis for any conversion with FLAM. The compressed file, the FLAMFILE, is stored by default as a sequential file in accordance with the above-mentioned principle (for random access, index sequential storage is also possible). The problems which occur with uncompressed files when the requirements are comparable must therefore not be simply ignored due to the fact that FLAM is being used. Some of these problems are made easier to solve by the FLAM concept. Others remain despite FLAM and must therefore be solved along application-specific or organizational lines as before, the only difference being that the original file can be substituted by a FLAMFILE. FLAM does not solve the problem of incompatible record and field structures in heterogeneous environments. Sometimes users are totally unaware about these problems. FLAM offers user exits that allow to embed special conversion routines. As FLAM is designed as an open system, it will be possible to offer standard solutions for special problem areas in the future. FLAM requires, that the original records are passed to FLAM record by record. The chosen compression method implies that FLAM works asynchronously. N original records may result in k compressed records where n is unequal k. This can be a problem in special cases. The FLAMFILE is always created with a fixed record length that may be specified by the user. This results usually in compressed records of equal length. This is necessary because some DP systems only support files with equal record length. This is also true for some data transmission products. The smallest record length is 80 byte. This allows to process the FLAMFILE in the punched card format (RJE file transfer!). The upper limits are defined by the environment: where the file is stored and which products are used to transfer the file. The upper limit defined by FLAM is 32764 byte. However, the user may define the format of the record as "variable length" or "fixed length". For a fixed-length record that is not filled up, padding is applied. It is also possible to define different block sizes to optimize the I/O operations, file transfer and storage requirements. This flexibility makes it mostly possible to find a suitable solution for all participating software and hardware environments as well as for special applications. The FLAMFILE is principally a binary file where all 256 bit combinations are allowed. With this code the FLAMFILE can only be transmitted in transparent mode (MODE=CX8 and VR8). If transmission on a 7-bit line is performed, file transfer products expand such binary files in a way that ASCII compatibility is guaranteed. Some products convert each half byte into one byte, other products expand 3 bytes into 4 bytes. If the original file contains only printable characters, FLAM can provide a different code format of the compressed file (MODE=CX7). In this case characters from the original file are not combined with FLAM descriptors but simply copied into the compressed file. This mode is mostly more efficient than MODE=CX8 with following expansion 3 to 4. The FLAM descriptors itself under MODE=CX7 consist only of printable characters that are unambiguous in the international code systems ASCII and EBCDIC. These are all small and big Latin letters, the ten digits and the blank. Any kind of control character, special characters, umlauts are excluded. The advantage of this method is that the resulting FLAMFILE can be converted freely between ASCII and EBCDIC and vice versa at any time between compression and decompression. If the conversion is not handled by the data transmission system or on the data transmission path, the user may use the FLAM user exits and can apply the appropriate code conversions as part of the compression/decompression process. FLAM always works in MODE=CX7 in respect to the code system of its host. If source and target are of the same code system (ASCII or EBCDIC) no conversion is necessary. If source and target are of different code systems, FLAM requires that the FLAMFILE is converted to the code system of the target host system first to decompression. If the FLAMFILE in MODE=CX7 shall be transmitted via a 7-bit line as well as via an 8-bit line, it will be necessary to analyse the actual situation to maintain ull compatibility. It must be considered, that FLAM offers integrated code conversion functions not on all platforms. However, this problem can be solved with MODE=CX7. Because the FLAMFILE has records of equal length, the last record is filled with a padding character if fixed record format is used. In MODE=CX7 this padding character is blank, otherwise binary zero. If a variable record format is used, no padding is done for the last record - the last record is shortened instead. Each record in the FLAMFILE has (internal) overhead: the FLAM syntax, which organizes a frame for the compressed data to fulfil the different requirements. The overhead is the same for each record: 4 byte in 7-bit format and 6 byte in 8-bit format. The user should consider this when defining the record length for the FLAMFILE. The shorter the record length, the bigger the overhead. In addition, the FLAMFILE contains the following syntactical elements: an optional file header for each original file, a compulsory block header for each matrix, etc. Usually the FLAMFILE starts with a file header. This file header consists of a system independent on a system dependent part. The file header contains various information about the original file. During decompression FLAM will use this information - if not provided otherwise - to create the decompressed target file. It is possible to concatenate multiple compressed files. In this case the FLAMFILE contains multiple file header. The FLAM utility ignores these file headers during decompres-sion, and will use only the first one. However, the others appear in the protocol. With this feature FLAM is prepared for the insertion of identical file headers into archive files, a feature that may help to identify a file even in the case of hardware faults. If the FLAM record level interface is used, the different files can be separated during decompression. An empty file is converted into a FLAMFILE that consists only out of a file header. This implies that an empty file no longer must be treated as a special case. The usual problems with empty files in the job command language and during file transfer belong to the past. With a parameter it can be specified if and which file header shall be created during compression. The decompression function can be used to only print out the file header without doing the actual decompression. This allows a quick information about the origin of the compressed file. One block header is created per matrix. This block header contains all the necessary information for proper decompression even without a file header. However, in this case the user has to specify the target format via parameter, JCL or catalogue entry if another format than sequential and variable record length shall be created. The block header contains also all information that is needed by the FLAM nucleus for decompression, e.g. MODE, version, matrix size, etc. This information guarantees the upward compatibility of FLAM. The individual records of the FLAMFILE contain their length redundantly. If the FLAMFILE has variable record length an additional length field of 2- or 4-bytes length is part of each record. In MODE=CX7 on PC standard record separator characters of length 1 or 2 are used. Therefore the record length is not a physically unique size within a heterogeneous environment. Each FLAMFILE record created in 8-bit code is protected against manipulations via a 16-bit checksum. In addition, so called block pointers allow synchronisation (and restart on block level), if data cannot be decompressed properly because of manipulation or hardware faults. A FLAMFILE created in 7-bit code does not contain checksums, because this would inhibit code conversion from ASCII to EBCDIC and vice versa. Instead the number of bytes per record is checked. This function detects for example if a non 1:1 code conversion was applied. This may happen if printer control characters or tabulator characters are not converted 1:1. However, this does not comply with the preposition that the file must only contain printable characters. It has a clear advantage to use the 8-bit format if it is not really necessary to work in 7-bit format. Compression is faster, the compression ratio is better, the compressed file is better protected in respect to data security and data integrity and transmission of these files is faster in transparent mode. Also more possibilities for data encryption exist. The reason is, that a 7-bit FLAMFILE can only be encrypted by randomising the character string. Other encryptions do not comply with the purpose for which these files were created (see above). A FLAMFILE in 8-bit format, however, can be encrypted with all kinds of additional encryption methods to create a FLAMFILE not compatible with the market version. For such conversions, the original state of the FLAMFILE as created by FLAM must be restored before the FLAMFILE is decompressed. In MODE=CX7 the FLAMFILE in addition must be converted into the code system of the host where the decompression is performed. In the case that 1:1 code conversions shall be performed before compression or after decompression, FLAM offers the possibility to convert from EBCDIC to ASCII and vice versa as well from one EBCDIC dialect to another EBCDIC dialect. These conversions are implemented via code tables that can be replaced by the user. It is possible to use user defined code tables as well for encryption purposes. For all conversion problems not mentioned so far the user has the possibility to use the user exits for manipulation of uncompressed data. This is independent from the MODE parameter. The required conversions can be performed in connection with record processing. Independently from the user exits, the record level interface provides access on record level before compression and after decompression. Using this possibility the user may process original data that otherwise could not be handled by FLAM. This interface also allows the integration of FLAM with user applications and software packages. Also in the case when a FLAMFILE was created without a file header (HEADER=NO) FLAM can decompress this FLAMFILE. Principally a restoration of a damaged FLAMFILE is possible but requires the consultation of an expert from the manufacturer. Such damages are caused exclusively by hardware or material faults or by unauthorised manipulation. .Group file The ability to store several compressed files in one FLAMFILE has been realized in the form of the group file. If several files are read during compression, FLAM generates for each input file a file header (parameter HEADER=YES,default) in the FLAMFILE. A number of FLAMFILEs are written physically, in sequential order one after the other. (If the parameter HEADER=NO is set, no details about the respective file are stored in the group file. When decompressed, this file is then no longer recognized as a FLAMFILE containing a number of compressed files and it can only be decompressed altogether.) The file type and format of a group file can be adapted to suit any requirements, exactly as in FLAMFILEs. The parameter SHOW=DIR can be used to display the details of all of the compressed files in this group file, without the group file being decompressed. FLAM is able to decompress each individual file of this group file by specifying a selection rule (see chapter 3.1.4.3). The decompressed file can be specified by means of a command or FLAM creates the file dynamically and catalogues it. Libraries are compressed into a group file by FLAM on a member-by-member basis, i.e. it would be possible to decompress each member into a separate file using an appropriate conversion rule. Accordingly, a number of individual files can be used to generate library members. This group file allows libraries that have been generated under a range of different operating systems to be exchanged between heterogeneous operating systems in a compatible way. If neither a selection rule nor a conversion rule is specified, the compressed files are decompressed into a specified file as in earlier FLAM versions; i.e. all of the originally different files are now positioned one after the other, decompressed. Conversion is executed in accordance with the file attributes of the output. NOTE: If `FILEINFO=NO` was set when the group file was being created, no file name was stored for the compressed data in question. This also means that there would be no file name available for creating the files. Despite this, the compressed data can still be accessed and appropriate conversion rules compiled by means of the internal file names FILE0001 (for the 1st file) through FILE9999 (for the 9999th file). .Heterogeneous data exchange Compressed files may be ported to a target system via file transfer or via tapes, cartridges, etc. It is not necessary that source system and target system are of the same type. However, necessary requirement is, that a file transfer feature exists or that a compatible tape or cartridge format exists. If these requirements are fulfilled, data exchange is possible on both systems if FLAM exists and is installed. All FLAM versions are upward compatible. That means, that systems with both FLAM older and newer version can compress/decompress in the old manner. Since version 2.x the different formats of the compressed data are compatible on all systems where FLAM exists. For data exchange between heterogeneous and homogeneous systems only logical data formats should be compressed with FLAM. Physical data formats cannot be reproduced identically on a physically different system. Different methods for compression exist. With ADC, VR8 and CX8 data is compressed in 8-bit mode, with CX7 in 7-bit mode. For data exchange between mainframes any mode can be used therefore. It is also necessary to check if the file transfer works transparently for 8-bit binary data. If yes, an 8-bit method (which can be decompressed on the target system) should be chosen for compression. If file transfer is not transparent the CX7 mode must be selected. The file is only allowed to contain printable characters that can be converted 1:1 during the file transfer. Also in this case it should be checked if the selected compression mode is available on the target system. For file transfer also transfer mode, record length and record format (variable or fixed) must be considered. It is possible that on the target system it is necessary to insert or delete length fields before decompression. Some file transfer products allow only certain record length and certain record formats. One parameter that must be provided with the same value on both systems is the buffer size (MAXBUFFER) for the compression of one data block. This parameter has on mainframes the maximum value of 2.5 MB. FLAM uses on mainframes two alternating buffers, so double the memory is needed. File attributes of the original file are meaningless for data exchange. The reason is that the file transmitted in sequential form is the FLAMFILE. Within the target system the decompressed data can be stored in a file with any valid file organization. This may be an organization that allows for sequential, index sequential or random access. Important is, that the data must comply to this organization (e.g., a record key must be sorted in ascending order for index sequential organization). Files can be compressed immediately during or after processing and can remain stored in compressed form until they are transmitted. Or they can be stored in uncompressed form and are compressed directly before transmission. .Code Conversion During compression and decompression any 1:1 code conversions can be applied to the original data. For the conversion from EBCDIC to ASCII a standard table is provided. It is also possible to load a user defined table by specifying its name (TRANSLATE). Generally spoken it is better to apply code conversion during decompression because the compression algorithm treats some frequent characters (like zero or blank) in a special way. Code conversion could reduce the compression effect. Also, because of the smaller character set of the ASCII code, it may happen that characters not contained in the ASCII code set are lost during code conversion, which cannot be reconstructed for decompression. A delicate problem during the exchange of compressed data are index sequential files. Caused by the conversion the binary or alphanumeric keys may be out of order because of different collating sequences in the code systems. No problem, however, exists for keys that consist out of printable letters or for printable numeric keys. A special conversion before or after processing with FLAM is necessary for index sequential files that contain binary and alphanumeric keys. .Transformation of file formats Target files may be created during decompression with a file organization and record format differing from the original file. This is especially true for compressed files received from another operating system. If no other specifications are made by the user, all files that were compressed under the same operating system are reconstructed using the information in the file header. In addition it is possible to convert the compressed file into any file format supported by FLAM on the target system. However, dependencies between file organization and record format may exist: If the file is transformed to fixed record length, the original records can be longer or shorter than the new record length. Longer original records are truncated if parameter TRUNCATE=YES is specified. Shorter original records are padded up to the new fixed record length with blanks (PADCHAR). If an index sequential file is transformed into a sequential file, the keys are removed if parameter KEYDISP=DEL is specified. If a sequential file is transformed into an index sequential file, the original records must contain a field with key properties (unique and sorted in ascending order). Otherwise a printable key or arbitrary length can be inserted at the key position (KEYDISP=NEW). Records of length 0 or gaps within relative files are removed if the file is converted into index sequential format. If relative files are converted into sequential files, gaps are converted into records with length 0. For a conversion into fixed format gaps are removed. If files are converted into relative format, records of length 0 are represented as gaps, except if records of length 0 can be represented in the relative file format. LDS files are managed on the disk by VSAM in units of 4096 bytes. If there is an "internal" format it is known only to the user and it is not used by VSAM. In this regard, FLAM offers support not only for compression, but also for decompression. Through the use of the parameters IRECSIZE, IBLKSIZE, and IDSORG=LDS for input and ORECSIZE, OBLKSIZE and ODSORG=LDS for output, it is possible to specify "internal" fixed record lengths with appropriate blocking. There is no requirement for the block size to be an exact multiple of the record length, any remainder will be ignored. These specifications enable every file to be converted into an LDS format; that is, a considerably greater degree of efficiency can be achieved during compression. ... print CLE_DOCTYP_CHAPTER === Compression Mode === 3.3. Compression Mode FLAM supports four compression modes: adc, cx7, cx8, and vr8. In the adc, cx8, and vr8 modes, it creates the FLAMFILE as a binary file. Mode adc is the most efficient one. It compresses the datastream by removing the redundancy of repetitive sequences and can be used universally. The other modes mainly eliminate "vertical redundancy", i.e. their efficiency relies on the recognition of re-occurring column contents in table-structured data. Compression mode adc can be combined with one of two encryption methods supported by FLAM. The first is AES, that has been declared standard for US government agencies by NIST (National Institute of Standards and Technology), the second - flamenc - is a proprietary algorithms of limes datentechnik gmbh. None of the other compression methods allows encryption. In all binary modes the compressed information is largely independent of the characteristics of the system on which the compression took place, i.e. the FLAMFILEs must not be modified, for example, if they are transmitted on a line which translates codes automatically. FLAMFILEs which are compressed in the cx7 mode, on the other hand, are insensitive to code translations, since they represent all control information by means of alphanumeric characters, the codes for which are standardized internationally in the ASCII and EBCDIC systems. Providing the original data is character-coded and suitable for exchange between systems with different character sets, the information content of the FLAMFILE remains fully intact after it has been converted to the character set of the destination system, despite the altered binary contents, thanks to the character-oriented interpretation. The compression mode is identified automatically when the file is decompressed. The user thus does not need to know the compression mode in order to decompress a file. ... print CLE_DOCTYP_CHAPTER === FLAM and AES === 3.4. FLAM and AES Below you can fined some remarks for the use of AEs within the different compression modes of FLAM4. .with MODE=ADC/NDC FLAM uses this algorithm for encrypting compressed data. Keys of up to 64 characters can be specified (see also the description of the PASSWORD parameter in version 3). Internally, a key of 128 bits is derived (AES- 128) and data security is enhanced by the insertion of verification fields created also with AES (hash-MACs). This encryption method is activated by setting the parameters CRYPTOMODE=AES and CRYPTOKEY=key and is available with compression modes ADC and NDC (MODE=ADC or MODE=NDC). With CRYPTO-MODE=AES the compression mode defaults to ADC when no MODE-parameter is set. This fast algorithm, combined with the ADC compression, enables the user to encrypt large amounts of data with a worldwide accepted algorithm. Using CRYPTOMODE=AES and MODE=ADC/NDC the full FLAMFILE is secured. Any manipulationes will be detected. Therfore a VSAM-KSDS-FLAMFILE cannot be used with this feature. It is created to insert, update, delete any records in the file. This is not allowed. .with MODE=CX8/VR8 In some cases it is not necessary to guarantee the completeness of the whole file (file transfer!) but the records must be encrypted. The file itself is not moved. Using CRYPTOMODE=AES and MODE=CX8/VR8 the full functionality as without encryption is possible for a VSAM-KSDS-FLAMFILE. You can update, delete, insert any original record as before. All records are com-pressed in CX8/VR8 mode, encrypted with an AES-256 algorithm. It is still a FLAMFILE, but all segments are secured independent from each other and not all together. At least FLAM (MVS) V4.7 is required for decryption. ... print CLE_DOCTYP_CHAPTER === Components and Details === 3.5. Components and Details The following sections describe several components of FLAM4. ==== FLAM Utility FLAM is a utility that is able to compress and encrypt whole files or to expand compressed and to decrypt files. By using the parameters COMPRESS or UNCOMPRESS/DECOMPRESS, you select the operation mode: Compression of an uncompressed file or expansion of a compressed file. .File compression using FLAM As FLAM compresses a file it will write the result into a sequential or index sequential file, the FLAMFILE. This file may have a header that will store information about the original file. FLAM is able to process all PS, PO and VSAM files. To adapt the compression process to the requirements of the user, it is possible to specify parameters during the program call interactively. It is also possible to define the parameters using a parameter file or while generating the system. FLAM creates a job protocol for each execution (on screen or into a log file). During compression FLAM processes a set of 1-4095 (logical) records within one block (Matrix). FLAM can process input and output from both disc and magnetic tape or cartridge. This is also true for the compressed file, the FLAMFILE. FLAM always compresses a set of records in one step. The size of the intermediate buffer can be specified via the parameter `MAXBUFFER`. FLAM will read only as many input records in one step as can be stored within the intermediate buffer. For compatibility reasons to all FLAM versions the size of the intermediate buffer is restricted by 32 kB. However, if the target computer (i.e., the computer on which the compressed file will be decompressed) is known and the FLAM version installed on this computer allows for it, a bigger buffer size for optimal compression can be defined. It is also possible to restrict the number of records within one set using the `MAXRECORDS` parameter. If you define `MAXRECORDS=1`, FLAM will use a serial context free compression method that is only useful with long records. The typical compression ratios are usually achieved with 16-32 records within one set. With a bigger set a slightly better compression ratio can be achieved (which also leads to less CPU utilization!), but the intermediate buffer has to be bigger. The better the compression ratio is, the less CPU time is needed! The compression method is always the same - based on the Frankenstein-Limes-Access-Method. Only the treatment of the matrix columns and the representation of the compressed files differs from case to case. This is controlled with the MODE parameter. With MODE=CX8 FLAM will only compress repeating characters (horizontally and vertically). With MODE=VR8 the remaining data is compressed in addition using the FL-B(4) code. This process first translates the characters into a special 8-bit code and then homogenizes them using specially designed logical operations. The results are bit strings that can be compressed efficiently. One reason for this is that the data resulting from the Frankenstein-Limes-Access-Method can be grouped into partial equalcharacter classes. In both cases, the compressed file, the FLAMFILE, is a sequence of 8-bit combinations that are written into sequential or index sequential files. Record size, record format, block size can be defined by the user. Each record of a file is in addition protected with a checksum. Code conversions within the FLAMFILE are not allowed. During transmission the file has to be treated as a binary file. For files that contain only printable characters and shall be transmitted using a 7-bit-line, FLAM offers MODE=CX7. This mode creates a compressed file which will "behave" during transmission in the same way as the uncompressed file. FLAM does not check, if the original file is suitable for transmission, but will create a compressed file using a very restricted character set that is neutral to the different code conversion utilities on the marketplace. Using this mode one can create compressed files that can be converted from EBCDIC to ASCII and vice versa (e.g., during a file transfer). However, it is necessary that the code conversion is reversible without any changes. Otherwise FLAM will signal a syntax error in the compressed file due to differences in the byte numbers and will stop decompression. Such cases are possible, if the original file contains printer control characters or tabulator characters, which are not converted 1:1. Apart from this, code conversion can be performed integrated with the compression step. FLAM offers the possibility of code conversion (before compression, or after decompression). Code conversion is controlled via standard code tables or user defined code tables. For special cases where a 1:1 code conversion is not possible for all characters, user exits are provided. .Decompression of files using FLAM FLAM reads a compressed file (FLAMFILE), decompresses the content and writes the result into a target file. FLAM automatically configures itself according to the parameters (e.g., buffer size or record set size) used during compression. The general layout of the compressed file is described in a separate chapter. FLAM version 2.7 can decompress all FLAMFILES created with FLAM V1.0 - 2.6 (upward compatibility). FLAM V2.0/ V2.1 can decompress sequential FLAMFILES created with FLAM V2.5 - V2.7 (downward compatibility). To adapt the decompression process to the requirements of the user, it is possible to specify parameters during the program call interactively. It is also possible to define the parameters using a parameter file or while generating the system. FLAM creates a job protocol for each execution (on screen or into a log file). During decompression the characteristic parameters of the original files are restored depending on the information in the file header. By specifying certain parameters it is possible to generate a target file differing from the original file. All conversions are possible provided that the target system caters for the appropriate access method. That the FLAMFILE eventually was created in a different environment (e.g., different operating system) lacks influence on the operation of FLAM. Data is decompressed into equivalent file formats or in user-defined file formats. By specifying code conversion tables it is possible to change code systems after compression. To be even more flexible, FLAM provides user exits that allow the user to apply user defined post processing. .FLAM Utility synopsis FLAM is able to compress complete files and to reconstruct complete files from compressed data. For original files all file and record formats of type PS, PO or VSAM for disc or tape are supported. The user interface for file IO (`DEVICE=USER`) allows to support additional access methods. The user exits allow to pre- and post process original data as well as compressed data. The user exits are implemented as subprograms that are loaded dynamically during execution time from a module library (STEPLIB). Using predefined or dynamically loaded conversion tables, character conversion can be applied to the original data. It is possible to change file and record formats during decompression. Conversion from variable to fixed format or from sequential to index sequential organization is possible. The compressed data (FLAMFILE) can be stored in sequential or index sequential format in any record or file format. The record and file format of the compressed data is independent from the record and file format of the original data. An index sequential FLAMFILE provides efficient random access to original data by using the record level interface. At the other hand, sequential organization of the FLAMFILE is better suited for file transfer between computers with different operating systems. FLAM compressed data is always heterogeneously compatible. This means that compressed data, which was generated under a certain operating system, can be decompressed on all operating systems supported by FLAM. Depending on the available record and file formats on the target systems, record and file conversions may be necessary. FLAM can be executed online as well as in batch. It can be adapted in a flexible way to the requirements of the users. Several parameters are supplied for this purpose. These parameters may be entered or may be stored via the `PARM` interface. An additional parameter file is also possible. Default values for the parameters may be created during installation (see: Generation of Default Values). File attributes can be also defined via the DD command. During processing, parameters are evaluated in the following sequence: * First the installation parameters are used. * During decompression this values are substituted by the values from the file header of the FLAMFILE, if available. * Then the values from the parameter file are used. The input PARM in the EXEC statement overwrites these values. * File attributes defined in the `DD` or `ALLOCATE` command overwrite again. According to this hierarchy, a very flexible mode of operation is possible. Please be aware that the sequence is not always chronological: For example, it is possible to select a parameter file via `PARM` input. This parameter file is read in after the input despite of the fact, that the `PARM` inputs will overwrite the specifications of the parameter file. .FLAM Utility Parameters Independently from the input medium, parameters are evaluated always using the same syntax. Only upper case letters are allowed. Parameters may be passed in one or more lines or records. In each line the parameter interpretation ends with the first blank position. After the blank any type of comment may follow. Single parameters must not be separated by end of line. Evaluation of the parameters is ended at the keyword `END` or by an empty input line (length = 0) or by `EOF` for the input medium. There are parameters with or without keywords. Keywords and values can be abbreviated. Within brackets [] alternative keywords are presented as used in other operating systems (z/OS, DOS/VSE, BS2000). Keywords from previous versions are presented within angle brackets. This keywords should not be used any longer. Because of compatibility reasons all parameters are presented although not all parameters are evaluated under z/OS. Keyword parameters can be specified in two different modes as usual under z/OS: ------------------------------------------------------------------------ parameter0,parameter1=value1,parameter2=value2,... ------------------------------------------------------------------------ or: ------------------------------------------------------------------------ parameter0,parameter1(value1),parameter2(value2),... ------------------------------------------------------------------------ A string value (file names, module names, password, ...) may be passed as C'.....' (characters) or X'....' (hexadecimal). All string parameters are assigned with blanks if "(NONE)" or no value is specified: ------------------------------------------------------------------------ parameter=(NONE), or parameter(NONE),... ------------------------------------------------------------------------ or: ------------------------------------------------------------------------ parameter=,... or parameter(),... ------------------------------------------------------------------------ The sequence of parameters is arbitrary if not otherwise specified. Only parameters that differ from the default values must be specified. In the following we describe all parameters in alphabetic order: Parameter may be abbreviated as long as they stayunique. Otherwise the first matching entry of the following summary is used. For more about every parameter see section <>. .FLAM Utility Variables It is also possible to control FLAM via environment or system variables. An environment variable overwrites a system variable. [width="90%",cols="^1,4",align="center"] |==== |*Environment variables* |are stored in ------------------------------------------ //CEEOPTS DD DSN=filename,... ------------------------------------------ or set by an LE-Routine (like a C-program). They are used only in the actual step. |`FL_ZEDC_SUPPORT` |controls how to use a zEDC-Hardware. *Possible values:* :OFF: do not use the hardware, even if one is found. `MODE=ADC` is in effect. :ON: use it always *Default:* no value. Use it, if found. *Valid for:* * compression * decompression. NOTE: When a hardware is not found and the algorithm is required for decompression, a software routine is used. |*System variables* |are stored in `SYS1.PARMLIB(IEASYMxx)`. When for the same function an environment variable is found, the environment variable is used. |`&FLZEDC` |Controls how to use a zEDC-Hardware system wide. This is same as the environment variable `FL_ZEDC_SUPPORT`. *Possible values:* :OFF: don't use the hardware, even if one is found. `MODE=ADC` is in effect. :ON: use it always *Default:* no value. Use it, if found. *Valid for:* * compression * decompression. NOTE: When a hardware is not found and the algorithm is required for decompression, a software routine is used. |`&FLOGQL1` |First qualifier in logstream name of LOGR. *Possible values:* + * 1-8 characters as with IXCMIAPU defined. Activates FLAM logging. |`&FLOGQL2` |Second qualifier in logstream name of LOGR. *Possible values:* * 1-8 characters as with IXCMIAPU defined |`&FLOGQL3` |Third qualifier in logstream name of LOGR. *Possible values:* * 1-8 characters as with IXCMIAPU defined *Valid for:* * compression* decompression. *Example:* + IXCMIAU defines the logstream name FLAM.USAGE ----------------------------------------- (DEFINE LOGSTREAM NAME(FLAM.USAGE) ...). ----------------------------------------- Then in Parmlib-Member must be stored: + ----------------------------------------- SYMDEF(&FLOGQL1='FLAM') SYMDEF(&FLOGQL2='USAGE') ----------------------------------------- NOTE: Without any system variable FLAM cannot log any data. Look for some examples to define a logstream in FLAM.JOBLIB. |==== .JCL for FLAM This description is valid for batch processing. Appropriate commands must be used for dialogue processing (TSO). FLAM is started via the EXEC command: ------------------------------------------------------------------------ //stepname EXEC PGM=FLAM,PARM= ... ------------------------------------------------------------------------ Parameter specification must be made according to the JCL conventions (max. 100 characters, inclusion in quotes in case of special characters, like '=', '(', etc. ). FLAM parameters specified via `PARM=` will overwrite FLAM parameters stored in the FLAM parameter file. If the FLAM modules are not stored within a system library a FLAM load library must be specified: ------------------------------------------------------------------------ //STEPLIB DD DSN=user.FLAM.LOAD,DISP=SHR ------------------------------------------------------------------------ The input and output files can be assigned to FLAM via DD cards. The DD names used are predefined default names. Other DD names can be chosen via parameters. The following file types are supported by FLAM: * Physical sequential PS (also members of a PO library) * PDS / PDSE libraries (but no PDSE-LOADs) * VSAM ESDS / KSDS / RRDS / LDS The following record formats are supported: * V / VB / VS / VBS / F / FB / FS / FBS / U Print files (A or M) are also supported. The specification of a parameter file is possible but not mandatory (usually the PARM= instruction is sufficient): ------------------------------------------------------------------------ //FLAMPAR DD DSN=parameter_file,DISP=OLD ------------------------------------------------------------------------ It is also possible to define the parameter file directly embedded into the JCL: ------------------------------------------------------------------------ //FLAMPAR DD * parameter0,parameter1 parameter2 /* ------------------------------------------------------------------------ For compression, an input file must be specified: ------------------------------------------------------------------------ //FLAMIN DD DSN=uncompressed_input_file, // DISP=SHR ------------------------------------------------------------------------ This file must already exist and must be catalogued. It is allowed to be "logically empty", which means that it does not contain a data record. For the creation of a compressed file (FLAMFILE) it is sufficient to write: ------------------------------------------------------------------------ //FLAMFILE DD DSN=compressed_file, // DISP=(NEW,CATLG), // UNIT=...., // SPACE=.... ------------------------------------------------------------------------ With this card a sequential data set with a fixed record length of 512 and a system determined block length will be created. However, this may depend on the default parameters specified or on additional parameters supplied with the job. ------------------------------------------------------------------------ //FLAMFILE DD DSN=compressed_file, // DISP=(NEW,CATLG), // UNIT=...,SPACE=..., // DCB=(LRECL=1024,BLKSIZE=26624) ------------------------------------------------------------------------ Using DCB attributes this assignment overwrites the `MAXSIZE` or `BLKSIZE` parameters for the FLAMFILE. The file is created according to the specifications in the DD statement. One example for using flexible configurations with the FLAMFILE comes from the area JES / NJE: ------------------------------------------------------------------------ //FLAMFILE DD SYSOUT=F,DEST=(node,userid), // DCB=(LRECL=80,BLKSIZE=3120),... ------------------------------------------------------------------------ If the FLAMFILE is already catalogued (`DISP=OLD`) FLAM will use the configuration details from the catalogue entry (even, if it is a VSAM file). For decompression an output file must be assigned. ------------------------------------------------------------------------ //FLAMOUT DD DSN=decompressed_file, // DISP=(NEW,CATLG), // UNIT=...,SPACE=... ------------------------------------------------------------------------ With this specification a decompressed file will be created that has identical characteristics regarding file format, record and block length with the original file. If a PO library has been compressed and a new PO file is to be defined for decompression by JCL, the selection rule `FLAMOUT=PO-file-name(*)` must always be specified. Otherwise member names are not known. ------------------------------------------------------------------------ //.. EXEC PGM=FLAM,PARM='D,FLAMOUT=PO-file-name(*)' //FLAMOUT DD DSN=po_file,DISP=OLD ------------------------------------------------------------------------ If the original file came from a foreign system (VM, VSE, UNIX, Windows, ...), FLAM will choose a format closest to the original file. If the FLAMFILE does not contain information about the original file, e.g. caused by parameter `HEADER=NO` during compression, the output file is created as variable blocked (VB) and with a maximum record length of 32756 bytes and a block length of 32760 bytes. Each DCB specification within a DD statement will overwrite the values automatically chosen by FLAM or specified as FLAM parameter. If the output file is already catalogued the catalogue entry will specify the file characteristics: ------------------------------------------------------------------------ //FLAMOUT DD DSN=output_file,DISP=OLD ------------------------------------------------------------------------ All messages are written into a message file by default: ------------------------------------------------------------------------ //FLPRINT DD DSN=list_file,DISP= ... ------------------------------------------------------------------------ or ------------------------------------------------------------------------ //FLPRINT DD SYSOUT=* ------------------------------------------------------------------------ If no file is specified despite the fact that the parameter `MSGDISP=MSGFILE` is set, a message is written to the console via `WTO, ROUTCDE=11` and the program is terminated with a condition code. If messages shall be suppressed, `SHOW=NONE` must be specified in the `PARM=` entry within the EXEC card. Each file can be assigned as: ------------------------------------------------------------------------ //ddname DD DUMMY ------------------------------------------------------------------------ This will suppress file output or inhibit file input. Since this does not affect the general processing of FLAM, it can be used conveniently for testing purposes: * evaluation of compression ratio * general testing of control flow * using own read and write routines from user exits (we recommend the USERIO interface instead, but in principle the possibility mentioned above can be used, too). The DUMMY assignments are not suitable for benchmarks. Because all physical disc or tape I/O is suppressed, this leads to a much shorter elapsed time, especially in case of later tape assignments or VSAM I/Os. NOTE: The record length in the DD statement must always include the length of the record length field in case of variable length records. Opposite to that, FLAM requires the actual length of the data (without length field) in it's own parameters and displays also only the actual length of the data in the protocol. To make this difference clear the parameter name `LRECL` has been changed to `MAXSIZE` for the FLAMFILE parameter printout. `IRECSIZE` and `ORECSIZE` are equivalent notations for the input file and the output file. The FLAM user interface for TSO users removes the task of writing JCL tatements from the user (including creation of VSAM files). All specifications can be made interactively. Also batch job streams can be generated with this facility. .Dynamic file allocation The specified files are allocated automatically by FLAM (dynamic allocation, SVC 99) by means of parameter input (`FLAMIN=filename`, `FLAMFILE=filename`, `FLAMOUT=filename`, ...), as long as a DD statement has not been specified. If the files are already catalogued, FLAM will use the configuration details from the catalogue entry. For reading the file is assigned with `DISP=SHR`, for writing with `DISP=OLD`. If the file is not already catalogued, it is created (`DISP=(NEW,CATLG)`). For decompression (`FLAMOUT=...`) data such as file organization, record length, block length, format and file size are taken from the FLAM file header, which means that the output file essentially corresponds to the original file. Parameter inputs, however, take priority over the stored data. When the parameter `FLAMOUT=<*>` is set, the file names stored in the file header are used (see section <>). The storage assignment contains the file size as primary specification, 1/4 of the file size being the secondary specification. In the case of PO libraries, the number of directory entries is also known. The file is thus stored in one extent on the disk. If the file size is not known, because e.g. a FLAMFILE is assigned or the compressed data has not been generated under MVS (and therefore does not contain the file size), default values are used (TRK,(30,900),RLSE). If the default values are inadequate, a DD statement specifying other values must be made or the file can be created before the job is run (e.g. with function 3.2 in ISPF). All file types supported by FLAM (PS, PO, and VSAM) can also be created dynamically again. However, since important information required for creating new files is missing (such as UNIT and VOLUME), SMS must be used. The fact that this information is missing is on the one hand due to data protection reasons (data exchange via file transfer!); on the other hand, they would usually not be useable due to transferral of the compressed data to other computers. After all, compressed data from foreign operating systems (VMS, VSE, UNIX, etc.) cannot even contain these specifications. NOTE: If parameters are specified and FLAM finds a DD statement via the DD-NAME, all the specifications in this statement take priority over the parameters specified or values stored in the file header. .Condition Codes For job control FLAM returns the following condition codes: [width="70%",cols="^1h,4",align="center",options="header"] |==== |Condition code|Description |0 | Normal termination (no error) |4 | Not all I/O files were processed during processing of group files |8 | Less severe error occurred (e.g. parameter error) |12| Usually a data management error |16| Severe error during compression or decompressi on |80| Compression ratio was less than the specified limit (see <> parameter) |88| The file assigned was not a FLAMFILE |==== Only for condition code 0 and 80 compression was done properly. In all other cases no or a faulty compressed file was created. We recommend to rename this file in order to avoid using it for processing at a later time. If a condition code greater 0 was returned an error message was already displayed by FLAM. Condition code 16 could be caused by a FLAM system error. The library `FLAM.SRCLIB` contains the call module FLAM. This can be customized as required, allowing other condition codes to be returned. .FLAM Utility File names Specifying a filename is an alternative to assigning the file by means of a DD statement. The specification can contain a conversion rule for filenames. The name can be specified in wildcard syntax for decompression. The character `>` in front of the filename means that the flamfile contains a list of filenames to be decompressed. `DUMMY` acts like the DD-statement `//.. DD DUMMY` .Filename In principle, FLAM is able to process all filenames via JCL valid in z/OS. FLAM stores the name of the compressed file into the FLAMFILE, if required. To avoid any conflicts with national character sets or naming conventions in other systems, all filenames stored in ASCII character set are translated for message and selection in the following way: All national characters are translated to 'X', a backslash '\\' to slash '/', and whitespace ' ' to underline '_'. So it is easier to enter foreign filenames that are unsupported in the z/OS environment. The filename itself remains unchanged in the FLAMFILE. Entering `DUMMY` as a filename causes FLAM to use this file as dummy like the JCL command `//ddname DD DUMMY`. I.e. reading an input file leads to EOF (end of file), writing to an output file has no effect. So, DD-statements are not necessary for DUMMY files. .Filename list By prefixing the '>' character (greater than) to the filename or DD-NAME, the FLAM parameters `FLAMIN` and `IDDN` are able to specify a list of files for compression instead of one single file. It is also possible to specify a file list for decompression by means of the `FLAMFILE` or `FLAMDD` parameter. In this list, each filename must be contained in a separate record; leading or trailing space characters (X'40') are ignored. A comment can be inserted after the first space character following the filename. Empty records or records with an asterisk '*' in the first column are regarded as comment lines. All filenames usable in z/OS are legal. Wildcard syntax is allowed. *Example:* If the records in the file USER.DAT.LIST contain the following filenames: ------------------------------------------------------------------------ USER.DAT.PS USER.VSAM.ESDS USER.POLIB.* USER.PO(MEMBER) ------------------------------------------------------------------------ the specification ------------------------------------------------------------------------ //... EXEC PGM=FLAM, // PARM=´C,FLAMIN=>USER.DAT.LIST´ ------------------------------------------------------------------------ leads to all the files specified being compressed into one FLAMFILE (group file). The file that contains the list of filenames can have any format supported by FLAM and be of any type. For 'in-stream files', i.e. input files temporarily created by JES, it is advisable to assign the filenames by means of DD names: ------------------------------------------------------------------------ //... EXEC PGM=FLAM,PARM=´C,IDDN=>DDNAME´ //DDNAME DD * USER.DAT.PS USER.VSAM.ESDS USER.POLIB.* USER.PO(MEMBER) /* ------------------------------------------------------------------------ This allows the filename list to be specified directly in the job. .Wildcard syntax Filenames can be specified in FLAM by means of parameters written with wildcard syntax. Correspondingly, the entry `FLAMIN=USER.*.DATA.%BC` leads to the compression of all files with the 1st qualifier USER, any 2nd qualifier, DATA as the third part of their name, and a three-digit 4th qualifier which ends with BC and begins with any character. The asterisk "*" stands for any (even an empty) character string. The percent character "%" stands for any character. These special characters are also allowed within members of one or more PO libraries: ------------------------------------------------------------------------ USER.POLIB(FL*) ------------------------------------------------------------------------ specifies all members of the library USER.POLIB, that begin with FL; ------------------------------------------------------------------------ USER.*D.LIB(A%%B) ------------------------------------------------------------------------ specifies all members whose 4-digit names begin with A and end with B and whose libraries have the identification USER, LIB as their final qualifier, and D as the last character of the second to the last part of their name. All input files are stored in this way in one compressed file (group file). Correspondingly, the wildcard entry in the selection rule for decompression ------------------------------------------------------------------------ FLAMOUT=USER.DAT.*LIB ------------------------------------------------------------------------ leads to the decompression of only those items of compressed data whose original names conform to the syntax specified. (Comment: this procedure requires a FLAM file header (`HEADER=YES`) and the file information `FILEINFO=YES`) for compression). *Example:* ------------------------------------------------------------------------ C,FLAMFILE=USER.DAT.CMP,FLAMIN=USER.*B.*LIB,... ------------------------------------------------------------------------ All files with names that conforming to the FLAMIN specification are to be compressed into the file USER.DAT.CMP. If the files ------------------------------------------------------------------------ USER.DATA.ALIB USER.DATAB.BLIB USER.DATCB.CLIB USER.DATCB.DLIB ------------------------------------------------------------------------ are catalogued, the 1st file does not conform to the wildcard syntax and is ignored during compression. If ------------------------------------------------------------------------ $$D,FLAMFILE=USER.DAT.CMP,FLAMOUT=USER.DATCB.*LIB$$ ------------------------------------------------------------------------ is now specified for decompression, only the files `USER.DATCB.CLIB` and `USER.DATCB.DLIB` will be decompressed. Correspondingly, it is also possible to select a number of FLAMFILEs for decompression: ------------------------------------------------------------------------ ..D,FLAMFILE=USER.CMP.*.VR8,... ------------------------------------------------------------------------ means all FLAMFILEs with the identification USER, CMP as their 2nd qualifier, any character string as the 3rd part of their name and VR8 as their last qualifier are to be decompressed. .Selection rule for decompression (!! deprecated !!) NOTE: Pointed brackets '<>' explained blow are dropped in FLAM 5, because this conversion rules cause too much data transfer in case of remote processing. If pointed brackets '<>' are used, a migration for FLAM 5 is necessary. See section <>. During decompression, the files can be created and cataloged by FLAM automatically. This requires the existence of a valid filename in the file header of the FLAMFILE (i.e. the parameters `HEADER` and `FILEINFO` must not be set to `NO` for compression). In addition, a FLAMFILE can contain several files (i.e. it is a group file) and it is possible to decompress specific files from such a group FLAMFILE on a targeted basis by specifying a selection rule. In order to distinguish it from a 'genuine' filename, a selection rule s written between angle brackets '<>' (!! deprecated !!). ------------------------------------------------------------------------ FLAMOUT= (!! deprecated !!) ------------------------------------------------------------------------ With the above rule set, the file `USER.FILE.ORG` is decompressed from the FLAMFILE. (Comment: if thepointed brackets were left out, the entire FLAMFILE would be decompressed and written into the file `USER.FILE.ORG`). This name must be contained in a file header of the FLAMFILE. If the FLAMFILE contains other compressed files, they are ignored due to the unique selection rule set. If several files are to be decompressed from a group file, a wildcard syntax can be specified. The simplest entry possible is a lone asterisk: ------------------------------------------------------------------------ FLAMOUT=<*> ------------------------------------------------------------------------ This decompresses all of the files from the FLAMFILE and locates them on the disk under their original names. A selection rule is implicitly regarded by FLAM as having an asterisk at the beginning and at its end, i.e.: ------------------------------------------------------------------------ $$ corresponds to <*DAT*ABC*>$$ ------------------------------------------------------------------------ When analyzing the file names, FLAM synchronizes itself to the character string specified. *Example:* The FLAMFILE contains the data `USER.DAT1.PS` and `USER.DAT2.PO`. The specification ------------------------------------------------------------------------ D,FLAMOUT=,... ------------------------------------------------------------------------ decompresses only the file `USER.DAT1.PS`. Since there is no asterisk specified in the name, decompression is termimated after the first hit. NOTE: If, in addition to the selection rule, a file is assigned by means of JCL, the JCL specification takes priority. I.e., the above specification together with the DD statement ------------------------------------------------------------------------ //FLAMOUT DD DSN=USER2.POLIB(MEMBER),DISP=... ------------------------------------------------------------------------ leads to decompression of the USER.DAT1.PS file from the FLAMFILE and it is written into the PO library `USER2.POLIB` as the member `MEMBER`. This, then, allows the changing of file names during decompression by means of JCL specifications. .Conversion rule (!! deprecated !!) NOTE: Angle brackets '<>' explained below are dropped in FLAM 5, because this conversion rules cause too much data transfer in case of remote processing. If pointed brackets '<>' are used, a migration for FLAM 5 is necessary. See section <>. This simple method of selecting files for decompression by means of a selection rule can, however, not normally be used for compressed data that has been generated under a different operating system (heterogeneous exchange of compressed data). The file names do not normally conform to the rules of the MVS operating system and can therefore not be used without being modified. For this purpose, it is possible to specify a conversion rule as the parameter for the output file. This character string describes how a new name is to be generated from a file name selected. At the same time, precisely those files that conform to the rule (selection rule) are selected. A conversion rule is a selection rule that is extended by an equals sign '=' and a second character string. In order to distinguish it from a 'genuine' filename, it must be written between pointed brackets '<>'. The rule comprises a character string that may contain an asterisk '*' as a substitute character for any number of characters or a percent sign '%' as a substitute for precisely one character. An ignore character (apostrophe ') is also defined. Each asterisk '*' or percent sign '%' in the selection rule must be assigned an asterisk or percent sign or an apostrophe in the conversion rule. The asterisk means that the character string in the input file is to be transferred unchanged to the output file. Correspondingly, the '%' transfers precisely that character located in this position. The apostrophe sees to it that the character string or character represented in the input file by an asterisk or percent sign respectively, is not transferred to the output file. The remaining characters from the input file are translated into the corresponding characters from the conversion rule. The length of the character string an be changed as desired. ------------------------------------------------------------------------ $$$$ ------------------------------------------------------------------------ With this rule, all file names beginning with `USER` are converted to the new identification `USER2`. The rest of the name is retained. ------------------------------------------------------------------------ $$$$ ------------------------------------------------------------------------ With this rule, all files with the identification `USER` are given the prefix `DEC` before their old name. In the process, only those files are affected whose second part of their name begins with `DAT`, is followed by any character and ends with `B`. This `B` is converted to `C` in the name of the output file. Above all, empty character strings are also allowed in the conversion rule, so that characters can be deleted. E.g.: ------------------------------------------------------------------------ $$$$ ------------------------------------------------------------------------ Old name: `USER.FLAMUP00` + New name: `USER.CMPFLAM00` The UP part of the name is not mentioned in the output name and is therefore left out. A conversion rule is implicitly supplemented by FLAM, e.g.: ------------------------------------------------------------------------ $$ corresponds to <*ASM.*='CMP.*>$$ ------------------------------------------------------------------------ This can be particularly useful when converting the filenames from other systems. *Example:* The FLAMFILE created under DEC/VMS contains the following filenames: ------------------------------------------------------------------------ DUA1:[ABC]DE0051.;7 DUA1:[ABC]DE0052.;4 DUA1:[ABC]DE0080.;2 DUA1:[ABC]DE0152.;4 ------------------------------------------------------------------------ This corresponds to specification of the file versions of the user ABC on disk volume `DUA1`, with the version number stated after the semicolon. In order to be able to create these files in MVS, the following conversion rule could be specified: ------------------------------------------------------------------------ $$FLAMOUT=$$ ------------------------------------------------------------------------ This implicitly deletes the name prefix `DUA1:[ABC]`, takes over the part of the name beginning with DE and supplements it with the identification, and deletes the rest of name: ------------------------------------------------------------------------ USER.DE0051 USER.DE0052 USER.DE0080 USER.DE0152 ------------------------------------------------------------------------ The conversion rule is also important with respect to generating members of a PO library: ------------------------------------------------------------------------ $$FLAMOUT=$$ ------------------------------------------------------------------------ This rule means that the 2nd name qualifiers of the original file are used as member names of the PO library. So far, the conversion rule has only been described for decompression from a (group) FLAMFILE. It can, however, also be used for compression with simultaneous generation of several items of compressed data in different files. In this respect, the conversion rule relates to the FLAMFILEs whose names are generated from the file names of the input files (FLAMIN). This is how all compressed data could be given a prefix: ------------------------------------------------------------------------ $$C,FLAMIN=USER.*.LIST,FLAMFILE=$$ ------------------------------------------------------------------------ It is, however, also possible to store each item of compressed data as a member of a library by means of the following specification: ------------------------------------------------------------------------ $$C,FLAMIN=USER.*.LIST,FLAMFILE=$$ ------------------------------------------------------------------------ The member name is, in this case, the 2nd qualifier of the input file. Also valid here: if a library is specified for the FLAMFILE by the JCL, the name of the library as specified in the rule is ignored and only the members in the specified PO file are generated. The reverse also applies: file names of the FLAMFILEs are used to generate file names for the decompressed files: ------------------------------------------------------------------------ $$D,FLAMFILE=USER.CMP.*,FLAMOUT=$$ ------------------------------------------------------------------------ NOTE: If a group file has been created with `HEADER=YES` but `FILEINFO=NO`, no filename has been stored for this file. The individual files can then be accessed for decompression via the internal file name: the name is in the range from `FILE0001` (for the first file) to `FILE9999` (for the 9999th file). ------------------------------------------------------------------------ $$D,FLAMOUT=,..$$ ------------------------------------------------------------------------ is for the third file in the group file; or ------------------------------------------------------------------------ $$D,FLAMOUT=,..$$ ------------------------------------------------------------------------ is for decompressing all files according to the conversion rule. *Comment:* As a "final rescue option" for automatic generation of the decompressed data with "impossible" filenames from foreign operating systems, the parameter `FILEINFO=NO` can be specified for the decompression process. With this parameter, the stored filenames are ignored and the internal names `FILE0001` to `FILE9999` are generated. These must then be converted to valid filenames by means of a conversion rule. ==== Subprogram FLAMUP FLAMUP differs from FLAM in that respect, that it can be called as a subprogram out of an application. All accesses to data are performed via FLAMUP. All parameters can be passed to FLAMUP using the CALL interface or methods as provided by FLAM (interactive parameter definition or parameter file) can be used. Using FLAMUP within a driver program allows for example to select a certain set of files and to compress/decompress all the selected files in one step. A typical example is, to select only files that were modified after a certain date (archiving). ==== I/O User Interface This interface allows the user to integrate own access methods into FLAM. One possibility is to pass the compressed records immediately to post-processing routines without creating a compressed file. Vice versa, during decompression the compressed records may be received from a preprocess instead of reading them from a file. A direct application could be the integration of FLAM with a file transfer application avoiding the creation of intermediate files. Generally this interface allows to intercept all input and output data of both FLAM and FLAMUP. This allows the user to adapt FLAM easily to specific access methods. ==== User exits .Original data input EXK10 This user exit interfaces the record passed to FLAM for compression. Special processing can be defined for: Start of file, record level, end of file. Records can be passed on, modified, deleted or inserted. This exit can be used to modify records in a structure dependent way. EXK10 is only available in FLAM and FLAMUP and corresponds with EXD10 during decompression. .Compressed data output EXK20 This exit interfaces the compressed data before it is written into the FLAMFILE. Special processing can be defined for: Start of file, record level, end of file. This exit can be used to modify records in a structure-independent way. With this exit it is possible, to modify the data with an own encryption routine, or a special code translation can be applied if a non-transparent file transmission method shall be used.It is possible to insert own records in front of the compressed records, for example archiving control records or origin information. Another possibility is the extension of records to append specific revision information. EXK20 is available in FLAM, FLAMUP and FLAMREC and corresponds with EXD20 during decompression. .Original data output EXD10 This exit interface the decompressed record immediately before it is written into the target file. Special processing can be defined for: Start of file, record level, end of file. Records can be passed on, modified, deleted or inserted. This exit can be used to modify records in a structure dependent way. EXD10 is only available in FLAM and FLAMUP and corresponds with EXK10 during compression. .Compressed data input EXD20 This user exit interfaces the compressed data immediately after it is read from the FLAMFILE. Special processing can be defined for: Start of file, record level, end of file. This exit can be used to modify records in a structure-independent way. With this exit it is possible, to decrypt the data with an own decryption routine or to apply the reverse code translation as used during compression. For a proper operation of FLAM it is indispensable that all changes applied to the compressed data are reversible. User exit EXD20 must deliver exactly the same data as user exit EXK20 received. All modifications applied to compressed data with EXK20 must be undone with EXD20. EXD20 is available in FLAM, FLAMUP and FLAMREC and corresponds with EXK20 during decompression. .Key management KMEXIT This user exit returns a key to the FLAM utility for en/decryption of a FLAMFILE. So it is possible to enter any PASWORD/CRYPTOKEY in a secure way without notice to the JCL or the protocol. This exit is implemented as an interface to special key management systems, without influence to the FLAM utility programs. ==== Utilities Some utilities are released that improves support form FLAM and FLAMFILEs for z/OS only. .FLAMCKV FLAMCKV analyses a cataloged VSAM-KSDS FLAMFILE. It displays the procentual distribution of record lengths and the number of records needed for one FLAM matrix (in FLPRINT, RECFM=VB,LRECL=124). This is very important for direct access to a VSAM-KSDS FLAMFILE. _Please remember:_ FLAM needs a complete matrix (i.e. a block of self-contained compressed records) for decompression. Even for direct access to a single record this complete matrix is requirerd. So performance is best when this matrix is stored in one VSAM record. If the VSAM record is too small to fit a complete matrix, FLAM has to read or write mutiple records. This decreases perfomance. These requirements are not so important for a small amount of data. But if thousands or millions of records are stored it becomes more and more relevant. *Example:* ------------------------------------------------------------------------ $$//CKV EXEC PGM=FLAMCKV$$ $$//STEPLIB DD DSN=FLAM.LOAD,DISP=SHR$$ $$//FLPRINT DD SYSOUT=*$$ $$//FLAMFILE DD DSN=USER.XMLDAT1.ADC,DISP=SHR$$ ------------------------------------------------------------------------ .FLAMCTAB (!! deprecated !!) Utility FLAMCTAB should not be used anymore in order to create translation table modules. All common translation table modules are provided and can be downloaded from our website under: https://flam.de/en/technology/download/addons/[FLAM4 Translation Tables]. Please use further information described there and the precompiled load modules for the most common convertions you can find under download. Another alternative under FLAM5/FLCL is usage of command 'FLAM' with keyword 'FROM' (conversation from CCSID) and 'TO' (conversation to CCSID [UTF-8]). .FLAMDIR FLAMDIR reads a FLAMFILE (DD-name FLAMFILE) and logs the directory of the compressed/encrypted files. The output (DD-name FLPRINT) looks like an ISPF panel 3.4 or option 'I' in FLAM start panel or FLTOC clist. Using the FLAM-parameter `D,SHOW=DIR` you still get a full protocol of the directory of the FLAMFILE. But using FLAMDIR the protocol is a conveniently laid out short summary of the directory of a Group FLAMFILE. *Example:* ------------------------------------------------------------------------ //DIR EXEC PGM=FLAMDIR //STEPLIB DD DSN=FLAM.LOAD,DISP=SHR //FLPRINT DD SYSOUT=* //FLAMFILE DD DSN=USER.ARCHIV.ADC,DISP=SHR ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER === Matrix Buffer Details === 3.6. Matrix Buffer Details The size of the matrix buffer and the number of records which are stored there can be specified for the compression procedure using the maxbuffer and maxrecords parameters. As a general rule, the efficiency of the compression procedure increases the more records are compressed together in the same block. If there are no other factors which need to be taken into account, it is normally best to choose the highest possible number (255 or 4095, resp.). This preset number is, however, merely an upper limit. There may be fewer records if the space in the matrix buffer is exhausted or if the end of the original file is reached. The size which must be set for the matrix buffer is dependent on the number and size of the records that need to be buffered.The space requirement for files with fixed-size records is calculated simply as the product of the number of records and the record size. If the files have a variable record size, it is not normally possible to predict the memory requirement exactly. The value which is chosen is bound to be approximate. However, the matrix buffer must always be at least large enough to hold every individual record in the file. ... print CLE_DOCTYP_CHAPTER === FLAMFILE Details [[flam4.main.flamfile.details]] === 3.7. FLAMFILE Details Below you can find more details about the construction of a FLAMFILE. .Structure of the FLAMFILE In order to achieve the aim of universal interchangeability, the structure of the FLAMFILE must be such that it can be represented in any operating system. This "least common denominator" is the sequential file with fixed-size records. Although other FLAMFILE record formats are allowed for the FLAMFILE in addition to the fixed format, the compressed data which is generated by the compression procedure is broken down into "abstract" records - known as FLAM records - with an identical size; these records are then mapped to the "concrete" structures of the installed operating system. In addition to the compressed data, each FLAM record contains a FLAM record size field at the beginning and either a checksum or a check character at the end. The FLAM record size field and the checksum or the check character are elements of the FLAM syntax. This permits the start and end of a FLAM record to be identified irrespective of the record format in which the FLAMFILE has been stored by the operating system. In FLAM4 terminology, a FLAM record is the same as a record if the FLAMFILE has fixed and variable record formats. The specified FLAM record size is thus the same as the record size or the maximum record size. The residual data in a compressed block belonging to a FLAMFILE is merged with the data in the next block to form a FLAM record with the fixed or maximum size. With the possible exception of the final record, all the FLAM records have the same length. This enables optimum use to be made of the storage medium; however, it is not possible to assign the records of the original file to the records of a FLAMFILE uniquely. A FLAMFILE of this type can therefore only ever be decompressed in its entirety. FLAM supports fixed and variable or stream record formats for the FLAMFILE. Irrespective of the record format, the minimum permissible size of a FLAM record is 80 bytes and the maximum size 32,760 bytes (4095 for `mode=cx7`), providing a lower upper limit has not been set by the user. This span is sufficient in practice to allow any FLAMFILE to be transferred, even in situations where this would not necessarily be the case with the original file, owing to restrictions imposed on the record size or the record format by the file transfer product. Even though many data exchange problems can be eliminated to a greater or lesser extent with the FLAM4 concept, there are some which still cannot be solved. If a file is unsuitable for uncompressed transfer with a particular file transfer product, for example on account of the binary data it contains, it will still not be possible to transfer it after it has been compressed with FLAM, even if it is compressed in the cx7 mode. The reason for this is that every bit combination which occurs in the original file also occurs in the FLAMFILE. .The FLAM File Header In addition to the compressed data, a FLAMFILE may contain other information which is stored in the FLAM file header. FLAM distinguishes between two information categories in two different types of file header: Common information is stored in the common FLAM file header. This includes all information about the original file, such as the file organization, record format, length and position of the primary key, etc., as well as details of the FLAM version and the operating system in which the FLAMFILE was created. User-specific information of any kind is stored in the user-specific FLAM file header. This header type can be inserted additionally if the record interface is used, for example for information which must be evaluated by the application program. Creation of the common FLAM file header is supported by all interfaces (command, subprogram and record) by means of parameters and arguments. A user-specific FLAM file header can only be created if the record interface is used. This interface incorporates the following functions for creating and evaluating the various file header types: [width="60%",cols="^1,^1,^1",align="center",options="header"] |=========== |FLAM file header type| Created by| Evaluated by |Common |flmphd |flmghd |User-specific |flmpuh |flmguh |=========== It is not mandatory to insert FLAM file headers in the FLAMFILE. If they are inserted, it is essential to observe the stipulated order when invoking the functions of the record interface. FLAM file headers are always inserted in the FLAMFILE directly preceding the compressed version of the file to which they belong. If a FLAMFILE contains compressed versions of several files, a FLAM file header can be inserted in front of each compressed file. When the files are decompressed, the compressed data originating from different files can be distinguished and decompressed into separate files. .Secure FLAMFILEs Each record in a FLAMFILE is protected against manipulations and transmission errors by an appended checksum. In compliance with increased requirements regarding cryptographic security, additional security features for encrypted FLAMFILEs have been implemented which improve the protection against a greater variety of attacks. A FLAMFILE provided with these features is called "Secure FLAMFILE". * The new protection mechanisms work on three levels * the segment level, * the member level, and * the file level. Here, "segment" denotes the compressed result of a single matrix and "member" the entirety of segments from a single original file. In a Secure FLAMFILE, there are byte and record counts both for each member as well as for the entire file, members are numbered consecutively, and a timestamp marks the creation time. When encrypted with AES, MACs (Message Authentication Codes) are attached to the FLAMFILE. Those are checksums calculated with a secret key that allow verifying the authenticity of the data. A MAC is calculated for each segment. To ensure the completeness of a member, a second MAC is chained through all segments of the same member, and a third MAC, chained through all members of the FLAMFILE, ensures the completeness with regard to the file. Similar features are included when FLAMFILEs are created with `mode=flamenc`. To include the information necessary for these checks, such as counts, timestamps, MACs, etc., a member header is inserted in front of each member and each member is followed by a member trailer. In addition, a file trailer is appended at the end of every Secure FLAMFILE. Secure FLAMFILEs may not be concatenated since this would create discontinuities of numbering, counters, and MAC chaining that would make FLAM assume integrity violations and break off processing. Individual members of a FLAMFILE, however, can be extracted, since checking of the member chaining is suspended, whenever members are skipped, and only the segment chaining is then tested. .File organization of FLAMFILEs All formats and file organizations for the original file are supported for the compressed file, called FLAMFILE (PS, IS, VSAM-ESDS, -KSDS, -RRDS). Compressed files in VSAM KSDS format can be modified record by record using record level interface. With parameter `CRYPTOM=AES`, `MODE=CX8` or `VR8` it is possible to update, insert, delete original records in a compressed and AES-encrypted VSAM-KSDS FLAMFILE. The use of this record level interface and the facility to store compressed data in index sequential files (VSAM KSDS) allows a fast random access to compressed data. This is well suited for setting up online archives for documents and similar data. ... print CLE_DOCTYP_CHAPTER === Interfaces === 3.8. Interfaces FLAM provides a set of interfaces that allow the use of the product within different environments and for different applications. The simplest application is the execution of FLAM via the EXEC command. This allows to compress or decompress complete files. In addition, FLAM provides a set of subprogram interfaces for integration with other programs and products. This also allows to develop tailored applications where FLAM is embedded in dedicated control programs. User exits provide the pre- and postprocessing of the original data as well as of the compressed data without the additional step via intermediate files. All interfaces are designed in regard to higher level programming languages like COBOL. Only in cases where the usage of address pointers is a conditio sine qua non, assembler (or equivalent, e.g. C) interfaces must be used. FLAM can be embedded in user-defined applications or vice versa by means of calls at different levels. The available interfaces are listed in the table below: [width="90%",cols="1,1,2",align="center",options="header"] |=========== |Interface | Call level | Purpose |*FLAM utility* via EXEC command | System | Compress/decompress entire files interactively or in procedures |*flamup subprogram* | Application programs | Compress/decompress entire files in user-defined applications |*flamrec record interface* | Application programs | Transfer and recall individual records in user-defined applications |*User-defined input/output* | FLAM | Replace FLAM input/output routines |*User exits and File access exits* | FLAM | User-defined file and record pre-processing and post-processing for compressed and non-compressed data Key management |=========== These interfaces have a hierarchical order, which defines which interface can be used or invoked from where. The interfaces are subdivided into two categories: active and passive. Active interface calls are initiated either by the user or by his application program; they activate the FLAM routines which support them. These routines include the command and the subprogram and record interfaces. FLAM uses passive interfaces to activate user routines which are required to perform a particular range of functions, but whose calls are not synchronized with the logic of the application program. This category includes the user-defined input/output and the user exits. The next few sections contain a brief description of the ways in which the different interfaces can be used, without covering every single detail. Each of the following chapters contains a detailed description of one of the interfaces and can be used as a reference document. All the program interfaces, i.e. all interfaces other than the command, are represented in C notation in the reference chapters. However, these interfaces do not support C programs only. Rather, they can be used with any programming language, since all arguments are also referenced by addresses instead of the typical C referencing by values. Addtional beside the C calling interface and the load module interface (COBOL, PLI, ASSEMBLER) Java and other object oriented class library interfaces are available. Even though, in a strict C context, a name without an "*" operator still identifies the argument address with this form of transfer, all references to such arguments in the descriptions below in fact relate to the argument values. ==== The FLAM Utility FLAM as a utility can be started on job control level via a job control command. Parameters define the mode of operation. Depending on the operating system, parameters may be supplied as a part of the command or may be entered via the screen. In addition can be parameters read from a parameter file. This file can be assigned to the process using job control commands or command parameters. The FLAM command can be used to compress or decompress individual files or groups of files either interactively or by means of procedures. The compress or decompress parameter specifies which operation is to be performed. All the information necessary to execute the command can be specified in the form of inputs made directly in the command line, in a parameter file or as property values. The FLAM command permits a separate FLAMFILE to be created from each original file. It is also possible to compress a group of original files into one FLAMFILE. The attributes of the decompressed file, such as its organization and record format, may otherwise differ from those of the original file. FLAM can thus also be used for file conversions. The FLAM command and all its parameter is described in detail in this manual below. ==== The FLAMUP Subprogram The interface to the FLAMUP subprogram permits roughly the same operations as with the FLAM Utility to be initiated in an application program. This interface makes the same range of functions available to the application programmer as to the user, with the exception of the built-in functions. FLAM parameters are used to specify these settings, similar to the procedure for the parameter file. These parameters are specified consecutively in an character string, which is transferred to the subprogram in the form of an argument. Once again, it is possible to refer to a parameter file using the parameter parfile=parameter file. If any information is omitted, the same default settings and properties are used as for the FLAM Utility command. When flamup has been executed in its entirety, the invoking program receives a return code indicating either the success of the call or the cause of the error. The FLAMUP load module interface and the flamupc interface is described in detail in the FLMUP interface spezification. See also: https://flam.de/en/technology/products/flam/api-subprogram/[FLMUP] ==== The FLAMREC Record Interface The FLAMREC record interface permits file processing and compression to be integrated in an application program. Decompressed records can be read sequentially from a FLAMFILE. A new FLAMFILE is created when the compressed data is output. A FLAMFILE can thus be opened either for read accesses only (decompression) or for write accesses only (compression). FLAM's record interface consists of a class of functions, which allow the application program to access and process compressed data in the same way as is possible with ordinary input/output instructions in the case of non-compressed data. The use of these functions is governed by rules similar to those which apply to the standard input/output. After the file has been opened, records can be read or written; the file is closed again when all processing has been completed. The compression or decompression that takes place during processing is completely transparent to the programmer, so that compared to programs using conventional input/output operations, there are almost no differences in the program logic. By using the record interface, compressing (or decom-pressing) the processed data no longer requires an extra step after (or before) running the application. In addition, it is no longer necessary to provide sufficient temporary storage space for both the compressed and the uncompressed copy of a file. The FLAMREC load module interface is described in detail in the FLMREC interface spezification. See also: https://flam.de/en/technology/products/flam/api-record-interface/[FLAMREC] ==== The User-Defined Input/Output FLAM supports all inputs and outputs to and from disk files using standard input/output instructions. However, it also allows the user to replace these standard input/output routines with routines of his own. This may be a good idea, for example, if there is a possibility of an input/output device other than a hard disk being used or if the file contains a record format which is not supported. It is not possible to describe every single aspect of every function of the user-defined input/output in detail, since these vary according to the device and to the nature of the data. Data which is received via a modem line, for example, requires different actions from data supplied by measuring instruments. FLAM considers the origin of the data to be "file", even though in a particular instance other objects may be involved. The functions of the user-defined input/output are as follows: [width="60%",cols="^,^,^",options="header",align="center"] |=========== |Function |Purpose |Corresponding I-O calls |usrcls |Terminating processing of file |fclose |usrget |Reading one record |fgets |usropn |Opening file |fopen |usrpos |Positioning to record |fseek |usrput |Writing one record |fputs |=========== FLAM specifies a range of functions based on the file processing requirements for this interface, in accordance with the above point of view. It is up to the user of the interface to optimize the design of the functions, to ensure that the desired results are achieved with them. In order to be able to use a user-defined input/output, the routines concerned must be linked to the application program. This option is therefore only supported by the program interfaces, i.e. by the subprogram and record interfaces. The support by the record interface of course only applies to accesses to the FLAMFILE, since these are the only types of access which take place via it. A user-defined input/output is initiated by invoking flmopd with device=7. This input/output can be activated at the subprogram interface for all the files which are concerned by the compression or decompression procedure, by means of the FLAM parameters user_io, inuser_io and outuser_io. Whereas, with the functions of the record interface, the programmer is obliged to observe a series of rules regarding the order and consistency of the calls, the correct interaction of the functions of the user-defined input/output is the responsibility of FLAM, which also takes the initiative for the calls. The programmer of the user-defined input/output routines is merely required to make sure that the actions and behavior of the individual functions are correct. The description in this chapter is therefore restricted to these aspects. usropn is invoked first of all once, and once only, for each assigned file. The workio argument causes a work area of 1024 bytes to be made available as file-specific memory. This area is adopted automatically for all subsequent calls until the usrcls. The openmode argument specifies the desired access type (input, output). The record_format, record_size, etc. arguments specify the file and record attributes, which can be adapted to each particular file if necessary. The successful termination of the function and any special states or errors can be reported by means of predefined and freely assignable return codes. Each return code is evaluated by FLAM, and passed on to the invoking programs if an error is established. A record is transferred for writing with usrput. If the record cannot be written with the specified size, the return code must include the information that it has been either shortened or padded with the characters specified by padchar in the usropn. FLAM requests the next record with usrget. The maximum number of characters which can be transferred is specified in the buffer_length parameter. If the record needs to be shortened as a result, this must be indicated by the return code. The return code must also indicate when the end of the file is reached. The record size must be returned for each record which is read (even if the record format is fixed). usrcls causes the file to be closed. The work area for this file is released again by FLAM after control has been handed back. ==== The User Exits User exits allow pre- and post processing of records before compression and after decompression under FLAM as utility as well as pre- and post processing of compressed records under both FLAM as utility and FLAM record level interface. E.g., this user exits can be used for encryption purposes or for selective processing of original data. A user exit is a user-provided program that is invoked by FLAM at specific events during the processing via the defined interface. These user-provided program has to be linked statically. FLAM supports two catagories of user exits: file access exits and an exit for automatic key management. ==== File Access Exits (s. also FLAM4 manual in paper form) These exits permit the user to postprocess the records output by FLAM (both those contained in the FLAMFILE and those in the decompressed file) before they are finally saved, i.e. they can be modified, deleted or given additional records. The records which are read by FLAM can be prepared for processing by FLAM in the same way. The term "access" refers in this context either to an input/output instruction or to a corresponding call of the user-defined input/output, and not to a call of a record interface function. Thus - at least as far as the FLAMFILE is concerned - the user exits are not synchronized with the application program, since the latter is not able to establish, for example, whether or not a flmget call actually does initiate one or more read operations. These user exits can be activated via the subprogram interface or the record interface, the latter being able to activate the user exits for FLAMFILE accesses only. The programs invoked through these interfaces must be staticly linked to the application. A user exit is invoked each time the file is opened, read, written or closed. The functioncode argument indicates to the user exit the access type, to permit it to respond in the appropriate manner to each particular situation. FLAM supports the following user exits for file accesses: [width="40%",cols="^,^",options="header",align="center"] |=========== | User exit | Invoked for accesses to | EXK10 | Original file (compress) | EXK20 | FLAMFILE (compress) | EXD10 | Decompressed file (decompress) | EXD20 | FLAMFILE (decompress) |=========== User exits exk10 and exk20 can be used during the compression procedure for accesses to the original file or to the FLAMFILE, whereas exd10 and exd20 can be used during the decompression procedure to access the decompressed file or the FLAMFILE. If records contained in the FLAMFILE were modified by user exit exk20 when they were compressed, they can only be decompressed if the modifications are undone again by the complementary user exit exd20, i.e. if every FLAM record is reset by exd20 to the state in which it was originally transferred by FLAM at user exit exk20. ==== User Exit for Automatic Key Management Automatic key management facilitates handling encrypted files by relieving the user from duties like generating, exchanging and storing passwords, in whatever way. When invoking the key management exit routine for encryption, FLAM also passes an optionally user-supplied character string via the interface. The exit returns a key value which is used by FLAM for encryption. In addition, the exit may also return a byte string that FLAM attaches to the encrypted FLAMFILE. When decrypting, FLAM passes this string back to the exit for the retrieval of corresponding decryption key. This exit can be activated by the flam command or the subprogram interface. It is also available via the record interface. Programs invoked through these interfaces must be created as shareable objects and are linked dynamically. IMPORTANT: Throughout this manual the terms "password" and "(encryption) key" are used synonymously. An exapmle for a professinal solution which use the FLAM Key Management Extension (FKME) together with Hardware/High Security Modules (HSM) for PCI DSS conform handling of card holder data in paymant systems can be found on the link below (in german only): https://www.flam.de/ftp/specifications/SpecFKME_HYB_SYM_AES.pdf[FKME] The interface specification for FLAM4 FKME can be found at: https://www.flam.de/ftp/FL5/doxygen/FKME/[FKME] ... print CLE_DOCTYP_CHAPTER === Installation === 3.9. Installation The product FLAM with all its components (FLCL, FLCC, FLSSRV, ...) is delivered in two parts. The main package contains the whole software and is available as download or via package repository (various Linux distributions). The second part is the license, which determines the features and platforms that can be used, where and for what you can use the software. If only the main package is installed, then a limited DEMO license can be used. ... print CLE_DOCTYP_CHAPTER ==== License ==== 3.9.1. License FLAM is protected against unauthorized use. The authorized usage of FLAM is only possible with a licence module provided by Limes Datentechnik. A licence allows the usage of FLAM on one or multiple computers, capacities, users or per usage. There is a difference between test licences for a limited period and unlimited (or limited) production licences. A test licence allows the testing and benchmarking of FLAM with all functions for a defined period (e.g. 30 days). * Test licenses must not be given to a third party. * During the test period, no backup copies of the test programs are allowed. * After the test period is expired all test programs must be deleted. A production licence allows the limited or unlimited usage of FLAM on any computer for which a licence was obtained. FLAM compresses structure-oriented using an algorithm that is a part of the Frankenstein-Limes method. This method has been patented in the Federal Republic of Germany and the United States of America and at the European Patent Bureau, registered by the inventor on 19th September 1985. ... print CLE_DOCTYP_CHAPTER ==== Download ==== 3.9.2. Download FLAM(R) can be downloaded from our website with the following link: https://flam.de/en/download/flam/ INSTALL and README text files are included as guides for installation. The download area distinguishes several operating systems: [width="50%",cols="2,5",options="header"] |==== |Operating system | Link to installation instructions |AIX | https://flam.de/en/download/flam/latest/aix/install.txt |Linux | https://flam.de/en/download/flam/latest/linux/install.txt |Solaris | https://flam.de/en/download/flam/latest/solaris/install.txt |Windows | https://flam.de/en/download/flam/latest/windows/install.txt |zSeries Linux | https://flam.de/en/download/flam/latest/zSeries/linux/install.txt |zSeries USS | https://flam.de/en/download/flam/latest/zSeries/uss/install.txt |zSeries z/OS | https://flam.de/en/download/flam/latest/zSeries/zos/install.txt |==== Manuals can be found at: https://flam.de/en/technology/download/documentation/ ... print CLE_DOCTYP_CHAPTER ==== Replace External Libraries ==== 3.9.3. Replace External Libraries External libraries under LGPL are linked in dynamic form and can be replaced by another dynamic library which fulfills the corresponding interface. This can be done by deleting the library in the program directory if the replacement library is located in the library path or by overwriting the library in the program directory. The list below contains all LGPL based libraries used by FL5 per platform. glibc on Linux: This library is part of the operating system and a prerequisite to run the FL5 infrastructure on Linux systems. FL5 can be used with different version and distributions of linux. Replacement of glibc will be done by the normal update process of your linux installation. libfuse on Unix: This library is part of the operating system and a prerequisite to run the FL5 file systems on Linux systems. The file systems are delivered for libfuse version 2 and 3. Replacement of libfuse will be done by the normal update process of your Unix installation. libssh: The libssh library can be found in the installation directory. A customized version is used that is not compatible with the official libssh releases. The modified libssh source can be found at: https://github.com/limes-datentechnik-gmbh/libssh NOTE: In all cases we recommend using the tested and supported libraries from this package. If a supplied library is replaced, the license terms and warranty of the new library become valid, replacing the license and warranty of the replaced library. ... print CLE_DOCTYP_CHAPTER === Filename Handling === 3.10. Filename Handling All filenames are mapped by CLE/P, which means that all replacement mechanisms it supports are also valid for each filename used within a command. See <> of <>. All filenames specified in command strings are parsed and handled in a unified way across the different platforms. Standard replacements by shells are usually not usable within the command syntax. There are some special attributes that can be appended to a filename when using paths or the URL-like syntax by using one of the following separators followed by a value string: * `/?` - member name (to access a member in an archive by name) * `/:` or `/#` - member index (to access a member in an archive by position) * `/&` - CCSID that the file's contents are encoded in If the URL-like syntax is used, special characters can be URL-encoded as a percent character followed by two hex digits (%xx). If no schema is provided, but a raw path, URL-encoded characters are not decoded because the string is not a URL. The schema `file://` can be used to access a local file on the system in URL-like syntax. Providing a CCSID is mainly intended to be used for parameter files which are read to memory from a remote location which may require conversion to the local character set. The filename is the only way to specify the CCSID for this type of files. Filenames in archives (ZIP, TAR, FLAM) or certain file headers (GZIP, PGP) are sanitized when reading or writing to prevent directory traversal attacks. This type of attack aims at writing files in arbitrary locations in the filesystem by storing specially crafted paths in the archive or file header. For example, extracting a ZIP file containing paths such as `/etc/passwd` or `../../../../../../etc/passwd` as user root on a Unix-based system could lead to a lock-out of all users. Path sanitization removes all path components that lead to an extraction outside the desired output directory. This includes turning absolute paths into relative ones and removing all `..` directory components. Both example paths above would be transformed to `etc/passwd` which are safe for extraction. The filename handling works as follows: .On mainframe systems (POSIX(OFF)): Usually, a filename has to be specified fully qualified, because a dynamic allocation is done in most cases. The following substitutions can be used, which are automatically replaced by their respective value during execution: ------------------------------------------------------------------------ - to expand an environment variable ------------------------------------------------------------------------ If the environment variables below are not defined, the following defaults are used: ------------------------------------------------------------------------ - The current user id (Example: file='.TEST.DATA') - The current user id (Example: file='DD:') - The current home directory (Example: file='/dat.bin') - The current user id, but normally the default owner 'limes' is defined - The default is simply the first letter of SYSCLONE or if not defined 'T' for test, normally not required for file names ------------------------------------------------------------------------ The syntax to define a DD name is: ------------------------------------------------------------------------ DD:name - a DD-Statement for name (Example: file='DD:INDAT') ------------------------------------------------------------------------ A replacement may also be used in a DD name definition: ------------------------------------------------------------------------ Example: file='DD:' ------------------------------------------------------------------------ Additionally, you can specify member names in a DD name definition if a PDS(E) is allocated. The member name can be fully qualified or contain wildcards. Member names for archives can be specified as part of the URL-like notation or it can be specified separately. ------------------------------------------------------------------------ Example: file='DD:INPUT(*B*)' file='DD:INPUT(EGBERT)' file='DD:INPUT(EGBERT)/?*HOGO*' name='DD:INPUT(*GB*)' member='*HOGO*' ------------------------------------------------------------------------ When using static allocation (DD name) for writing, the DCB parameter must be provided if the file does not exist. Otherwise, the defaults of the runtime environment (often `PS RECFM=UNDEFINED LRECL=0 BLKSIZE=6144`) are used. A static allocation is only useful to write the logical byte stream in a certain physical file format. In all other cases, we recommend to use dynamic allocation when writing new files. Since version 5.1.22-28565 the DD name must not be prefixed with 'DD:' anymore. If only one qualifier given, if it a valid DD name and if a allocation for it found, then the qualifier will be used as DD name. If you use FILE=STREAM when reading the file descriptor STDIN (DD:SYSIN) and when writing STDOUT (DD:SYSPRINT) is used. All other messages are printed by default to STDERR (DD:SYSOUT). For read the default DD name 'FLAMIN' and for write the default DD name 'FLAMOUT' is preferred if no file specification given. If such a allocation not found at write the default dataset name (original name plus extension) is used. For read or write of a FLAMFILE additional the DD name 'FLAMFILE' are supported to be backward compatible with the old FLAM utility. A Unix path name must contain a forward slash (`/`). To use a file in the current directory `./filename` must be used. Paths relative to the home directory may start with a tilde character (`~`) as an abbreviation for for UNIX pathnames or for host dataset names. ------------------------------------------------------------------------ Example: file='~/mydata.txt' file='~.po.dataset(test)' file='DD:~' ------------------------------------------------------------------------ NOTE: Starting with version 5.1.5, the plus sign (`+`) can no longer be used as abbreviation for the user's home directory. The tilde (`~`) remains unaffected and is now the only valid abbreviation. The plus sign (`+`) is now a wildcard character. It was supported as alternative for EBCDIC systems, because tilde is located on different code points depending on the EBCDIC code page. Since version 5.1.5, we support the correct interpretation of diacritical characters on EBCDIC systems based on the CCSID defined in the environment variable LANG. see <> and <> of <> To specify the casing (mainly useful for USS on ZOS) for a user ID, the following definitions can be used (if not defined as environment variable): ------------------------------------------------------------------------ CUSER - The current user ID in upper case (same as ) Cuser - The current user ID in title case cuser - The current user ID in lower case ------------------------------------------------------------------------ .On Unix, Linux, Windows and other non-mainframe systems (including USS) A filename may be an absolute or relative path and may contain any of the substitutions (including ~) mentioned in the previous section. To use a smaller than character (`<`) as part of a filename, the character must be escaped by doubling it. We recommend not to use special or whitespace characters in filenames. The ampersand character (`&`) can be used instead of at sign (`@`) because the letter has different code points in EBCDIC. (the alternative character on EBCDIC, if `@` is not working, is `§` (the command line parser converts the `@` corresponding to the environment variable LANG on EBCDIC systems, if the value not set or wrong, then you can use `&` or try `§`)). On USS and Micro Focus EDZ, DD names are supported. A UNIX path name is distinguished from a data set name in the same way as it is done in the z/OS runtime environment. Example on USS or EDZ: ------------------------------------------------------------------------ file="//DS:user.test(data)" file="//'user.test(data)'" file="//test(data)" file="//DD:name" file="DD:name" ------------------------------------------------------------------------ .Within Microfocus Enterprise Server (Linux or Windows) If the variable FLAM4MF is set to `yes` or a valid CCSID, a DD name is assumed unless it contains a dot character and is used as the dataset name. Otherwise a DD name must be specified as DD:name and a dataset is given with `//'name'` ------------------------------------------------------------------------ Examples: file="//'user.test(data)'" file=DD:name file=readme.txt ------------------------------------------------------------------------ This corresponds with the syntax for host dataset names on USS of z/OS. With the Microfocus support for Windows and Linux, you can work on x86 Windows and Linux systems with the local MF-EDZ-Enterprise-Server like you work with the catalog access on USS for MVS files. If you work with EBCDIC on the record interface in a EDZ environment then you must set the FLAM4MF environment variable to the used CCSID (e.g. IBM1141). If a CCSID is defined and starts with `IBM`, this will also override the default CCSID in the character conversion component and mark it as z/OS platform. In an error situation, it is possible to enable writing a trace to a file, which can be used to analyze problems. The environment variable `FLAM4MF_TRACEFILE` must contain a valid filename to write the trace output. To set the missing system variables for MF-EDZ, several solutions are implemented including `DD:SYSVAR` and the environment variables `FLAM4MF_STATIC/DYNAMIC_SYSVAR` documented above. .Local and remote file access (via SSH) with URLs (all platforms) Instead of local filenames, complete URLs defining all communication parameters and (if required) the member name or index can be used. Optionally a CCSID for this file can be added. The syntax is: ------------------------------------------------------------------------ schema://userid:authdata@hostname:port/filename/&ccsid schema://userid:authdata@hostname:port/filename/?membername/&ccsid schema://userid:authdata@IPv4:port/filename/:index/&ccsid schema://userid:authdata@[IPv6]:port/filename/#index/&ccsid ------------------------------------------------------------------------ If the URL-like syntax is used, special characters can be URL-encoded as a percent character followed by two hex digits (%xx). If no schema is provided, but a raw filename, URL-encoded characters are not decoded as the string is not a URL. The schema `file://` can be used to access a local file on the system in URL-like syntax. Currently, remote file access via SSH is supported for all file types including FLAM4 archives. The `ssh://` prefix indicates remote access via SSH. The userid is optional, the default is the current login user ID. You can use replacements also in the connection parameter. I.e. `` gets replaced with the lower case login user id, which is useful mainly on mainframes, where the SYSUID is in upper case. The password is optional. If no password is specified, public key authentication is attempted. The hostname/IP is mandatory and the port is optional, defaulting to the SSH standard port 22. The hostname and the remote file path are separated by a slash. An IPv6 address must be put in square brackets (e.g. `[::1]`). An absolute path requires another slash (i.e. // after the hostname). Otherwise, the path is relative to the current directory after login to the SSH server. See examples below: ------------------------------------------------------------------------ file='ssh://user:password@hostname:port//path/file.xxx' name='ssh://:password@hostname/path/file.xxx' name='ssh://[::1]//path/file.*/?*/member.*' ------------------------------------------------------------------------ This URL notation is a simplified form of accessing files via SSH. For even more connection options, you can use the `net.ssh()` object which allows, e.g., the configuration of the public/private key file path or different host key check procedures. See the documentation for the `net.ssh()` object for further details. If both, URL notation and `net.ssh()` object, are used to configure the SSH connection, communication parameters specified in the URL have higher priority. So, if a username is given within the URL as well as the through the object, the username from the URL is used. The file for an inverse command cannot be on a remote system. If a remote report, info or log file is specified, the output data written to the remote system is in the local system's character set. When reading user tables, checksum files, password files or other miscellaneous text files, an automatic character conversion to the local character set is performed. When accessing files via SSH from a mainframe system, the remote file is handled like a file from a non-mainframe platform that does not support datasets since SSH file access is block-oriented and not record-oriented. This means that, if record I/O is used, records are written or expected in one of the open formats on the remote system. For example: ------------------------------------------------------------------------ read.record(file='DATA.SET.PSVBA133') write.record(file='ssh://@server1/[name]') ------------------------------------------------------------------------ This statement for the CONV command of FLCL reads a local physical sequential variable block dataset with a logical record length of 133. A pair of 4 byte length field followed by the record data is written to the remote file for each read record. This is because the default mapping of the host record format (`VB`) to a non-host platform is `OPN_VAR`. .Using file name lists With version 5.1.21 the file name specification was extended to an array. This make it possible to provide a list of files/URLs in the forms below: ------------------------------------------------------------------------ file=hugo.txt file='ssh://user@server/berta.dat file=s*.txt file[hugo.txt,'ssh://user@server/berta.dat,s*.txt] file=hugo.txt,'ssh://user@server/berta.dat,s*.txt file=>list.txt file=>'DD:FLIST' ------------------------------------------------------------------------ In the last case a parameter file containing the list with the URLs can be provided using dynamic or static allocation (DD: only for z/OS). In the case a lot of URLs with several connections can be used. FLAM supports now a connection cache with the result that only the required connections are open. The `NET` overlay provides the default settings for each different URL. In each file specification wildcards are still possible. The feature can be used to combine different files in one archive. ... print CLE_DOCTYP_CHAPTER === Directory Support === 3.11. Directory Support .WHEN READING Multiple input files can be processed at once by using wildcard patterns as input filename and recursion into directories. It is also possible to select only certain members of FLAMFILEs by pattern or index and to specify how directories are traversed. Input file or member names can contain the wildcards below. The wildcards are supported on subprogram level, i.e. wildcards are available in the batch utility and in subprogram calls. Wildcards are not supported on record or byte interface level. ------------------------------------------------------------------------ '*' - zero or more characters '+' - one or more characters '?' - exactly one character '%' - exactly one character (backward compatible with FLAM4 and conform with ISPF (not in URLs)) '#' - exactly one digit (not supported on MVS, because '#' could be part of a qualifier) '=' - exactly one digit (supported on all platforms but mainly for MVS) '^' - escape character (put it before a wildcard character if it is part of the file/member name) ------------------------------------------------------------------------ If one of the wildcard characters is part of the filename itself, you can escape the wildcard character with the caret (`^`). For MVS datasets each qualifier is separated with a period (`.`). If you want to let the asterisk (`*`) or plus sign (`+`) match more than one qualifier, you must put it twice (i.e. `**` or `++`). On non dataset based systems (UNIX, Windows, USS, Linux) this mechanism is not available as there are no qualifiers in filenames and hence periods are handled like any other character. Below you can find some examples: ------------------------------------------------------------------------ file='*.txt' - all files with extension .txt including the file named .txt file='+.txt' - all files with extension .txt and at least one other character file='?.txt' - all files with one character and the extension .txt file='dat^*.txt' - the file "dat*.txt" file='**.G====V==' - all datasets ending with GnnnnVnn (all GDG datasets) file='.**(my*)' - all members starting with "MY" of all PO datasets under current user file='.**(-1)' - all previous generation of all GDG datasets under current user ------------------------------------------------------------------------ The directory support for MVS includes Generation Data Groups (GDG). If you want to read only the previous generation of all GDG's under your SYSUID, you must type as follows: ------------------------------------------------------------------------ file='~.**(-1)' - all GDG's with generation -1 under my SYSUID ------------------------------------------------------------------------ Is a wildcard used at read and the different files should be written into a generation data group as several versions. The file index in minimal amount of digest '[ind0]' could be used the define the next generations '(+[ind0])'. ------------------------------------------------------------------------ infile='/path/to/*.txt' pattern='.GDG(+[ind0])' outfile='USER.GDG(+n)' ------------------------------------------------------------------------ The enclosing quotation marks are required because the file name string contains round brackets. Strings without quotation marks are terminated at the first space character or curved or squared bracket. NOTE: If you use a wildcard in the first/high level qualifier, the catalog search interface (CSI) is used to return each dataset in the catalog. All these datasets are then compared against the corresponding pattern. This can consume a lot of time. Be careful with wildcards in the first/high level qualifier at ZOS dataset names. NOTE: Using wildcards at the beginning of a pattern (for example '**(MEMBER)') could result in opening a dataset which is stored on a tape which is not available immediately. This could result in the process waiting for console input. By default, datasets on tapes are not included in the dataset list retrieved by the wildcard mechanism. If you want to include datasets on tapes, then you must enable the corresponding switch. However, this will not prevent a console request. Files in catalog where the volume is not available, will result in the same behavior. Be careful with wildcards at the beginning of the search pattern. The escape character can be used for datasets, but this is not required. If the match string contains parenthesis `()`, only PO datasets are matched. To match GDG, the first character inside the parenthesis must be `+`, `-` or a number. In all other cases, a PO dataset is assumed. Relative generation numbers are currently not supported for PO datasets. To process only certain members inside archives, a match string for members can be used. A member can be defined as part of the URL with `?` as separator. Alternatively, for FLAMFILEs, a dedicated member specification can be used. In this case a leading question mark (`?`) is ignored. For a question mark as first wildcard, specify two of them. ------------------------------------------------------------------------ file='test*.adc/?*FLAM*' - All members that have "FLAM" in their name in files starting with "test" and ending with ".adc" file='*.adc member=*FLAM' - All members that have "FLAM" at the end of their name in all files ending with ".adc" file='*.adc member=?FLAM*' - All members whose names start with "FLAM" in all files ending with ".adc" file='*.adc member=??FLAM*' - All members whose names start with any character followed by "FLAM" in all files ending with ".adc" ------------------------------------------------------------------------ You can also select members from archives based on the member index. In this case, the member name must start with a hash (`#`) or colon (`:`) and can contain a list of indexes and/or index ranges (two indices separated by minus (`-`), inclusive) separated by comma. For example: ------------------------------------------------------------------------ read.flam(file=*.adc/#3,5,8-10,15) read.flam(file=*.adc/:3,5,8-10,15) read.flam(file=*.adc member=:3,5,8-10,15) read.flam(file=*.adc member=#3,5,8-10,15) ------------------------------------------------------------------------ All four variants read the 3rd, 5th, 8th, 9th, 10th and 15th member from files in the current directory that end with ".adc". To define how wildcard strings are matched, there is an object called 'DIR' that contains some switches. These include recursion into sub-directories, recursion into archives, inclusion of hidden files, symbolic link, etc. See the chapter to the 'DIR' object below for more information. .WHEN WRITING Missing directories are created automatically. To prevent this the switch 'NOMKDAR' must be activated. Overwriting of existing files can be prevented with the 'REMAIN' switch (see below). To convert the file names matched to a certain file name for writing see <>. ... print CLE_DOCTYP_CHAPTER === Input to Output Name Mapping [[flcl.input.output.name.mapping]] === 3.12. Input to Output Name Mapping When creating/extracting an archive, converting files or (more generally) doing batch processing, one might wish to automatically generate new filenames based on a pattern applied to the input filenames. Example: You might want to read a member from a FLAMFILE which was created on a mainframe and contains dataset names. You wish to convert the dataset name into a path-based file name. You can achieve this by defining a pattern for the output file or member name. .Pattern syntax: An output filename pattern can consist of simple text strings and tokens. Text strings are used as-is as output filename. Tokens are enclosed in square brackets (`[]`) and describe processing rules applied to the original file or member name. The pattern can consist of an arbitrary number of text strings and tokens (including none at all). A token can consist of one or more processing rules. Multiple processing rules are separated by a pipe character (`|`). Every processing rule within a token receives the output of the previous rule as its input. This allows chaining of processing rules. Characters inside tokens can be escaped by a preceding caret (`^`) to avoid interpretation as pattern syntax. Outside of tokens, carets are treated literally, except in front of squared brackets to treat a bracket literally and not as token delimiter. The keywords of the tokens below are not case sensitive. This is the list of supported processing rules: * [path] - Extracts the string before the last slash or backslash, i.e. the path without trailing (back)slash * [name] - Extracts the part after the last (back)slash, i.e. the filename * [base] - Extracts the part between the last (back)slash and the last dot, i.e. the filename without extension * [ext] - Extracts the part after the last dot, i.e. the file extension * [member] - Extracts a string enclosed in round brackets from the filename part of the string, e.g. a PDS member name on z/OS * [copy] - Simply copies the input string (useful if you only want to add an extension) * [upper] - Converts all characters to upper case * [lower] - Converts all characters to lower case * [title] - Converts the first character to upper case, all others to lower case * [cutX] - Cuts off the string after the X-th character, where X is a positive or negative number indicating how many characters to keep, counting from the front (positive) or back (negative) * [indN] - Inserts the current index as decimal number of at least length N (padded with preceding zeros) N is any number between 0 and 9, where 0 results in variable length numbers with their shortest possible representation (no preceding zeros) * [rndN] - Inserts random numbers of length N (with preceding zeros) N is any number between 0 and 9, where 0 results in variable length numbers (no preceding zeros) * [AX] - A is a place holder for a single-byte character, X is a positive or negative number; Extracts the string part between the X-th occurrence of the specified character and the next occurrence of that same character. If X is negative, search starts at the back and in reverse. If X is 0, the part before the first occurrence of A is extracted. * [AX-] - A is a placeholder for a single-byte character, X is a positive or negative number; Extracts the string part after the X-th occurrence of the specified character. If X is negative, search starts at the back and in reverse. If X is 0, the input string is copied. * [AX-BY] - A and B are placeholders for a single-byte character, X and Y are positive or negative numbers; Extracts the string part between the X-th occurrence of A and (relative to that occurrence) the Y-th occurrence of B. Like above, negative numbers mean counting from the back. * [search=replace] - Replaces the first occurrence of the string 'search' with the string 'replace' * [search=*replace] - Replaces all occurrences of the string 'search' with the string 'replace' * [regex:/pattern/replace/modifiers] - String replacement with Perl-compatible regular expressions * [table] - Evaluates to the current table name if end of table handling is activated (only when reading tables) .Using regular expressions Regular expressions (short: regex) can be used to apply arbitrarily complex matching rules and substitute these matches using the syntax `[regex:/pattern/replace/modifiers]`. They can be chained like any other processing rules using the pipe character, e.g. `[regex:/pattern/replace/modifiers|upper]` ..Regex delimiters The character immediately following the keyword `regex:` is a delimiter to separate the pattern, replacement string and optional modifiers. The delimiter can be any single-byte character that does not occur in the pattern or replacement string and is expected exactly three times. You can also use pairs of brackets as delimiters in which case the matching closing bracket is expected to follow an opening bracket. A few examples using different delimiters, all performing the same replacement: * `[regex:/pattern/replace/modifiers]` * `[regex:#pattern#replace#modifiers]` * `[regex:{pattern}{replace}modifiers]` The pattern and replacement strings must not be escaped with carets (`^`), even though they are inside a token (`[]`). ..Regex patterns Regular expression patterns in the Perl flavor are matched against the input string to find the substrings that are to be replaced. Writing regular expressions may seem like a daunting task if you never did so before. Understanding the basics is not as complicated as it may seem. You can find a tutorial that will get you started quickly here: + https://www.regular-expressions.info/quickstart.html Countless other tutorials and documentation for regular expressions can be found on the web. We recommend to consult these ressources to familiarize yourself with regular expressions. Advanced users can find the complete syntax documentation for writing regular expression patterns by following this URL: + https://www.pcre.org/current/doc/html/pcre2pattern.html ..Regex replacement strings Replacement strings are mostly string literals, except for the dollar character. A dollar character is an escape character that can specify the insertion of characters from capture groups and names from `(*MARK)` or other control verbs in the pattern. In other words, it can be used to insert parts of the match into the replacement string. Capture groups in a regex pattern are sub-expressions in parentheses. These can be accessed by their numerical index starting at 1 (`$1`, `$2`, ...). The capture group accessed by `$0` always references the whole match. Capture groups can also be referenced by name. A named group in a regex pattern uses the syntax `(?subpattern)`. This group can then be referenced in the replacement with `\$\{name}`. ..Regex modifiers Modifiers change the behavior of regular expression matching and/or replacement. Modifiers are optional and must be placed after the final delimiter. Multiple modifiers can be combined. Currently, these modifiers are supported: * `g`: Enables global matching. The regular expression is applied repeatedly and all matches are replaced. By default, regular expression matching stops after the first match has been replaced. * `E`: Fail with an error if the regular expression does not produce any match. By default, the input string remains unchanged if there is no match. * `R`: Instead of replacing matches in the input string, only the replacement string is returned. * `i`: Enables case insensitive matching. * `m`: Enables multi-line matching. By default `^` and `$` only match at the start/end of the entire input string. With multi-line matching, they also match after/before newlines. Note, however, that unless the `s` modifier is set, the "any character" metacharacter (`.`) does not match at a newline. * `s`: If set, a dot metacharacter in the pattern matches any character, including one that indicates a newline. However, it only ever matches one character, even if newlines are coded as CRLF. By default, a dot does not match at a newline. * `x`: Enables extended matching mode. Most whitespace characters in the pattern are ignored except when escaped or inside a character class. Characters between an unescaped `#` outside a character class and the next newline, inclusive, are also ignored, which makes it possible to include comments inside complicated patterns. * `n`: Disables numbered capturing groups (inside parentheses) in the pattern. Any opening parenthesis that is not followed by `?` behaves as if it were followed by `?:` but named parentheses can still be used for capturing (and they acquire numbers in the usual way). .Examples for using filename patterns: ------------------------------------------------------------------------ infile='/path/to/file.ext' pattern='text_without_tokens' outfile='text_without_tokens' infile='/path/to/file.ext' pattern='text^[path^]' outfile='text[path]' infile='/path/to/file.ext' pattern='[PATH]' outfile='/path/to' infile='/path/to/file.ext' pattern='[Name]' outfile='file.ext' infile='/path/to/file.ext' pattern='[ext]' outfile='ext' infile='/path/to/file.ext' pattern='[UPPER]' outfile='/PATH/TO/FILE.EXT' infile='/path/to/file.ext' pattern='[cut5]' outfile='/path' infile='/path/to/file.ext', pattern='FILE[ind4]' outfile='FILE0001' infile='/path/to/file.ext', pattern='FILE[rnd4]' outfile='FILE0815' infile='/path/to/file.ext' pattern='[/2]' outfile='to' infile='/path/to/file.ext' pattern='[/-1]' outfile='file.ext' infile='/path/to/file.ext' pattern='[/2-]' outfile='to/file.ext' infile='/path/to/file.ext' pattern='[/3-.1]' outfile='file' infile='/path/to/file.ext' pattern='[path=directory]' outfile='/directory/to/file.ext' infile='/path/to/file.ext' pattern='[regex:/\.ext$/.bin/]' outfile='/directory/to/file.bin' infile='/path/to/file.ext' pattern='[regex:#/([^/]+)$#$1#R]' outfile='file.ext' infile='USER.DATA.PDS(MYMEMBER)' pattern='[member]' outfile='MYMEMBER' infile='USER.DATA.PDS(MYMEMBER)' pattern='[.0|lower]/[.1|lower]/[member|lower].[ext|(0|lower]' outfile='user/data/mymember.pds' infile='/verylongpath/to/longfilename.ext' pattern='[path|verylong=|/=*.|.1-|upper].[base|cut-8|upper]' outfile='PATH.TO.FILENAME' infile='/path/to/*.txt' pattern='.GDG(+[ind0])' outfile='USER.GDG(+n)' whereby n starts with 1 and is incremented for each file ------------------------------------------------------------------------ The last example shows a wildcard used at read and the different targets should be written into a generation data group as several versions. The file index in minimal amount of digest '[ind0]' could be used to define the next generations '(+[ind0])'. .More information: Patterns can be used for output file and member names. This mapping mechanism is very powerful. For example, it can be used to compress each directory into a separate FLAMFILE, with each FLAMFILE containing all files of that directory. ------------------------------------------------------------------------ write.flam(file='[path]/?[name]') ------------------------------------------------------------------------ If you define an output filename pattern that results in the same path being generated more than once, the second and further occurrences will append the data to previously written files. If no pattern defined, this means that all matching files on input, will be written to one output file or archive, the append flag is automatically enabled starting with the second file. ... print CLE_DOCTYP_CHAPTER === Key Label Handling === 3.13. Key Label Handling For cryptographic operations (signing, encryption) keys are required. Beside passwords and passphrases FLAM supports professional key management solutions, based on different cryptographic infrastructures and hardware security modules. For example, on z/OS ICSF on other IBM platforms CCA and on distributed platforms PKCS#11 are supported. If a key protected by a shielded hardware module it must be normally referenced by a name called key label. These names or better the convention for these key labels can be different from client to client. To support different key label conventions FLAM supports key label templates. Key label templates using the same replacement mechanismen for environment variables () described in the file name handling above. Only the sepcial characters used as abbrevation are differ. ------------------------------------------------------------------------ '!' = - the environment qualifier (T-TEST or P-PROD) '~' = - current user id (logon name) '^' = - current owner id (defined over the configuration) ------------------------------------------------------------------------ FLAM use hard coded default key label templates for the different encryption solutions. For the OpenPGP support for example: ------------------------------------------------------------------------ '!FLAMPGP.%u.%t' ------------------------------------------------------------------------ Depending on the application several %x replacments can be supported. These qualifier (%u/%t) are replaced with the corresponding variables. In the OpenPGP example above %u are replaced with the USERID or KEYID and %t are replaced with the internal used key type definition. If you provide as USERID for example '~' or '' then the current logon name is used a user id. First the %x is replaced by the variable and then the replacement with the defined environemnt variables is done. Below you can see a few examples with an environment id 'T' for TEST and the internal types 'PUBLIC' or 'PRIVATE': ------------------------------------------------------------------------ TEMPLATE='!FLAMPGP.%u.%t' USERID='FREI' -------->'TFLAMPGP.FREI.PRIVATE TEMPLATE='!FLAMPGP.%u.%t' USERID='' #this makes not realy sense, but it is possible# -------->'TFLAMPGP./home/frei.PUBLIC TEMPLATE='!FLAMPGP.%u.%t' USERID='~' -------->'TFLAMPGP.frei.PRIVATE TEMPLATE='!FLMPGP..%u.T%t' USERID='~' -------->'TFLAMPGP.limes.frei.TPUBLIC TEMPLATE='!FLMPGP.^.%u.TYP.%t.MYKEY' USERID='~' -------->'TFLAMPGP.limes.frei.TYP.PRIVATE.MYKEY ------------------------------------------------------------------------ The internal type is defined by the application (mainly the used FLAM key management extension version 5 (FKM5)). Over the key template only the position of the internal type can be defined, the value is given by FLAM. The key template handling and limitations (how long, which characters, how many qualifiers also) are depending on the FKM5. The underlying cryptographic infrastructure defines what is possible. For example, a key label in PKCS#11 can be free defined and for ICSF/CCA it must conform to host data set names. Have a look at the FKM5BOOK for these details. Since version 5.1.16, the replacement characters are no longer case sensitive. The two templates below are equivalent ------------------------------------------------------------------------ TEMPLATE='!FLAMPGP.%u.%t' TEMPLATE='!FLAMPGP.%U.%tT' ------------------------------------------------------------------------ This change is making it easier to define such templates in JCL (CAPS ON) on z/OS. ... print CLE_DOCTYP_CHAPTER === Password Handling === 3.14. Password Handling Several components allow entering passwords, for example to encrypt files or establish SSH connections. Putting such sensitive information inside a script, a JCL or entering it directly on the command line can be serious security issue. Therefore, any string assingment within a command can be sourced out to a file which is much easier protected against prying eyes. ------------------------------------------------------------------------ password=f'pwdfile.txt' ------------------------------------------------------------------------ The prefix 'f' tells CLE/P that the string must be read from the specified filename. The file itself must contain the CLE/P string value. With SSH you can provide the password as part of the URL: ------------------------------------------------------------------------ ssh://user:password@host:port ------------------------------------------------------------------------ In this case, it may not be very useful to put the whole URL inside a file. Instead, the password can be specified through the corresponding NET object, where the password is a separate parameter. The rest of the connection information can remain in the URL: ------------------------------------------------------------------------ net.ssh(password=f'pwdfile.txt') file=ssh://@server.com/myfile.txt ------------------------------------------------------------------------ If you wish, you could also put all connections parameters inside the NET object, so that the URL form is not needed: ------------------------------------------------------------------------ net.ssh(user=max host=server.com password=f'pwdfile.txt') file=myfile.txt ------------------------------------------------------------------------ If a connection parameter is specified in the URL as well a in the SSH NET object, the values from the URL takes precedence. The special replacement capabilities (~) for file names, however, are only available for the URL. In the NET.SSH object only the simple string replacements () can be used. Sometimes it is convenient to use a parameter file for an object, overlay or command which contains the password. ------------------------------------------------------------------------ net.ssh=sshparm.txt net=netparm.txt conv=convpar.txt ------------------------------------------------------------------------ All these capabilities can be used to protect passphrases, but we recommend to use public key authentication with SSH and the use of our key management extension (FKME/FKM5) for PGP or FLAMFILEs to protect the keys by professional cryptographic infrastructure (e.g. with a HSM (hardware security module)). ... print CLE_DOCTYP_CHAPTER === Handling of Empty Records [[flcl.handling.empty.records]] === 3.15. Handling of Empty Records The handling of empty records is complicated on z/OS. FLAM5 uses the language environment (LE) to read and write records. If the environment variable '_EDC_ZERO_RECLEN' is not defined, it defaults to Y(es) for the record I/O operations. When reading, this results in an empty record being converted to a neutral FLAM5 record element. When writing, however, an empty variable length record is not written at all by the LE. To circumvent this, a padding character (default: 0x00) is written for an empty variable length record. To suppress this added padding character when reading the output file later, a record of length 1 with 0x00 or 0x40 will result in an empty record of length 0 if the environment variable '_EDC_ZERO_RECLEN' is set to 'Y'. In any other case, empty variable length records are converted to a record of length 1 containing the padding character by default on z/OS (binary files: 0x00, text files: whitespace character (0x40)). On platforms which support writing empty variable length records, when writing fixed-length records or when print control characters are used, this kind of dirty padding of empty variable length records is not done. ... print CLE_DOCTYP_CHAPTER === Handling of Spanned Records [[flcl.handling.spanned.records]] === 3.16. Handling of Spanned Records Spanned records are supported by FLAM5 components with a record length of up to 1 MiB. This length can be extended on request. The old FLAM4 components do not support such large records. To write large records to the old FLAMFILE, the matrix with the records must be convert to a block with record length fields in order to write this block into a FLAMFILE. The FLAM4 components will handle such a block as an undefined byte stream. When reading, the conversion from block to records must be requested with the same method to restore the original records. With XCNV, block/record formatting or conversion capabilities can be used. The CONV command offers the LARGER switch when reading and writing FLAMFILEs to activate the this type of conversion. When LARGER is used the block conversion is performed with the B4X method. There is currently no automatic approach implemented for it. The following example uses the XCNV command to read a VBS file and write the large records to a FLAMFILE with block formatting. ------------------------------------------------------------------------ //FLCLXCNV EXEC PGM=FLCL,REGION=0M,PARM='XCNV=DD:PARM' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * INPUT( SAVE.FILE( FIO.REC( NAME='.VBS.LR' ) ) ) OUTPUT( SAVE.FILE( FMT.BLK( METHOD=B4X ) FIO.FL4( NAME='.VBS.LR.ADC' ) ) ) /* ------------------------------------------------------------------------ To read the created FLAMFILE back to records and write it as VBS, the job below can be used, using record conversion. ------------------------------------------------------------------------ //FLCLXCNV EXEC PGM=FLCL,REGION=0M,PARM='XCNV=DD:PARM' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * INPUT( SAVE.FILE( FIO.FL4( NAME='.VBS.LR.ADC' ) CNV.REC( METHOD=B4X RECLEN=200KiB INICNT=4 ) ) ) INPUT( SAVE.FILE( FIO.REC( NAME='.NEWVBS' FALLOC( RECFORMAT=VBS RENEW ) ) ) ) /* ------------------------------------------------------------------------ The READ/WRITE.FLAM4() objects with the LARGER switch actived will do the same, but without the possibility to change the format of the length field with the METHOD parameter. The FLAM command of FLCL and the new FLAM4 utility will also support the LARGER switch when reading and writing FLAMFILEs. By default, the record length for VS/VBS is 1 MiB. To save memory, this can be reduced with the RECLEN parameter. The FLAM5 I/O will read large V(B)S records by default. This can be deactivated for record I/O by using the switch NOSPAN. This switch is active for the FLAM command of FLCL, or the new FLAM4 utility, if the LARGER switch is not enabled to ensure backwards compatibility. NOTE: Automatic record length detection will not work with large records (>=32768). This requires to set the record formatting method when reading such a FLAMFILE. ... print CLE_DOCTYP_CHAPTER === Use of IBM Unicode Services [[flcl.use.of.unicode.serives]] === 3.17. Use of IBM Unicode Services Since version 5.1.27, FLAM supports the use of IBM Unicode Services on system Z. The Unicode Services are used only if no user table, transliteration, case mapping, subsets, whitespace handling, reporting, normalization or other complex conversion is requested. If a simple conversion is done, in a few cases the implementation of FLAM is faster and FLAM will still use this faster implementation. In all other cases, the IBM Unicode Services will be used by default to reduce CPU and memory utilization. It is important to note the differences in behavior of the IBM Unicode Services: * If an incomplete character is encountered and the error handling is not STOP, the FLAM character conversion module ignores or substitutes this incomplete character. The IBM Unicode Services does return an EINVAL error in this case. * IBM Unicode Services can produce errors for technically valid code points when processing UTF-8 encoded data. The FLAM character conversion module first decodes UTF-8 and verifies the code point afterwards, independent of the encoding. This means you can encode code points in multiple forms (e.g. the character 0 as 0x30 or 0xFC80808080B0). The IBM Unicode Services support only the shortest form (0x30) of an encoding for a code point and all other possible encodings are not accepted and will result in an error (invalid character). * Depending on the Unicode Services version (APAR) a BOM character in the data might remain for UTF character sets or might be replaced by a substitution character for single byte character sets. When calling the Unicode Service FLAM will request the BOM removal, and FLAM will remove a BOM character if it is causing an invalid character error, but both will not work with certain versions of Unicode Services. * No error is caused by a wrong BOM (invalid character) in the middle of the data and a BOM change from LE to BE or vice versa is also not detected. For data with mixed endianness the Unicode Services should be deactivated. To prevent these changes in behavior you must set the system symbol `&FLIBMUS` or the environment variable `FL_IBM_UNICODE_SERVICE` to `OFF`, but this will result in more CPU and memory utilization for simple character conversions (see chapter <>). The environment variable's default is `AUTO` which ensures that only the minimal behavior changes above are in place. To use the Unicode Services also for character conversion mode `STOP` the environment variable has to be set to `ON`. This will cause FLAM to use the round trip conversion technique of the Unicode Services, which results in the behavior changes below: * A lot of non-convertible characters will be converted to a unique character in the target character set without an error using the round trip conversion scheme. * A substitution character is only converted with activated substitution. Without substitution, the substitution character 0x1A in ASCII/UTF or 0x3F in EBCDIC will cause an error. Based on that behavior FLAM will request the substitution if Unicode Services are used. We recommend to set FL_IBM_UNICODE_SERVICE=ON to maximize performance if the limitations described above are acceptable because checking each character if it is valid or not has a significant performance impact. ... print CLE_DOCTYP_CHAPTER === Use of Built-in Functions === 3.18. Use of Built-in Functions The generated description of the built-in functions are independent of the program, owner and command names. This may make it difficult to use them. Below, you can find some examples for the use of built-in functions of FLAM4. ------------------------------------------------------------------------ To get the syntax for command CONV: :> flam4 syntax conv To get the syntax for the overlay read of command CONV: :> flam4 syntax conv.read To get the syntax for the object text of overlay read of command CONV: :> flam4 syntax conv.read.text To get the complete syntax (not only one level) of overlay write of command CONV: :> flam4 syntax conv.write all To get the help for command INFO: :> flam4 help info To get the complete help for the overlay GET of command INFO: :> flam4 help info.get all To get the manual page for the overlay GET of command INFO: :> flam4 help info.get man To generate the manpage for command CONV: :> flam4 manpage conv=manconv.txt To generate the manual for command CONV: :> flam4 gendocu conv=docconv.txt To generate the complete user manual for FLCL: :> flam4 gendocu=docflcl.txt To change the properties for command FLAM: :> flam4 chgprop flam mode=VR8 inrecformant=undefined cut=ON To generate and activate a property file for command CONV: :> flam4 genprop conv=conv.prop :> flam4 setprop conv=conv.prop To list all defined properties: :> flam4 getprop To list all properties for command INFO: :> flam4 getprop info depall To list only the defined properties for command INFO: :> flam4 getprop info defall Delete a property file for command CONV from the configuration: :> flam4 delprop conv To define "HUGO" as owner: :> flam4 setowner HUGO To get the current owner: :> flam4 getowner To define a environment variables: :> flam4 setenv LANG=DE_DE.CP1252 To list all defined environment variables: :> flam4 getenv To delete a environment variable: :> flam4 delenv LANG To activate tracing in a file: :> flam4 trace file=trace.txt :> flam4 trace on To deactivate the trace: :> flam4 trace off To list all the defined data in the configuration file: :> flam4 config To delete all the configuration settings from the configuration file: :> flam4 config clear ------------------------------------------------------------------------ All built-in functions not listed here do not require any parameters. On mainframe systems, using built-in functions in JCL looks like: ------------------------------------------------------------------------ //* SET ENVIROMENT VARIABLE LANG (IMPORTANT FOR AUTO CONVERSIONS) //SETENVL EXEC PGM=FLCL,PARM='setenv LANG=DE_DE.IBM-1141' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //* ------------------------------------------------------------------------ For documentation, help and syntax you can also use the ISPF commands FLDOC, FLHLP and FLSYN under menu 6. ------------------------------------------------------------------------ FLDOC CONV FLHLP XCNV.INP FLSYN UTIL ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER === Security Considerations === 3.19. Security Considerations Safety is an important issue for FLAM. FLAM can help to fulfill many security requirements (e.g. PCIDSS, ISO27001, ...), using encryption (OpenPGP, SSH), hardware security modules (PKCS#11 and IBM CCA/ICSF for key exchange and signing), strong access control and checksums, antivirus scanning and much more, nevertheless the following points must be observed for the secure operation of FLAM. //// TODO: .SOC support With a maintenance contract you can register in the customer portal for a newsletter about security vulnerabilities. This newsletter will be sent if a security vulnerability for FLAM comes up. This is useful in supporting your secure operating center (SOC). //// .Related software In the `install.txt` for your platform under prerequisites you will find the software on which FLAM is based or depends. If nothing is listed, then only the operating system with the language environment and the corresponding libc is affected. In the `about.txt` you can see all the external libraries and their versions that FLAM integrates. Both pieces of information can be used to determine whether the use of FLAM might be affected by an emerging security vulnerability. .Check coherence With the call `flcl version` a version of each software component is written. The installation of FLAM will be coherent if all build numbers are the same. The build number is the last number of the version behind the minus character. Especially for dynamic linked parts of the software it will be important to check if the installation was well done. We recommend establishing an installation verification process which runs the built-in function `VERSION` and verifies that the build numbers are all the same. .Code signing limes support code signing for modules and installation packages if it is possible on a platform. If not limes provides secure checksums to allow verification of the packages after download. .Environment There are a lot of environment variables or system symbols which are security related. Below you can find some important examples (please note: this list is incomplete): * FL_PGP_PREF_SYMALG_AES128 - Invert order of preferred symmetric algorithms from AES256/192/128 to AES128/192/256 in PGP certificates * FL_ALLOW_LOW_ENTROPY - If defined and no source of entropy is available, the random number generated is seeded with a low entropy which may result in weak encryption keys * FL_DEACTIVATE_HW_CRYPTO_ACCELERATION - Deactivate hardware acceleration for cryptographic operations (enforce use of software implementation) * FL_SSH_HOME_DIR - Directory containing SSH key files and known_hosts file (/home/user/.ssh) * FL_KEY_EXPIRE_WARN_DAYS - Amount of days to warn before a key will expire (default is 90 (3 months)) See <> and verify these settings. To get a list of the current environment please use the command below: ----------------------------------------------------------------------- flcl info get.system ----------------------------------------------------------------------- .SAF support For operating systems implementing a secure access facility (e.G. z/OS) FLAM supports a lot of policies and additional resources for better control about the usage of FLAM. For more information about this see <> and the `install.txt`. Unfortunately, there is no such additional protection of application-specific resources for other operating systems. With the SAF support it is possible to ensure minimal key, password or hash length. Critical operations like input logging, packet capturing, antivirus scanning etc. can be deactivated and more. There are a lot of policies and resources defined for the supported cryptographic features (PGP, EDC, SSH), which will not be explained below in more details. .Credentials Credentials (Passwords, PINs, Keys, ...) entered using the CLP strings are flagged as protected and will not be printed in clear form. Such values are replaced by `****** SECRET ******` unless the input logging is activated. In this case the complete command string is written to the log and might contain critical values. After parsing the command line FLAM knows the critical values and is replacing them. Additionally it is better to provide such sensitive input within parameter files. The file can be better protected. But the best solution in such cases is to use a hardware security module (HSMs). .Support of hardware security modules (HSMs) FLAM supports hardware security modules (HSMs) over a service provider interface for session key procedures and signing (PGP, FLAMFILE, ...). Currently, the use of PKCS#11 and IBM CCA (including ICSF) are supported. We recommend using the HSM support instead of clear passwords for key stores or such critical things. The misuse of the service provider interface can be prevented completely with SAF support. .Verification of cryptographic algorithms All software cryptographic operations are done using a central crypto kernel (CryCore). This singleton determines the optimal implementation depending on the computer used. It supports hardware acceleration if possible. But before the concrete algorithm is used the first time a self test against the NIST test vectors is done, to ensure that the loaded functions do implement the algorithm correctly. For IBM mainframes mainly the CPACF functions are used to implement the cryptographic algorithms. For all other platforms the library libcrypto of the OpenSSL project is used. .Secure randomness To generate keys (mainly for SSH) it is important to have a high entropy. By default, FLAM ensure a high entropy which could result in an error, if only weak randomness is available. The environment variable `FL_ALLOW_LOW_ENTROPY` allows low entropy to prevent such errors and the policy `LOW.ENTROPY` can be used to prevent this. In some cases the quality of randomness is not so important, in such cases it could be useful to allow low entropy and in another context it will be dangerous, and then it will be better to prevent this. .Secure delete of sensitive data FLAM supports secure delete of sensitive data (keys, passwords, pins). This means that erasing the memory is done as early as possible and the overwriting of the memory will not be eliminated by the optimizer for the release code. .Antivirus scanning FLAM supports antivirus scanning over a service provider interface. It will be required by certain security standards to scan the clear content of a file before it is encrypted and stored on a remote system. In such a case the antivirus scanner can be used. Our standard implementation uses clamAV, but the interface can be implemented against each kind of antivirus scanner. Simultanously, this interface is a backdoor to intercept clear data and should be closed for all other cases. FLAM implements a complete set of SAF control mechanism which can be used to ensure this. .Input logging The input logging will write the complete input parameter string to the log (this is not done by default and only important for debugging in a few cases). In this case the content is not known and might contain critical data or credentials. The SAF policy `LOG.INPUT` can be used to prevent the activation of input parameter logging. .Packet capturing Packet capturing is useful to find errors for remote connections (e.g. SSH) but packets might contain critical or sensitive data. With the SAF policy `SSH.PCAP.ALLOWED` you can control the usage of the PCAP feature and prevent this kind of backdoor for normal processing. .Command pre- and post-processing The command pre- and post-processing is a powerful feature of FLAM and can be used in once or per file, locally or remote at read or write. The feature was implemented mainly for process automation but to run commands (often executable code) on remote systems could be dangerous. To prevent misuse FLAM has implemented a set of SAF controls. .FLAM started task (FLAMSTC) The FLAM started task (FLMSTC) on z/OS is an active component running in the background providing services to implement the subsystems, including the LE-less interface for the FLUC record interface. The FLAMSTC provides the scheduling of SRBs in another enclave for the zIIP support and the possibility to run authorized functions. This is realized with three different PC routines. The FLAMSTC does not implement own supervisor calls (SVCs). Functions of the FLAM and FLUC subsystems are called as part I/O SVCs. The QA process is using zACS to penetrate the PC routine and the corresponding recovery routines. The only findings of the zACS scan are regular program terminations of the FLAMSTC with 0C4. //// TODO: .FLUC file system (flucFS) sollte nur under eigenen user laufen //// .Error trace The error trace is an important feature of FLAM which help a lot to analyze errors. For a good cause determination the data involved in the error is often written to the error trace. This means that the error trace could include parts of sensitive data items. In a case of an error this trace might need protection. It is similar to a dump processing in an error situation. Such a dump could contain critical data and must be protected. .Quality assurance limes is using vulnerability scanners and static code analysis for conformance with certain Coding Standards to prevent critical mistakes in our programs. Additional CPPCHECK and ScanBuild are used for static code analyzes. For each platform limes uses the standard memory checker (e.g. valgrind on linux) for dynamic code analysis running over all test cases. But there are a lot of platforms without a tool for runtime analyzes. Therefore, limes has implemented an own resource checker (memory, file handles, stack, ...) in the debug code to detect stack and memory overflows, missing frees(leaks) or unclosed or misused files. Limes is using Coco(r) to determine the code coverage of FLAM and ensure that each part of the code is tested. Squish(r) is used to automate testing of our GUIs. For automated testing of each ISPF panel and limes command we use the API of the x3270 terminal program. For all testing limes is using an own test framework which runs all the unit and regression tests on all platforms. The cross-reference tests nsures that data produced by standard tools (e.g. gzip, openssl enc, gpg) is understood on each platform and produced output of FLAM from each platform can be read with the standard tools. We use continues integration, this means that all the build tests are running after each build, a rebuild must be successful before the code is taken over from the developer repository to the main repository and all the long-running tests (performance, large files, ...) will be started automatically each night and anything will be checked each morning. This allows us to deliver rolling releases, where a customer can request a new parameter on Wednesday, and we can deploy a new revision or release on Monday. For this each weekend is currently used to clean anything, rebuild anything, tests anything, package anything, install the new package to a clean system, test anything again. And if anything was successful then we can publish the new build for our users. Sometimes it takes long to get all tests to run with success, and we cannot release a new version each week. All regression tests must be successful before we publish a new version of FLAM. The publishing include an encrypted code escrow for the complete source used to build this release. All build results, all test results anything will be stored in an archive for this publication. So we can review the complete state if a customer has a problem with a certain build including all tests results, all compile and link lists and so on. We can install from this repository easily each published build, to set up the customer environment to reproduce an error or a behavior. .Methodology limes is using a mixture of SCRUM and team concert with strict coding guidelines where requirement, design, development, build, test, packaging, installation, testing again and the deployment runs in parallel. Version control is done with GIT and anything is integrated in an eclipse workspace under Arch Linux for our developer. The software development process ensures that all described QA measures are adhered to. .Documentation The documentation is generated from the source code and based on that it must be totally in sync with the application and components used. The quality of the description for a parameter or the manual page for an object or overlay could be bad, but any implemented feature is documented and vice versa. We use the FLAMCLEP command string parser and anything what this compiler is accepting is defined in tables and the help message and the manual page must be provided otherwise a table error occurs. The same method is used for interface specifications (API), where each function and each parameter for each function must be documented in form of comments in the code otherwise the docu generation will fail. This process ensures that the documentation is always up-to-date and in sync with the implementation and the user can trust that anything is documented. ... print CLE_DOCTYP_CHAPTER === Performance Considerations === 3.20. Performance Considerations Performance (CPU/memory utilization and throughput) is an important topic when using FLAM. The following points describe some major factors that contribute to the overall performance of FLAM. They should give you a good starting point if you experience performance issues. .Pre-load architecture In order to minimize consumed CPU cycles during mass data processing, FLAM pre-calculates as much as possible before processing any data, which requires memory on the one hand and spends some CPU time on the other. If you process many small files with FLAM, this can have a negative effect on performance. FLAM is not designed and developed to process small amounts of data. .Hardware accelerator support FLAM uses special instructions of the respective hardware platforms to be able to accelerate some algorithms. On IBM's mainframes, for example, this concerns the use of CPACF and zEDC. .Outsourcing of computing power to zIIP For IBM mainframe systems, it is possible to outsource memory-to-memory functions (compression, conversion, formatting) as SRBs to so-called zIIP processors. Although this does not result in a performance advantage, it does save chargeable CPU time (MSU). .Record count and block size FLAM processes data in self-sufficient segments, which represent a block or a certain number of records. The amount of data that is processed in one go determines the throughput performance. It is important to understand that there is an overhead per segment, so the segments should not be too small. However, if too much data is put into a segment, then the processing constantly takes place outside the caches, which also reduces the performance. Here, the block size or the record count must be optimally selected for the respective platform. For historical reasons, FLAM works with a default block size of 64 KiB and a record count of 512. This is a good average value across all platforms, but not always optimal. It should also be noted that such a segment is expanded after decompression, i.e. in this case one should always start with a smaller block size. On many systems, 256 KiB is now a good block size. The optimal record count is the optimal block size divided by the average record length. .Character set conversion Character set conversion can take up a lot of CPU time. By default, FLAM checks each character for validity and stops processing with an error if invalid or unconvertible characters are found (MODE=STOP). This form of error handling is relatively expensive in terms of CPU utilization, but ensures detection of incorrectly encoded text data. Enabling error handling that leads to the replacement of these incorrect characters, processing can be accelerated. The character set conversion is faster if all characters of the input alphabet result in data of the same length in the output alphabet. With the environment variable `FL_DEFAULT_CHAR_MODE` (or the system variable `&FLCHARM` on z/OS), the default mode for error handling during character set conversion can be changed from `STOP` to `SUBSTITUTE`, for example, in order to achieve higher performance here. On z/OS, FLAM uses the IBM Unicode Services as long as it supports the required character set conversion and is known to be faster than FLAM's implementation. This only works for simple conversions without mappings, subsets, reporting or transliterations. The environment variable `FL_IBM_UNICODE_SERVICE` (or the system variable `&FLIBMUS`) can be used to enable/disable the use of IBM Unicode Services. By setting the variable to ON instead of the default AUTO, the IBM Unicode Services will also be used if STOP is selected as the character conversion mode, which saves a lot of CPU cycles, but at the cost of no longer noticing some errors in the character data. .Logging FLAM writes many log messages by default, which also takes up CPU time and I/O. The `QUIET` or `SILENT` keywords and the `MESSAGE` object can be used to control the amount of output and thus limit or increase the amount of work involved. Often, during the transition from test to production, people forget to reduce FLAM's chattiness, which cost both disk space and CPU cycles in production. .Literal cache FLAM uses a global literal cache to save memory and CPU when comparing strings. Here a hashtable is used. Its size can be defined via the environment variable `FL_LITERAL_CACHE_SIZE` (or system variable `&FLITCS`). In the debug log, the appropriate statistics are output, which may help you to make adjustments if necessary. .ZIP archives and PS-VB on Mainframes When creating ZIP archives, offsets and statistics values must eventually be written into the member headers, which requires repositioning (fseek()) within the file. This is possible for host datasets on z/OS via so-called RBA access. In a PS-VB file, this requires a tree in the background where the OS remembers which relative byte address (RBA) is in which block, in which records at which offset in the record. This is extremely time-consuming and if there are many members in the ZIP archive, this can increase the runtime considerably. The maintenance of this tree also costs a lot of CPU, which is not charged to the application. To avoid this, you should use file types for ZIP archives that are suitable for RBA access. Under z/OS, these are primarily never files in the USS. Alternatively, FLAM also supports streamed ZIP archives, as they were developed for piping under Unix systems. With the keyword `STREAM`, you can force a purely sequential write, which then also works with a PS-VB, but you should check whether ZIP tool used for extraction understands this streamed format. With PS-FB datasets, the RBA access is faster because you do not need a tree, but there is padding at the end of the ZIP file. Since ZIP archives have to be read from behind, many other tools cannot cope with this padding because they expect the ZIP directory and not 0 bytes or blanks. With FLAM this is not a problem, but as already mentioned, many other tools then recognize the ZIP file as incomplete or not even as a ZIP file. .z/OS CEEOPTs (from install.txt) For memory allocation, don't use the realloc control feature of CEE. ------------------------------------------------------------------------ _CEE_REALLOC_CONTROL=bound,percentage ------------------------------------------------------------------------ FLAM is only tested with default settings. With some memory allocation strategies out of memory errors can occur. For good performance and enough memory we recommend the IBM default runtime options except for run CEEOPTS below: ------------------------------------------------------------------------ //CEEOPTS DD * ALL31(ON) STACK(1M,1M,ANYWHERE,KEEP,512K,128K) HEAP(1M,1M,ANYWHERE,KEEP,8K,4K) /* ------------------------------------------------------------------------ With `ALL31(OFF)` and `HEAP(,,BELOW,,)` the available memory is too small for the stack in some cases, especially when using the FLUC subsystem, table or large-record support. Since version 5.1.25, FLAM uses application-specific run-time option defaults with the `CEEUOPT` assembly language source program to set these application-specific option defaults. The `CEEUOPT3` member used for 31 bit and the `CEEUOPT6` member used for 64 bits are located in hlq.FLAM.SRCLIB. We recommend building and linking the examples if you are going to write applications that use our APIs (e.g. FLCREC). For batch processing we recommend to create a storage report (RPTSTG(ON)) and define the CEEOPTS like recommended in the report. ... print CLE_DOCTYP_CHAPTER === JCL Considerations === 3.21. JCL Considerations If you use JCL to run FLCL batch programs, it is recommended to use the parameter file feature for inlining commands. Parameter files can also be used for each object and overlay (see example below): .Example ------------------------------------------------------------------------ //FLCLXCNV EXEC PGM=FLCL,REGION=0M,PARM='XCNV=DD:PARM' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //PARM DD * INPUT( SAV.FILE ( FIO.REC ( NAME=s'~input.dataset' ) ) ) OUTPUT( SAV.FILE ( FMT.BLK ( METHOD=L4X #conform to PKWARE# ) CNV.ZIP ( ) FIO.ZIP=DD:ZIPARCH ) ) /* //ZIPARCH DD * ( NAME=s'~.output.ziparch' APPEND #add member to archive# ) /* //STDENV DD * LANG=de_DE.IBM1141 /* ------------------------------------------------------------------------ The parameter files (DD:PARM, DD:ZIPARCH) are read by the command line parser and executer (CLE/P). This works only in the expected manner if NUMBER=OFF is specified for this job in the ISPF editor. With NUMBER=ON, the numbering is part of the inline code and will result in a syntax error from the CLE/P. The same is valid for the DD name STDENV. In a parameter file, you can add a semicolon at the end of each line to handle the line number as comment. Since version 5.1.20-23372: On command level if no parameter given and no parameter file assigned, then the DD name 'FLAMPAR' is read. See the example below: ------------------------------------------------------------------------ //FLCLINFO EXEC PGM=FLCL,REGION=0M,PARM='INFO' //STEPLIB DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR //SYSOUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //FLAMPAR DD * GET.SYSTEM /* ------------------------------------------------------------------------ To execute our commands we recommend the stack and heap definitions for the memory management of the language environment below (the IBM defaults are in comment above). If for example the increments to low (128 bytes), then the run and CPU time could increase significantly. ------------------------------------------------------------------------ //CEEOPTS DD * *RPTSTG(ON) ALL31(ON) *STACK(128K,128K,ANYWHERE,KEEP,512K,128K) LE default STACK(1M,1M,ANYWHERE,KEEP,512K,128K) *HEAP(32K,32K,ANYWHERE,KEEP,8K,4K) LE default HEAP(1M,1M,ANYWHERE,KEEP,8K,4K) /* ------------------------------------------------------------------------ Please use the storage report (above as comment) to determine the best values for your usage. With does a lot of work depending on the parameter given. It could possible depending on what is done with FLAM that better values increase performance and lower CPU utilization. With ALL31(OFF) and HEAP(,,BELOW,,) the available memory is too small for the stack in some cases, especially when using the FLUC subsystem, table or large-record support. Since version 5.1.25, FLAM uses application-specific run-time option defaults with the CEEUOPT assembly language source program to set these application-specific option defaults. The CEEUOPT3 member used for 31 bits and the CEEUOPT6 member used for 64 bits are located in hlq.FLAM.SRCLIB. We recommend to build and link the examples if you are going to write application that uses our APIs (e.g. FLCREC). To interpret commands correctly, some of the punctuation characters are on different code points depending on the EBCDIC CCSID used to enter these values. The environment variable LANG must be set to the correct CCSID to interpret tilde and other characters correctly. see <> of <> Since z/OS 2.1, a new service (CEEGTJS) is available to determine the values for exported JCL symbols in the application. The service is called dynamically and the language environment must be in the steplib concatenation to use it. If the service is not available, the exported JCL symbols cannot be used. If the environment variable replacement fails the exported JCL symbols are tried. This feature allows you to use exported JCL symbols in the control statements of FLCL like environment variables. This is mainly useful to build powerful JCL procedures. ------------------------------------------------------------------------ //E0 EXPORT SYMLIST=(CHAROUT,HASHALGO,HASHLEN) //S1 SET CHAROUT=IBM1141 //S2 SET HASHALGO=SHA512 //S3 SET HASHLEN=128 //CONV EXEC PGM=FLCL,PARM='CONV=DD:PARM' //STEPLIB DD DSN=FLAM.LOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSOUT DD SYSOUT=* //PARM DD *,SYMBOLS=JCLONLY READ.RECORD(FILE=DD:INPUT) WRITE.TEXT(FILE=DD:GZIP COMP.GZIP() HASH(OUTPUT=DD:HASH FORMAT=HEX CUT=/2 # important out-bytes=HASHLEN*2 if hex # CCSID= ALGO=&HASHALGO. ) ) //INPUT ... //GZIP ... //HASH ...DCB=(RECFM=FB,LRECL=&HASHLEN.)... ------------------------------------------------------------------------ The environment variables have a higher priority. If there is an environment variable with the same name as an exported JCL symbol, the value of the environment variable is used. System variables available as JCL symbols (if SYSSYM=ALLOW is specified in the class definition) can be exported and used in the control statements. Since version 5.1.17 of FLAM, the system symbols are generally available for replacment (e.g '...<&LDAY>...') if no environment variable with the same name exists. This means that the system symbols have the lowest priority in the replacement process. To export all JCL symbols for an application, you can use 'SYMLIST=*'. The "&name." replacment of JESS (DD *,SYMBOLS=JCLxxxx) can also be utilized if inline coding is used. For an external parameter file, however, the JCL symbols are only available with "". ... print CLE_DOCTYP_CHAPTER === SAF Consideration [[flcl.saf.considerations]] === 3.22. SAF Consideration Beginning with version 5.1.21 a lot of additional checks against a System authorization facility (SAF) can be activated. These SAF support gives for example an RACF administrator on z/OS the possibility to activate different security policies and control a lot of resources used by FLAM. * Several security policies (disable PCAP, ensure finger prints, minimal key length, ...) * CLE/P parameter strings (complete control about the functionality) * Control access to pre- and post-processing commands (executed on remote or local systems) * To secure all service provider interfaces (FAVE, FCPE, FKME) ** Control to which AV daemon the clear data is transfered * Control access to complete URLs containing connection, path, filename and member ** Allow to protect members of PDS/E, ZIP or FLAM archives * Secure SSH connections (Control user, server and port) * PGP key management (User ID to import, export, generate, delete) * Control access to PGP user and key IDs for signing and key exchange * Control methods, algorithms, modes and a lot more of the en/decryption component * Activate and enforce additional SMF logging (e.g. record type 119 (SSH)) * All feature codes of the FLAM license module The complete SAF setup is documented in the corresponding install.txt and can be simply extended on user request. For z/OS with RACF we recommend at minimum the definitions below (be careful with this example, we are no RACF specialists): ------------------------------------------------------------------------ //S01 EXEC PGM=IKJEFT01,DYNAMNBR=75,TIME=100,REGION=6M //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTERM DD DUMMY //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.BRODCAST,DISP=SHR //SYSTSIN DD * RDEFINE CDT + $FLAM + CDTINFO(CASE(ASIS) /* mixed case profile names */ + DEFAULTRC(4) /* default */ + DEFAULTUACC(NONE) /* default */ + FIRST(ALPHA) /* profile name first char */ + GENERIC(ALLOWED) /* default */ + KEYQUALIFIERS(0) /* default */ + MAXLENGTH(246) /* maximum prof name length */ + OPERATIONS(NO) /* default */ + OTHER(ALPHA, /* prof name allows A-Z,a-z */ + NUMERIC, /* 0-9 */ + NATIONAL, /* #,@,$ */ + SPECIAL) /* all except blank ( ) , ; */ + POSIT(202) /* in user range 128-527 */ + RACLIST(ALLOWED) /* setr raclist allowed */ + SECLABELSREQUIRED(NO) /* default */ + SIGNAL(NO) /* default (no ENF signals) */ + ) SETROPTS CLASSACT(CDT) SETROPTS RACLIST(CDT) SETROPTS RACLIST(CDT) REFRESH //S02 EXEC PGM=IKJEFT01,DYNAMNBR=75,TIME=100,REGION=6M //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTERM DD DUMMY //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.BRODCAST,DISP=SHR //SYSTSIN DD * PERMIT $FLAM CLASS(CDT) RESET /* clear permit list */ RALTER CDT $FLAM OWNER(SYS1) /* change ownership */ SETROPTS RACLIST(CDT) REFRESH //S03 EXEC PGM=IKJEFT01,DYNAMNBR=75,TIME=100,REGION=6M //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTERM DD DUMMY //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.BRODCAST,DISP=SHR //SYSTSIN DD * SETROPTS GENERIC($FLAM) RDEFINE $FLAM + ** /* cover all entities */ + OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.MUSTDEF OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.POLICIES OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.SMF.LOGGING OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.CLP.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.CMD.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.AVE.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.CPE.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.KME.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.EDC.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.FIO.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.FL4.CONTROL OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.SSH.CONNECT OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.PGP.KEYMNGM OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.PGP.USERID OWNER(SYS1) UACC(NONE) RDEFINE $FLAM GLOBAL.PGP.KEYID OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.LOW.ENTROPY OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.LOG.INPUT OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SSH.AUTH.*.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SSH.PCAP.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SSH.WEAK.HOSTKEY.CHECK.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SSH.CUSTOM.HOSTKEY.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SSH.CUSTOM.KNOWN.HOST.FILE.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SSH.CUSTOM.CONFIG.FILE.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.KEY.GENERATE.PGP.EXPIRD.REQIRED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.KEY.GENERATE.PGP.ALGO.RSA.KEYLEN OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.KEY.GENERATE.PGP.ALGO.DSA.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.KEY.IMPORT.PGP.KIDVER OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.KEY.IMPORT.PGP.DOUBLE.KIDS OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.PGP.PWD OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.PGP.PWD.LEN.ENC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.PGP.PWD.LEN.DEC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.PGP.ENFORCE.ENCRYPTION OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.PGP.ENFORCE.INTEGRITY OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.PGP.ENFORCE.SIGNING OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.EDC.PARM.FILE.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.EDC.KEY.PRINT.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.EDC.KEY.INVERSE.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.EDC.NO.HEADER.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.EDC.NO.SALT.ALLOWED OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SYM.KEY.LEN.ENC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SYM.KEY.LEN.DEC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SYM.PWD.LEN.ENC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM POLICY.SYM.PWD.LEN.DEC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM FAVELIB.libfave OWNER(SYS1) UACC(NONE) RDEFINE $FLAM FAVEFUC.CLAMAVC OWNER(SYS1) UACC(NONE) RDEFINE $FLAM FAVEPAR.clamd.limes.de:3310 OWNER(SYS1) UACC(NONE) RDEFINE $FLAM FAVEIPH.clamd.limes.de OWNER(SYS1) UACC(NONE) RDEFINE $FLAM FAVEIPS.3310 OWNER(SYS1) UACC(NONE) PERMIT ** CLASS($FLAM) RESET /* clear permit list (optional)*/ PERMIT GLOBAL.MUSTDEF CLASS($FLAM) ID(*) ACC(CONTROL) /* PERMIT GLOBAL.POLICIES CLASS($FLAM) ID(*) ACC(NONE) ACTIVE */ PERMIT GLOBAL.SMF.LOGGING CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.CLP.CONTROL CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.CMD.CONTROL CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ /* PERMIT GLOBAL.AVE.CONTROL CLASS($FLAM) ID(*) ACC(NONE) ACTIVE */ PERMIT GLOBAL.CPE.CONTROL CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.KME.CONTROL CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.EDC.CONTROL CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.FIO.CONTROL CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ /* PERMIT GLOBAL.FL4.CONTROL CLASS($FLAM) ID(*) ACC(ALTER) ACTIVE */ PERMIT GLOBAL.SSH.CONNECT CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.PGP.KEYMNGM CLASS($FLAM) ID(*) ACC(ALTER)/*INACTIV*/ PERMIT GLOBAL.PGP.USERID CLASS($FLAM) ID(*) ACC(ALTER) /*INACTIV*/ PERMIT GLOBAL.PGP.KEYID CLASS($FLAM) ID(*) ACC(ALTER) /*INACTIV*/ PERMIT POLICY.SSH.AUTH.PUBLICKEY CLASS($FLAM) ID(*) ACC(READ) PERMIT POLICY.SSH.AUTH.PASSWORD CLASS($FLAM) ID(*) ACC(UPDATE) /* PERMIT POLICY.SSH.PCAP.ALLOWED CLASS($FLAM) ID(*) ACC(NONE)*/ PERMIT POLICY.SSH.WEAK.HOSTKEY.CHECK.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.SSH.CUSTOM.HOSTKEY.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.SSH.CUSTOM.KNOWN.HOST.FILE.ALLOWED CLASS($FLAM) ID(*) ACC(CONTRL) PERMIT POLICY.SSH.CUSTOM.CONFIG.FILE.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.KEY.GENERATE.PGP.EXPIRD.REQIRED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.KEY.GENERATE.PGP.ALGO.RSA.KEYLEN CLASS($FLAM) ID(*) ACC(UPDATE) /* PERMIT POLICY.KEY.GENERATE.PGP.ALGO.DSA.ALLOWED CLASS($FLAM) ID(*) ACC(NONE)*/ PERMIT POLICY.KEY.IMPORT.PGP.KIDVER CLASS($FLAM) ID(*) ACC(UPDATE) /* PERMIT POLICY.KEY.IMPORT.PGP.DOUBLE.KIDS CLASS($FLAM) ID(*) ACC(READ)*/ PERMIT POLICY.PGP.PWD CLASS($FLAM) ID(*) ACC(ALTER) PERMIT POLICY.PGP.PWD.LEN.ENC CLASS($FLAM) ID(*) ACC(CONTRL) PERMIT POLICY.PGP.PWD.LEN.DEC CLASS($FLAM) ID(*) ACC(ALTER) /* PERMIT POLICY.PGP.ENFORCE.ENCRYPTION CLASS($FLAM) ID(*) ACC(NONE)*/ /* PERMIT POLICY.PGP.ENFORCE.INTEGRITY CLASS($FLAM) ID(*) ACC(NONE)*/ PERMIT POLICY.PGP.ENFORCE.SIGNING CLASS($FLAM) ID(*) ACC(ALTER) PERMIT POLICY.EDC.PARM.FILE.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.EDC.KEY.PRINT.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) /* PERMIT POLICY.EDC.KEY.INVERSE.ALLOWED CLASS($FLAM) ID(*) ACC(NONE)*/ PERMIT POLICY.EDC.NO.HEADER.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.EDC.NO.SALT.ALLOWED CLASS($FLAM) ID(*) ACC(READ) PERMIT POLICY.EDC.STATIC.KEY.ALLOWED CLASS($FLAM) ID(*) ACC(ALTER) PERMIT POLICY.EDC.STATIC.IV.ALLOWED CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.SYM.KEY.LEN.ENC CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.SYM.KEY.LEN.DEC CLASS($FLAM) ID(*) ACC(UPDATE) PERMIT POLICY.SYM.PWD.LEN.ENC CLASS($FLAM) ID(*) ACC(CONTRL) PERMIT POLICY.SYM.PWD.LEN.DEC CLASS($FLAM) ID(*) ACC(ALTER) PERMIT FAVELIB.libfave CLASS($FLAM) ID(*) ACC(READ) PERMIT FAVEFUC.CLAMAVC CLASS($FLAM) ID(*) ACC(READ) PERMIT FAVEPAR.clamd.limes.de:3310 CLASS($FLAM) ID(*) ACC(READ) PERMIT FAVEIPH.clamd.limes.de CLASS($FLAM) ID(*) ACC(READ) PERMIT FAVEIPS.3310 CLASS($FLAM) ID(*) ACC(READ) SETROPTS CLASSACT($FLAM) SETROPTS RACLIST($FLAM) SETROPTS RACLIST($FLAM) REFRESH ------------------------------------------------------------------------ First (in step 1 and 2) the general resource class $FLAM must be setup. The last step defines the rights for the certain entities. The recommendation defines all the policies and enable anti virus scanning only with the defined ClamAV daemon (DSN must be adjusted). The entities for command execution or parameter string can include whitespace or one of the other 4 invalid characters '(),;'. If such an invalid character is encountered in an entity string, it is replaced by an underscore '_' before the SAF check. For example the PDS(E) 'HLQ.PO(HUGO)' results in the resource name 'FIOURL.HLQ.PO_HUGO_'. The global pre-write pre-processing command 'ls -l' results in the resource name 'CMD.PRE.WRITE.GLOBAL.ls_-l'. ... print CLE_DOCTYP_PGMSYNOPSIS === FLAM4 Synopsis === 3.23. FLAM4 Synopsis The synopsis collects the help, syntax and other important information for a program, function or parameter. It is a kind of quick reference and will used over the complete hierarchies in this document. Below you can find for example the synopsis for program FLAM4. ----------------------------------------------------------------------- HELP: Frankenstein Limes(R) Access Method (FLAM(R)) 4 Utility PATH: limes TYPE: PROGRAM SYNTAX: > flam4 COMMAND/FUNCTION ... ----------------------------------------------------------------------- ... print CLE_DOCTYP_PGMSYNTAX === Syntax of FLAM4 === 3.24. Syntax of FLAM4 Below you can see the main syntax for this program. This content is printed on the screen if the built-in function <> is used. Each command can be entered as an argument list on the command line or in a parameter file. To define a parameter file the assignment character (`=`) must be used. For an argument list a blank or if the command (main table) is of type overlay a dot (`.`) or if the command is of type object a round bracket open (`(`) must be given. If the round bracket open is used then the corresponding round bracket close (`)`) at the end of the command is mandatory. The dot and parenthesis are in accordance with the normal syntax. If an argument list is used all parameters (argv[]) are concatenated to one long string and this string is passed to the parser, to allow usage of all features of the shell to build the command line input. To make sure that anything you enter is passed one by one to the parser please use double (WINDOWS(R), UNIX) or single (z/OS(R)) quotation marks at the beginning and end of the argument list. If the dot for overlays or parenthesis for objects are used then the complete command must be included in double quotation marks. The examples below show all possible command entries for the type object: ----------------------------------------------------------------------- :> FLAM4 command temp() dummy='str' :> FLAM4 command(temp() dummy='str') :> FLAM4 command "temp() dummy='str'" :> FLAM4 "command(temp() dummy='str')" ----------------------------------------------------------------------- For a command of type overlay it looks like: ----------------------------------------------------------------------- :> FLAM4 command temp() :> FLAM4 command.temp() :> FLAM4 command "temp()" :> FLAM4 "command.temp()" ----------------------------------------------------------------------- In a parameter file you can start with a dot for an overlay or with parenthesis for an object or with the argument (overlay) or argument list (object) directly. ------------------------------------------------------------------------ Syntax for program 'flam4': --| Commands: INFO/FLAM --|--| flam4 [OWNER=oid] command "... argument list ..." [MAXCC=[max][-min]] [QUIET/SILENT] --|--| flam4 [OWNER=oid] command=" parameter file name " [MAXCC=[max][-min]] [QUIET/SILENT] --|--| flam4 [OWNER=oid] command=>"parameter file name " [MAXCC=[max][-min]] [QUIET/SILENT] --|--| You can optionally specify: --|--|--| the owner id for this command (to use custom configuration files) --|--|--| the maximum condition code (max) to suppress warnings --|--|--| the minimum condition code (min), zero is returned if the condition code would be smaller --|--|--| QUIET disables the normal log output of the command line executer --|--|--| SILENT disables log and errors messages of the command line executer --| Built-in functions: --|--| flam4 SYNTAX [command[.path] [DEPTH1 | ... | DEPTH9 | ALL]] --|--| flam4 HELP [command[.path] [DEPTH1 | ... | DEPTH9 | ALL]] [MAN] --|--| flam4 MANPAGE [function | command[.path][=filename]] | [filename] --|--| flam4 GENDOCU [command[.path]=]filename [NONBR] [SHORT] --|--| flam4 HTMLDOC [path] [NUMBERS] [TYPES] [SHORT] --|--| flam4 GENPROP [command=]filename --|--| flam4 SETPROP [command=]filename --|--| flam4 CHGPROP command [path[=value]]* --|--| flam4 DELPROP [command] --|--| flam4 GETPROP [command[.path] [DEPTH1 | ... | DEPTH9 | DEPALL | DEFALL]] --|--| flam4 SETOWNER name --|--| flam4 GETOWNER --|--| flam4 SETENV variable=value --|--| flam4 GETENV --|--| flam4 DELENV variable --|--| flam4 LSTENV --|--| flam4 HLPENV --|--| flam4 TRACE ON | OFF | FILE=filename --|--| flam4 CONFIG [CLEAR] --|--| flam4 GRAMMAR --|--| flam4 LEXEMES --|--| flam4 LICENSE --|--| flam4 VERSION --|--| flam4 ABOUT --|--| flam4 ERRORS ------------------------------------------------------------------------ ... print CLE_DOCTYP_PGMHELP === Help for FLAM4 === 3.25. Help for FLAM4 Here you can see the static main help for this program. This content is printed on the screen if the built-in function <> is used without any further arguments. ------------------------------------------------------------------------ Help for program 'flam4': --| Commands - to execute powerful subprograms --|--| flam4 INFO - Provides various information --|--| flam4 FLAM - Flam4 utility command --| Built-in functions - to give interactive support for the commands above --|--| flam4 SYNTAX - Provides the syntax for each command --|--| flam4 HELP - Provides quick help for arguments --|--| flam4 MANPAGE - Provides manual pages (detailed help) --|--| flam4 GENDOCU - Generates auxiliary documentation (ASCIIDOC text format) --|--| flam4 HTMLDOC - Generates the complete HTML documentation into a folder --|--| flam4 GENPROP - Generates a property file --|--| flam4 SETPROP - Activate a property file --|--| flam4 CHGPROP - Change a property value in the currently active property file --|--| flam4 DELPROP - Remove a property file from configuration --|--| flam4 GETPROP - Show current properties --|--| flam4 SETOWNER - Defines the current owner --|--| flam4 GETOWNER - Show current owner setting --|--| flam4 SETENV - Set an environment variable (defined in the config file) --|--| flam4 GETENV - Show the environment variables (defined in the config file) --|--| flam4 DELENV - Delete an environment variable (defined in the config file) --|--| flam4 LSTENV - List status of all possible usable environment variables --|--| flam4 HLPENV - List help message for all possible usable environment variables --|--| flam4 TRACE - Manage trace capabilities --|--| flam4 CONFIG - Shows or clear all the current configuration settings --|--| flam4 GRAMMAR - Shows the grammar for commands and properties --|--| flam4 LEXEMES - Shows the regular expressions accepted in a command --|--| flam4 LICENSE - List license information for the program --|--| flam4 VERSION - List version information for the program --|--| flam4 ABOUT - Show information about the program --|--| flam4 ERRORS - Show information about return and reason codes of the program For more information please use the built-in function 'MANPAGE' ------------------------------------------------------------------------ ... print CLE_DOCTYP_COMMANDS == FLAM4 Commands == 4. FLAM4 Commands Commands are used to run powerful subprograms. The command line processor compiles a table defined syntax in a corresponding preinitialized data structure. This structure will be mapped to the parameter structure of the corresponding subprogram. The subprogram can be executed with the defined arguments/parameters. To support these variable commands several built-in functions are available and described in a separate <>. Each supported command is explained in a separate section in this document. Each section contains a synopsis including the help message, the path, the type and the syntax followed by a detailed description. If an argument of this command is an object or overlay or, if a detailed description is available for this argument, a separate section with synopsis (help, path, type, syntax) and description is written, otherwise a bullet list is printed which contains the keyword, the syntax and the help message. For the syntax of an overlay braces `{}` are used to keep all possible arguments of an overlay logically together. These braces are not part of the real syntax. The braces are only written to demonstrate clearly that one of these arguments must be selected with the DOT operator (optional) for defining the overlay. To be compatible with certain shells, the features below are implemented: * Strings can be enclosed with single '' or double "" quotation marks * Integrated strings (without spaces) can also defined without quotes * Keywords can also start with "-" or "--" in front of the qualifier * If it unique then parenthesis and the dot can be omit for object and overlays Commands can be declared in deep hierarchical depth. In order to simplify their handling the path is a powerful instrument for managing only relevant parts of it. Because of that the path is printed for each synopsis. To run commands under different owners, the owner id can be defined in front of the command. To suppress the outputs of the command line executer you can set QUIET or SILENT as the last parameter. Additional to QUIET, SILENT suppressed also the error messages. These options has no effect of the outputs of the commands and built-in functions. For job control, the maximum (and minimum) condition code (MAXCC=[max][-min]) of the command execution can optionally be set as the last parameter for each command. The value 'max' is the maximum condition code. If the condition code, that would be returned if MAXCC was not specified, would be greater than 'max', it is reduced to the value of 'max'. If the condition code would be smaller than 'min', the actually returned condition code is reduced to 0. Both, 'max' and 'min', values are optional. To only specify the minimum, simply omit the 'max' value, i.e. the MAXCC value starts with a hyphen. Example: The clause `MAXCC=8-4` will set each condition code greater than 8 to 8 and each condition code smaller than 4 to 0, while `MAXCC=-4` only sets condition codes smaller than 4 to 0. ------------------------------------------------------------------------ flam4 [OWNER=oid] command "... argument list ..." [MAXCC=[max][-min]] [QUIET] flam4 [OWNER=oid] command=" parameter file name " [MAXCC=[max][-min]] [QUIET] ------------------------------------------------------------------------ The parameter for each command can be provided as argument list on the command line or as parameter file. [[CLEP.COMMAND.INFO]] === 4.1. Command 'INFO' .Synopsis ----------------------------------------------------------------------- HELP: Provides various information PATH: limes.flam4 TYPE: OBJECT SYNTAX: > flam4 INFO(NET.{},GET.{},FMT=LIST/XML,OUT='str'/STDOUT/STDERR,DIR(),LOG()) ----------------------------------------------------------------------- .Description The INFO command provides further information about local or remote files, CCSIDs among others. To use CONV and other commands it is sometimes useful to know which CCSIDs are supported or which members are in a FLAMFILE or how the file attributes are named. The system information (environment, default values, hardware support) may be helpful information to diagnose errors. To get syntax information, please use: ----------------------------------------------------------------------- flam4 SYNTAX INFO ----------------------------------------------------------------------- To get help for a parameter, please use: ----------------------------------------------------------------------- flam4 HELP INFO.parameter[.parameter[...]] ----------------------------------------------------------------------- To read the manual page for a parameter, please use: ----------------------------------------------------------------------- flam4 MANPAGE INFO.parameter[.parameter[...]] or flam4 HELP INFO.parameter[.parameter[...]] MAN ----------------------------------------------------------------------- To generate the user manual for a command, please use: ----------------------------------------------------------------------- flam4 GENDOCU INFO=filename ----------------------------------------------------------------------- Parameters can be defined by the command line (directly or per file) or by properties read from the corresponding property file. .Examples ----------------------------------------------------------------------- flam4 INFO get.system flam4 INFO get.ccsids flam4 INFO get.file='test.adc' out='list.txt' flam4 INFO get.file='ssh://@server1/test.adc' out='list.txt' flam4 INFO get.file='~.*' dir(rec) out='list.txt' flam4 INFO get.fkme(help) ----------------------------------------------------------------------- .Arguments * `NUMBER: FMT=LIST/XML - Specify formatting of information [LIST]` ** `LIST - Provide information as list` ** `XML - Provide information in XML-structure` [[CLEP.COMMAND.FLAM]] === 4.2. Command 'FLAM' .Synopsis ----------------------------------------------------------------------- HELP: Flam4 utility command PATH: limes.flam4 TYPE: OBJECT SYNTAX: > flam4 FLAM(NETINP.{},NETOUT.{},COMPRESS/C,DECOMPRESS/UNCOMPRESS/D,REMAIN,APPEND/ADD,FLAMFILE/FL['str'/STREAM/DUMMY...],MEMBER/MEM='str',FLAMIN/FLAMI['str'/STREAM/DUMMY...],FLAMOUT/FLAMO='str'/STREAM/DUMMY,OUTPATH/OPATH='str',MODE/MO/CMPMODE=CX7/CX8/VR8/ADC/NDC/AES/FLAMENC,NDC,CRYPTOMODE/CRYMODE/ENCMODE/CRYPTMODE/CRYPTOM=OFF/AES/FLAMENC,CRYPTOKEY/CRYPTKEY/CRYPTOK='bin',PASSWORD/PASSW='bin',PAASCII='bin-ascii',PAEBCDIC='bin-ebcdic',PAXWORD='bin-hex',PARFILE/PARF/PAFILE/PARAMETER_FILE='str',PARDDN/PADDN/PARD='str',RECDELIM/RECDEL/RECD='bin-hex',RECFORMAT/RECFORM/RECFMT/RECFM/RECF=FIX/F/VAR/V/TEXT/STREAM/UNDEFINED/UNDEF/U/FIXBLK/FB/FIXS/FBS/VARBLK/VB/VARSPAN/VBS,RECSIZE/MAXSIZE/MAXS=num,INRECDELIM/IRECDEL/IRECD/IREC='bin-hex',INRECFORMAT/IRECFORM/IRECFMT/IRECFM/IRECF=EAF/FIX/DTEXT/TEXT/STREAM/VARIABLE/UNDEFINED/U/F/FIXBLK/FB/V/VARBLK/VB/FIXS/FBS/VARSPAN/VBS/VAR_2B/VAR2B_DATA/VAR_4B/VAR_ASCII/VAR_EBCDIC,INRECSIZE/IRECSIZE/IRECS=num,OUTRECDELIM/ORECDEL/ORECD/ORECDELIM/OUTDELIM/ORDELIM='bin-hex',OUTRECFORMAT/ORECFORM/ORECFMT/ORECFM/ORECF=EAF/FIX/DTEXT/TEXT/STREAM/VARIABLE/UNDEFINED/U/F/FIXBLK/FB/V/VARBLK/VB/FIXS/FBS/VARSPAN/VBS/VAR_2B/VAR2B_DATA/VAR_4B/VAR_ASCII/VAR_EBCDIC,OUTRECSIZE/ORECSIZE/ORECS/ORSIZE/OUTSIZE=num,BLKSIZE/BLKS=num,IBLKSIZE/INBLKSIZE/IBLOCKSIZE/INBLOCKSIZE/IBLK=num,OBLKSIZE/OUTBLOCKSIZE/OBLOCKSIZE/OBLK=num,ATTRIBUTES=NONE/COMMON/ALL,OPTION=CUT/NOCUT/SUPPRESS/NOSUPPRESS,CUT/TRUNCATE/TRU[=YES/NO],NOCUT/NOTRUNCATE,SUPPRESS,NOSUPPRESS,PADCHAR/PAD_CHAR/PAD='bin-hex',TRIMCHAR='bin-hex',MAXBUFFER/MAXB=num,MAXRECORDS/MAXR=num,FLAMCODE/FLAMC=SYSTEM/ASCII/EBCDIC,KMDLL/KMLIB/KMELIB='str',KMEXIT/KMEEXIT/KMEFUC/KME/KMFUNC='str',KMPARAMETER/KMEPAR/KMPARM/KMP='bin',HEADER/HE=YES/NO,COMMENT/COMM/LABEL='str',MSGFILE/LOGFILE/MSGF='str',SECUREINFO/SEC=YES/NO/IGNORE/MEMBER,MSGDISP/MSGD=TERMINAL/MSGFILE/SYSTEM,SHOW/SH=ALL/DIR/ATTRIBUTE/ERROR/NONE,INFO/I=YES/NO/HOLD,CHECKALL/CHECKA,CHECKFAST/CHECKF,HELP/LIST,FILEINFO/FI=YES/NO,CODETABLE='str',TRANSLATE/TRA='str'/E/A/A/E/NONE,NOPATH,CLIMIT/CLI=num,IPRCNTRL=DETACH/RETAIN/ERASE/REPLACE,IRPLFFD/IRP=num,OPRCNTRL=DETACH/RETAIN/ERASE/REPLACE,ORPLFFD/ORP=num,NOMKDIR/NOM,EXK10/EXK1='str',EXK20/EXK2='str',EXD10/EXD1='str',EXD20/EXD2='str',SPACE1=num,SPACE2=num,OSPACE1=num,OSPACE2=num,UNIT='str',OUNIT='str',STORCLASS/STORCLAS/STORC='str',OSTORCLASS/OSTORCLAS/OSTORC='str',VOLUME/VOL='str',OVOLUME/OVOL='str',DATACLASS/DATAC='str',ODATACLASS/ODATACLAS/ODATAC='str',MGMTCLASS/MGMTCLAS/MGMTC='str',OMGMTCLASS/OMGMTCLAS/OMGMTC='str',DSORG/DS/FCBTYPE=SEQ/PS/EDS/ESDS/KDS/KSDS/RDS/RRDS/LDS,IDSORG/IFCBTYPE=SEQ/PS/LIB/PDS/USS/EDS/ESDS/KDS/KSDS/RDS/RRDS,ODSORG/ODSO=SEQ/PS/LIB/PDS/USS/EDS/ESDS/KDS/KSDS/RDS/RRDS,ICLOSDISP/ICLO=REWIND/UNLOAD/LEAVE,CLOSDISP/CLO=REWIND/UNLOAD/LEAVE,OCLOSDISP/OCLO=REWIND/UNLOAD/LEAVE,DISPS=NEW/SHR/MOD/OLD,DISPN/DISPNORMAL=CATALOG/CATLG/DELETE/KEEP/UNCATALOG/UNCATLG,DISPA/DISPABNORMAL=CATALOG/CATLG/DELETE/KEEP/UNCATALOG/UNCATLG,ODISPS=NEW/SHR/MOD/OLD,ODISPN/ODISPNORMAL=CATALOG/CATLG/DELETE/KEEP/UNCATALOG/UNCATLG,ODISPA/ODISPABNORMAL=CATALOG/CATLG/DELETE/KEEP/UNCATALOG/UNCATLG,KEYDISP/KEYD=OLD/NEW/DEL,KEYLENGTH/KEYLEN/KEYL=num,OKEYPOSITION/OKEYPOS/OKEYP=num,OKEYLENGTH/OKEYL=num,ORGKEYPOS/ORGKEYPO/ORGKEYP=num,ORGKEYLEN/ORGKEYL=num,IKEYPOSITION/IKEYPOS/IKEYP=num,IKEYLENGTH/IKEYLEN/IKEYL=num,DEVICE/DEV=FILE/DISK/TAPE/FLOPPY/STREAMER/USER,IDEVICE/IDEV=FILE/DISK/TAPE/FLOPPY/STREAMER/USER,ODEVICE/ODEV=FILE/DISK/TAPE/FLOPPY/STREAMER/USER,SPLITMODE/SPLITM=NONE/SERIAL/PARALLEL,SPLITNUMBER/SPLITN=num,SPLITSIZE/SPLITS=num,FLAMDDN/FLAMD='str',IDDN['str'...],ODDN='str',MSGDDN/MSGDD='str',REMOVE/DELETE/INDELETE/IDELETE,FROM='str'/DEFAULT/ASCII/EBCDIC/BOMUTF/BOMUCS/SYSTEM/LOCAL,CHRMODE/SUBMODE=STOP/IGNORE/SUBSTITUTE/IDENTITY/TRANSLIT,TO='str'/DEFAULT/ASCII/EBCDIC/SYSTEM/LOCAL,LARGER,DIR(),LOG(),NORUN) ----------------------------------------------------------------------- .Description The FLAM command can be used to compress or decompress individual files or groups of files either interactively or by means of procedures. The compress or decompress parameter specifies which operation is to be performed. All the information necessary to execute the command can be specified in the form of inputs made directly on the command line, in a parameter file or as installation-specific default values. The FLAM command is backward-compatible with the FLAM4 utility and designed to work with FLAM4 archives, also called FLAMFILEs. There are a few differences to the old FLAM4 utility: * Completion/Condition codes are defined by CLE/P ** Return code 0, 4 and 8 have the same meaning (OK, WARNING, ERROR) ** Higher return codes are different and correspond to more critical issues ** There are also completion codes between 0 and 4 or 4 and 8, for example 1 if a warning in the log (can be deactivated) ** Special condition codes (>64) are the same ** Please use the built-in function 'ERRORS' or refer to the appendix 'Return Codes' for more information ** The main difference in completion codes are: *** DMS errors result in a return code of 8 and not 12 with FLAM5 *** If directory walk (wildcards) fail for all files FLAM5 returns 8 instead of 4 (it returns 4 like FLAM4 if not all files processed successfully) * The log output is in FLAM5 syntax and looks different to the old FLAM4 utilities but is now the same on each platform * The default value file is replaced by the FLAM5 property management (and much more intelligent default behavior) * The default behavior differs a little bit and if required the property management must be used to adjust it * If squared, curly brackets or other operators are required, the parameters must be enclosed in quotation marks ** This was minimized as much as possible to be compatible with the old syntax * If keywords used insaide a string value in a parameter assignment the string must enclose in quotation marks ** IDDN=FLAMDD will result in a syntax error, because FLAMDD is a valid keyword and CLP try to assign the value of FLAMDD ** IDDN='FLAMDD' will work because 'FLAMDD' is interpreted a string value * The selection of members to read from a FLAMFILE is now made via the member definition of the FLAMIN parameter ** As part of the URL: FLAMIN='FLAM.FILE.ADC/?HUGO.PDS(*)' ** Or by the new parameter MEMBER: MEMBER='HUGO.PDS(*)' ** Instead of: FLAMOUT= * The file name mapping (selection and replacement rules) is different and works like documented below (see INPUT TO OUTPUT NAME MAPPING) * No support of VSAM-LDS as FLAMIN or FLAMOUT (original data) on z/OS (not supported by LE) * The new FL5 system header is not supported by the old FLAM4 components ** VSAM data sets archived with FLAM4 can be reproduced with FLAM5 but not vice versa ** Sizes of original files are only known by FLAM5 (FLAM4 uses hard-coded defaults for primary (2/4MiB) and secondary space (40/50(MiB))) * Recreation of archived PDS load libraries is not yet supported within FLAM5 ** FLAM4 archived PDS load libraries and can be recreated ** FLAM4 does not support this for PDSEs ** FLAM5 does not support it at all till the XMIT support will be available * FLAM5 uses the CEE runtime environment (LE) and FLAM4 works LE-less on z/OS, which may result in different behavior ** Writing KSDS with new key position and length with FLAM5 results in re-ordered records and in a FIOPUT error on FLAM4 if reordering is required ** Empty records are really empty records (length=0) with FLAM4, with FLAM5 (LE) a empty record is written as one padding byte (see <>) * FLAM4 recognizes some errors only after an open of an output file, which may result in an empty file ** FLAM5 detects a lot of such errors before opening to avoid creation of empty files * FLAM5 recognizes from the combination of specified parameters whether compression or decompression is requested ** FLIMIN FLAMFILE (=COMP) and FLAMFILE FLAMOUT (=DECO) ** On z/OS, COMPRESS or DECOMPRESS must be specified for FLAM4 ** On Unix and Windows DECOMPRESS is the default for FLAM4 * The print control character handling is more intelligent in FLAM5 ** To be backward compatible: + In the case you set a certain record format without print control character and the original file contains print control character you must define OPRNCTR=RETAIN to ensure the print control character as part of the record like in FLAM4. If this new parameter not defined to RETAIN the print control character is not part of the record in this case. ** The provided record length in FLAM5 don't include the print control character (is one byte shorter then in FLAM4) * The default padding character is in FLAM4 always x'40' and in FLAM5 x'40' only used if text (ASA/MCC or CCSID defined) else it is x'00' * With FLAM4 for undefined files the record length must be provided by the caller, FLAM5 use the block size instead * The LIST switch on Windows will be the same as HELP. To get the current default values (properties) the built-in function GETPROP must be used. * FLAM4 on z/OS often requires upper case. FLAM5 syntax is not case sensitive,i.e. lower case can often be used * Given IRECFM and IRECSIZE are override by FLAM4 if the file is cataloged, FLAM5 try to allocate and open the file with the given parameter ** This result sometimes in errors with FLAM5 because the read file don't match with the given parameter (file attributes in conflict) ** Sometimes the runtime environment converts the data to the requested record format (original is VBA and requested is VB) * The FLAM4 UserIO is no longer supported for original data (`IDEVICE=USER` and `ODEVICE=USER` is ignored) * FLAM4 don't support large files (`DSNTYPE=LARGE`) at dynamic allocation, a corresponding data class must be used in this case ** FLAM5 supports large files by default if the amount of required tracks above the limit ** With version 5.1.22 the FLAM4 components set the DSNTYPE too large if more than 64K tracks allocated ** Only FLAM5 supports since version 5.1.22 extended attribute datasets (up to 16 TiB) * FLAM4 works with hard-coded defaults for space allocation (primary 2/4MiB secondary 40/50MiB) ** FLAM5 determines the file size and allocates depending on the space used by the original file by default ** This does not work for multi-volume datasets, in this case only the space used of the first volume will be allocated ** If you write a PO and you will extend this dataset later it is recommended to provide custom space values * FLAM5 supports also the default DD name 'FLAMOUT' at compress if 'FLAMFILE' not allocated and the same for 'FLAMIN' at read If the FLAM command is specified with the 'list' parameter, a list of the currently active settings of the default values is displayed; they can be modified by the system manager using the '-defaults' parameter. The FLAM command permits a separate FLAMFILE to be created from each original file (this requires input name mapping, which is differs to FLAM4). It is also possible to compress a group of original files into one FLAMFILE. The input files which are to be (de)compressed can either be specified by their fully qualified name or by patterns with wildcards. Output filenames can be explicit or generated by substitution rules. When decompressing, by default a file is to be restored with its original name and attributes. With the exception of the '-list' and '-defaults' parameters, there are equivalent FLAM parameters for all the elements of the command syntax, so that each setting can be entered either directly in the command line or in a parameter file which is referred to in the command by the parameter parfile=parameter file. Using a parameter file permits complicated keyboard inputs to be avoided, for example if settings which differ from the installed default values are required frequently. The following features are supported: * Different kinds of remote access protocols (SSH) * Different kinds of data sources (Files, Streams, Tables) * Different kinds of I/O methods (Binary, Character, Text, Record, FLAM) * Different compression modes (ADC, CX7, CX8, VR8 and NDC) * Different encryption modes (AES and FLAMENC) * Powerful character conversion based on CCSIDs If the FLAM command is not sufficient, you can use the CONV command which supports FLAMFILEs and other compression formats and provide various conversion and formatting modes. FLAM and CONV parameter are not identical. To get syntax information, please use: ----------------------------------------------------------------------- flam4 SYNTAX FLAM ----------------------------------------------------------------------- To get help for a parameter, please use: ----------------------------------------------------------------------- flam4 HELP FLAM.parameter[.parameter[...]] ----------------------------------------------------------------------- To read the manual page for a parameter, please use: ----------------------------------------------------------------------- flam4 MANPAGE FLAM.parameter[.parameter[...]] or flam4 HELP FLAM.parameter[.parameter[...]] MAN ----------------------------------------------------------------------- To generate the user manual for the command, please use: ----------------------------------------------------------------------- flam4 GENDOCU FLAM=filename ----------------------------------------------------------------------- Parameters can be defined via command line (directly or through a parameter file) or via properties taken from the corresponding property file. .Example For the FLAM4 utility the command name 'flam' is default and can be omitted. The FLCL utility has no default command, in this case the command name 'FLAM' must be defined. ----------------------------------------------------------------------- flam4 FLAM comp flamin=swift.txt flamfile=swift.fla inrecformat=stream ----------------------------------------------------------------------- Reads file in record format stream (usual text), compresses with default compression mode ADC, writes FLAMFILE. ----------------------------------------------------------------------- flam4 FLAM deco flamfile=swift.fla flamout=swift.txt outrecformat=stream ----------------------------------------------------------------------- Reads and decompresses FLAMFILE, writes output file in record format stream. .Arguments * `SWITCH: NORUN - Don't run the command only show parsed parameter` ... print CLE_DOCTYP_CHAPTER == FLAM Subsystem for z/OS == Appendix A. FLAM Subsystem for z/OS This appendix describes the handling of the FLAM subsystem in IBM z/OS (OS/390, MVS) environment. FLAM-sub is an interface to FLAM (z/OS), the compression and encryption utility, and may only be used in conjunction with a license for FLAM-utility. FLAM as a subprogram offers the same functionality as if used as a standalone application. The difference is, that FLAM is called from a custom application or a driver program. It is possible to specify parameters with the call. === FLAM as a subsystem FLAM-sub supports the subsystem interface of the command language (JCL) in z/OS. Files can thus be processed in compressed an encrypted form without having to modify the associated programs. The additional compression and decompression steps which were necessary in the past can now be dispensed with. The calling program is not aware of any differences as compared with the conventional mode of file processing. It receives a record for a read call in the same way as before, while for a write call the data management system (DMS) receives the record as usual. The subsystem decompresses before reading and compresses before writing. The compression results are similar to those of the FLAM utility (approx. 70-90%). The FLAM subsystem even enables programs to be supported with file formats for which they were not originally written, in other words the program and the data formats can be separated from one another. The FLAM subsystem is loaded almost entirely in the high address space (above 16 MB); only a small tuning module for 24-bit addressing works in the low address space. Since 24-bit addressing is used for the DMS accesses to PS files, a memory area corresponding to the length of one data record is created in the low address space. FLAM creates all work areas above the 16 MB limit, so that the usual memory area is still made available to the applications. No changes are normally necessary in the calling programs. The compressed files created by the subsystem (FLAMFILEs) can be decompressed by the FLAM utility at any time, and all FLAMFILEs created by the FLAM utility are accepted and processed logically by the FLAM subsystem. The same applies likewise to FLAMFILEs that are created/read via the record interface of FLAM (see sections <> and <>. === Subststem call ------------------------------------------------------------------------ //ddname DD DSN=filename, // DISP=OLD, // SUBSYS=(FLAM,'flam-parameter') ------------------------------------------------------------------------ This call designates the cataloged file filename for processing with the FLAM subsystem. FLAM parameters can be specified in addition, in the same way as with the FLAM utility. Invalid parameters are rejected by FLAM as JCL errors and the job they refer to is not even started. The DD command is still verified by JES. The catalog entry for the file is not modified by FLAM. All entries such as the file name, the record and block lengths, the volume, etc. remain unchanged. The files that must be processed with the FLAM subsystem are thus still verified by the check mechanisms, such as SMS and RACF. There is no FLAM restriction on the number of files that can be processed "simultaneously" by the subsystem. The only restrictions are those imposed by the system itself, such as the maximum amount of available memory or the maximum possible number of DD statements. If files for which the SUBSYS specification in the DD statement is not supported must be processed (for example, JES files), the problem can be overcome by specifying a FLAM parameter and a second DD statement: ------------------------------------------------------------------------ //ddname DD SUBSYS=(FLAM,'FLAMDDN=ddname1'), // DCB=(LRECL=....,BLKSIZE=....) //ddname1 DD SYSOUT=G,DEST=(.....), // DCB=(LRECL=80,.....) ------------------------------------------------------------------------ Specifying a DD name ddname1 as a FLAM parameter causes the subsystem to process the file assigned by means of this name (in this case by writing in it). This file must be specified in the JCL. Similarly, ddname1 could be used as a decompression input. The calling program, on the other hand, uses the file assigned with ddname! This method permits reading or writing in any file (JES, RJE, other subsystems, magnetic tapes, temporary files, etc.) which cannot otherwise be processed with the usual method (see <>). If DCB attributes are specified for ddname, FLAM interprets them as values for the original (uncompressed) file. DCB specifications for ddname1 apply to the compressed file (the FLAMFILE). Record and block formats which are completely different from one another can thus be set for the original and compressed files depending on the particular problem, in other words the subsystem allows an application to use files that have absolutely nothing in common with the entries in the program (see <>) and that without the subsystem might have to be converted. If the FLAMFILE is accessed without a SUBSYS specification, it behaves like a "normal" file, in other words it can be read (copied, transferred) by means of utilities or file transfer programs without being decompressed! Similarly, a FLAMFILE which is transferred by a file transfer program can be read and processed via the subsystem. [[flam4.flamsub.preconditions]] === Preconditions FLAM must be installed on the computer both as a utility (V4.0 or higher) and as a subsystem. Data records can be written, read, modified, inserted or deleted. The data set organization may be of PS or VSAM. The accesses must be logical, in other words either in sequential order or according to a key. Accesses according to the RBA (relative byte address) of a data record or an alternate index are not possible! A DCB for PS files or an ACB for VSAM files can be coded in the calling program. The programs (or more precisely the file control block DCB) must be stored in the low address space. A FLAMFILE in KSDS format must have the same structure as in the FLAM utility (see <>), i.e. * Relative key position: 0 * Key length: one byte longer than the original * Record length: between 80 and 32760 bytes * Control interval size: any, depending on record length Although FLAM supports a wide range of record length, please take care on performance views. FLAM compresses a number of data records in a row (depending on MAXR and MAXB parameter). MAXB=64 means 64 KB of Data will be compressed. Assume a compression rate of 90 %, this will lead to a compression result of 6.4 KB compressed data. If you had defined the VSAM-KSDS-FLAMFILE like your original file (i.e. with record size of 400 byte) FLAM has to write 17 VSAM records for the compressed data, each with a new built key. So has FLAM to read 17 records to decompress one original record. So it is better to define a larger RECSIZE for the FLAMFILE, in this example you should use RECSIZE(7168). NOTE: The runtime system of COBOL (PL/I, C) checks all assigned VSAM files. The FLAMFILE must therefore be tuned to the VSAM file type if COBOL programs are used: for accesses to VSAM-ESDS it must also have been created as ESDS, while for accesses to VSAM-KSDS a KSDS-FLAMFILE must exist as well. With Assembler programs, on the other hand, the FLAMFILE and the original file need not necessarily be of the same type (providing this is also meaningful). === Principle of operation All the JCL specifications are checked before a batch job is started by JES. The parameters for the subsystem are transferred and verified by FLAM-sub. If an error is detected, the DD statement is rejected as a JCL error. In addition, a message is output in the JCL list. The job is thus only started if there are no errors; otherwise it is aborted in the usual way. An OPEN command call in the program causes a connection to be set up to the FLAM subsystem. FLAM- sub decides at this stage whether the file must be compressed or decompressed. An open input means decompression (read only), while an open output means compression (write only). An open I/O allows full I/Os to the FLAMFILE via the key of the original records. At the same time, the record and block lengths are taken from the calling program (or from the DCB specifications in the JCL) and - in the case of an open output - transferred to the FLAM file header of the FLAMFILE (unless the `HEADER=NO` parameter has been specified). The FLAM parameters specified in the JCL are also activated at the time of the open and the compression/ decompression routines set accordingly. All read and write accesses take place in accordance with the specifications in the program. Asynchronous VSAM calls are executed synchronously by the subsystem, while return codes are not returned (or the error routines activated) until the CHECK call. It makes no difference whether the file is accessed by the program as BSAM, QSAM or VSAM. FLAM-sub maps all calls to the currently available FLAMFILE. The record format can always be either variable or fixed. If an error occurs, the error routine remains active in the calling program and is not overlaid by the subsystem. The error return codes that are returned by the subsystem correspond to those of the DMS (see also: https://www-01.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R3SC236852/$file/idad500_v2r3.pdf[MVS/DFP Macro Instructions for VSAM Data Sets] for VSAM error codes). In other words, the error routines need not be modified; the IEC ... messages of the DMS (see also: http://192.234.241.151/mvs/iea2m720.pdf[z/OS MVS System Messages Volume 7 (IEB - IEE)]) are still traced if necessary. Additional messages are output by FLAM-sub with `WTO,Routcde=11` for FLAM errors and subsystem errors (see section <>). They are documented both in the system log and in the JCL list. No other messages are generated (the `FLAM SHOW=ALL` parameter of the utility has no effect). A CLOSE for the file causes FLAM to close the FLAMFILE and free all the work areas again that were requested with the OPEN. New OPEN calls are now allowed. === Restrictions It is not normally necessary to modify the JCL as a result of using the FLAM subsystem. If specifications such as the record and block lengths are omitted from the programs (utilities such as IEBGENER and SORT function in this way), however, DCB specifications must be included in the DD statement! The usual principle, whereby the values are automatically taken from a catalog entry in such cases, does not work if a subsystem is used. An OPEN routine only recognizes the actual presence of a subsystem file and no longer accesses a catalog entry (even if one exists). ==== In general [width="100%",cols="^1,12",align="center"] |==== | *ADC* | `MODE=ADC` is not allowed in update mode. | *DGD* | Data Generation Groups must be specified with the absolute name (`FILE.G0004V00`), not with the relative name (`FILE(+1)`). | *DISP* | The DISP parameter in the DD statement (e.g. `DISP=NEW` or `DISP=(..., DELETE)` cannot be interpreted and thus has no effect. All files that are assigned directly via FLAM-sub (`DSN=filename, SUBSYS=FLAM`) must be cataloged. If new files must be created, a second DD statement can be used as shown in the example JCL below this table. Alternatively, the file must have been created in a previous STEP. All types of access (GET, WRITE, PUT, POINT, ERASE, CHECK) with appropriate RPL modifications for VSAM are possible via the subsystem. The following restrictions apply (see also: https://www-01.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R3sc236852/$file/idad500_v2r3.pdf[manual DFSMS/MVS Macro Instructions for Data Sets]). |==== ------------------------------------------------------------------------ //ddname DD SUBSYS=(FLAM,'FLAMDDN=newfile') //newfile DD DSN=..., DISP=(NEW,CATLG), UNIT=...,SPACE=... ------------------------------------------------------------------------ ==== DCB calls (PS files) [width="100%",cols="^1h,12",align="center"] |==== |CLOSE |All attempts to write in LOCATE mode are rejected. |LOCATE |Data Generation Groups must be specified with the absolute name (FILE.G0004V00), not with the relative name (FILE(+1)). |POINT |A POINT in conjunction with BSAM accesses is rejected with the error message IEC141I 013-BC .. (it is not even passed on to the subsystem). |VBS |A file with RECFM=VS or VBS (spanned records) can be written in but not read (IEC141I 013-A8 ...). |==== ==== ACB/RPL calls (VSAM files) [width="100%",cols="^1h,12",align="center"] |==== |CHECK |Asynchronous calls are executed synchronously by FLAM-sub (though, as usual, no response to errors until CHECK). |CNVTAD |CNVTAD returns invalid specifications or zero. |ENDREQ |ENDREQ (terminate a request) has no effect (FLAM does not support locks and only holds the last key). |RPL |Accesses with more than one RPL (RPL list) are not supported (only the first RPL is interpreted). |SHOWCB |`SHOWCB ACB=...` + The CINV, KEYLEN, LRECL and RKP fields contain the values for the original file, while all other specifications (such as ENDRBA, HALCRBA, etc.) refer to the FLAMFILE. |VERIFY |VERIFY (synchronize end of data) has no effect (VSAM control blocks are not updated). |VSAM-KSDS |If a VSAM-KSDS file must be loaded via FLAM-sub (OPEN OUTPUT in the program), the position and length of the key must be notified to the subsystem (important: the key position is 1 byte higher for FLAM than the RKP for IDCAMS!) see example JCL following this table. The reason for this is as follows: These specifications are normally transferred by means of IDCAMS when the file is cataloged. With VSAM accesses these specifications are contained in the catalog entry and not in the ACB. The FLAMFILE has been cataloged there with completely different values however, so that the parameters of the original file are no longer available. This information is contained in the compressed file for all subsequent accesses (INPUT or I/O) and does not need to be specified again. |==== ------------------------------------------------------------------------ //... DD ..., SUBSYS=(FLAM,'OKEYP=value1,OKEYL=value2') ------------------------------------------------------------------------ [[flam4.flamsub.used.parameters]] === Used FLAM parameters Parameters for FLAM are transferred in the DD statement of the SUBSYS specification: ------------------------------------------------------------------------ SUBSYS=(FLAM,'parameter1=value1,parameter2=value2,..') ------------------------------------------------------------------------ or ------------------------------------------------------------------------ SUBSYS=(FLAM,'parameter1=value1','parameter2=value2','...') ------------------------------------------------------------------------ The used parameters correspond to those of the FLAM utility (see <>). Parentheses () can also be used instead of the equals sign '=', e.g. MO(ADC). Any apostrophes contained within the parameter string literal must be duplicated (e.g. SUBSYS=(FLAM,'CRYPTOKEY=C''PASS WORD WITH BLANKS'''). Valid FLAM utility parameters available in FLAM-sub: [width="90%",cols="2,1,4",align="center",options="header"] |==== |Parameter | short form | Link (use z/OS parameter only) |CRYPTOMODE | CRYPTOM | <> |CRYPTOKEY | CRYPTOK | <> |FILEINFO | FI | <> |FLAMDDN | FLAMD | <> |HEADER | HE | <> |KMEXIT | KME | <> |KMPARM | KMP | <> |MAXBUFFER | MAXB | <> |MAXRECORDS | MAXR | <> |MAXSIZE | MAXS | <> |MODE | MO | <> |MSGDDN | MSGD | <> |ODSORG | ODSO | <> |OKEYLEN | OKEYL | <> |OKEYPOS | OKEYP | <> |SECUREINFO | SEC | <> |SPLITMODE | SPLITM | <> |SPLITNUMBER | SPLITN | <> |SPLITSIZE | SPLITS | <> |TRANSLATE | TRA | <> |==== It is possible to control FLAMSUB via environment or system variables as follows. An environment variable overwrites a system variable. [width="90%",cols="^1h,4",align="center"] |==== |Environment variables |are stored in -------------------------------- `//CEEOPTS DD DSN=filename,...` -------------------------------- or set in a C program. It is used in the actual STEP only and must be repeated for every other STEP. |FL_ZEDC_SUPPORT |controls use of ZEDC hardware algorithm. *Possible values:* :OFF: switch to `always NO ZEDC`. ADC is still in effect on compression. Decompression will be done in software, if ZEDC-compressed data is found. :ON: switch to `always ZEDC`. Instead of ADC, ZEDC is used. Without hardware, software is used. *Default:* without variable, ZEDC hardware is used when found. Otherwise, ADC is still in effect. *Valid for:* * compression * decompression. NOTE: When ZEDC hardware is not found and the algorithm is required, a software routine is used. |System Variable |are stored in `SYS1.PARMLIB(IEASYMxx)`. When system- and environment variable are found, the environment variable is used. A system variable is used system-wide in any STEP. |&FLZEDC |Controls using of the ZEDC hardware-algorithm. *Possible values:* + :OFF: switch to `always NO ZEDC`. ADC is still in effect on compression. Decompression will be done in software, if ZEDC compressed data is found. :ON: switch to `always ZEDC`. Instead of ADC, ZEDC is used. Without hardware, software is used. *Default:* without variable, ZEDC hardware is used when found. Otherwise, ADC is still in effect. *Valid for:* * compression * decompression |&FLOGQL1 |First qualifier in logstream with name LOGR. Without this parameter no logging is done. *Possible values:* + * 1-8 characters as in IXCMIAPU defined. Activates FLAM logging. |&FLOGQL2 |Second qualifier in logstream with name LOGR. *Possible values:* + * 1-8 characters as in IXCMIAPU defined. |&FLOGQL3 |Third qualifier in logstream name of LOGR. *Possible values:* + * 1-8 characters as with IXCMIAPU defined *Valid for:* * compression * decompression *Example:* IXCMIAU defines the logstream name FLAM.USAGE ------------------------------------------------ `(DEFINE LOGSTREAM NAME(FLAM.USAGE) ...)`. ------------------------------------------------ Then in Parmlib member must be stored: ------------------------------------------------ `SYMDEF(&FLOGQL1='FLAM')` + `SYMDEF(&FLOGQL2='USAGE')` ------------------------------------------------ NOTE: Without any system variable FLAM cannot log any data. Look for some examples for how to define a logstream in FLAM.JOBLIB. |==== === Subsystem control parameter These parameters are keyword parameters that must be specified separately from the FLAM parameters. They serve control of the subsystem and have nothing to do with FLAM: ------------------------------------------------------------------------ ...SUBSYS=(FLAM,*parm*,'flam-parameter') ------------------------------------------------------------------------ [width="90%",cols="^1h,9",align="center"] |==== |IG10 |Causes return code 10 (not a FLAMFILE) to be ignored when opening for reading. The records are transferred to the calling program without being decompressed. This parameter is meaningful in systems which can contain both compressed and uncompressed files (e.g. for tests). |TRACE |Activates the TRACE function. All function calls to FLAM and the control blocks (ACB, RPL) of the calling program are traced. The trace file must be specified in the JCL. The FLAM parameter `MSGDDN=ddname` specifies the DD name (default: `FLPRINT`). |==== [[flam4.subsystem.messages]] === Subsystem messages FLAM-sub only outputs a message on the console in the event of an error (by means of WTO ,ROUTCDE=11). This message is also documented in the JCL log. The letter after the message number specifies the time at which the error was detected: [width="50%",cols="^1h,4",align="center"] |==== |I |When the subsystem was initialized |C |When the JCL was analyzed |A |When the file was allocated |O |When the file was opened |D |During the I/O operation |E |When the file was closed |==== Except when the subsystem is initialized during the IPL of the operating system, errors only affect the specified file for a particular job, in other words the subsystem remains active for the other files. [width="100%",cols="^1h,12",align="right"] |==== |FLM0500I |*INITIALIZATION OF SUBSYSTEM FLAM COMPLETED* The FLAM subsystem has been initialized correctly and is now available. Response: None |FLM0501I |*SUBSYSTEM FLAM NOT INSTALLED ON THIS SYSTEM* The subsystem cannot be initialized. It has not been installed on the computer. Response: Insert the entry FLAM,FLSSIPL in SYS1.PARMLIB(IEFSSN..). |FLM0502I |*SUBSYSTEM FLAM ALREADY INITIALIZED* The program for initializing the subsystem was restarted after the IPL and then rejected. The subsystem that was already active is still active. Response: FLAM-sub need not be restarted. |FLM0503I |*NO MEMORY RECEIVED FOR INITIALIZATION* The operating system cannot make any memory available for initialization. The subsystem is not active. Response: An analysis is necessary (system dump). |FLM0504I |*SUBSYSTEM COULD NOT FREE SYSTEM MEMORY* The memory made available by the operating system cannot be freed. The subsystem has nevertheless been initialized and is now available. Response: An analysis is necessary (system dump). |FLM0505I |*SUBSYSTEM FLAM FUNCTION MODULE NOT FOUND* A FLAM-sub module required to initialize the subsystem is missing. The module was named by a previous operating system message. The subsystem has not been initialized. Response: Store the subsystem modules in a library belonging to the LINKLIST concatenation. |FLM0506I |*MODULE IEFJSVEC NOT FOUND* The IEFJSVEC operating system module required to initialize the subsystem is missing. The subsystem has not been initialized. Response: Set the missing module in a library that has been linked with the LINKLIB. |FLM0507I |*IEFJSVEC: NO MEMORY FOR SSVT* The subsystem initialization procedure was aborted by the IEFJSVEC operating system module because there is insufficient memory. Response: An analysis is necessary (system dump). |FLM0508I |*LOGIC ERROR IN IEFJSVEC* The subsystem initialization procedure was aborted by the IEFJSVEC operating system module due to an internal error. Response: An analysis is necessary (system dump). |FLM0510I |*INTERNAL ERROR. RC = no* + An internal error has been detected. The subsystem has not been initialized. Response: Please inform your sales partner. |FLM0515I |*INITIALIZATION OF SUBSYSTEM FLAM FAILED* The subsystem cannot be initialized due to an error. The error situation was documented in a previous message. |FLM0519I |*FLAM SUBSYSTEM IS NOT LICENSED* The license does not allow use of the subsystem. |*FLM0520C*/ *FLM0502C* | *PARAMETER ERROR: ...* / *SYNTAX ERROR: ...* An invalid parameter was entered in the DD statement of the JCL for FLAM-sub Response: Correct <> and restart job. |FLM0523C |*NO MEMORY RECEIVED FOR JCL-CONVERTION* A subsystem module was not allocated any memory by the operating system. Response: An analysis is necessary (system dump). |FLM0530A |*PARAMETER ERROR: ...* / *SYNTAX ERROR: ...* An invalid parameter was entered in the DD statement of the JCL for FLAM-sub. The invalid parameter is displayed. Response: Correct <> and restart job. |FLM0531A |*NO MEMORY RECEIVED FOR ALLOCATION* A subsystem module was not allocated any memory by the operating system. Response: An analysis is necessary (system dump). |FLM0540O |*OPEN ERROR. DDNAME=ddname. RC= no (FLAM)/(VSAM)* An error was detected at the time of the open for the file assigned in the JCL with ddname. FLAM return codes are specified in decimal format and VSAM errors in hexadecimal format see also <>, <> and DFSMS Macro Instructions). This message is followed by the IEC141I 013-xx message of the data management system, whereby the DD name used by the program is output. The message IEC161 may be output in connection with VSAM accesses. The reported FLAM message numbers are equal to the numbers in FLAM utility (ch. 8 in the manual FLAM(MVS)). In the table below are some examples. |FLM0541O |*ALLOCATION FAILED FOR DSN filename SVC99 ERROR CODE = no1. INFO CODE = no2.* An error was detected when an attempt was made to allocate the file filename dynamically. The codes are specified in hexadecimal format. Response: Analyse the error and info codes of the SVC99 as described in the https://www-01.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R3sa231371/$file/ieaa800_v2r3.pdf[MVS Programming: Authorized Assembler Services Guide, GC28-1763]. e.g. error code = 1708, info code = 2 or 5 means: File is not cataloged (remember: only cataloged files are allowed using FLAM-sub). |FLM0543O |*TRANSLATE-PARAMETER INVALID WITH OPEN I/O - IGNORED* The TRANSLATE parameter is ignored in connection with an OPEN I/O. Input and output records can only be translated according to the specified table with an OPEN INPUT or an OPEN OUTPUT. Processing continues without translating the data. |FLM0544O |*FORCED BY PARAMETER TO ACCESS AS UNCOMPRESSED DATA* No FLAMFILE was identified when a subsystem file was opened (message: FLM0440O with RC=10). The set subsystem parameter IG10 forces processing to continue on the basis of an uncompressed (original) file. Important: If there is no original (uncompressed) file, unpredictable errors may occur. |FLM0552D |*I/O ERROR. DDNAME = ddname. RC = no (FLAM)/(VSAM) file-name* An error was detected by the FLAM subsystem while reading or writing in the specified file file-name. FLAM return codes are specified in decimal or hexadecimal format and VSAM errors in hexadecimal format. This message may be followed by the IEC020I 001-.. message of the data management system. You can find a list of VSAM return codes in the manual entitled 'DFSMS Macro Instructions'. All FLAM return codes are listed in the section <>. Examples can be found in a following table. |FLM0553D |*NO MEMORY RECEIVED FOR I/O OPERATIONS* A subsystem module was not allocated any memory by the operating system. Consequently, no other information is available. Response: Specify a higher value for REGION in the EXEC statement. |FLM0570E |*CLOSE ERROR. DDNAME = ddname. RC = no file-name* An error was detected when an attempt was made to close the specified file. Since compressed-file records may have to be written as well when a file is closed, see also FLM0552D message. |==== [width="90%",cols="^1,12",align="center"] |==== |no: |Description (FLM0540O) |-1 |Not enough memory (license rights may have been violated) |10 |The file is not a FLAMFILE (input or I/O |11 |FLAMFILE format error |12 |Record length error |13 |File length error |14 |Checksum error |21 |Illegal matrix buffer |22 |Illegal compression method |23 |Illegal CODE in FLAMFILE |24 |Illegal BLOCKMODE |25 |Illegal record length |31 |File not assigned (DD statement missing) |33 |Invalid file type |34 |Invalid record format |35 |Invalid record length |36 |Invalid block length |40 |Cannot load module or table (TRANS parameter) |60 - 78 |FLAM syntax error |120 |Name generation in error. Generating a new file name is impossible due to the lack of digit numbers in the name (i.e. only 1 digit number is in the name, cannot generate more than 9 file names). |121 |One fragment of the split FLAMFILE is missing. |122 |Sequence check error of a serially split FLAMFILE. The fragments are not in ascending order. |123 |Fragments of the split FLAMFILE do not belong together. |124 |The FLAMFILE was split in more fragments than the actual version can handle. |130 |Security violation: FLAMFILE is not in original state during read (i.e. concatenation of FLAMFILEs, updates, inserts). Allowed to be ignored (SEC=IGNORE). |131 |Missing records in a FLAMFILE. Allowed to be ignored. |132 |A member was inserted into a group FLAMFILE. Allowed to be ignored. |133 |Sequence error: FLAMFILE records are not in ascending order. Allowed to be ignored. |134 |Security error: FLAMFILE starts without security information, but then one were found. Cannot be ignored. + Perhaps concatenation of FLAMFILEs without and with security information. |531 |Some fragments missing during decompression and parallel split |Response: |If RC = -1 and correct license, specify a higher value for REGION in the EXEC statement; if RC = 11 - 14, the file is no longer available in its original state, the data contents have been modified (possibly by a file transfer program) or another type of error has occurred and must be rectified by following the instructions provided. Some security violations are allowed to be ignored, but take care of the situation. |==== [width="50%",cols="^1,4",align="center",options="header"] |==== |no: |Description (FLM0552D) |11 |FLAMFILE format error |12 |Record length error |13 |File length error |14 |Checksum error |==== If one of the above codes is returned, the data in the FLAMFILE has been mutilated (e.g. as a result of a file transfer). [width="50%",cols="^1,4",align="center",options="header"] |==== |no: |Description |7 |Password not declared |15 |Record length greater than 32 KB |16 |Record length greater than MAXB - 4 |25 |Illegal record length |29 |Password in error |123-134 |see FLM0540O |==== Detecting a security violation the FLAM error code is reported as a hexadecimal value (nn**mmmm**). + *nn* describes, where the error was detected: [width="50%",cols="^1,4",align="center",options="header"] |==== |nn |Description |01 |Header |02 |Segment |03 |Membertrailer |04 |Filetrailer |==== *mmmm* is the error: [width="50%",cols="^1,4",align="center",options="header"] |==== |*mmmm* |Description |0001 |MAC1, the data MAC |0002 |MAC2, the chaining MAC |0003 |MAC3, the MAC over MACs |0010 |data missing |0020 |data inserted |0040 |data updated |0080 |record counter compressed data |0100 |byte counter compressed data |0200 |record counter original data |0400 |byte counter original data |0800 |chaining on FLAM encryption |==== The error codes are ORed, e.g. 030180 means: record and byte counter of the compressed data do not fit with the stored information, detected in the member trailer. It is allowed to ignore these violations (SEC=IGNORE), but take care of the situation. === Installing FLAM-sub FLAM-sub installation including installation instructions can be downloaded from our website: https://flam.de/ or by using the following link: https://flam.de/en/download/flam/latest/zSeries/zos/31bit/ There you can find downloads using SMP/E (System Modification Program/Extended) and XMIT (Transfering Load Modules between Mainframes). FLAM-sub is an interface to FLAM (z/OS), the compression and encryption utility, and may only be used in conjunction with a license for the FLAM utility. [[flam4.flamsub.examples]] === Examples ==== Input/output with cataloged files The USER.EXAMPLE.INPUT file has been created as a compressed file (FLAMFILE), for example with the FLAM utility. It must now be read by another program. The USER.EXAMPLE.OUTPUT file was created by this program previously and must now likewise be compressed. Both files are cataloged. Two different procedures are possible. a) When the file was opened, the program specified the record and block lengths in the ACB or the DCB (or in the FD section if COBOL programs are used). Solution: The SUBSYS specification in the DD statement is sufficient. ------------------------------------ //stepname EXEC PGM=program //INPUT DD DSN=USER.EXAMPLE.INPUT, // DISP=SHR, // SUBSYS=FLAM //OUTPUT DD DSN=USER.EXAMPLE.OUTPUT, // DISP=OLD, // SUBSYS=FLAM ------------------------------------ Since the "correct" values are specified in the calling program, the data records are transferred to or from the subsystem in accordance with them. b) The calling program assumes that an "original" catalog entry exists, i.e. neither a record length nor a block length is specified in the program (IEBGENER, SORT and numerous utilities function in this way). Solution: Specify additional DCB attributes in the DD statement. ---------------------------------------------------- //stepname EXEC PGM=program //INPUT DD DSN=USER.EXAMPLE.INPUT, // DISP=SHR, // DCB=(RECFM=FB,LRECL=121,BLKSIZE=3025), // SUBSYS=FLAM //OUTPUT DD DSN=USER.EXAMPLE.OUTPUT, // DISP=OLD, // DCB=(RECFM=VB,LRECL=438,BLKSIZE=4096), // SUBSYS=FLAM ---------------------------------------------------- The program is presented with the specified DCB attributes, irrespective of the actual catalog entry. The subsystem transfers records in accordance with these specifications. FLAM, on the other hand, reads or writes compressed-file records on the disk in cataloged format. The subsystem activity is confirmed by the system messages in the JCL listing. ---------------------------------------------------- IEF237I FLAM ALLOCATED TO INPUT IEF237I FLAM ALLOCATED TO OUTPUT . . IEF285I USER.EXAMPLE.INPUT SUBSYSTEM IEF285I USER.EXAMPLE.OUTPUT SUBSYSTEM ---------------------------------------------------- ==== FLAM parameters and subsystem Cf. example 1, except that a specific compression method (ADC) must be set for the output file. Solution: Enter the necessary parameters in the SUBSYS specification of the DD statement. ---------------------------------------------------- //stepname EXEC PGM=program //INPUT DD DSN=USER.EXAMPLE.INPUT, // DISP=SHR, // SUBSYS=FLAM //OUTPUT DD DSN=USER.EXAMPLE.OUTPUT, // DISP=OLD, // SUBSYS=(FLAM,'MO=ADC') ---------------------------------------------------- The FLAM utility and the ''D,SHOW(ATT)'' parameters or the I OPTION in the FLAM panels allow you to check whether the compression procedure is successful. ==== Creating a new file A file called USER.EXAMPLE.NEWDAT must be created. It is not cataloged: ---------------------------------------------------- //OUTPUT DD DSN=USER.EXAMPLE.NEWDAT, // DISP=(NEW,CATLG,), // UNIT=SYSDA,SPACE=(CYL,(24,24)), // DCB=(RECFM=FB,LRECL=121,BLKSIZE=3025) ---------------------------------------------------- Solution: Specify a second DD statement for FLAM-sub. The original file description is given a new DD name, to which a FLAM parameter in the SUBSYS specification refers. ---------------------------------------------------- //OUTPUT DD SUBSYS=(FLAM,'FLAMDD=NEWDAT'), // DCB=(RECFM=FB,LRECL=121,BLKSIZE=3025) //NEWDAT DD DSN=USER.EXAMPLE.NEWDAT, // DISP=(NEW,CATLG,), // UNIT=SYSDA,SPACE=(CYL,(6,6)), // DCB=(RECFM=FB,LRECL=121,BLKSIZE=3025) ---------------------------------------------------- The compressed file USER.EXAMPLE.NEWDAT is thus created exactly the same as the original. The advantage of this method is that any subsequent copy or transfer programs which function without the SUBSYS specification will find the same catalog entry as for the original and therefore do not need to be specially tuned. The new DD statement requires the DCB specification, since it was also mandatory in the original. Otherwise, the compressed file can be set in any way, e.g.: ---------------------------------------------------- //OUTPUT DD SUBSYS=(FLAM,'FLAMDD=NEWDAT'), // DCB=(RECFM=FB,LRECL=121,BLKSIZE=3025) //NEWDAT DD DSN=USER.EXAMPLE.NEWDAT, // DISP=(NEW,CATLG), // UNIT=SYSDA,SPACE=(CYL,(6,6)), // DCB=(RECFM=FB,LRECL=1024,BLKSIZE=26624) ---------------------------------------------------- As far as the calling program (and the subsystem) is concerned, the original records are 121 bytes long. The FLAMFILE, on the other hand, has longer records and blocks for performance reasons (this is the normal, recommended case). The file descriptions can thus be separated from one another. ==== Temporary files FLAM-sub cannot process any temporary files directly. A second DD statement is necessary for this purpose. ---------------------------------------------------- //DDNAME DD SUBSYS=(FLAM,'FLAMDD=tempfile') //tempfile DD DSN=&&file,DISP=... ---------------------------------------------------- It is not necessary to specify a file name on account of the FLAMDD specification for DDNAME. A temporary file name is generated automatically by JES. FLAM-sub branches directly to the temporary file. ==== Other subsystems Other subsystems, such as JES for SYSOUT/SYSIN, must be activated by means of a second DD statement. ---------------------------------------------------- //DDNAME DD SUBSYS=(FLAM,'FLAMDD=subfile') //subfile DD SYSOUT=A,DEST=(....),... ---------------------------------------------------- Depending on the program which is used, it may be necessary to specify DCB parameters in order to process the original file (DDNAME statement, cf. example 1). ==== Loading a VSAM-KSDS file If a KSDS file is loaded, the file is opened for writing (OUTPUT). The keys must be transferred to VSAM in ascending order (this is also checked by FLAM-sub). It is possible to load the file in I/O mode, in other words it must contain at least one record. At the same time, this leads to a deterioration in performance and an increase in the number of accesses. The file itself is normally created using IDCAMS and is already cataloged when the job is executed. The simplest method is to create a compressed KSDS file with the FLAM utility. ===== Load a real VSAM-KSDS file A ''real VSAM-KSDS file'' means, an ACB is provided to open the data set. When the file is loaded via the subsystem, the key description of the original data must be specified in the form of parameters: ---------------------------------------------------- //OUTPUT DD DSN=USER.EXAMPLE.OUTPUT, // SUBSYS=(FLAM,'OKEYPOS=21,OKEYLEN=64,MAXS=7168'), // DCB=(LRECL=512,BLKSIZE=20480) ---------------------------------------------------- The original records are up to 512 bytes long in this example, while the control interval takes up 20480 bytes and the key begins at position 21 and is 64 bytes long. The subsystem should use the full record length of the VSAM file (7168 bytes). NOTE: FLAM counts the key position starting at 1 (corresponds to RKP=0). The VSAM file itself requires the entries (see also <>) for the example: ---------------------------------------------------- KEYS(65 0) RECSZ(7168 7168) ---------------------------------------------------- Important: Even if a higher value is specified for `RECSZ`, the FLAMFILE record length is restricted to 7168 bytes by the `MAXS` parameter. NOTE: A VSAM file must be empty when it is loaded, in other words either it must not contain any records or the REUSE parameter must have been specified for IDCAMS, to allow the file to be overwritten. ===== Load a VSAM-KSDS file via a utility Most utilities (like IEBGENER or SORT) use a DCB to access a subsystem data set. FLAM subsystem then normally declares the data as physical sequential (PS) and has no knowledge about a key description. Subsystem parameter overwrite this situation and force the subsystem to recognize the output data as key sequenced records: ---------------------------------------------------- //OUTPUT DD DSN=USER.EXAMPLE.OUTPUT, // SUBSYS=(FLAM,'ODSORG=KSDS,OKEYPOS=21,OKEYLEN=64,MAXS=7168'), // DCB=(LRECL=512,BLKSIZE=20480) ---------------------------------------------------- The original records are up to 512 bytes long in this example, while the control interval takes up 20480 bytes and the key begins at position 21 and is 64 bytes long. The subsystem shall use the full record length of the VSAM-KSDS file USER.EXAMPLE.OUTPUT (7168 bytes). NOTE: FLAM counts the key position starting at 1 (corresponds to RKP=0). The VSAM file USER.EXAMPLE.OUTPUT itself requires the entries (see also <>) for the example: ---------------------------------------------------- KEYS(65 0) RECSZ(7168 7168) ---------------------------------------------------- *Important:* Even if a higher value is specified for RECSZ, the FLAMFILE record length is restricted to 7168 bytes by the MAXS parameter. NOTE: FLAM subsystem checks the ascending order of the key sequence of the output records and returns the VSAM return code, if in error. The utility normally is unaware of this situation and returns a 'WRITE ERROR' without any further information. ==== TRACE function A trace function is activated for both the input file and the output file for test purposes: ---------------------------------------------------- //INPUT DD DSN=filename1,SUBSYS=(FLAM,TRACE) //OUTPUT DD DSN=filename2, // SUBSYS=(FLAM,TRACE,'MSGDDN=FLTRACE') //* //FLPRINT DD SYSOUT=* //FLTRACE DD DSN=filename3,DISP=(NEW,CATLG), // SPACE=(TRK,(12,12),RLSE),UNIT=SYSDA ---------------------------------------------------- Different files must be specified for the trace. FLTRACE is the assignment for the OUTPUT file trace. FLPRINT is taken as the default name, since no parameter has been specified in the DD statement. The trace for INPUT is as follows (the records have been shortened here): --------------------------------------------------------------------------- FLAM - TRACE FUNCTION - COPYRIGHT 2012 BY LIMES DATENTECHNIK GMBH ACB DURING OPEN: 009E13A0 (A000004C 00000000 00000000 54000000 00000000 00000000 48900008 0000000 00000000 00000000 002C0041 009E12A8 03000000 0000521C 00000000 0C30005 00000000 00000000 00000000) FLMOPN, ON ENTRY: 1:03700C04(037008A0) 2:03700C08(8349A192) 3:0349AA90(00000001) 4:0349AA FLMOPN, ON RETURN: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:0349AA90(00000001) 4:0349AA FLMOPD, ON ENTRY: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:0349AA90(00000001) 4:03700C 5:03700CC4() 6:03700C20(00000000) 7:03700C24(00000009) 8:03700C28(00000200) 9:03700C 10:03700C5C(00000000 00000000 00000001 00000008 00000000 00000000 00000000 00000000 00000000 000000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 11:03700C34(00001800) 12:03700C38(00000000) 13:03700C3C(00000000) FLMOPD, ON RETURN: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:0349AA90(00000001) 4:03700CC 5:03700CC4() 6:03700C20(00000000) 7:03700C24(00000008) 8:03700C28(00000100) 9:03700CC 10:03700C5C(00000000 00000000 00000000 00000000 00000001 00000000 00000000 00000000 00000000 000000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 11:03700C34(00001800) 12:03700C38(00000000) 13:03700C3C(00000000) FLMOPF, ON ENTRY: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:03700C40(00000000) 4:03700C4 7:03700C50(00000001) 8:03700C54(000000FF) 9:03700C5C(00000000 00000000 00000001 00000008 00000001 00000000 00000000 00000000 00000000 000000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 10:03700C58(00000001) 11:03700CC4( ) 12:03700CC( ) FLMOPF, ON RETURN: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:03700C40(000000C8) 4:03700C4 7:03700C50(00000001) 8:03700C54(000000FF) 9:03700C5C(00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 10:03700C58(00000001) 11:03700CC4( ) 12:03700CC( ) FLMGHD, ON ENTRY: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:03700E54(00000036) 4:03700E58(FLAM.DAT.CMP ) 5:03700E4C(00000000) 6:03700E48(00000009) 7:03700E3C(00000050) 8:03700E0 9:03700C5C(00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 10:03700E40(00000C30) 11:03700E38(00000000) 12:03700E28(01010000) FLMGHD, ON RETURN: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:03700E54(0000001C) 4:03700E58(FLAM.FLAMV27C.CLIST(ORGFLAM)) 5:03700E4C(00000000) 6:03700E48(00000009) 7:03700E3C(00000050) 8:03700E0 9:03700C5C(00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000000 10:03700E40(00000C30) 11:03700E38(00000000) 12:03700E28(01010000) I/O-REQUEST: 00000000 GET ACB: 000052F0 (A000004C 000147C0 8003E170 54000000 00000000 00000000 48900008 0000000 00000000 00000000 002C0041 009E12A8 12000000 0000521C 00000000 0C30005 00000000 00000000 00000000) RPL: 00005390 (0000004C 00000000 00000000 00000000 00000000 00000000 000052F0 0000000 00045E5C 00000000 20000000 00000000 00000050 00000050 00000000 0000000 00000000 00000000 000053EC) FLMGET, ON ENTRY: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:000053C0(00000050) 4:00045E5C( 5:000053C4(00000050) FLMGET, ON RETURN: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:000053C0(00000050) 4:00045E5C(PROC 0 5:000053C4(00000050) I/O-REQUEST: 00000000 GET ACB: 000052F0 (A000004C 000147C0 8003E170 54000000 00000000 00000000 48900008 000000 00000000 00000000 002C0041 009E12A8 12000000 0000521C 00000000 0C3000 00000000 00000000 00000000) RPL: 00005390 (0000004C 00000000 00000000 00000000 00000000 00000000 000052F0 000000 00045EAC 00000000 20000000 00000000 00000050 00000050 00000000 000000 00000000 00000000 000053EC) FLMGET, ON ENTRY: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:000053C0(00000050) 4:00045EAC( 5:000053C4(00000050) FLMGET, ON RETURN: 1:03700C04(0372CD88) 2:03700C08(00000000) 3:000053C0(00000050) 4:00045EAC(CONTROL NOLIST NOSYMLIST NOCONLIST NOFLUSH NOMSG NOCAPS 5:000053C4(00000050) . . FLMCLS, ON ENTRY: 1:03700C04(0372CD88) 2:03700C08(00000002) FLMCLS, ON RETURN: 1:03700C04(FFFFFFFF) 2:03700C08(00000000) --------------------------------------------------------------------------- The names of the FLAM functions (FLMOPN, FLMGET, etc.) and their parameters are equal to those listed in the manual for FLAM V4 (Chapter 3, Interfaces). The parameters are numbered in the same way as in the manual. The next specification is the parameter address. The parameter contents are enclosed in parentheses ''()''. The control blocks ACB and RPL are specified by the subsystem interface, even if a PS file is accessed. Please refer to the IBM manuals for further details of these control blocks, e.g. ''Macro Instructions for VSAM Data Sets'', ''Data Areas'', or to the ''SYS1.AMODGEN(IFGACB)'' and ''SYS1.AMODGEN(IFGRPL)'' macros. ==== FLAMFILE split ===== Serial split To split a file in several fragments, depending on the amount of data, the serial split is used. --------------------------------------------------------------------------- //output DD DSN=FLAM.SUBDAT.A01.ADC, // SUBSYS=(FLAM,'MO=ADC,SPLITM=SER,SPLITS=200') --------------------------------------------------------------------------- The file FLAM.SUBDAT.A01.ADC is cataloged, the filename has two numeric characters (at least 1 numeric character is necessary). Compression method ADC is used for higher compression. One file must not exceed 200 MB data. When more than 200 MB data are to be written, file FLAM.SUBDAT.A01.ADC is closed and file FLAM.SUBDAT.A02.ADC is searched and allocated, or if not found in the catalog, created. With two numeric characters it is possible to generate up to 99 files. If your organisation requires special filenames or a higher flexibility for data set allocation is needed, assigning with DD-names is recommended. Only the first DD- name is passed to the subsystem. --------------------------------------------------------------------------- //output DD SUBSYS=(FLAM,'SPLITM=SER,SPLITS=200,FLAMDD=CMP01') //* //CMP01 DD DSN=filename1,DISP=OLD //CMP02 DD DSN=filename2,DISP=OLD //CMP03 DD DSN=filename3,DISP=(NEW,CATLG), // UNIT=TAPE,... --------------------------------------------------------------------------- All files must have the same record length, but the data set organization or format may differ. When more than 200 MB are to be written, the file with DD-name CMP01 is closed and CMP02 is opened. To read a split FLAMFILE it is not necessary to pass any parameter to the subsystem. The filename of the first fragment is named in the DD-statement: ------------------------------------------------------------------------ //input DD DSN=FLAM.SUBDAT.A01.ADC, // SUBSYS=FLAM ------------------------------------------------------------------------ All following files will be allocated dynamically by the subsystem. Assigning files via DD-statement is possible as well. The DD-name must have at least one numeric character, the first name is passed to the subsystem: --------------------------------------------------------------------------- //input DD SUBSYS=(FLAM,'FLAMDD=CMP01') //* //CMP01 DD DSN=filename1,DISP=SHR //CMP02 DD DSN=filename2,DISP=SHR //CMP03 DD DSN=filename3,DISP=SHR --------------------------------------------------------------------------- NOTE: The files must be assigned in the same order as they were written. The subsystem checks the sequence of the files. In case of errors, message FLM0540O and RC=122 is reported, and the data management system detects an open error. ===== Parallel split Parallel split means: up to 4 files are written or read in parallel. All files must be cataloged and assigned within JCL! Therefore, it is necessary to assign the subsystem files via the DD-statement. The DD-name must have at least one numeric character, the first name is passed to the subsystem. --------------------------------------------------------------------------- //output DD DSN=any, // SUBSYS=(FLAM,'MO=ADC,SPLITM=PAR,SPLITN=3,FLAMDD=CMP01') //* //CMP01 DD DSN=filename1,DISP=OLD //CMP02 DD DSN=filename2,DISP=OLD //CMP03 DD DSN=filename3,DISP=OLD --------------------------------------------------------------------------- All files must have the same record length, but the data set organization or format may differ. JCL required for reading a split FLAMFILE in parallel: --------------------------------------------------------------------------- //input DD DSN=any, // SUBSYS=(FLAM,'FLAMDD=CMP01') //* //CMP01 DD DSN=filename1,DISP=SHR //CMP02 DD DSN=filename2,DISP=SHR //CMP03 DD DSN=filename3,DISP=SHR --------------------------------------------------------------------------- The files may be in different order than they were written, the subsystem uses the original order automatically. NOTE: If one file is missing, message FLM0540O with RC=531 is reported by the subsystem and the data management system detects an OPEN-error. Using `DISP=NEW` in the assigned DD-statements will lead to a system error 50D, so it is really necessary to catalog all files. ==== Encryption ===== Encryption via CRYPTOKEY parameter To encrypt/decrypt the file a key is passed (`CRYPTOKEY=...`) to the subsystem. The cryptographic method is chosen by parameter `CRYPTOMODE`. Without this parameter, method FLAM is used (compatible to FLAM-sub V3): --------------------------------------------------------------------------- //DDNAME DD DSN=filename, // SUBSYS=(FLAM,'CRYPTOM=AES,CRYPTOK=OTTO') --------------------------------------------------------------------------- Using parameter PASSWORD instead of CRYPTOKEY is allowed and compatible to FLAM-sub V3. Please remember: Any apostrophes contained within the parameter string literal must be duplicated: --------------------------------------------------------------------------- //DDNAME DD DSN=filename, // SUBSYS=(FLAM,'CRYPTOM=AES,CRYPTOK=X''AE01EA''') --------------------------------------------------------------------------- ===== Encryption via KME-module A program name that supports the <> will be used as followes: --------------------------------------------------------------------------- //DDNAME DD DSN=filename, // SUBSYS=(FLAM,''CRYPTOM=AES'', // 'KME=name,KMP=C''parameter for KME''') --------------------------------------------------------------------------- This method benefits from a fully automatic encryption/decryption without any manual user action. The key used for encryption is never seen in any protocol. ===== Encryption using FKMEFILE Program FKMEFILE is an example for using the KME interface. It reads a record containing a key from a sequential file and returns it to the subsystem. A parameter ddname is needed for the file to read. The key must be the first record of the file. The syntax is as of parameter CRYPTOKEY: the key may be a string, a C'string with blanks', a A'key in ascii', or a X'hexvalue'. Trailing blanks are ignored. --------------------------------------------------------------------------- //DDNAME DD DSN=filename, // SUBSYS=(FLAM,'CRYPTOM=AES', // 'KME=FKMEFILE,KMP=C''ddname1''') //* THIS is the key-file //ddname1 DD DSN=keyfilename,DISP=SHR --------------------------------------------------------------------------- Just to test it on the fly: --------------------------------------------------------------------------- //ddname DD * C'this is my key' /* --------------------------------------------------------------------------- ==== Using zEDC ===== Automatically With `MODE=ADC` the zEDC accelerator card zEDC will be used, when it is installed. Nothing else is to be done. On decompression: If no card is found and the compression mode was zEDC a software routine decompresses the FLAMFILE. --------------------------------------------------------------------------- //DDNAME DD DSN=filename,SUBSYS=(FLAM,'MODE=ADC') --------------------------------------------------------------------------- ===== Never Setting an environment variable for the actual job, the zEDC algorithm will not be used, although the card was found. --------------------------------------------------------------------------- //DDNAME DD DSN=filename,SUBSYS=(FLAM,'MODE=ADC') //CEEOPTS DD * ENVAR(''FL_ZEDC_SUPPORT=OFF'') /* --------------------------------------------------------------------------- Same for all jobs when a system variable is defined in SYS1.PARMLIB(IEASYMxx): `*SYMDEF(&FLZEDC='OFF')*` ===== Always zEDC Setting an environment variable for the actual job, the zEDC algorithm will be used, although the card was not found. --------------------------------------------------------------------------- //DDNAME DD DSN=filename,SUBSYS=(FLAM,'MODE=ADC') //CEEOPTS DD * ENVAR(''FL_ZEDC_SUPPORT=ON'') /* --------------------------------------------------------------------------- Same for all jobs when a system variable is defined in SYS1.PARMLIB(IEASYMxx): `*SYMDEF(&FLZEDC='ON')*` ... print CLE_DOCTYP_CHAPTER == User I/O Interface == Appendix B. User I/O Interface The user interface for I/O allows FLAM to use user-defined file access methods instead of the operating system defined ones. This is possible when using the FLAM utility for the original file and when using the *FLAM record interface* for both original file and compressed FLAM file. One possibility is to pass the compressed records immediately to post-processing routines without creating a compressed file. During decompression, the compressed records may be received from pre-processing step instead of reading them from a file. A direct application could be the integration of FLAM with a file transfer application, avoiding the creation of intermediate files. Generally this interface allows intercepting all input and output data of both FLAM and FLAMUP. This allows the user to adapt FLAM easily to specific access methods. === Using user I/O interface The user can replace the FLAM file accesses methods with custom file access methods by using the User I/O interface. These access methods are used by the FLAM utility for accessing the original file, the compressed file and the target file. In the record level interface, however, only access routines for the compressed file exist. The use of user-defined access routines can be specified for each file separately via the parameters `DEVICE=USER` or `IDEVCE`, `ODEVICE`. However, the user-provided I/O routines must be linked to the FLAM utility or to the FLAM record level interface before. The following routines must be provided by the user: * Open and Close (USROPN, USRCLS) * Sequential Read and Write (USRPUT, USRGET) * Optionally random access Read and Write (USRPKY, USRGKY) * Delete and Positioning (USRDEL, USRPOS). ==== How does it work [width="100%",cols="h,1*",align="right"] |==== |`USROPN` |For each allocated file this function is called once and only once. A working area of 1024 bytes is passed to the routine. This working area acts as file-specific memory and is passed from function to function until USRCLS is called. Files are identified via symbolic file names. The access mode is specified in parameter `OPENMODE`: INPUT, OUTPUT, INOUT, OUTIN. File format and file attributes are specified in the parameters `RECFORM`, `RECSIZE`, `BLKSIZE`, etc. These settings may be adapted to special requirements. Via pre-defined and user-defined return codes, the successful execution as well as special status information and errors can be reported to the higher layers. |`USRCLS` |This function is called to close the file. The 1024 bytes working area reserved for this file is de-allocated after returning control to FLAM. |`USRGET` |This function is called to retrieve the next record. The maximum amount of characters that may be returned is specified in parameter `BUFLEN`. If it is necessary to truncate the record, this must be signalled via a return code. Also, the end of file condition must be signalled via a return code. For each record returned, the record length must be returned as well (also for records with fixed record length!). |`USRPUT` |This function is called to write a record. If it is not possible to write the record in full length, the return code record truncated must be reported to the higher layers. Another possibility is to fill the record with the padding character (`PADCHAR`) as specified in `USROPN` and to return the corresponding return code. |`USRPOS` |This function is called to move the current read or write pointer. Relative positioning (forward and backward) from the current position as well as absolute positioning from file start or file end is possible. |`USRGKY` |With this function, a record with a specified key is read. The corresponding key is contained in the record area at the position and in the length as specified with the parameter `KEYDESC` during `USROPN`. Reading by key also sets the current read pointer for following `USRGET` function calls. If no record is found, this must be signalled via a return code. Afterwards it is possible to read the record with the nearest higher key with function `USRGET`. |`USRPKY` |With this function, a record with a specified key is updated or inserted. If the record has the same key as the record last read, the old record is replaced with the new record. Otherwise, the record is inserted. If this is not possible (duplicate keys may be forbidden), this must be signalled via an appropriate return code. The writing of records with a key also updates the position of the current write pointer. |`USRDEL` |With this function the last read record is deleted. |==== ==== Compression (schematic) image::pics/flam4_user_io_compression.png[User IO compression,align="center"] *Parameter for FLAM or FLAMUP:* COMPRESS, IDEVICE = USER, DEVICE = USER ==== Decompression (schematic) image::pics/flam4_user_io_decompression.png[User IO decompression,align="center"] *Parameter for FLAM or FLAMUP:* DECOMPRESS, ODEVICE = USER, DEVICE = USER === Examples ==== ASSEMBLER example This example sets up a DUMMY device that returns immediately the return code END OF FILE during read. During write, all records are accepted and OK is always returned without actually writing the records to a storage medium. The functions `USRGKY` and `USRPOS` always end with the return code INVALID KEY or INVALID POSITION. The function `USRDEL` always returns the code INVALID FUNCTION. This functionality is equivalent to a file assignment of DUMMY. By filling in appropriate code into the sequences marked with three periods, this routine can be used as a template for specific user written I/O routines. ------------------------------------------------------------------------ FLAMUIO START TITLE 'FLAMUIO: USER-I/O-MODULE FOR FLAM' *********************************************************************** * NAME: FLAMUIO * * FUNCTION: * * DUMMY MODULE AS EXAMPLE FOR AN USER-IO-MODULE * * INTERFACES: * * USROPN OPEN DATA SET * * USRCLS CLOSE DATA SET * * USRGET READ SEQUENTIAL * * USRGKY READ WITH KEY * * USRPUT WRITE SEQUENTIAL * * USRPKY WRITE WITH KEY * * USRDEL DELETE ACTUAL RECORD * * USRPOS POSITION IN DATA SET * * NOTES: * * ALL FUNCTIONS ARE REENTRANT. * * WE NEED NO RUNTIME SYSTEM. * * INDEPENDENT FROM ANY /370-SYSTEM. * *********************************************************************** * * ADDRESSING -/ RESIDENCY MODE * FLAMUIO AMODE ANY FLAMUIO RMODE ANY * * RETURN CODES * OK EQU 0 NO ERROR * EQU -1 REQM-ERROR; INVALID HANDLE * OR INVALID FUNCTION CUT EQU 1 RECORD TRUNCATED EOF EQU 2 END OF DATA SET GAP EQU 3 GAP IN RELATIVE DATA SET FILL EQU 4 RECORD PADDED INVKEY EQU 5 KEY NOT FOUND RCEMPTY EQU 30 INPUT DATA SET EMPTY RCNEXIST EQU 31 DATA SET DOES NOT EXIST RCOPENMO EQU 32 INVALID OPEN MODE RCFCBTYP EQU 33 INVALID FILE FORMAT RCRECFOR EQU 34 INVALID RECORD FORMAT RCRECSIZ EQU 35 INVALID RECORD LENGTH RCBLKSIZ EQU 36 INVALID BLOCK SIZE RCKEYPOS EQU 37 INVALID KEY POSITION RCKEYLEN EQU 38 INVALID KEY LENGTH RCDSN EQU 39 INVALID DATA SET NAME * EQU X'0FXXXXXX' OTHER ERRORS * *********************************** * REGISTER EQUATES * *********************************** R0 EQU 0 R1 EQU 1 R2 EQU 2 R3 EQU 3 R4 EQU 4 R5 EQU 5 R6 EQU 6 R7 EQU 7 R8 EQU 8 R9 EQU 9 R10 EQU 10 R11 EQU 11 R12 EQU 12 R13 EQU 13 R14 EQU 14 R15 EQU 15 * DC C'*** MODULE FLAMUIO. ' DC C'USER-I/O-MODULE FOR FLAM ' DC C'TIME - DATE ASSEMBLED: ' DC C'&SYSDATE - &SYSTIME ***' TITLE 'USROPN' USROPN DS 0D ENTRY USROPN USING USROPN,R10 ************************************************************************* * NAME: USROPN * * FUNCTION: * * OPEN DATA SET * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA, INITIALIZED WITH X'00'. * * THIS AREA IS CONNECTED TO THIS DATA SET. * * USABLE AS WORK AREA DURING THE DIFFERENT CALLS * * FOR THE ACTUAL DATA SET. * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = 30 INPUT DATA SET IS EMPTY * * = 31 DATA SET NOT CONNECTED OR DOES NOT EXIST * * = 32 ILLEGAL OPEN MODE * * = 33 ILLEGAL DSORG * * = 34 ILLEGAL RECORD FORMAT * * = 35 ILLEGAL RECORD LENGTH * * = 36 ILLEGAL BLOCK SIZE * * = 37 ILLEGAL KEY POSITION * * = 38 ILLEGAL KEY LENGTH * * = -1 UNSUPPORTED FUNCTION; GETMAIN ERROR * * = X'0FXXXXXX' OTHER ERROR CODE * * 3 -> OPENMODE F OPEN MODE * * = 0 INPUT (SEQUENTIAL READ) * * (DATA SET MUST EXIST) * * = 1 OUTPUT (SEQUENTIAL WRITE) * * (DATA SET WILL BE OVERWRITTEN) * * = 2 INOUT (READ OR WRITE SEQUENTIAL OR WITH KEY) * * (DATA SET MUST EXIST) * * = 3 OUTIN (WRITE OR READ SEQUENTIAL OR WITH KEY) * * (DATA SET WILL BE OVERWRITTEN) * * 4 -> DDNAME CL8 DD-NAME * * 5 <-> DSORG F DATA SET ORGANIZATION * * = 0; 8; 16 ... SEQUENTIAL * * = 1; 9; 17 ... INDEX SEQUENTAIL * * = 2; 10; 18 ... RELATIVE * * = 3; 11; 19 ... DIRECT * * = 4; 12; 20 ... UNSTRUCTURED * * = 5; 13; 21 ... LIBRARY * * 6 <-> RECFORM F RECORD FORMAT * * = 0; 8; 16 ... VARIABELE (V) * * 8 = BLOCKED 16 = BLOCKED/SPAN NED * * = 1; 9; 17 ... FIX (F) * * 9 = BLOCKED 17 = BLOCKED/SPANNED * * = 2; 10; 18 ... UNDEFINED (U) * * * * = 3; 11; 19 ... STREAM (S) * * 11 = DELIMITER 19 RECORD DESCRIPTOR WORD * * 7 <-> RECSIZE F DATA LENGTH (WITHOUT DELIMTER OR RDW) * * = 0 - 32767 * * RECFORM = V: MAX. RECORD LENGTH OR 0 * * RECFORM = F: RECORD LENGTH * * RECFORM = U: MAX. RECORD LENGTH OR 0 * * RECFORM = S: LENGTH DELIMITER OR RDW * * 8 <-> BLKSIZE F BLOCK SIZE * * = 0 UNBLOCKED * * 9 <-> KEYDESC STRUCT KEY DESCRIPTION * * * * KEYFLAGS F OPTIONS * * = 0 NO DUPLICATE KEYS * * = 1 DUPLICATES ALLOWED * * KEYPARTS F NUMBER OF KEY PARTS * * = 0 - 8 * * KEYPOS1 F 1. BYTE OF 1. KEYPART * * = 1 - 32766 * * KEYLEN1 F LENGTH OF 1. KEYPART * * = 1 - 255 * * KEYTYPE1 F DATA TYPE OF 1. KEYPART * * = 0 PRINTABLE CHARACTER * * = 1 BINARY * * . * * . * * . * * KEYPOS8 F 1. BYTE OF 8. KEYPART * * = 1 - 32766 * * KEYLEN8 F LENGTH OF 8. KEYPART * * = 1 - 255 * * KEYTYPE8 F DATA TYPE OF 8. KEYPART * * = 0 PRINTABLE CHARACTER * * = 1 BINARY * * * * 10 <-> DEVICE F DEVICE TYPE * * = 7; 15; 23 USER DEFINED * * 11 <-> RECDELIM XL RECORD DELIMITER * * 12 -> PADCHAR XL1 PADDING CHARACTER * * 13 <-> PRCTRL F PRINTER CONTROL CHARACTER * * = 0 NONE * * = 1 ASA-CHARACTER * * = 2 MACHINE SPECIFIC CHARACTER * * 14 -> CLOSDISP F CLOSE PROCESSING * * = 0 REWIND * * = 1 UNLOAD * * = 2 RETAIN / LEAVE * * 15 -> ACCESS F ACCESS METHOD * * = 0 LOGICAL (RECORD BY RECORD) * * = 1 PHYSICAL * * 16 <-> DSNLEN F LENGTH OF DATA SET NAME OR BUFFER FOR NAME * * 17 <-> DSN CL DATA SET NAME * * (DATA SET NAME SHOULD BE RETURNED, IF 1. BYTE * * OF GIVEN NAME IS C' ' OR A DIFFERENT DATA SET * * IS ALLOCATED). * ************************************************************************ * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R2,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * OPEN DATA SET * * . * . * . * * SET RETURN CODE TO 'NO ERROR' * LA R0,OK ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREA REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** * LTORG DROP R10 TITLE 'USRCLS' USRCLS DS 0D ENTRY USRCLS USING USRCLS,R10 *********************************************************************** * NAME: USRCLS * * FUNCTION: * * CLOSE DATA SET * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * * OR DMS-ERROR CODE * *********************************************************************** * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R2,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * CLOSE DATA SET * * . * . * . * * SET RETURN CODE TO 'NO ERROR' * LA R0,OK ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREA REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** * LTORG DROP R10 TITLE 'USRGET' USRGET DS 0D ENTRY USRGET USING USRGET,R10 *********************************************************************** * NAME: USRGET * * FUNCTION: * * READ A RECORD (SEQUENTIAL) * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = 1 RECORD TRUNCATED * * = 2 END OF FILE * * = 3 EMPTY SLOT IN RELATIVE RECORD DATA SET * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * * 3 <- RECLEN F RECORD LENGTH IN BYTES * * 4 <- RECORD XL RECORD * * 5 -> BUFLEN F LENGTH OF RECORD BUFFER IN BYTES * *********************************************************************** * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R5,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * READ A RECORD * * . * . * . * * HERE: RETURN CODE 'END OF FILE' * LA R0,EOF ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREA REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** * LTORG DROP R10 TITLE 'USRGKY' USRGKY DS 0D ENTRY USRGKY USING USRGKY,R10 *********************************************************************** * NAME: USRGKY * * FUNCTION: * * READ RECORD WITH GIVEN RECORD-KEY * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = 1 RECORD TRUNCATED * * = 2 END OF FILE * * = 5 KEY NOT FOUND * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * * 3 <- RECLEN F RECORD LENGTH IN BYTES * * 4 <- RECORD XL RECORD WITH SEARCH KEY * * 5 -> BUFLEN F LENGTH OF RECORD BUFFER IN BYTES * *********************************************************************** * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R5,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * READ RECORD * * . * . * . * * HERE: RETURN CODE 'RECORD NOT FOUND' * LA R0,INVKEY ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREAS REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** * LTORG DROP R10 TITLE 'USRPUT' USRPUT DS 0D ENTRY USRPUT USING USRPUT,R10 *********************************************************************** * NAME: USRPUT * * FUNCTION: * * WRITE A RECORD (SEQUENTIAL) * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = 1 RECORD TRUNCATED * * = 4 RECORD FILLED WITH PADDING CHARACTER * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * * 3 -> RECLEN F RECORD LENGTH IN BYTES * * 4 -> RECORD XL RECORD * *********************************************************************** * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R4,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * WRITE THE RECORD * * . * . * . * * RETURN CODE: 'NO ERROR' * LA R0,OK ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREA REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * ********************************************************************** * LTORG DROP R10 TITLE 'USRPKY' USRPKY DS 0D ENTRY USRPKY USING USRPKY,R10 *********************************************************************** * NAME: USRPKY * * FUNCTION: * * WRITE A RECORD WITH GIVEN KEY (INDEX SEQUENTIAL) * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = 1 RECORD TRUNCATED * * = 4 RECORD FILLED WITH PADDING CHARACTER * * = 5 INVALID KEY * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * * 3 -> RECLEN F RECORD LENGTH IN BYTES * * 4 -> RECORD XL RECORD * * NOTES: * * IF THE GIVEN KEY IS THE SAME LIKE THE LAST KEY READ * * THE RECORD SHALL BE OVERWRITTEN (REWRITE). * * OTHERWISE THE RECORD SHALL BE INSERTED. * *********************************************************************** * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R5,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * WRITE THE RECORD * * . * . * . * * RETURN CODE: 'NO ERROR' * LA R0,OK ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREA REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** * LTORG DROP R10 TITLE 'USRDEL' USRDEL DS 0D ENTRY USRDEL USING USRDEL,R10 *********************************************************************** * NAME: USRDEL * * FUNCTION: * * DELETE ACTUAL RECORD * * PARAMETER: * * 1 <-> WORKAREA 256F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 NO ERROR * * = 5 NO ACTUAL RECORD READ * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * *********************************************************************** * * SAVE REGISTER AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R2,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * DELETE RECORD * * . * . * . * * HERE: RETURN CODE 'NO ACTUAL = RECORD READ' * LA R0,INVKEY ST R0,0(R2) * * RETURN TO CALLER * LM R14,R12,12(R13) BR R14 * * RELEASE WORK AREA REGISTER * DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** * LTORG DROP R10 TITLE 'USRPOS' USRPOS DS 0D ENTRY USRPOS USING USRPOS,R10 *********************************************************************** * NAME: USRPOS * * FUNCTION: * * POSITION IN DATA SET * * PARAMETER: * * 1 <-> WORKAREA F WORK AREA * * 2 <- RETCO F RETURN CODE * * = 0 OK * * = 5 ILLEGAL POSITION * * = -1 UNSUPPORTED FUNCTION * * = X'0FXXXXXX' ELSE * * 3 -> POSITION F RELATIVE POSITION * * = 0 NO NEW POSITION * * = - MAXINT TO BEGINNING OF DATA SET * * ( -2147483648 OR X'80000000') * * = + MAXINT TO END OF DATA SET * * ( +2147483647 OR X'7FFFFFFF') * * = - N N RECORDS BACKWARD * * = + N N RECORDS FORWARD * * NOTES: * * YOU CAN CREATE EMPTY SLOTS (GAPS) USING FORWARD POSITIONING * * IN A RELATIVE DATA SET IN OUTPUT MODE. * *********************************************************************** * * SAVE REGISTERS AND LOAD PROGRAM REGISTER * STM R14,R12,12(R13) LR R10,R15 * * LOAD PARAMETER * LM R1,R5,0(R1) * * ADDRESS WORK AREA * LR R12,R1 USING WORKAREA,R12 * * POSITION RECORD * * . * . * . * * HERE: RETURN CODE -1 UNSUPPORTED FUNCTION * LA R0,0 BCTR R0,0 ST R0,0(R2) * * RETURN * LM R14,R15,12(R13) BR R14 * * RELEASE WORK AREA REGISTER DROP R12 * *********************************************************************** * LOCAL CONSTANTS * *********************************************************************** LTORG DROP R10 TITLE 'FLAMUIO: DUMMY SECTIONS' *********************************************************************** * DUMMY SECTIONS * *********************************************************************** * WORKAREA DSECT *********************************************************************** * WORK AREA ON DOUBLE WORD BOUNDARY * *********************************************************************** * DS XL1024 * LWORK EQU *-WORKAREA LENGTH; MAXIMAL 1024 BYTES EJECT *********************************************************************** * DUMMY SECTION * *********************************************************************** OPNPAR DSECT *********************************************************************** * PARAMETERLIST FOR USROPN * NOTE: ADDRESSES ARE GIVEN, NOT THE VALUES. *********************************************************************** ADWORKA DS A WORK AREA ADRETCO DS A RETCO ADOPMO DS A OPENMODE ADDDN DS A DDNAME ADDSORG DS A DSORG ADRECFO DS A RECFORM ADRECSI DS A RECSIZE ADBLKSI DS A BLKSIZE ADKEYDE DS A KEYDESC ADEVICE DS A DEVICE ADRECDE DS A RECDELIM ADPADC DS A PADCHAR ADPRCTL DS A PRCNTRL ADCLOSDI DS A CLOSDISP ADACC DS A ACCESS ADDSNLEN DS A LENGTH DSN ADDSN DS A DATA SET NAME EJECT *********************************************************************** * DUMMY SECTION * *********************************************************************** KEYDESC DSECT KEY DESCRIPTION KEYFLAGS DS F KEYFLAGS KEYPARTS DS F NUMBER OF KEYPARTS KEYPOS1 DS F KEYPOSITION OF 1. KEYPART KEYLEN1 DS F LENGTH OF 1. KEYPART KEYTYPE1 DS F DATATYPE OF 1. KEYPART KEYPOS2 DS F KEYLEN2 DS F KEYTYPE2 DS F KEYPOS3 DS F KEYLEN3 DS F KEYTYPE3 DS F KEYPOS4 DS F KEYLEN4 DS F KEYTYPE4 DS F KEYPOS5 DS F KEYLEN5 DS F KEYTYPE5 DS F KEYPOS6 DS F KEYLEN6 DS F KEYTYPE6 DS F KEYPOS7 DS F KEYLEN7 DS F KEYTYPE7 DS F KEYPOS8 DS F KEYPOSITION OF 8. KEYPART KEYLEN8 DS F LENGTH OF 8. KEYPART KEYTYPE8 DS F DATATYPE OF 8. KEYPART END ------------------------------------------------------------------------ ==== COBOL example The user I/O can also be implemented in COBOL or in another higher programming language. The following example implements two different functions that can be selected via the symbolic file name (LINKNAME or DDNAME). Using the DD-name DATBASE, ten records can be read with the content: "THIS IS A DATA BASE RECORD FROM THE USER I/O" Then return code END OF FILE is returned. Using DD-name "USER...", 20 records can be read with the content: "THIS IS A USER RECORD FROM THE USER I/O" Then return code END OF FILE is returned. In both cases, the call protocols are written to the terminal. This allows to observe precisely the sequence of the different calls. ------------------------------------------------------------------------ IDENTIFICATION DIVISION. PROGRAM-ID. USERIO. AUTHOR. LIMES DATENTECHNIK GMBH. * * USERIO IS AN EXAMPLE FOR AN USER I/O MODULE TO CONNECT * TO FLAM. * * THE PROGRAM IS WRITTEN TO SUPPORT 2 DIFFERENT DATA SETS IN * THE SAME MODULE DISTINGUISHED BY THE DD-NAME (DATBASE OR * USER....) * ENVIRONMENT DIVISION. * CONFIGURATION SECTION. * SPECIAL-NAMES. SYSOUT IS OUT-PUT. * DATA DIVISION. * WORKING-STORAGE SECTION. * 77 ALL-OK PIC S9(8) COMP VALUE 0. 77 FUNCTION-ERR PIC S9(8) COMP VALUE -1. 77 REC-TRUNCATED PIC S9(8) COMP VALUE 1. 77 END-OF-FILE PIC S9(8) COMP VALUE 2. 77 REC-NOT-FOUND PIC S9(8) COMP VALUE 5. 77 NEW-HEADER PIC S9(8) COMP VALUE 6. 77 FILE-EMPTY PIC S9(8) COMP VALUE 30. 77 FILE-NOT-EXIST PIC S9(8) COMP VALUE 31. 77 OPEN-MODE-ERR PIC S9(8) COMP VALUE 32. 77 FILE-NAME-ERR PIC S9(8) COMP VALUE 39. * 77 EXAMPLE-USER-RECORD PIC X(72) VALUE "THIS IS A USER RECORD FROM THE USER I/O". 77 EXAMPLE-DATBAS-RECORD PIC X(72) VALUE "THIS IS A DATA-BASE RECORD FROM THE USER I/O". 77 RECLEN PIC S9(8) COMP VALUE 80. **************************************************** / LINKAGE SECTION. * 01 USER-WORK. 03 W-DDNAME PIC X(8). 03 W-COUNTER PIC S9(7) COMP-3. 03 W-ELSE PIC X(1012). 01 RETCO PIC S9(8) COMP. 01 OPENMODE PIC S9(8) COMP. 88 OP-INPUT VALUE 0. 88 OP-OUTPUT VALUE 1. 01 DDNAME. 03 DDNAME-1 PIC X(4). 03 FILLER PIC X(4). * * IN THIS EXAMPLE WE DO NOT NEED THE FOLLOWING PARAMETERS * *01 DSORG PIC S9(8) COMP. *01 RECFORM PIC S9(8) COMP. *01 RECSIZE PIC S9(8) COMP. *01 BLKSIZE PIC S9(8) COMP. *01 KEYDESC. * 03 KEYFLAGS PIC S9(8) COMP. * 03 KEYPARTS PIC S9(8) COMP. * 03 KEYENTRY OCCURS 8 TIMES. * 05 KEYPOS PIC S9(8) COMP. * 05 KEYLEN PIC S9(8) COMP. * 05 KEYTYPE PIC S9(8) COMP. *01 DEVICE PIC S9(8) COMP. *01 RECDELIM PIC X(4). *01 PADCHAR PIC X. *01 PRCTRL PIC S9(8) COMP. *01 CLOSMODE PIC S9(8) COMP. *01 ACCESS PIC S9(8) COMP. *01 DSNLEN PIC S9(8) COMP. *01 DATA-SET-NAME PIC X(44). * * USED FOR READING * 01 DATALEN PIC S9(8) COMP. 01 DATA-AREA. 03 DATA-1 PIC X(72). 03 DATA-2 PIC X(8). 01 BUFFLEN PIC S9(8) COMP. * / PROCEDURE DIVISION. * USROPN-MAIN SECTION. * * OPEN ROUTINE * USROPN-MAIN-1. ENTRY "USROPN" USING USER-WORK, RETCO, OPENMODE, DDNAME. * * IN THIS EXAMPLE WE DO NOT USE THE OTHER PARAMETERS, SO IT IS * NOT NECESSARY TO MENTION THEM. * FLAM STANDARDS ARE USED: * SEQUENTIAL, * VARIABLE LENGTH UP TO 32752 BYTE (BUT WE ONLY USE 80 BYTE) * * * WE ONLY SUPPORT OPEN INPUT IN THIS EXAMPLE, * CHECK THE OPEN MODE * IF OP-INPUT THEN NEXT SENTENCE ELSE MOVE OPEN-MODE-ERR TO RETCO DISPLAY "USER I/O CANNOT WRITE TO " DDNAME UPON OUT-PUT GO TO USROPN-MAIN-99. * * FOR FURTHER USE, WE STORE THE DD-NAME IN THE * GIVEN WORKAREA * MOVE DDNAME TO W-DDNAME. * * WE SUPPORT DIFFERENT DATA SETS, * CHECK FOR DDNAME "DATBASE", OR THE FIRST 4 BYTE FOR "USER" * IF DDNAME = "DATBASE" THEN PERFORM OPN-DATBASE ELSE IF DDNAME-1 = "USER" THEN PERFORM OPN-USER ELSE MOVE FILE-NAME-ERR TO RETCO DISPLAY "USER I/O DOES NOT SUPPORT " DDNAME UPON OUT-PUT. USROPN-MAIN-99. * * GO BACK TO FLAM * GO BACK. / OPN-DATBASE SECTION. * * OPEN-ROUTINE FOR A DATA BASE * OPN-DATBASE-1. * * HERE YOU HAVE TO PROCESS THE OPEN, * * * INITIALISE COUNTER-FIELD IN WORK AREA * MOVE ZERO TO W-COUNTER. * * WE ONLY DISPLAY A MESSAGE * DISPLAY "USER I/O: OPEN FOR DATABASE IS DONE" UPON OUT-PUT. OPN-DATBASE-90. * * SET THE RETURN CODE * MOVE ALL-OK TO RETCO. OPN-DATBASE-99. EXIT. / OPN-USER SECTION. * * OPEN ROUTINE FOR THE OTHER EXAMPLE * OPN-USER-1. * * HERE YOU HAVE TO PROCESS THE OPEN, * * INITIALISE COUNTER-FIELD IN WORK AREA * MOVE ZERO TO W-COUNTER. * * WE ONLY DISPLAY A MESSAGE * DISPLAY "USER I/O: OPEN FOR " DDNAME " IS DONE" UPON OUT-PUT. OPN-USER-90. * * SET THE RETURN CODE * MOVE ALL-OK TO RETCO. OPN-USER-99. EXIT. / USRCLS-MAIN SECTION. * * CLOSE ROUTINE * USRCLS-MAIN-1. ENTRY "USRCLS" USING USER-WORK, RETCO. * * WE SUPPORT DIFFERENT DATA SETS, * CHECK FOR DDNAME * IF W-DDNAME = "DATBASE" THEN PERFORM CLS-DATBASE ELSE PERFORM CLS-USER. USRCLS-MAIN-99. * * GO BACK TO FLAM * GO BACK. / CLS-USER SECTION. * * CLOSE ROUTINE FOR THE OTHER EXAMPLE * CLS-USER-1. * * HERE YOU HAVE TO PROCESS THE CLOSE, * * WE ONLY DISPLAY A MESSAGE * DISPLAY "USER I/O: CLOSE FOR " W-DDNAME " IS DONE" UPON OUT-PUT. CLS-USER-90. * * SET THE RETURN CODE * MOVE ALL-OK TO RETCO. CLS-USER-99. EXIT. / CLS-DATBASE SECTION. * * CLOSE ROUTINE FOR A DATA BASE * CLS-DATBASE-1. * * HERE YOU HAVE TO PROCESS THE CLOSE, * * WE ONLY DISPLAY A MESSAGE * DISPLAY "USER I/O: CLOSE FOR DATA BASE IS DONE" UPON OUT-PUT. CLS-DATBASE-90. * * SET THE RETURN CODE * MOVE ALL-OK TO RETCO. CLS-DATBASE-99. EXIT. / USRGET-MAIN SECTION. * * ROUTINE FOR READING RECORDS * USRGET-MAIN-1. ENTRY "USRGET" USING USER-WORK, RETCO, DATALEN, DATA-AREA, BUFFLEN. * * WE SUPPORT DIFFERENT DATA SETS, * CHECK FOR DDNAME * IF W-DDNAME = "DATBASE" THEN PERFORM GET-DATBASE ELSE PERFORM GET-USER. USRGET-MAIN-99. * * GO BACK TO FLAM * GO BACK. / GET-DATBASE SECTION. * * GET-ROUTINE FOR A DATA BASE * GET-DATBASE-1. * * WE RETURN ALWAYS THE SAME RECORD * * AFTER THE 10. RECORD WE FINISH (EOF) * IF W-COUNTER +10 THEN MOVE EXAMPLE-DATBAS-RECORD TO DATA-1 MOVE W-DDNAME TO DATA-2 MOVE RECLEN TO DATALEN ADD +1 TO W-COUNTER MOVE ALL-OK TO RETCO ELSE MOVE ZERO TO DATALEN MOVE END-OF-FILE TO RETCO. GET-DATBASE-99. EXIT. / GET-USER SECTION. * * GET ROUTINE FOR THE OTHER EXAMPLE, * GET-USER-1. * * WE RETURN ALWAYS THE SAME RECORD, * * AFTER THE 20. RECORD WE FINISH (EOF) * IF W-COUNTER +20 THEN MOVE EXAMPLE-USER-RECORD TO DATA-1 MOVE W-DDNAME TO DATA-2 MOVE RECLEN TO DATALEN ADD +1 TO W-COUNTER MOVE ALL-OK TO RETCO ELSE MOVE ZERO TO DATALEN MOVE END-OF-FILE TO RETCO. GET-USER-99. EXIT. ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER == User Exits == Appendix C. User Exits To be even more flexible, FLAM provides user exits which can be used for pre- and post-processing of uncompressed records as well as FLAMFILE records. User exits are needed for inserting, deleting and editing records or fields before or after a compression or decompression procedure. FLAMFILEs can also be modified to make it personal and incompatible by swapping bytes via FLAM user exits. The effect is basically the same like enciphering (by symmetrical exit modules). Code conversion is controlled via standard code tables or user defined code tables. For special cases where a 1:1 code conversion is not possible for all characters, user exits can be used, too. === Specific Addressing Modes User exits can be written for any addressing mode (`AMODE=ANY`, `AMODE=31`, `AMODE=24`, no mode specified). The addressing mode only has to be taken into account if FLAM is loaded with `AMODE=31` and, for some reason, the user exit is only able to run with `AMODE=24`. Only in this case, the addressing mode must be switched in the user exit itself. It must be noted that the save area, the return address, the parameter list and the parameters can be addressed only in `AMODE=31`. The addressing mode used by FLAM is stored in the most significant bit of register R14 and can be looked up there. In all other cases, the addressing mode is already set correctly and it is switched over again by FLAM after the return, if this is necessary. It is irrelevant whether the return is executed with a BR14 or a BSM 0,14 (Branch and Set Mode is a return branching instruction that set the addressing mode). === Original Data Input EXK10 This user exit serves as an interface between the data record that is transferred to FLAM for compression. Special processing can be defined for: * start of file * record level * end of file Records can be passed on, modified, deleted or inserted. This exit can be used to modify records in a structure dependent way. EXK10 is only available in FLAM and FLAMUP and corresponds to EXD10 during decompression. ==== Principle of Operation EXK10 In this user exit, the original records (which shall be compressed) are passed to a user module immediately after they are read from the input file. This user exit can be used with the FLAM utility or with the subprogram FLAMUP. Records can be accepted, modified, inserted and deleted by this exit. The exit is activated via the parameter `EXK10=`. The user exit module must be contained in the library that has been assigned with the STEPLIB command. [width="70%",cols="^2,^3",align="center"] |==== |*user module name* |free choice (max. 8 characters) |*Register usage* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="90%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |First call for file (after OPEN) | | 4 |Record read and passed | | 8 |Last call for file (before CLOSE) |**2. RETCO** |*F* |Return code | | 0 |Accept record / no error | | 4 |Do not accept record | | 8 |Insert record | | 12 |Enforce end of compression | | 16 |Error in user exit; abnormal termination |**3. RECPTR** |*A* |Record pointer |**4. RECLEN** |*F* |Record length (maximum 32760) |**5. EXWORK** |*256F* |During the first call, the working storage area contains the symbolic file name of the original file within the first 8 characters. The rest of the area is padded with x00. This area can be used by the user exit module for any purpose. With each call this working storage area is made available again with the old content. NOTE: If a record shall be extended or inserted, the necessary working storage area must be provided by the user exit module. Return code 12 is only necessary if the compression shall be finished before the end of the input file is reached. For function code 0 and 8, no record is passed to the module. For function code 8, it is allowed to insert a record with return code 8. For return code 8, the record provided by the user exit module is processed. Then the user exit module is called again for the old record from the input file. The following table shows the *valid function and return codes:* |==== [width="45%",cols="^2,^1,^1,^1,^1",align="center"] |==== |Function code | | 0 | 4 | 8 |Return code | 0 | x | x | x | | 4 | | x | | | 8 | | x | x | | 12 | | x | | | 16 | x | x | x |==== === Compressed Data Output EXK20 This user exit treats compressed data before it is written to a FLAMFILE. Special processing can be defined for: * start of file * record level * end of file This exit can be used to modify records in a structure independent way. With this exit it is possible to modify the data with an own encryption routine or a special code translation can be applied if a non-transparent file transmission method shall be used. It is possible to insert own records in front of the compressed records, for example archiving control records or origin information. Another possibility is the extension of records to append specific revision information. EXK20 is available in FLAM, FLAMUP and FLAMREC and corresponds to EXD20 during decompression. ==== Principle of Operation EXK20 In this user exit, the compressed records are passed to a user module immediately before they are written to the compressed file. This user exit can be used with the FLAM utility or with the subprogram FLAMUP. This user exit can accept, modify, insert and delete records. The exit is activated via the parameter `EXK20=`. The user exit module must be contained in the library that has been assigned with the STEPLIB command. [width="70%",cols="^2,^3",align="center"] |==== |*user module name* |free choice (max. 8 characters) |*Register usage* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="90%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |First call for file (after OPEN) | | 4 |Record passed | | 8 |Last call for file (before CLOSE) |**2. RETCO** |*F* |Return code | | 0 |Accept record / no error | | 4 |Do not accept record | | 8 |Insert record | | 12 |Enforce end of compression | | 16 |Error in exit; abnormal termination |**3. RECPTR** |*A* |Record pointer |**4. RECLEN** |*F* |Record length (maximum 32760) |**5. EXWORK** |*256F* |During the first call the working storage area contains the symbolic file name of the original file within the first 8 characters. The rest of the area is padded with x00. This area can be used by the user exit module for any purpose. With each call, this working storage area is made available again with the old content. NOTE: If a record shall be extended or inserted, the necessary working storage area must be provided by the user exit module. Return code 12 is only necessary if the compression shall be finished before the end of the input file is reached. For function code 0 and 8, no record is passed to the module. For function code 8, it is allowed to insert a record with return code 8. For return code 8, the record provided by the user exit module is written. Then the user exit module is called again for the old compressed record. |==== Table of valid function codes and return codes: [width="45%",cols="^2,^1,^1,^1,^1",align="center"] |==== |Function code | | 0 | 4 | 8 |Return code | 0 | x | x | x | | 4 | | x | | | 8 | | x | x | | 12 | | x | | | 16 | x | x | x |==== === Original Data Output EXD10 This exit treats the decompressed records immediately before they are written to the target file. Special processing can be defined for: * start of file * record level * end of file Records can be passed on, modified, deleted or inserted. This exit can be used to modify records in a structure dependent way. EXD10 is only available in FLAM and FLAMUP and corresponds to EXK10 during compression. ==== Principle of Operation EXD10 In this user exit, the decompressed records are passed to a user module immediately before they are written to the output file. This user exit can be used with the FLAM utility or with the subprogram FLAMUP. This user exit can accept, modify, insert and delete records. The exit is activated via the parameter `EXD10=`. The user exit module must be contained in the library that has been assigned with the STEPLIB command. [width="70%",cols="^2,^3",align="center"] |==== |*user module name* |free choice (max. 8 characters) |*Register usage* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="90%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |First call for file (after OPEN) | | 4 |Record passed | | 8 |Last call for file (before CLOSE) |**2. RETCO** |*F* |Return code | | 0 |Accept record / no error | | 4 |Do not accept record | | 8 |Insert record | | 12 |Enforce end of compression | | 16 |Error in exit; abnormal termination |**3. RECPTR** |*A* |Record pointer |**4. RECLEN** |*F* |Record length (maximum 32760) |**5. EXWORK** |*256F* |During the first call the working storage area contains the symbolic file name of the original file within the first 8 characters. The rest of the area is padded with x00. This area can be used by the user exit module for any purpose. With each call this working storage area is made available again with the old content. NOTE: If a record shall be extended or inserted, the necessary working storage area must be provided by the user exit module. Return code 12 is only necessary if decompression shall be finished before the end of the compressed file is reached. For function code 0 and 8, no record is passed to the module. For function code 8, it is allowed to insert a record with return code 8. For return code 8, the record provided by the user exit module is written. Then the user exit module is called again for the old record. A change of the record length is accepted if the output file is defined with `RECFORM=V`. |==== Table of valid function codes and return codes: [width="45%",cols="^2,^1,^1,^1,^1",align="center"] |==== |Function code | | 0 | 4 | 8 |Return code | 0 | x | x | x | | 4 | | x | | | 8 | | x | x | | 12 | | x | | | 16 | x | x | x |==== === Compressed Data Input EXD20 This user exit treats the compressed data immediately after it is read from the FLAMFILE. Special processing can be defined for: * start of file * record level * end of file This exit can be used to modify records in a structure independent way. With this exit it is possible, to decrypt the data with an own decryption routine or to apply the reverse code translation as used during compression. For a proper operation of FLAM it is indispensable that all changes applied to the compressed data are reversible. User exit EXD20 must deliver exactly the same data as user exit EXK20 received. All modifications applied to compressed data with EXK20 must be undone with EXD20. EXD20 is available in FLAM, FLAMUP and FLAMREC and corresponds with EXK20 during decompression. ==== Principle of Operation EXD20 In this user exit, the compressed records are passed to a user module immediately after they are read from the compressed file. This user exit can be used with the FLAM utility or with the subprogram FLAMUP and in the record level interface FLAMREC. This user exit can accept, modify and delete records. The exit is activated via the parameter `EXD20=`. The user exit module must be contained in the library that has been assigned with the STEPLIB command. [width="70%",cols="^2,^3",align="center"] |==== |*user module name* |free choice (max. 8 characters) |*Register usage* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="90%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |First call for file (after OPEN) | | 4 |Record passed | | 8 |Last call for file (before CLOSE) |**2. RETCO** |*F* |Return code | | 0 |Accept record / no error | | 4 |Do not accept record | | 8 |Insert record | | 12 |Enforce end of decompression | | 16 |Error in exit; abnormal termination |**3. RECPTR** |*A* |Record pointer |**4. RECLEN** |*F* |Record length (maximum 32760) |**5. EXWORK** |*256F* |During the first call the working storage area contains the symbolic file name of the original file within the first 8 characters. The rest of the area is padded with x00. This area can be used by the user exit module for any purpose. With each call this working storage area is made available again with the old content. NOTE: If a record shall be extended or inserted, the necessary working storage area must be provided by the user exit module. Return code 12 is only necessary if decompression shall be finished before the end of the compressed file is reached. Because of the necessary synchronisation with the construction of the matrix, this return code is not always possible. For function code 0 and 8, no record is passed to the module. |==== Table of valid function codes and return codes: [width="45%",cols="^2,^1,^1,^1,^1",align="center"] |==== |Function code | | 0 | 4 | 8 |Return code | 0 | x | x | x | | 4 | | x | | | 8 | | x | x | | 12 | | (x) | | | 16 | x | x | x |==== [[flam4.appendix.kmexit]] === Key Management KMEXIT This user exit returns a key to the FLAM utility for en-/decryption of a FLAMFILE. So, it is possible to enter any PASWORD/CRYPTOKEY in a secure way without notice to the JCL or the protocol. This exit is implemented as an interface to special key management systems, without influence to the FLAM utility programs. ==== Principle of Operation KMEXIT This user exit is an interface to a special (e.g. user written) key management system. On encryption, parameters (`KMPARM=...`) are passed to the module. It returns a key for encryption of the FLAMFILE and a string of up to 512 bytes. This data is stored in the FLAMFILE as a user header (see parameter COMMENT or function FLMPUH). On decryption, parameters (`KMPARM=...`) and the data stored in the user header are passed to the exit. The module returns the same key as on encryption. It is up to the module, how to create a key and what kind of information is to be stored into the user header of the FLAMFILE. This data will help the module to find the correct key on decryption. The exit is activated via the parameter `KMEXIT=`. The user exit module must be contained in the library that has been assigned with the STEPLIB command. [width="70%",cols="^2,^3",align="center"] |==== |*user module name* |free choice (max. 8 characters) |*Register usage* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="90%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |Decryption | | 1 |Encryption |**2. RETCO** |*F* |Return code | | 0 |No error | | else |Error(s) detected |**3. PARMLEN** |*F* |Length of parameter (up to 256 byte) |**4. PARAM** |*XLn* |Parameter |**5. DATALEN** |*F* |Decryption: + -> Length of data + Encryption: + -> Buffer length of field DATA (512) + <- Length of returned data (max. 512) |**6. DATA** |*XLn* |PData (in length of DATALEN) |**7. CKYLEN** |*F* |Length of key for en-/decryption + -> Buffer length of field CRYPTOKEY (64) + <- Length of returned key (max. 64) |**8. CRYPTOKEY** |*XLn* |Returned key (in length of CKYLEN) |**9. MSGLEN** |*F* |Message length + -> Length of message buffer (field MESSAGE) (128) + <- Length of returned Message length (max. 128) |**10. MESSAGE ** |*CLn* |Returned message (in length of MSGLEN) If a message is returned (MSGLEN > 0), it is sent to the protocol (FLM0445 ...). The returned key is not sent to the protocol. The data DATA is stored 'as is' in the user header of the FLAMFILE. If special security is required, it has to be done by the exit. Usage of this exit overwrites the parameter COMMENT and CRYPTOKEY, if any. The exit is called only once if encryption of many files into a Group-FLAMFILE `(C,FLAMIN=user.*)` is required. It is called only at the beginning of the first file. The exit is called many times for decryption of many FLAMFILEs `(D,FLAMFILE=user.*.aes)`. It is called after opening each FLAMFILE. In the DD-statement, concatenated FLAMFILEs are treated as one FLAMFILE! NOTE: Look for an example in FLAM.SRCLIB (KMX-SAMPL). |==== === User exits used by FLAM utility ==== Compression with EXK10, EXK20 image::pics/flam4_user_exit_comp_utility.png[User exit compression,align="center"] During compression, it is possible to call additional routines for pre-processing of original records and for post-processing of compressed records. E.g., pre-processing may perform a selection of specific records or fields. Post-processing may introduce additional encryption of the compressed record. Instead of using the more complicated record level interface record-orientated processing can be done often with user exit EXK10. ==== Decompression with EXD10, EXD20 image::pics/flam4_user_exit_deco_utility.png[User exit decompression,align="center"] During decompression, it is possible to call additional routines for pre-processing of compressed records and for post-processing of original records. E.g., pre-processing may decrypt encrypted records. Post-processing may perform a selection of specific records or fields. Instead of using the more complicated record level interface record-orientated processing can be done often with user exit EXD10. === User exits used by record interface ==== Compression with EXK20 image::pics/flam4_user_exit_comp_rec_interf.png[User exit compression,align="center"] The user exit for compressed records can also be used under the record level interface. The actual interception of original records by the interface is not affected. ==== Decompression with EXD20 image::pics/flam4_user_exit_deco_rec_interf.png[User exit decompression,align="center"] The user exit for compressed records can also be used under the record level interface. The actual interception of original records by the interface is not affected. === User exit examples ==== EXK10/EXD10-user exit The following exit routine can be used for compression as well as for decompression and has the purpose of modifying fields within records. The sample code is found in the library FLAM.SRCLIB. ------------------------------------------------------------------------ TITLE 'SEPARATE: EXIT FOR FLAM COMPRESSION' SEPARATE CSECT SEPARATE AMODE ANY SEPARATE RMODE ANY ********************************************************************** * THIS PROGRAM SEPARATES FIELDS WITHIN RECORDS WHICH CAN * BE SEPARATED BY DELIMITER CHARACTERS INTO DIFFERENT FLAM RECORDS. * THIS ENABLES A BETTER COMPRESSION. * THE DESIGN OF THE PROGRAM ALLOWS TO MODIFY THE DELIMITER AND * EVEN THE LENGTH OF THE DELIMITER BY CHANGING ONLY ONE STATEMENT. * * THE DELIMITERS ARE REMOVED FROM THE RECORD AND ARE REPLACED * BY FLAM SYNTAX. * IF THE RECORD DOES NOT CONTAIN DELIMITERS IT IS PASSED TO * FLAM WITHOUT MODIFICATIONS. * * THE ROUTINE SEPARATE IS ACTIVATED VIA PARAMETER 'EXK10=SEPARATE' * DURING THE CALL OF FLAM OR FLAMUP. * * THE FIELDS CONSIST OF PRINTABLE CHARACTERS SEPARATED BY * A TWO BYTE LONG DELIMITER (X'0D25'). * * THE DATA COMPRESSED IN THIS WAY IS TRANSMITTED VIA * FILE TRANSFER TO A PC, DECOMPRESSED FIELD BY FIELD * USING FLAM AND WRITTEN TO THE STORAGE MEDIUM (WITH * DELIMITER OF THE OPERATING SYSTEM, X'0D0A' WITH MSDOS * OR X'0A' WITH UNIX). * * NOTE: * * FOR DECOMPRESSION ON A MAINFRAME A FILE WITH * VARIABLE RECORD LENGTH MUST BE SPECIFIED. * EACH FIELD SEPARATED DURING COMPRESSION IS WRITTEN * AS A SEPARATE RECORD. THE DELIMITERS ARE NOT CONTAINED * IN THE DECOMPRESSED RECORDS. * THIS MEANS THAT THE ORIGINAL FILE CANNOT BE RECONSTRUCTED * ON A MAINFRAME. * * THIS MODULE IS REENTRANT AND REUSABLE * * *--------------------------------------------------------------------- * * AUTHOR: LIMES DATENTECHNIK GMBH * D-61381 FRIEDRICHSDORF/TS. * TEL. 06172-5919-0 * FAX 06172-5919-39 ********************************************************************** * * INTERFACE: R1 POINTS TO PARAMETER LIST * * 0(R1) - A(FUNCTION CODE) * 4(R1) - A(RETURN CODE) * 8(R1) - A(A(RECORD)) RECORD POINTER * 12(R1) - A(RECORD LENGTH) * 16(R1) - A(WORK AREA) NEW WITH FLAM V2.5 * ********************************************************************** EJECT STM R14,R12,12(R13) SAVE REGISTERS LR R12,R15 ENTRY ADDRESS USED AS PROGRAM BASE USING SEPARATE,R12 ASSIGN BASE REGISTER USING WORKAREA,R2 BASE REGISTER WORK AREA LA 15,0 INITIALISE RETURN CODE WITH 0 * L R3,0(,R1) LOAD A(FC) CLC 0(4,R3),FCSATZ PASS RECORD ? BE SATZUEB == YES CLC 0(4,R3),FCOPEN OPEN ? BNE RET == NO * * AT OPEN TIME RESET WORK AREA FIELDS * L R2,16(,R1) A(WORKAREA) MVI FLAG,X'00' RESET FLAGS B RET SATZUEB DS 0H * * RECORD WAS PASSED * L R10,8(,R1) A(A(RECORD)) TO R10 L R4,0(,R10) LOAD A(RECORD) L R11,12(,R1) A(RECORD LENGTH) L R5,0(,R11) LOAD RECORD LENGTH LA R9,0(R5,R4) A(RECORD END) L R2,16(,R1) A(WORK AREA) * TM FLAG,SATZDA RECORD ALREADY PRESENT ? BNO BEGINN == NO TM FLAG,LOESCH DELETE RECORD ? BO LOESATZ == YES * BEGINNA DS 0H RECORD WAS ALREADY PROCESSED L R4,SATZPTR A(FIELD) FROM LAST TIME * BEGINN DS 0H OI FLAG,SATZDA INDICATE RECORD PRESENT LR R7,R4 SAVE A(FIELD BEGIN) LR R6,R9 A(FIELD END) SR R6,R7 - A(FIELD BEGIN) = L'REMAINDER BZ LEERSATZ L'= 0, PASS EMPTY RECORD C R6,LTRENNKZ BNL SUCH L' L'DELIMITER - HAS NO DELIMITER OI FLAG,LOESCH INDICATE DELETE OPERATION FOR NEXT RUN LR R4,R9 A(RECORD END) B SUCHEND SUCH DS 0H LA R8,1 INCREMENT FOR BX INSTRUCTION S R9,LTRENNKZ FOR BX INSTR. SET RECORD END -L' SUCHLOOP DS 0H * * SEARCH STRING IS (DELIMITER) * CLC 0(L'TRENNKZ,R4),TRENNKZ DELIMITER ? BE ISTDA == YES BXLE R4,R8,SUCHLOOP NEXT CHARACTER * OI FLAG,LOESCH INDICATE DELETE OPERATION FOR NEXT RUN LA R4,L'TRENNKZ-1(R4) FIELD IS BIGGER BY L'-1 B SUCHEND * ISTDA DS 0H LA R6,L'TRENNKZ(R4) INCREMENT RECORD POINTER ST R6,SATZPTR SAVE RECORD POINTER SUCHEND DS 0H * * FILL FLAM PARAMETER LIST * SR R4,R7 FIELD LENGTH ST R4,0(R11) IS RECORD LENGTH FLAM ST R7,0(R10) RECORD ADDR FOR FLAM LA R15,8 RETURN CODE: INSERT RECORD * RET DS 0H * * RETURN TO FLAM * L R3,4(,R1) LOAD A(RC) ST R15,0(,R3) PASS RC L R14,12(R13) RESTORE REGISTERS LM R0,R12,20(R13) BR R14 RETURN * LOESATZ DS 0H LA R15,4 RETURN CODE: DELETE RECORD MVI FLAG,X'00' RESET FLAG B RET AND RETURN * LEERSATZ DS 0H AFTER DELIMITER AT RECORD END OI FLAG,LOESCH INDICATE DELETE OPERATION FOR NEXT RUN LA R4,0 RECORD IS EMPTY ST R4,0(R11) RECORD LENGTH FOR FLAM LA R15,8 RETURN CODE: INSERT RECORD B RET AND RETURN * * CONSTANTS AND WORK AREAS * * FCSATZ DC F'4' FUNCTION CODE RECORD PASSED FCOPEN DC F'0' OPEN LTRENNKZ DC A(L'TRENNKZ) LENGTH OF DELIMITER *-------------------------------------------------------------------- * * IN CASE OF DIFFERENT DELIMITER MAKE MODIFICATIONS HERE * TRENNKZ DC XL2'0D25' DELIMITER TO BE FOUND *-------------------------------------------------------------------- * * REGISTER * R0 EQU 0 R1 EQU 1 PARAMETER ADDRESS R2 EQU 2 BASE REGISTER FOR WORK AREA R3 EQU 3 R4 EQU 4 R5 EQU 5 R6 EQU 6 R7 EQU 7 R8 EQU 8 R9 EQU 9 R10 EQU 10 R11 EQU 11 R12 EQU 12 BASE REGISTER R13 EQU 13 A(SAVE AREA) R14 EQU 14 RETURN ADDRESS R15 EQU 15 ENTRY ADDRESS * LTORG * DC C'*** MODULE SEPARATE V1.02 FOR FLAM ' DC C' COPYRIGHT (C) 1990-91 BY LIMES DATENTECHNIK GMBH. ' DC C'DATE, TIME ASSEMBLED: ' DC C'&SYSDATE , &SYSTIME ' DC C'***' * * WORKAREA IS PROVIDED BY FLAM (1024 BYTES) * WORKAREA DSECT * DDNAME DS CL8 DD-NAME OF CURRENT FILE SATZPTR DS A RECORD POINTER FLAG DS X INDICATORS FOR PROCESSING SATZDA EQU 1 RECORD ALREADY PRESENT LOESCH EQU 2 DELETE RECORD END ------------------------------------------------------------------------ ==== EXK20/EXD20-user exit Since FLAM protects compressed files against manipulations by applying a checksum, it is possible to provide encryptions within the user exits with a very low overhead. Because the compressed data is already encrypted, simple deterministic character swapping within the compressed data cannot be detected easily by an unauthorized user. During decompression this character swapping - if not redone by an authorized user - will lead to a check sum error and the compressed file cannot be read. The symmetric construction of the user exits allows to use the same routine for encryption as well as for decryption provided that algorithms are used that will restore the original data when executed twice. This is the case with mutual character swapping. Similar results can be obtained with translate tables for cyclic (cycle length 2) character code exchange. ------------------------------------------------------------------------ TITLE 'EX20 (B) | VERSION 1.00:06/25/91 | ' *********************************** * COLUMBUS-ASSEMBLER * *********************************** * SYMBOLIC CONDITIONS FOR #IF,#WHEN,#WHIL(E),#TOR,#AND,#OR #LT EQU 4 LESS THAN #GT EQU 2 GREATER THAN #EQ EQU 8 EQUAL #NE EQU 7 NOT EQUAL #LE EQU 13 LESS OR EQUAL #GE EQU 11 GREATER OR EQUAL #LZ EQU 4 LESS THAN ZERO #GZ EQU 2 GREATER THAN ZERO #ZE EQU 8 ZERO #NZ EQU 7 NOT ZERO #ON EQU 1 ONES #MI EQU 4 MIXED #ZO EQU 11 ZEROS OR ONES #ZM EQU 14 ZEROS OR MIXED #OM EQU 7 ONES OR MIXED #F EQU 15 TRUE IN ANY CASE * FLOATING POINT REGISTERS, GENERAL REGISTERS, COLUMBUS REGISTERS FA EQU 0 FB EQU 2 FC EQU 4 FD EQU 6 R0 EQU 0 R1 EQU 1 R2 EQU 2 R3 EQU 3 R4 EQU 4 R5 EQU 5 R6 EQU 6 R7 EQU 7 R8 EQU 8 R9 EQU 9 R10 EQU 10 R11 EQU 11 R12 EQU 12 R13 EQU 13 R14 EQU 14 R15 EQU 15 R#PAR EQU R1 R#BASE EQU R10 R#STACK EQU R13 R#EXIT EQU R14 R#PASS EQU R15 EJECT EX20 CSECT USING EX20,R#PASS *********************************************************************** * NAME: EX20 VERSION: 13.03.91 * * FUNCTION: * * FLAMFILE IS ENCRYPTED AND DECRYPTED IN A SIMPLE WAY. * * * * THE 16TH AND 17TH CHARACTER ARE SWAPPED WHICH * * CHANGES THE CHECKSUM. THE COMPRESSED DATA * * CAN ONLY DECOMPRESSED IF THE SWAP OPERATION * * DONE AGAIN. * * PARAMETER * * 1 -> ID F IDENTIFICATION * * 2 <- RETCO F RETURN CODE * * 3 -> RECPTR A RECORD POINTER * * 4 -> RECLEN F RECORD LENGTH * *********************************************************************** * * SAVE REGISTERS AND LOAD BASE REGISTERS * STM R14,R12,12(R13) * * LOAD PARAMETERS * LM R1,R4,0(R1) * PASS COMPRESSED RECORD CLC 0(4,R1),F4 BC #F-#EQ,#F1001 * LOAD RECORD LENGTH L R4,0(R4) * RECORD LENGTH GREATER 16 LA R14,16 CR R4,R14 BC #F-#GT,#F1002 * * SWAP 16TH AND 17TH CHARACTER * L R3,0(R3) LA R14,0(R3,R14) IC R5,0(R14) MVC 0(1,R14),1(R14) STC R5,1(R14) #F1002 DS 0H #F1001 DS 0H * * RETURN CODE = ACCEPT RECORD / NO ERROR * LA R0,0 ST R0,0(R2) * * RETURN * LM R14,R12,12(R13) BR R#EXIT * * LOCAL CONSTANTS * F4 DC F'4' F16 DC F'16' LTORG DS 0D DROP R#PASS END ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER == BIFLAM == Appendix D. BIFLAM This appendix describes the line compression and decompression (BIFLAM) solution delivered with FLAM4 for the IBM z/OS environment. === Bi-/serial compression with BIFLAMK Bi-/serial compression does not employ FL-Matrices. The compression effect is achieved by comparing the original data with a sample and/or by using serial compression techniques. BIFLAMK works synchronously. Each call results immediately in output data. It does not need a memory-compression is done on a call-by-call or record-by-record basis. Bi-/serial compression is especially suitable for integration into other products or applications. The compression effect is considerably lower than compression based on the FL-matrix. The advantage is the independence of each record. In many environments, this independence is a necessary requirement for integration. Beside compression, BIFLAMK offers two more functions that become more and more important as requirements for data security and data integrity are rising. All compressed records are encrypted and protected against modification using checksums over both original and compressed file. BIFLAMK offers several compression modes that can be selected via a function code. First, a vanilla serial compression mode is provided that does not require a sample record. All compressed records are independent from each other and can be decompressed individually. Second, a bi-serial mode is provided that can be adapted to different environments. This compression mode is based on a comparison of the record with a sample record byte by byte. The result is a bitmap that denotes all positions where characters are equal and the remaining differing characters. The first option allows to control the post-processing of the remainder. A serial compression can be applied to the remainder or the remainder is simply encrypted. The serial compression of the remainder can be bypassed if the CPU overhead is considered too high or the compression ratio is satisfactory. The second option controls the processing of the sample. If a dynamic sample is used, a checksum is computed for each sample record and is inserted into the compressed record. This slightly reduces the compression effect and increases CPU overhead. On the other hand, data security is improved because modifications are detected more easily. For an exact error analysis it is even possible to decide whether the compressed record or the sample has been modified. If a static sample is used, no checksum of the sample is computed. In this case, modifications in the sample can only be detected by using the checksum of the original file. The third option allows the storage of samples within the compressed file. During decompression, these records are restored again as samples. With this method, BIFLAMK can create sequences of records that can be decompressed by BIFLAMD without additional information (samples). One possibility would be to provide one sample first. Then all records are compressed against this sample and bi-serial compression is used to compress the remainders as well. Another possibility is to use dynamic samples by using the processor of each record as the sample for the current record. This sequence result in good compression ratios if adjoining records are similar (reports, data entry lists). The disadvantage is that the individual records are not independent from each other. This sequence must be decompressed as a whole. However, additional information (like samples) is not necessary. It does not make much sense, to use a different sample for each record. Sample records can only be compressed with simple serial compression. In addition the compressed data of the original data has to be stored as well. ==== Principle of operation BIFLAMK BIFLAMK is used for compression of data record by record. The compressed data is always returned in the same call. BIFLAMK is reentrant. For operation, a working storage area is needed that must be provided by the calling program. The content of the area before the call is ignored by BIFLAMK. All calls to BIFLAMK are totally independent from each other. All areas can have any alignment. The working storage area for the input record and for the compressed record must not overlap. In-place compression is not possible. [width="45%",cols="^2,^3",align="center"] |==== |*Name* |BIFLAMK |*Parameters* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="80%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |Serial compression without sample | | 8 |Biserial compression with sample, serial post compression of the remainder and static sample | | 9 |Sample record for biserial compression with serial post compression | | 10 |Biserial compression with sample, serial post compression of the remainder and dynamic sample | | 11 |Sample record for biserial compression with serial post compression | | 12 |Biserial compression with sample, serial post compression and encryption of the remainder and static sample | | 13 |Sample record for biserial compression with encryption | | 14 |Biserial compression with sample, serial post compression and encryption of the remainder and dynamic sample | | 15 |Sample record for biserial compression with encryption |**2. RETCO** |*F* |Return code | | 0 |Function executed | | 2 |Invalid function code | | 3 |Length error + - working storage too small + - return area too small + - record bigger than 32767 bytes |**3. WORK** |*XLn* |Working storage area. The working storage area must be at least 512 bytes in size. For biserial compression the working storage area must be at least 512 bytes + length of return area. |**4. WRKLEN** |*F* |Length of working storage area in bytes |**5. BUFLEN** |*F* |Length of return area or maximum length of compressed record; this size must be at least 8 bytes + 1.1 * length of original record. |**6. RECIN** |*XLn* |Original record |**7. RECLEN** |*F* |Record length in bytes |**8. COMPREC** |*XLn* |Compressed record (length of this area = BUFLEN) |**9. COMPLEN** |*F* |Length of compressed record in bytes The next two parameters are only used for biserial compression: |**10. SAMPREC** |*XLn* |Sample |**11. SAMPLEN** |*F* |Sample length in bytes |==== === Bi-/serial decompression BIFLAMD BIFLAMD decompresses compressed records created with BIFLAMK. For the serial decompression, no sample record (including length) is used. This implies that BIFLAMD has two fewer parameters. Therefore, it is necessary to inform BIFLAMD during the call which mode (serial or bi-serial) should be used for decompression. For a proper decompression, it is necessary that BIFLAMD receives exactly the same compressed records and sample records as created with BIFLAMK. Modification (code translations) must not be applied to the compressed data and the sample records. If compressed data shall be transmitted between different computers, the file transmission must be transparent. BIFLAMD automatically detects if a record was compressed serial or bi-serial and will report an error message if the compression syntax does not correspond to the call function code. Also modifications in compressed data, sample records and original data are detected via the checksums. ==== Principle of operation BIFLAMD BIFLAMD is used for record by record decompression of compressed data that was created with BIFLAMK. BIFLAMD is reentrant. For operation, a working storage area is needed that must be provided by the calling program. The content of the area before the call is ignored by BIFLAMD. All calls to BIFLAMD are totally independent from each other. All areas can have any alignment. The working storage area for the compressed record, for the sample record, and for the output record must not overlap. In-place decompression is not possible. [width="45%",cols="^2,^3",align="center"] |==== |*Name* |BIFLAMD |*Parameters* | |R1 |Address of parameter list |R13 |Points to save area (18 words) |R14 |Contains return address |R15 |Contains call address (entry point) |==== [width="80%",cols="^3,^1,10",align="center",options="header"] |==== |*Parameter list* | | |**1. FUCO** |*F* |Function code | | 0 |Serial decompression without sample | | 8 |Biserial decompression with sample |**2. RETCO** |*F* |Return code | | 0 |Function executed | | 1 |Sample record for biserial decompression returned; no original record was written (only during biserial decompression) | | 2 |Invalid function code or record is compressed serially (function code = 8) or record is compressed biserially (function code = 0). | | 3 |Length error - working storage area too small - compressed record smaller than 3 bytes - return area too small. | | 4 |Checksum error in compressed record | | 5 |Checksum error in sample record (only for dynamic samples) | | 6 |Checksum error in original record | | 7 |Other errors in compressed record | | 8 |Sample record is shorter as during compression (only for biserial compression) | | 9 |Compressed record too short |**3. WORK** |*XLn* |Working storage area. The working storage area must be at least 512 bytes in size. For biserial compression, the working storage area must be at least 512 bytes + 1.125 * the sum of length of the return areas. |**4. WRKLEN** |*F* |Length of working storage area in bytes |**5. BUFLEN** |*F* |Length of return areas; maximum length of the original record or the sample record in bytes |**6. RECOUT** |*XLn* |Original record (length of area = BUFLEN) |**7. RECLEN** |*F* |Record length in bytes |**8. COMPREC** |*XLn* |Compressed record |**9. COMPLEN** |*F* |Length of compressed record in bytes The next two parameters are only needed for biserial decompression: |**10. SAMPREC** |*XLn* |Sample record (length of area = BUFLEN) |**11. SAMPLEN** |*F* |Length of sample record in bytes |**12. RETCO** |*F* |Return code |==== === Compression data flow with BIFLAMK image::pics/flam4_biflam_comp.png[BIFLAM compression,height=771,width=774,align="center"] BIFLAMK processes original records and sample records on a record by record basis. When only serial compression is used (function code 0), BIFLAMK only processes original records and compiles them into compressed records. If bi-serial compression with sample (function codes 8,10,12,14) is used, an original record is processed in combination with a sample record to create a compressed record. For storage of the sample records (function codes 9,11,13,15), only the sample record is processed to create a compressed record. === Compression data flow with BIFLAMD image::pics/flam4_biflam_deco.png[BIFLAM decompression,height=771,width=774,align="center"] BIFLAMD processes compressed records record by record if necessary in combination with a sample record - and creates an original record or a sample record. During serial decompression (function code 0), the compressed record is only processed and will always create an original record. Sample records are not necessary. During bi-serial compression (function code 8), always one compressed record is processed at a time. Depending on compression, the sample record is read in addition and the original record is created. If a sample record was compressed during compression, this sample record is reconstructed during decompression. This situation is signalled with a return code of 1. ... print CLE_DOCTYP_CHAPTER == FLAM version 1 == Appendix E. FLAM version 1 FLAM version 1 consisted of two batch utilities called FLKOMP and FLDECO for compressing and decompressing files. With FLAM version 2, these were replaced by the batch utility called FLAM. The FLAM utility in turn uses FLAMUP and so that FLAM1 customers do not have to adapt their jobs here, FLKOMP and FLDECO were reimplemented against the new FLAMUP so that the old syntax is still available here. With FLAM version 5, the syntax of the old FLAM utility was largely mapped to the new FLAM command of the FLCL, which is also available via the FLAM4 utility. For FLKOMP and FLDECO, no conversions of the old command syntax were introduced here. Therefore, here is a short instruction on how to transfer the old FLKOMP and FLDECO calls to FLAM4 or FLCL. Here is an example call of FLKOMP. ------------------------------------------------------------------------ //COMP1 EXEC PGM=FLKOMP,PARM=' PARAM MAXB=2,END' ------------------------------------------------------------------------ Firstly, the PARAM and END specifications must be omitted. Furthermore, FLKOMP and FLDECO work with the DD names INPUT and OUTPUT instead of FLAMIN and FLAMOUT, which may have to be adapted or you have to set INDD=INPUT and OUTDD=OUTPUT if you work with the FLAM4 utility and the old syntax from FLAM2 onwards. A FLKOMP in as FLCL call via the CONV command of FLCL would look like this: ------------------------------------------------------------------------ FLCOMP: READ.RECORD(FILE='DD:INPUT') WRITE.FLAM() FLDECO: READ.FLAM() WRITE.RECORD(FILE='DD:OUTPUT) ------------------------------------------------------------------------ If you work with FLAMIN and FLAMOUT as DD names, this is simplified even more: ------------------------------------------------------------------------ FLCOMP: READ.RECORD() WRITE.FLAM() FLDECO: READ.FLAM() WRITE.RECORD() ------------------------------------------------------------------------ But even today, FLKOMP and FLDECO are still included in the delivery of FLAM and can still be used. ... print CLE_DOCTYP_LEXEMES == Lexemes [[CLEP.APPENDIX.LEXEMES]] == Appendix F. Lexemes The regular expressions used by the command line compiler are shown below. This content is printed on the screen with the built-in function `LEXEMES`. To describe the regular expression the following character classes are used: * :print: - all printable characters * :space: - all blanks, tabs and new lines * :cntr: - all control characters * :alpha: - all letters ([[a-z]|[A-Z]]) * :alnum: - all letters and digits [0-9] * :digit: - all decimal numbers [0-9] * :xdigit: - all hexadecimal numbers [[0-9]|[a-f||[A-F]] All comments and separators are ignored by the scanner and not passed with the token to the parser. This means that the separators (SEP) are used in the grammar but will never be part of or own tokens. If a binary entry for a string is possible and no classification is done it is taken as a binary string in local character set representation (c) without null termination. This default can be changed. In this case the help message for this argument should be described the default interpretation of the binary string. For binary strings it is still possible to enter a null-terminated string, but then the prefix *s* must be used. If no binary entry is feasible, only a null-terminated string in local character representation (s) can be built and taken by default. ------------------------------------------------------------------------ Lexemes (regular expressions) for argument list or parameter file --| COMMENT '#' [:print:]* '#' (will be ignored) --| LCOMMENT ';' [:print:]* 'nl' (will be ignored) --| SEPARATOR [:space: | :cntr: | ',']* (abbreviated with SEP) --| OPERATOR '=' | '.' | '(' | ')' | '[' | ']' | (SGN, DOT, RBO, RBC, SBO, SBC) --| '=>'| '+' | '-' | '*' | '/' | '{' | '}' (SAB, ADD, SUB, MUL, DIV, CBO,CBC) --| KEYWORD ['-'['-']][:alpha:]+[:alnum: | '_']* (always predefined) --| NUMBER ([+|-] [ :digit:]+) | (decimal (default)) --| num ([+|-]0b[ :digit:]+) | (binary) --| num ([+|-]0o[ :digit:]+) | (octal) --| num ([+|-]0d[ :digit:]+) | (decimal) --| num ([+|-]0x[ :xdigit:]+) | (hexadecimal) --| num ([+|-]0t(yyyy/mm/tt.hh:mm:ss)) | (relativ (+|-) or absolut time) --| FLOAT ([+|-] [ :digit:]+.[:digit:]+e|E[:digit:]+) | (decimal(default)) --| flt ([+|-]0d[ :digit:]+.[:digit:]+e|E[:digit:]+) (decimal) --| STRING ''' [:print:]* ''' | (default (if binary c else s)) --| str [s|S]''' [:print:]* ''' | (null-terminated string) --| str [c|C]''' [:print:]* ''' | (binary string in local character set) --| str [a|A]''' [:print:]* ''' | (binary string in ASCII) --| str [e|E]''' [:print:]* ''' | (binary string in EBCDIC) --| str [x|X]''' [:print:]* ''' | (binary string in hex notation) --| str [f|F]''' [:print:]* ''' | (read string from file (for passwords)) --| Strings can contain two '' to represent one ' --| Strings can also be enclosed in " or ` instead of ' --| Strings can directly start behind a '=' without enclosing ('`") --| In this case the string ends at the next separator or operator --| and keywords are preferred. To use keywords, separators or --| operators in strings, enclosing quotes are required. --| --| The predefined constant keyword below can be used in a value expressions --| NOW NUMBER - current time in seconds since 1970 (+0t0000) --| MINUTE NUMBER - minute in seconds (60) --| HOUR NUMBER - hour in seconds (60*60) --| DAY NUMBER - day in seconds (24*60*60) --| YEAR NUMBER - year in seconds (365*24*60*60) --| KiB NUMBER - kilobyte (1024) --| MiB NUMBER - megabyte (1024*1024) --| GiB NUMBER - gigabyte (1024*1024*1024) --| TiB NUMBER - terrabyte (1024*1024*1024*1024) --| RNDn NUMBER - simple random number with n * 8 bit in length (1,2,4,8) --| PI FLOAT - PI (3.14159265359) --| LCSTAMP STRING - current local stamp in format: YYYYMMDD.HHMMSS --| LCDATE STRING - current local date in format: YYYYMMDD --| LCYEAR STRING - current local year in format: YYYY --| LCYEAR2 STRING - current local year in format: YY --| LCMONTH STRING - current local month in format: MM --| LCDAY STRING - current local day in format: DD --| LCTIME STRING - current local time in format: HHMMSS --| LCHOUR STRING - current local hour in format: HH --| LCMINUTE STRING - current local minute in format: MM --| LCSECOND STRING - current local second in format: SS --| LCOFFSET STRING - difference to Greenwich mean time: +/-HH --| LCOFFABS STRING - difference to GMT (absolute): HH --| GMSTAMP STRING - current Greenwich mean stamp in format: YYYYMMDD.HHMMSS --| GMDATE STRING - current Greenwich mean date in format: YYYYMMDD --| GMYEAR STRING - current Greenwich mean year in format: YYYY --| GMYEAR2 STRING - current Greenwich mean year in format: YY --| GMMONTH STRING - current Greenwich mean month in format: MM --| GMDAY STRING - current Greenwich mean day in format: DD --| GMTIME STRING - current Greenwich mean time in format: HHMMSS --| GMHOUR STRING - current Greenwich mean hour in format: HH --| GMMINUTE STRING - current Greenwich mean minute in format: MM --| GMSECOND STRING - current Greenwich mean second in format: SS --| GMOFFSET STRING - difference to local time: +/-HH --| GMOFFABS STRING - difference to local time (absolute): HH --| SnRND10 STRING - decimal random number of length n (1 to 8) --| SnRND16 STRING - hexadecimal random number of length n (1 to 8) --| --| SUPPLEMENT '"' [:print:]* '"' | (null-terminated string (properties)) --| Supplements can contain two "" to represent one " --| Supplements can also be enclosed in ' or ` instead of " --| Supplements can also be enclosed in ' or ` instead of " --| ENVIRONMENT VARIABLES '<'varnam'>' will replaced by the corresponding value --| Escape sequences for critical punctuation characters on EBCDIC systems --| '!' = '&EXC;' - Exclamation mark --| '$' = '&DLR;' - Dollar sign --| '#' = '&HSH;' - Hashtag (number sign) --| '@' = '&ATS;' - At sign --| '[' = '&SBO;' - Square bracket open --| '\' = '&BSL;' - Backslash --| ']' = '&SBC;' - Square bracket close --| '^' = '&CRT;' - Caret (circumflex) --| '`' = '&GRV;' - Grave accent --| '{' = '&CBO;' - Curly bracket open --| '|' = '&VBR;' - Vertical bar --| '}' = '&CBC;' - Curly bracket close --| '~' = '&TLD;' - Tilde --| Define CCSIDs for certain areas in CLP strings on EBCDIC systems (0-reset) --| '&' [:digit:]+ '; (..."&1047;get.file='&0;%s&1047;'",f) --| Escape sequences for hexadecimal byte values --| '&' ['X''x'] :xdigit: :xdigit: ';' ("&xF5;") ------------------------------------------------------------------------ ... print CLE_DOCTYP_GRAMMAR == Grammar [[CLEP.APPENDIX.GRAMMAR]] == Appendix G. Grammar Below you can see the grammar used by the command line compiler. This content is printed on the screen if the built-in function <> is used. There are two context free grammars, one is used for the command line the other for the parameter file. The parameter file contains mainly the same argument list as the command line. The property file defines the default settings. For this a different grammar is applied. Properties can be set for each argument, i.e. for arguments not used or arguments in overlays. Based on that, properties are set by using the path (dotted keyword list). A property is always a 'SUPPLEMENT' for the argument list on the command line or parameter file. Thus all defined properties have the same effect as arguments on the command line. A property sets this argument to the corresponding value if it is not defined on the command line. ------------------------------------------------------------------------ Grammar for argument list, parameter file or property file --| Command Line Parser --| command -> ['('] parameter_list [')'] (main=object) --| | ['.'] parameter (main=overlay) --| parameter_list -> parameter SEP parameter_list --| | EMPTY --| parameter -> switch | assignment | object | overlay | array --| switch -> KEYWORD --| assignment -> KEYWORD '=' value --| | KEYWORD '=' KEYWORD # SELECTION # --| | KEYWORD '=>' STRING # parameter file # --| object -> KEYWORD ['('] parameter_list [')'] --| | KEYWORD '=' STRING # parameter file # --| | KEYWORD '=>' STRING # parameter file # --| overlay -> KEYWORD ['.'] parameter --| | KEYWORD '=' STRING # parameter file # --| | KEYWORD '=>' STRING # parameter file # --| array -> KEYWORD '[' value_list ']' --| | KEYWORD '[' object_list ']' --| | KEYWORD '[' overlay_list ']' --| | KEYWORD '=' value_list # with certain limitations # --| It is recommended to use only enclosed array lists to know the end --| | KEYWORD '[=' STRING ']' # parameter file # --| | KEYWORD '[=>' STRING ']' # parameter file # --| value_list -> value SEP value_list --| | EMPTY --| object_list -> object SEP object_list --| | EMPTY --| overlay_list -> overlay SEP overlay_list --| | EMPTY --| A list of objects requires parenthesis to enclose the arguments --| --| value -> term '+' value --| | term '-' value --| | term --| term -> factor '*' term --| | factor '/' term --| | factor --| factor -> NUMBER | FLOAT | STRING --| | selection | variable | constant --| | '(' value ')' --| selection -> KEYWORD # value from a selection table # --| variable -> KEYWORD # value from a previous assignment # --| | KEYWORD '{' NUMBER '}' # with index for arrays# --| constant -> KEYWORD # see predefined constants at lexeme # --| For strings only the operator '+' is implemented as concatenation --| Strings without an operator in between are also concatenated --| A number followed by a constant is a multiplication (4KiB=4*1024) --| --| Property File Parser --| properties -> property_list --| property_list -> property SEP property_list --| | EMPTY --| property -> keyword_list '=' SUPPLEMENT --| keyword_list -> KEYWORD '.' keyword_list --| | KEYWORD --| SUPPLEMENT is a string in double quotation marks ("property") ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER == Properties [[CLEP.APPENDIX.PROPERTIES]] == Appendix H. Properties All properties can be written to a global property file with the built-in function <>: ------------------------------------------------------------------------ 'GENPROP filename'. ------------------------------------------------------------------------ A property file for a certain command can be generated with: ------------------------------------------------------------------------ 'GENPROP command=filename' ------------------------------------------------------------------------ The first command shows the valid root path for the property file. A property file can contain properties for more than one root path. The property file parser will only interpret properties where the root matches, i.e. if the property file is assigned to a certain owner and possibly to a certain command, then only properties for this owner, the program itself and possibly the command will be accepted, all other properties are ignored. The second long comment gives some help for handling a property file. Thereafter the generated properties for this root are listed for all supported commands. The format is: ------------------------------------------------------------------------ path = SUPPLEMENT # Type: tttttt Help: hhh...hhh # ------------------------------------------------------------------------ The path denotes the unique dotted keyword list for one argument. The 'SUPPLEMENT' is a string normally in double quotation marks containing the value list for this argument in accordance with the syntax of the command line parser (strings then normally in single quotation marks, a opposite usage of the quotation marks is possible but not recommended). This means for example that a string that requires apostrophes must be included in apostrophes `("'...'")`. The supplement string is returned by the corresponding command line parser if no value for this argument is entered on the command line or is included in the parameter file. Behind the property definition the type and help message are printed as comment to help in editing the generated property file. The property values can be related to the current default settings at time of generation, what means that if hard coded default values or another property file is already in use, then these values are taken over into the new property file. The property file can contain parameter which are not available on the command line. Such parameter are only documented in the property file. On the other side not all command line parameter are available as property. If a description for a property argument is available which is not a part of the user manual this description is written as comment in front of the corresponding property definition. Below you can see the documentation of all parameter which are only available as property followed by all defined default properties in form of a sample property file for all available commands. ... print CLE_DOCTYP_PROPREMAIN === Remaining Documentation === Appendix H.1. Remaining Documentation Below you can find the documentation for all parameters which are only available over the property definitions. These parameters are not a part of the description for each command above. This list is generated and can be empty. In this case all parameters are available over the command line. If available, the manual page otherwise the help message is printed. .DESCRIPTION FOR limes.flam4.INFO.LOG.DESTINATION.SYSTEM: (TYPE: OBJECT) Use system logging facility The SYSTEM destination allows the log messages to be written to the logging system of the operating system. On Unix and z/OS(R) systems the syslog() call is used. On Windows(R) the event logging facility is used. ----------------------------------------------------------------------- NOTE: On Windows(R) this requires the registry key entry: HKEY_LOCAL_MACHINE SYSTEM CurrentControlSet Services EventLog Application FLM ----------------------------------------------------------------------- This registry key will be created by using the DLL flevent.dll .HELP FOR limes.flam4.INFO.LOG.DESTINATION.SYSTEM: (TYPE: OBJECT) Use system logging facility .DESCRIPTION FOR limes.flam4.FLAM.LOG.DESTINATION.SYSTEM: (TYPE: OBJECT) Use system logging facility The SYSTEM destination allows the log messages to be written to the logging system of the operating system. On Unix and z/OS(R) systems the syslog() call is used. On Windows(R) the event logging facility is used. ----------------------------------------------------------------------- NOTE: On Windows(R) this requires the registry key entry: HKEY_LOCAL_MACHINE SYSTEM CurrentControlSet Services EventLog Application FLM ----------------------------------------------------------------------- This registry key will be created by using the DLL flevent.dll .HELP FOR limes.flam4.FLAM.LOG.DESTINATION.SYSTEM: (TYPE: OBJECT) Use system logging facility ... print CLE_DOCTYP_PROPDEFAULTS === Predefined Defaults === Appendix H.2. Predefined Defaults Below you can find a sample property file which contains all predefined default values for all supported commands when this documentation was created. ------------------------------------------------------------------------ # Property file for: limes.flam4 # #------------------------------------------------------------------- # # # # The property file can be used to overwrite the default values # # used if the argument is not defined on the command line. # # # #------------------------------------------------------------------- # # # # The values is assigned between "" conforming to the syntax. # # Example NUMBER array : "4711, 1, 2, 3" # # Example STRING array : "'str1' x'303000''str2',a'abc'" # # Example FLOAT array : "123.34, 1.58, PI" (PI as constant) # # # # The last example shows that one can use key words if a list # # of constants is defined for this argument. # # # # For a SWITCH one can use the special key words ON/OFF to enable or # # disable the SWITCH or define the value as a number # # Examples for a SWITCH : "ON"/"OFF"/"4711" # # # # For an OBJECT only the special key word INIT can be used to ensure # # that this OBJECT will be initialized if it was not set on the # # command line. If INIT is not set, then the elements of the OBJECT # # will be initialized only if you type at least obj() in the command # # line. If you want an object to be initialized when you don't use it# # on the command line then the special key word must be set. # # Examples for an OBJECT: ""/"INIT" # # # # For an OVERLAY it is the same logic as for objects but the keyword # # of the element to initialize must be used if the overlay is not # # set in the command line. # # Example OVERLAY array : "LINE TRIANGLE LINE RECTANGLE" # # Example for a OVERLAY : "RECTANGLE" # # # #------------------------------------------------------------------- # ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER == Return Codes [[CLEP.APPENDIX.RETURNCODES]] == Appendix I. Return Codes The return codes of the program are also called completion codes and returned at program termination. This content is printed on the screen with the built-in function <>. :0: command line, command syntax, mapping, execution and finish of the command was successful :1: command line, command syntax, mapping, execution and finish of the command was successful but a warning can be found in the log :2: command line, command syntax, mapping, execution was successful but cleanup of the command failed (may not happened) :4: command line, command syntax and mapping was successful but execution of the command returned with a warning :8: command line, command syntax and mapping was successful but execution of the command returned with an error :12: command line and command syntax was OK but mapping of the parameter for command execution failed :16: command line was OK but command syntax was wrong (error from command line parser) :20: command string was wrong (please use built-in function "SYNTAX" for more information) :24: initialization of the parameter structure for the command failed (may not happened) :28: configuration is wrong (determination of configuration data failed, configuration file might be damaged) :32: table error (something within the predefined tables is wrong (internal error)) :36: system error (e.g. load of environment or open of file failed) :40: access control or license error :44: interface error (e.g. parameter pointer equals NULL (only at call of subprogram interface)) :48: memory allocation failed (e.g. dynamic string handling) :64: fatal error (basic things are damaged) :>64: special condition codes from the command for job control The command line is interpreted by CLE the command string by CLP. For example: ------------------------------------------------------------------------ :> flcl conv "read.file='test.txt' write.flam(file='test.adc')" | | |-- the command string is compiled by CLP | | | and might result in a syntax error | |-- conv is part of the command line |-- flcl is part of the command line :> flcl conv=para.txt | | |-- para.txt is part of the command line, but the content | | of para.txt might contain a CLP syntax error | |-- conv is part of the command line |-- flcl is part of the command line ------------------------------------------------------------------------ A command execution (run) will return 0 or 1 on success, 4 if a warning occurs (only some of the files converted), 8 in case of an error or special condition codes greater then 64 can be returned. If the command execution was successful (return code 0) but a relevant warnings was logged then the return code is 1 (anything was fine, but please have a look in the log (e.g. a key is close to expire)). A return code of 2 indicates that a call to the finish function failed. Such a fail of the finish function is not recognizable if the run function (command execution) returns with a warning or an error. On the other side, if the finish failed then an additional warning in the log cannot determined because the condition code is still 2. All completion codes bigger than 8 and smaller or equal to 64 indicate an error in front of the command execution. A relevant warning is a warning written to the log by a used component. Warnings written by components which are removed from procession because the format does not fit or something else, are still in the log but not relevant for the completion code 1. ... print CLE_DOCTYP_SPECIALCODES === Special Condition Codes === Appendix I.1. Special Condition Codes Special condition codes are issued for a dedicated job/batch control. For error conditions, which must be handled in a special way, the condition codes below are defined. :80: Defined compression limit was not achieved but the requested operation was done (FLMRTC_LIM). :88: During decompression, the input file has not been recognized as being a FLAM compressed file. The very beginning of the file is corrupted to such an extent that the FLAM syntax cannot be recognized or the file is empty (FLMRTC_CFF). :112: On charset detection, a text encoded in UTF-8 may erroneously be detected as an ASCII-based charset if the first block of data contains only ASCII characters (<128). In consequence, character conversion will eventually run into an "malformed encoding" (ENOMSG) error on the first multi-byte character which results in this condition code. In this case, it makes sense to re-run the command with the correct ASCII CCSID, for example Latin-1 (FLMRTC_TAS). :160: The default (64 KiB) or the set block size is too small. Please restart the job with a larger block size. This could happen, for example, if PGP is used with BZIP2 compression and large (900 KiB) blocks. In this case, 1 MiB would be a reasonable block size (FLMRTC_NBS). ... print CLE_DOCTYP_REASONCODES === Reason Codes [[CLEP.APPENDIX.REASONCODES]] === Appendix I.2. Reason Codes Below you can find the list of reason codes returned after command execution. Each command returns a reason code for the corresponding warning (return code 4) or error (return code 8). This content is printed on the screen with the built-in function <>. * 1 - Fatal error (may not happen) * 2 - System error (wrong size of data types) * 3 - Handle error (not in right shape) * 4 - Interface error (parameter incorrect) * 5 - Activity not possible * 6 - Memory allocation failed (not enough memory) * 7 - Length error (need more data at read or more space at write) * 8 - Control code wrong * 9 - Version not supported * 10 - Function not supported * 11 - Mode not supported * 12 - Suite not supported * 13 - Confidentiality mode not supported * 14 - Integrity mode not supported * 15 - Verification mode not supported * 16 - Wrong header checksum * 17 - Wrong pot checksum * 18 - Wrong data checksum * 19 - Wrong checksum * 20 - Wrong message authentication code (MAC) * 21 - Wrong key test value * 22 - Syntax error (file not formatted correctly) * 23 - Count error (supplied or calculated amount wrong) * 24 - End of list (handle points behind last or first element) * 25 - Delete of file failed * 26 - Type error (supplied or calculated type wrong) * 27 - Rebuild required (flush not possible) * 28 - No change (informational) * 29 - Status error (handle not in right state) * 30 - Orientation not supported * 31 - Split method not supported * 32 - Maximum delimiter amount achieved * 33 - Error at compression * 34 - Error at decompression * 35 - Maximum header length achieved * 36 - Maximum data length achieved * 37 - Open of FLAMFILE failed * 38 - Record format not supported * 39 - Writing to file failed * 40 - Reading from file failed * 41 - Get position of file pointer failed * 42 - Set position of file pointer failed * 43 - End of file (informational) * 44 - Rename of FLAMFILE failed * 45 - Delete of FLAMFILE failed * 46 - Close of FLAMFILE failed * 47 - Seek in file does not work * 48 - Set of buffer size for FIO failed * 49 - FLAMFILE format not supported * 50 - No FLAMFILE * 51 - No entry found * 52 - Construct is empty * 53 - Connection error * 54 - Unsupported FLAM-ID * 55 - Remote procedure call failed * 56 - Method not supported * 57 - Open of file failed * 58 - Error from ZLIB (GZIP) * 59 - Character set not supported * 60 - Space too small * 61 - Logging facility not available * 62 - Error from character conversion module * 63 - Open of hash output file failed * 64 - Close of file failed * 65 - Error with FKME * 66 - Error at character conversion * 67 - Info not supported * 68 - Band not supported * 69 - Open of report file failed * 70 - Size error (supplied or calculated size wrong) * 71 - Invalid character encountered * 72 - Incomplete character encountered * 73 - Error with byte order mark (BOM) * 74 - Format wrong or not supported * 75 - Auto conversion not possible * 76 - Error from BZIP2 library * 77 - Environment variable are not or cannot be defined * 78 - Dynamic load of a function failed * 79 - Access control or license violation * 80 - Compression limit not reached * 81 - Error from LZMA/XZ library * 82 - Error from XML parser (EXPAT) * 83 - Whole first block is padding * 84 - Command line parser failed * 85 - Mapping failed * 86 - Open of temporary file failed * 87 - Wrong magic bytes * 88 - Corrupted or no FLAMFILE * 89 - Initialization error * 90 - Parameter not formatted correctly * 91 - Binary data detected * 92 - From and to are equal * 93 - Data is encrypted (key required) * 94 - Key value wrong * 95 - Reallocation required * 96 - Invalid username * 97 - Authentication failed * 98 - Hash calculation failed * 99 - Processing of one or more files failed * 100 - Hash/checksum file is empty * 101 - Read of hash/checksum file failed * 102 - Name of hash/checksum file is not valid * 103 - Hash/checksum file is not formatted correctly * 104 - Open of code table failed * 105 - Code table is not formatted correctly * 106 - Attribute settings not supported * 107 - Option wrong or not supported * 108 - Invalid parameter combination * 109 - Open of info file failed * 110 - Protocol wrong or not supported * 111 - Differences found * 112 - Try again with an ASCII code page (LATIN1) for read * 113 - Data incomplete * 114 - Procedure not supported * 115 - Key not found * 116 - OpenPGP error * 117 - Algorithm not supported * 118 - Key is inactive * 119 - Key is invalid * 120 - Key is expired * 121 - Positioning failed * 122 - Sequence error, incorrect order * 123 - Update not possible * 124 - Regular expression error * 125 - Value out of range * 126 - Usage measurement not possible * 127 - Dummy processing not possible * 128 - Open of inverse file failed * 129 - Internal error (may not happen) * 130 - Micro Focus support failed * 131 - Dynamic string library error * 132 - Open of usage log failed * 133 - Serialization failed * 134 - Deserialization failed * 135 - Program call (PC) failed * 136 - Row specification wrong * 137 - Record delimiter not supported * 138 - End of table (informational) * 139 - End of member (informational) * 140 - Have more data (informational) * 141 - Alignment error * 142 - Not enough space available (internal buffer too small) * 143 - Plausibility error * 144 - Unexpected remaining rest * 145 - Length parameter too small (the resulting space is not sufficient) * 146 - Virus found * 147 - Connection was closed * 148 - Error from anti virus scanner * 149 - Authorization failed * 150 - Command execution failed * 151 - Already exists * 152 - Tell offset in file does not work * 153 - Record too long * 154 - Record too short * 155 - Dynamic allocation of a file failed * 156 - Re-open of file failed * 157 - No such file or directory * 158 - Permission denied * 159 - No space left on device * 160 - Restart with larger block size required * 161 - Close/end encountered (special handling required) * 162 - Missing field definition * 163 - Element already deleted * 164 - Error from ZSTD library * 165 - Crypto error * 166 - Open of side/parameter file failed * 167 - Write of side/parameter file failed * 168 - Length mismatch * 169 - Compile of regular expression failed * 170 - Validation based on regular expression failed * 171 - Replacement based on regular expression failed * 172 - Ignore record * 173 - Provided offset invalid * 174 - Pot not valid * 175 - Reopen failed (possible no space for bigger tables) * 176 - User table fail (possible endless recursion defined * 177 - Maximum expansion achieved * 178 - Invalid argument at open or incomplete byte sequence at processing * 179 - Invalid character encountered * 180 - No message of the desired type or malformed character encoding * 181 - Error during whitespace handling * 182 - Error from IBM Unicode Services * 183 - Error from LZ4 library * 184 - Index not valid * 185 - Make path failed * 186 - Row not found * 187 - Column not found * 188 - Wildcrads found * 189 - Empty file written ... print CLE_DOCTYP_VERSION == Version [[CLEP.APPENDIX.VERSION]] == Appendix J. Version Here you can see the version of the program and its components. This content is printed on the screen if the built-in function <> is used. ------------------------------------------------------------------------ --| FL5-FLAM4 VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:47 --|--| FL5-FLUCRUN VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:53:56 --|--|--| FL5-FLCUPM VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:53:50 --|--|--|--| FL5-ORGLST VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--| FL5-ORGNET VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--|--| FL5-ORGIPN VERSION: 5.3.0-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--| FL5-ORGSAV VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--|--| FL5-ORGFIL VERSION: 5.2.0-33135 BUILD: RELEASE Mar 8 2024 20:54:02 --|--|--|--|--|--|--| FL5-ORGFIO VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--|--|--|--| FL5-FIODAT VERSION: 5.1.15-33135 BUILD: RELEASE Mar 8 2024 20:53:41 --|--|--|--|--|--|--|--| FL5-FIOBLK VERSION: 5.1.58-33135 BUILD: RELEASE Mar 8 2024 20:53:41 --|--|--|--|--|--|--|--|--| FL5-FIO VERSION: 1.1.23-33135 BUILD: RELEASE Mar 8 2024 20:53:31 --|--|--|--|--|--|--|--|--|--| FL5-BIOOPN VERSION: 1.1.20-33135 BUILD: RELEASE Mar 8 2024 20:53:29 --|--|--|--|--|--|--|--|--|--|--| FL5-FLSTDIO VERSION: 1.1.9-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--|--| FL5-FILELOCL VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--|--| FL5-FILESSH VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--|--|--| libssh VERSION: 0.8.90-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--|--|--|--| OpenSSL VERSION: 0x1010109F-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--|--| FL5-FLIOLOCL VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--|--| FL5-FLIOSSH VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:17 --|--|--|--|--|--|--|--|--|--|--|--| FL5-DRIOLOCL VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:17 --|--|--|--|--|--|--|--|--|--|--|--| FL5-DRIOSSH VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:17 --|--|--|--|--|--|--|--|--|--| FL5-RIOOPN VERSION: 1.1.20-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--|--|--|--|--|--|--| FL5-RIOEDZ VERSION: 1.1.3-33135 BUILD: RELEASE Mar 8 2024 20:53:40 --|--|--|--|--|--|--|--|--|--| FL5-TIOOPN VERSION: 1.1.19-33135 BUILD: RELEASE Mar 8 2024 20:54:04 --|--|--|--|--|--|--|--|--|--| FL5-YIOOPN VERSION: 1.2.15-33135 BUILD: RELEASE Mar 8 2024 20:54:04 --|--|--|--|--|--|--|--|--| FL5-CMDPRO VERSION: 1.0.0-33135 BUILD: RELEASE Mar 8 2024 20:54:18 --|--|--|--|--|--|--|--|--|--| FL5-FLMSSH VERSION: 1.0.0-33135 BUILD: RELEASE Mar 8 2024 20:54:18 --|--|--|--|--|--|--|--|--|--|--| FL5-CONNSSH VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:16 --|--|--|--|--|--|--|--|--|--|--| FL5-FLCRYPTO VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:27 --|--|--|--|--|--|--|--|--|--|--| FL5-FLHMAC VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:27 --|--|--|--|--|--|--|--|--|--| FL5-EXECCMD VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:17 --|--|--|--|--|--|--|--| FL5-FIOREC VERSION: 5.1.64-33135 BUILD: RELEASE Mar 8 2024 20:53:43 --|--|--|--|--|--|--|--| FL5-FIOTXT VERSION: 5.1.42-33135 BUILD: RELEASE Mar 8 2024 20:53:44 --|--|--|--|--|--|--|--| FL5-FIOFL4 VERSION: 5.2.101-33135 BUILD: RELEASE Mar 8 2024 20:53:42 --|--|--|--|--|--|--|--|--| FL5-FL4REC VERSION: 1.0.0-33135 BUILD: RELEASE Mar 8 2024 20:53:21 --|--|--|--|--|--|--|--| FL5-FIOZIP VERSION: 5.1.36-33135 BUILD: RELEASE Mar 8 2024 20:53:44 --|--|--|--|--|--|--|--|--| FL5-ZIP VERSION: 1.1.25-33135 BUILD: RELEASE Mar 8 2024 20:53:39 --|--|--|--|--|--|--|--|--|--| FL5-FLHASH VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:27 --|--|--|--|--|--|--| FL5-ORGCNV VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:02 --|--|--|--|--|--|--|--| FL5-CNVBLK VERSION: 5.1.7-33135 BUILD: RELEASE Mar 8 2024 20:53:31 --|--|--|--|--|--|--|--|--| FL5-FMTBIN VERSION: 5.1.10-33135 BUILD: RELEASE Mar 8 2024 20:53:57 --|--|--|--|--|--|--|--|--| FL5-FMTBLK VERSION: 5.1.11-33135 BUILD: RELEASE Mar 8 2024 20:53:57 --|--|--|--|--|--|--|--|--| FL5-FMTTXT VERSION: 5.2.45-33135 BUILD: RELEASE Mar 8 2024 20:53:58 --|--|--|--|--|--|--|--|--|--| libpcre2 VERSION: 10.39-33135 BUILD: RELEASE Mar 8 2024 20:53:58 --|--|--|--|--|--|--|--| FL5-CNVREC VERSION: 5.1.11-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--|--| FL5-FMTREC VERSION: 5.1.19-33135 BUILD: RELEASE Mar 8 2024 20:53:58 --|--|--|--|--|--|--|--| FL5-CNVBAS VERSION: 5.1.23-33135 BUILD: RELEASE Mar 8 2024 20:53:30 --|--|--|--|--|--|--|--|--| FL5-BAS VERSION: 1.0.0-33135 BUILD: RELEASE Mar 8 2024 20:53:29 --|--|--|--|--|--|--|--| FL5-CNVPGP VERSION: 5.1.26-33135 BUILD: RELEASE Mar 8 2024 20:53:35 --|--|--|--|--|--|--|--|--| FL5-PGP VERSION: 1.1.19-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--|--|--| FL5-PGPUTL VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:39 --|--|--|--|--|--|--|--|--|--| FL5-PGPPAC VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:38 --|--|--|--|--|--|--|--|--|--|--| FL5-SECUTL VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:27 --|--|--|--|--|--|--|--|--|--| FL5-PGPKEY VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:38 --|--|--|--|--|--|--|--|--|--| FL5-PGPCMP VERSION: 1.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--|--|--|--| zlib VERSION: 1.2.12-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--|--|--|--| libbzip2 VERSION: 1.0.8-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--| FL5-CNVEDC VERSION: 5.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:34 --|--|--|--|--|--|--|--| FL5-CNVGZP VERSION: 5.2.43-33135 BUILD: RELEASE Mar 8 2024 20:53:36 --|--|--|--|--|--|--|--| FL5-CNVBZ2 VERSION: 5.2.29-33135 BUILD: RELEASE Mar 8 2024 20:53:31 --|--|--|--|--|--|--|--| FL5-CNVLXZ VERSION: 5.1.26-33135 BUILD: RELEASE Mar 8 2024 20:53:36 --|--|--|--|--|--|--|--|--| liblzma VERSION: 5.2.5-33135 BUILD: RELEASE Mar 8 2024 20:53:36 --|--|--|--|--|--|--|--| FL5-CNVZST VERSION: 5.2.2-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--|--| libzstd VERSION: 1.5.2-33135 BUILD: RELEASE Mar 8 2024 20:53:37 --|--|--|--|--|--|--|--| FL5-CNVCHR VERSION: 5.2.11-33135 BUILD: RELEASE Mar 8 2024 20:53:34 --|--|--|--|--|--|--|--|--| FL5-CHR VERSION: 1.1.9-33135 BUILD: RELEASE Mar 8 2024 20:53:29 --|--|--|--|--|--|--|--|--|--| FL5-ICV VERSION: 1.2.8-33135 BUILD: RELEASE Mar 8 2024 20:54:05 --|--|--|--|--|--|--|--| FL5-CNVAVS VERSION: 5.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:36 --|--|--|--|--|--|--|--| FL5-CNVHSH VERSION: 5.1.13-33135 BUILD: RELEASE Mar 8 2024 20:53:36 --|--|--|--|--|--|--|--| FL5-CNVSIG VERSION: 5.1.1-33135 BUILD: RELEASE Mar 8 2024 20:53:36 --|--|--|--|--|--|--|--| FL5-CNVCMT VERSION: 5.1.0-33135 BUILD: RELEASE Mar 8 2024 20:53:35 --|--|--|--|--|--|--|--| FL5-CNVCPY VERSION: 5.1.3-33135 BUILD: RELEASE Mar 8 2024 20:53:34 --|--|--|--|--|--|--| FL5-ORGFMT VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:03 --|--|--|--|--|--|--|--| FL5-FMTXML VERSION: 5.1.40-33135 BUILD: RELEASE Mar 8 2024 20:53:58 --|--|--|--|--|--|--|--|--| libexpat VERSION: expat_2.4.9-33135 BUILD: RELEASE Mar 8 2024 20:53:58 --|--|--|--|--|--|--|--| FL5-FMTTAB VERSION: 5.1.42-33135 BUILD: RELEASE Mar 8 2024 20:53:59 --|--|--|--|--|--|--|--|--| FL5-ELMTAB VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:14 --|--|--|--|--|--|--|--|--|--| FL5-TABFIX VERSION: 5.1.6-33135 BUILD: RELEASE Mar 8 2024 20:54:19 --|--|--|--|--|--|--|--|--|--| FL5-TABLFD VERSION: 5.2.8-33135 BUILD: RELEASE Mar 8 2024 20:54:19 --|--|--|--|--|--|--|--|--|--| FL5-TABDLM VERSION: 5.1.9-33135 BUILD: RELEASE Mar 8 2024 20:54:19 --|--|--|--|--|--|--|--|--|--| FL5-TABVAR VERSION: 5.1.1-33135 BUILD: RELEASE Mar 8 2024 20:54:19 --|--|--|--|--|--|--|--|--|--| FL5-TABCSV VERSION: 5.2.11-33135 BUILD: RELEASE Mar 8 2024 20:54:20 --|--|--|--|--|--|--|--|--|--| FL5-TABTVD VERSION: 5.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:20 --|--|--|--|--|--|--|--|--|--| FL5-TABXML VERSION: 5.2.23-33135 BUILD: RELEASE Mar 8 2024 20:54:25 --|--|--|--|--|--|--|--|--|--| FL5-TABRST VERSION: 5.1.0-33135 BUILD: RELEASE Mar 8 2024 20:54:26 --|--|--|--|--|--|--|--|--| FL5-ELMPRC VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:14 --|--|--|--|--|--|--|--|--|--| FL5-CPYPRC VERSION: 5.1.1-33135 BUILD: RELEASE Mar 8 2024 20:54:26 --|--|--|--|--|--|--|--|--|--| FL5-MSKPRC VERSION: 5.1.1-33135 BUILD: RELEASE Mar 8 2024 20:54:26 --|--|--|--|--|--|--|--|--|--| FL5-CNVPRC VERSION: 5.1.1-33135 BUILD: RELEASE Mar 8 2024 20:54:26 --|--|--|--|--|--|--|--|--|--| FL5-EXTPRC VERSION: 5.1.1-33135 BUILD: RELEASE Mar 8 2024 20:54:26 --|--|--|--|--|--|--|--|--| FL5-ELMCNV VERSION: 5.1.28-33135 BUILD: RELEASE Mar 8 2024 20:54:14 --|--|--|--|--|--|--|--|--|--| FL5-BINCNV VERSION: 5.1.6-33135 BUILD: RELEASE Mar 8 2024 20:54:14 --|--|--|--|--|--|--|--|--|--| FL5-STRCNV VERSION: 5.1.11-33135 BUILD: RELEASE Mar 8 2024 20:54:14 --|--|--|--|--|--|--|--|--|--| FL5-INTCNV VERSION: 5.1.8-33135 BUILD: RELEASE Mar 8 2024 20:54:15 --|--|--|--|--|--|--|--|--|--| FL5-FLTCNV VERSION: 5.1.6-33135 BUILD: RELEASE Mar 8 2024 20:54:15 --|--|--| FL5-DIROPN VERSION: 1.1.6-33135 BUILD: RELEASE Mar 8 2024 20:54:28 --|--| FLAM-CLE VERSION: 1.3.87-33135 BUILD: RELEASE Mar 8 2024 20:53:46 --|--|--| FLAM-CLP VERSION: 1.3.135-33135 BUILD: RELEASE Mar 8 2024 20:53:45 ------------------------------------------------------------------------ ... print CLE_DOCTYP_ABOUT == About [[CLEP.APPENDIX.ABOUT]] == Appendix K. About This is the about information for the program (flam4). Its content is printed on the screen if the built-in function <> is used. ------------------------------------------------------------------------ --| The new FLAM4 Utility (identical to 'FLCL FLAM ...') (FL5-FLAM4) --| VERSION: 5.1.28-33135 Build: RELEASE Mar 8 2024 20:54:47 --| Copyright (C) limes datentechnik (R) gmbh --| All rights reserved --| The Frankenstein Limes Universal Converter (FLUC(R)) provides the new FLAM4 utility: --| This program uses the libraries below: --|--| FL5-Core (FL5-CORE) --|--| VERSION: 5.1.28-33135 Build: RELEASE Mar 8 2024 20:53:28 --|--| Copyright (C) limes datentechnik (R) gmbh --|--| All rights reserved --|--| This library uses the libraries below: --|--|--| zlib Version 1.2.12 based on zlib license --|--|--| Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler --|--|--| Thanks to Jean-loup and Mark to provide this software library --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICZLIB.txt in the FLAM install folder --|--|--| https://zlib.net/zlib_license.html --|--|--| https://zlib.net/ --|--|--| On IBM SystemZ we link the extended zlib (including zEDC support) of IBM --|--|--| libbzip2 Version 1.0.8, 13-Jul-2019 based on BSD like license --|--|--| Copyright (C) 1996-2010 Julian R Seward. --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICBZIP2.txt in the FLAM install folder --|--|--| https://sourceware.org/bzip2/ --|--|--| liblzma Version 5.2.5 of XZ Utils project is placed in public domain --|--|--| Copyright (C) 2013 Igor Pavlov and Lasse Collin --|--|--| Thanks to Igor Pavlov and Lasse Collin to provide this software library --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICLLZMA.txt in the FLAM install folder --|--|--| https://tukaani.org/xz/ --|--|--| LZMA SDK Version 21.07 of the 7zip project is placed in public domain --|--|--| Igor Pavlov : Public domain --|--|--| Thanks to Igor Pavlov and Lasse Collin to provide this software library --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICLZMAS.txt in the FLAM install folder --|--|--| https://www.7-zip.org/sdk.html --|--|--| FLAM(R) use only the liblzma of the LZMA SDK --|--|--| libzstd Version 1.5.2 based on BSD license --|--|--| Copyright (C) 2016-present, Facebook, Inc. All rights reserved. --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICZSTD.txt in the FLAM install folder --|--|--| https://github.com/facebook/zstd/blob/dev/LICENSE --|--|--| https://github.com/facebook/zstd --|--|--| liblz4 Version 1.9.3 based on BSD 2-Clause license --|--|--| Copyright (C) 2011-present, Yann Collet. --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICLZ4.txt in the FLAM install folder --|--|--| https://github.com/lz4/lz4/blob/dev/LICENSE --|--|--| http://www.lz4.org --|--|--| Expat Version expat_2.4.9 based on MIT license --|--|--| Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006 Expat maintainers. --|--|--| Copyright (C) 1998, 1999, 2000 Thai Open Source Software Center Ltd and Clark Cooper --|--|--| This library is used in modified form. --|--|--| for license an more information see: --|--|--| doc/txt/LICEXPAT.txt in the FLAM install folder --|--|--| https://github.com/libexpat/libexpat/blob/master/expat/COPYING --|--|--| https://libexpat.github.io/ --|--|--| The modified Expat source can be found at: https://github.com/limes-datentechnik-gmbh/libexpat --|--|--| PCRE2 Version 10.39 based on BSD license --|--|--| Copyright (c) 1997-2021 University of Cambridge. --|--|--| This library is used in unmodified form. --|--|--| for license an more information see: --|--|--| doc/txt/LICPCRE2.txt in the FLAM install folder --|--|--| https://www.pcre.org/licence.txt --|--|--| https://www.pcre.org/ --|--|--| libcrypto Version 0x1010109F-33135 based on SSLeay license --|--|--| This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/) --|--|--| This product includes cryptographic software written by Eric Young (eay@cryptsoft.com) --|--|--| This software is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICOPSSL.txt in the FLAM install folder --|--|--| https://www.openssl.org/source/license.html --|--|--| https://www.openssl.org/ --|--|--| FLAM(R) use the EVP interface (libcrypto) of OpenSSL 1.1.1i 8 Dec 2020 --|--|--| libssh Version 0.8.90 based on LGPL license --|--|--| Copyright (C) 2005-2008 by Aris Adamantiadis --|--|--| Copyright (C) 2008-2018 by Andreas Schneider --|--|--| This library is used in modified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICLSSH.txt in the FLAM install folder --|--|--| http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html --|--|--| https://www.libssh.org/ --|--|--| The modified libssh source can be found at: https://github.com/limes-datentechnik-gmbh/libssh --|--|--| The LGPL conform replacement of the libraries are described at the end of doc/txt/README.txt --|--|--|--| EVP interface (libcrypto) of OpenSSL 1.1.1i 8 Dec 2020 (see above) --|--|--| Lightning memory-mapped database library (liblmdb) --|--|--| Version LMDB 0.9.70: (December 19, 2015) based on OpenLDAP License --|--|--| Copyright 2011-2017 Howard Chu, Symas Corp. --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICLMDB.txt in the FLAM install folder --|--|--| http://www.OpenLDAP.org/license.html --|--|--| https://symas.com/lmdb/technical/ --|--|--| The glibc library based on LGPL license for Linux systems --|--|--| Copyright (C) 1991-2012 Free Software Foundation, Inc. --|--|--| This library is used in unmodified form. --|--|--| for license and more information see: --|--|--| doc/txt/LICLGPL.txt in the FLAM install folder or --|--|--| http://www.gnu.org/licenses/gpl-3.0.html --|--|--| http://www.fsf.org/ --|--|--| Part of the operating system (prerequisite, not part of this distribution) --|--| Frankenstein Limes Command Line Execution (FLAM-CLE) --|--| Version: 1.3.87-33135 Build: RELEASE Mar 8 2024 20:53:46 --|--| Copyright (C) limes datentechnik (R) gmbh --|--| This library is open source from the FLAM(R) project: http://www.flam.de --|--| for license see: https://github.com/limes-datentechnik-gmbh/flamclep --|--| This library uses the internal library below: --|--|--| Frankenstein Limes Command Line Parser (FLAM-CLP) --|--|--| Version: 1.3.135-33135 Build: RELEASE Mar 8 2024 20:53:45 --|--|--| Copyright (C) limes datentechnik (R) gmbh --|--|--| This library is open source from the FLAM(R) project: http://www.flam.de --|--|--| for license see: https://github.com/limes-datentechnik-gmbh/flamclep ------------------------------------------------------------------------ ... print CLE_DOCTYP_CHAPTER == Glossary [glossary] == Glossary :ADC: Abbreviation for Advanced Data Compression, which is the default compression algorithm of FLAM4. :AES: The Advanced Encryption Standard (AES) is a symmetric crypto system, the successor of DES and 3DES. It uses the Rijndael algorithm with a fixed block size of 128 bits and a variable key length of 128, 192 or 256 bits, providing a high grade of security. :ASCII: The American Standard Code for Information Interchange (ASCII) is a 7-Bit encoding, which is the source for later character sets based on more bits. :AVS: Anti virus scanning :BAS: Abbreviation for base encoding. :BCS: Banking Communication Standard. + Specification for remote data transmission between customers and credit institutions. The BCS is used, for example, in cash management systems to enable multi-bank capability, i.e. the processing of data from different credit institutions with software. :BIG ENDIAN: Endianness refers to the term byte order. A big-endian machine stores the most significant byte first, i.e. at the lowest byte address. This behavior is most important when two computers with different byte orders communicate with each other. The big-endian format is used by Motorola 6800 and 68k, Xilinx Microblaze, IBM POWER, System/360, System/370, ESA/390, z/Architecture and PDP-10. :BIN: Abbreviation for binary. :BOM: The Byte Order Mark (BOM) characterizes in which order the bytes are interpreted. This is important when exchanging data between different systems. For UTF-16 and UTF-32 a BOM sign at the beginning of the data stream is necessary, as more than one byte is used for the encoding of one character. UTF-8 uses the BOM sign "EF BB BF", which is used to mark a text file as UTF-8, even though byte order is not issue here. :BOMUTF: With this option of the FLCL, you can determine the correct UTF CCSID from the byte order mark (BOM). :BOMUCS: With this option of the FLCL, you can determine the correct UCS CCSID from the byte order mark (BOM). :BYTE ORDER: The byte order is the order in which bytes are interpreted for entities that consist of multiple bytes, e.g. Unicode characters. This is important when exchanging data between different systems. A big-endian machine stores the most significant byte first, i.e. at the lowest byte address. A little-endian machine, however, stores the least significant byte first. :BZ2: Abbreviation for BZIP2. :CANONICAL EQUIVALENCE: Canonical equivalence is a fundamental equivalence between individual Unicode characters and sequences of Unicode characters. :CASEMETHOD: FLCL offers a method for converting unicode characters to uppercase, lowercase or some special representation (CASE=UPPER/LOWER). :CCSID: This is an abbreviation used by IBM for "Coded Character Set Identifier". The encoding of a specific code page is referred to by a 16-bit number. see also: https://en.wikipedia.org/wiki/CCSID[CCSID] :CHAR: Abbreviation for the conversion of character streams. :CHR: Abbreviation for character :CLAMD: Clam Anti Virus Daemon :CNV: Abbreviation for conversion. Collects all conversion components which transform data stream in data stream (e.g. character conversion, compression, de/encryption, de/encoding, ...). :CODEPAGE: IBM introduced a numbering scheme to character encodings (16 bits). This scheme was taken over by other vendors like Microsoft, SAP or Oracle Corporation. The term Codepage is used synonymously for character encoding or character set. :CODE POINT: A code point or code position is a single character in a list of characters represented as numerical values that create the code space. The Unicode code space has a total size of 17 * 65536 code points. The same code space may have several codepages. :COMPATIBILITY EQUIVALENCE: One speaks of compatibility equivalence of two sets of characters if their full compatibility decompositions are identical. :CONV: The sub-program FLUC offers all possible conversion and formatting methods via the CONV command. These are various read and write procedures, many useful conversion tools and some special data formatting features. :CNV: Conversion component :CRY: Abbreviation for crypto(graphy). :CX7: Compression encoding as printable characters in FLAM4 :CX8: Compression encoding as binary data in FLAM4 :DAT: Abbreviation for data. :DIACRITIC: A diacritic is also called diacritical mark, diacritical point or diacritical sign. Typical diacritical marks are the acute (´) and grave (`), which are often called accents. Diacritical marks may be a dash, dot or tick above or below a letter, or in some other position such as within the letter or between two letters. :DIR: Abbreviation for directory (walk). :EBCDIC: The Extended Binary Coded Decimals Interchange Code (EBCDIC) was developed by IBM as 8-Bit-encoding, which, however, does not use all code words. EBCDIC originates from the older 4-Bit-Code BCD (Binary Coded Decimals) and is applied solely on Mainframes. The EBCDIC codepages are available in German, US, Swedish, Arabian, Latin-1 and other languages. Moreover, there exists a Unicode set of characters based on EBCDIC and it is called UTF-EBCDIC. :BLK: Abbreviation for block. :ENV: Abbreviation for environment. :FINGERPRINT: In public-key cryptography, a public key fingerprint is a short sequence of bytes used to identify a longer public key. Fingerprints are created by applying a cryptographic hash function to a public key. Since fingerprints are shorter than the keys they refer to, they can be used to simplify certain key management tasks. + see also: https://en.wikipedia.org/wiki/Public_key_fingerprint/[Public key fingerprint] :FIO: Abbreviation for File I/O. All components to read or write files. The C programming language has a general mechanism for reading and writing files abstracting all Input/Output to streams of bytes. The operating system offers redirection which can be very useful. Examples are the standard input stream, standard output stream and standard error stream. :FKME: FLAM Key Management Exit/Extension Service provider interface to connect FLAM till version 4 with different kinds of cryptographic infrastructure, hardware security modules and/or key management systems. :FL4: Abbreviation for FLAM in Version 4 or lower. :FL5: Abbreviation for FLAM in Version 5. :FLAM: Frankenstein Limes Access Method. FLAM is our access method for encrypted and compressed data. In particular, FLAM is a utility for converting, compressing, encrypting (with AES (Advanced Encryption Standard)), signing and formatting of mass data with simple interfaces for integration in all kinds of solutions. All this is done with a minimum of CPU utilization, utilization of co-processors and the use of professional key management solutions. Ordinarily, FLAM provides - when used as sub-program - the commands COMP and DECO and makes use of an element interface for the subsystems/plugins/drivers and also the sub-program. :FLAM4: This component writes a Flamfile record-oriented. You can state the name of a Flamfile or that of a member. :FLAMFILE: The FLAMFILE is composed like a package file, as multiple files can be compressed. It is structured clearly and divided hierarchically in records, segments and members. In this connection a segment is referred to the single matrix of a compressed file and a member to the entity of segments that are part of the compressed file. In order to ensure the universal exchangeability, the FLAMFILE has to be available in a format that is representable on all operating systems (so-called heterogeneous compatibility). Each compressed data segment is unique referring to data content, environment and time. This allows - despite compression/encryption - an access down to the record-level, without decompressing/decrypting. :FLCC: Frankenstein Limes Control Center (GUI).In contrast to FLCL this is the graphical user interface for running FLAM, FLUC and FLIES, quasi a dialogue utility :FLCL: Frankenstein Limes Command Line. FLCL offers a command line for the three sub-programs FLAM, FLUC and FLIES. It is a quasi-batch utility for the sub-programs. Moreover, FLCL is a universal data format and character set converter, which can be included easily in professional internationalization projects. It supports different input and output formats (GZIP, PDF, B64...) and can be integrated into scripts or as sub-program. More than 150 codepages are supported. FLCL can be applied to various commonly used platforms if working with processable data streams or file I/O. :FLIES: Frankenstein Limes Integrated Extended Security. The sub-program FLIES governs the compressed and encrypted segments using commands like FIND and CHNGE. In particular, it is useful for processing Flamfiles, which however cannot be decrypted or decompressed. For instance the data in the file header is managed by FLIES to re-encrypt the file, change the data access, use as backup, archive or as mediator/headend. :FLUC: Frankenstein Limes Universal Converter. FLUC is a universal text converter with special focus on the high requirements in professional internationalization projects, where Mainframes and external applications are used as central backend. :FLUCUP: FLUC Subprogram as C interface (DLL) for Data source to target conversion. :FMT: Abbreviation for formatting. All components to parse a data stream in FLAM5 element list or build a data stream from FLAM5 elements list. :FROM-CODE: The FROM-Code specifies the character encoding of the input data that used as source encoding for character conversion. :GZIP: GZIP is a compression program for nearly every operation system and uses the DEFLATE algorithm from the ZLIB library. GZIP is the short form for GNU zip. The used algorithm is a combination of LZ77 and Huffmann encoding. The source code is licensed under the GPL. The program has 9 compression steps providing either optimal CPU time or compression ratios. The compressed file has the ending ".gz". As GZIP compresses only single files. Several files must first be combined with (for example) "tar" to a tarball which can then be compressed to a ".tar.gz" file. GZIP files have a 10-byte header, containing a magic number, a version number and a timestamp. With FLCL you are able to read and write GZIP files, to mark content as text data in the header, set operating/file system or modified time (seconds since 1970). :GZP: Abbreviation for GZIP. :HOST: see MAINFRAME :HSH: Abbreviation for hash (checksum) :ICNV: Abbreviation for iconv conversion with simplified commands on the command line. :ICONV: With the standard program iconv, characters can be translated between different encodings on UNIX-like systems. As iconv is now part of the GNU C Library, actual Linux systems are equipped with iconv. Under Windows(R), iconv is provided only as DLL or static library, which is under the license of the GPL and can be used via Cygwin or GnuWin32. Iconv is also supported as Dynamic Link Library for PHP. :ID: Abbreviation for identifier. :ISO-8859: The ISO-8859 norm family contains some useful 8-bit character sets (e.g. Latin-1 to Latin-15). The frequently used Latin-1 character set is identical with the first 256 characters of the Unicode character set. These codepages are also called extended Ascii. :LITTLE ENDIAN: A little-endian machine stores the least significant byte first. The endianness of a 16, 32 or 64 bits word or even bits can occur in some Unicode representations. Also mixed forms of endianness are possible, for example when the byte order within a 16 bits word differs from that of a 16 bits word within a 32 bits word. The little-endian format is used by x86 (including x86-64), 6502 (including 65802, 65C816), Z80 (including Z180, eZ80 etc.), MCS-48, 8051, DEC Alpha, Altera Nios II, Atmel AVR, SuperH, VAX, and PDP-11. :LOGGING: In FLCL various messages (warnings, errors, debug information, important information, input data) can be logged. The messages can be redirected from stderr to stdout or a file. :LXZ: Abbreviation for LZMA XZ. :MAINFRAME: A mainframe is a very complex and powerful computer system. Its capacity is far beyond that of a common Personal Computer and that of most servers. They are applied for the highly reliable processing of mass data, mass transactions and company critical data and gain importance again in connection with server consolidation and cloud computing. :MODE: With the keyword MODE in FLCL you can select the mode of operation. :NDC: Abbreviation for No Data Compression, which can be considered as copying data in FLAM4. :NFC: This is Normalization Form C (Canonical Decomposition, followed by Canonical Composition). :NFD: This is Normalization Form D (Canonical Decomposition). :NFKC: This is Normalization Form KC (Compatibility Decomposition, followed by Canonical Composition). :NFKD: This is Normalization Form KD (Compatibility Decomposition). :Normalization: The transformation of a character sequence in one of the four normal forms is called normalization. Normalization is necessary, because there is more than one differing possibility for many characters to be represented as a sequence of Unicode characters. The four normal forms consist of one form for the canonical equivalence, one for the compatible equivalence, and both can be in a "composed" or "decomposed" form. :OS: Abbreviation for operation/file system in the header :PADDING: Padding are bytes inserted to align the data to certain lengths or storage positions. Padding bytes are most commonly appended after the the last data byte. When reading padded data, the padding bytes are discarded. :PDS: Partitioned data set. A PDSE consists of a directory and zero or more members, :PDSE: Partitioned data set extended. Exactly same as PDS in many respects. PDSE data sets can be stored only on DASD, not on tape. Interesting thing is the directory can expand automatically as needed. Additionally it has an index which helps to locate the members inside the PDSE faster. SPACE from deleted members are automatically reused in PDSE . :PGP: Abbreviation for pretty good privacy (OpenPGP). :PLATFORM NEUTRAL: The components of FLAM are platform neutral. This means that they work independently from the used operating system. Operating systems can be for example LINUX, UNIX-like systems, WINDOWS(R) or z/OS(R). :PO: Data set organisation partitioned organized. :PS: Data set physical sequential file. :REC: Abbreviation for record. :SEPA: The Single Euro Payments Area (SEPA) is a project to harmonize the way retail payments are made and processed in Euro. The motivation behind it is to make payments in Euro and across the EU as fast, safe and efficient as national payments are now. SEPA shall enable customers to make cashless Euro payments to anyone in the EU, for instance by credit transfer, direct debit or debit card. + see also: https://en.wikipedia.org/wiki/Single_Euro_Payments_Area/[SEPA] :SIG: Abbreviation for signature :SSH: Secure shell + see also: https://en.wikipedia.org/wiki/Secure_Shell/[SSH] :SUB CHARSET: A Sub Charset is a partial quantity of a character set. For instance the new identity card issued in Germany (nPa) requires that each printed character - even the diacritical characters - is "composed". It means that they remain in their original form and are not "decomposed" into several bytes. For that reason, the Sub Charset "Latin in Unicodes" with some more than 500 letters is used. :SUBCHR: FLCL can substitute characters that are unsupported in their target encoding by system specific characters (e.g. 0x1A for ASCII or 0x3F for EBCDIC pages) or by arbitrary characters. In the first case, you must use SUBCHR=SYSTEM, in the other you can define your own substitution characters. On z/OS(R), the utility iconv is available on the system and replaces all invalid characters by system specific characters, including the BOM sign. Note that it does not stop in contrast to LINUX or WINDOWS(R) but completes conversion. :SUBSTITUTE: If a character exists in the source code but not in the target code, it will be converted to a converter-defined substitute character. :SURROGATE: The Unicode code space is divided into 17 planes of 65536 code points each. Some code points have no character value, some are reserved for private use, and some are permanently reserved as non-characters. UTF-16 with code points beyond these planes is encoded in 16 bit surrogate pairs (High surrogate and low surrogate), which are in fact replacement characters. UTF-16 surrogate values are illegal in UTF-32. Note that the conversion of UTF-16 strings into UTF-8 byte sequences requires that the High and Low surrogate have to be combined correctly before they are transfered into UTF-8. :SYSTEM TABLE: FLCL makes use of a pre-loaded system substitution table for the transliteration of characters. This is comparable to "translit.h" of iconv under LINUX. :TAB: Abbreviation for table. :TAR: Tape Archiver; Short for Tape Archive, and sometimes referred to as tarball, a file that has the TAR file extension is a file in the Consolidated Unix Archive format. A program or command that can open archives is needed to open a TAR file. Because the TAR file format is used to store multiple files in one single file, it's a popular method for both archiving purposes and for sending multiple files over the internet, like for software downloads. The TAR file format is common in Linux and Unix systems, but only for storing data, not compressing it. TAR files are often compressed after being created, but those become TGZ files, using the TGZ, TAR.GZ, or GZ extension. :TARGET: Trans-European Automated Real-time Gross Settlement Express Transfer System. + see also: https://www.ecb.europa.eu/paym/target/target2/html/index.en.html/[TARGET] :TXT: Abbreviation for text. :TO-CODE: With few utilities you can convert text from one encoding to another. The TO-Code specifies the output encoding that characters are converted to. :TRANSCRIPTION: To transcribe the expression of a language means by dictionary, to represent it in another writing system rather in that, in which the language is usually written respectively in which it is available in written form. For the term of transcription it does not matter, in which medium the expression is available. Note that the transliteration of an expression is a particular type of transcription. :TRANSLATION: This is the transfer of a (mostly written) and fixed text from the original language into any target language; the process will be also denoted as "Translating". :TRANSLITERATION: To transliterate an expression represented in writing S1 into a writing system S2 means, that the characters contained in the expression are exchanged from S1 to S2 according to a fixed rule. In the simplest case, this can be a table which assigns each character of S1 a character of S2. Frequently, there are complications for the simple case, if for example a single character is assigned to a digraph or vice versa or if a character is mapped in context A onto a specific character, in context B however onto another character. For example: in a payment order from Germany, the name of the receiver must be written explicitly in Latin letters so that a transliteration of the original text with help of a table is required. :UNICODE: The so-called Unicode is an attempt, started in the 90s, to standardize a single character set that eliminates the need for different and incompatible encodings in different countries and cultures. The text should be always readable with a Unicode-supporting appropriate editor. This allows an error free data exchange (for the purpose of readability) also on a global and international level. For this character set, 8 bits (1 byte) are no longer sufficient to for a representation of every possible character. The length of characters encoded in Unicode can be between 1 and 4 bytes. ISO 10646 is a practically equal notation of the Unicode character set called Universal Character Set (UCS). :URL: Uniform Resource Locator + see also: https://en.wikipedia.org/wiki/URL/[URL] :USER TABLE: For character set conversion in FLCL you can define your own user substitution table to transliterate those characters that are not handled respectively replaced by the system table. :UTF-8: The 8-Bit Unicode Transformation Format is the most commonly used encoding for Unicode characters. The term Unicode is used equivalent to UCS. UTF-8 has a central meaning in the global character conversion for internet protocols. :UTF-16: In UTF-16 an encoded byte sequence with a length of 2 bytes is assigned to each Unicode character. UTF-16 is used in Java for internal character sequence representation. UTF-16 is the eldest format of the Unicodes. :UTF-32: UTF-32 (UCS-4) is encoded with 4 bytes and therefore the most simple Unicode representation, as other formats need variable byte lengths for their representation. One disadvantage of UTF-32 is the greater space requirement compared to UTF-8 and ISO-8859 character sets and the missing backward compatibility to ASCII.# :VSAM: Virtual Storage Access Method + see also: https://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zconcepts/zconcepts_169.htm/[VSAM] :VR8: Enhanced compression encoding as binary data in FLAM4 :ZIP: ZIP is an archive file format that supports lossless data compression. ... print CLE_DOCTYP_CHAPTER == Imprint [colophon] == Imprint ----------------------------------------------------------------------- limes datentechnik(R) gmbh Louisenstrasse 101 D-61348 Bad Homburg v.d.H. phone: +49(0)6172-5919-0 fax: +49(0)6172-5919-39 mail: info@flam.de web: www.flam.de or www.limes.de area of jurisdiction: Bad Homburg vor der Hoehe HRB 3288 (gegr. 1985) managing directors: Falk Reichbott, Ute Wiebach limes datentechnik(R): efficiency at the limit of possibility. -----------------------------------------------------------------------