Below you can find more details about the construction of a FLAMFILE.
In order to achieve the aim of universal interchangeability, the structure of the FLAMFILE must be such that it can be represented in any operating system. This "least common denominator" is the sequential file with fixed-size records. Although other FLAMFILE record formats are allowed for the FLAMFILE in addition to the fixed format, the compressed data which is generated by the compression procedure is broken down into "abstract" records - known as FLAM records - with an identical size; these records are then mapped to the "concrete" structures of the installed operating system. In addition to the compressed data, each FLAM record contains a FLAM record size field at the beginning and either a checksum or a check character at the end. The FLAM record size field and the checksum or the check character are elements of the FLAM syntax. This permits the start and end of a FLAM record to be identified irrespective of the record format in which the FLAMFILE has been stored by the operating system.
In FLAM4 terminology, a FLAM record is the same as a record if the FLAMFILE has fixed and variable record formats. The specified FLAM record size is thus the same as the record size or the maximum record size.
The residual data in a compressed block belonging to a FLAMFILE is merged with the data in the next block to form a FLAM record with the fixed or maximum size. With the possible exception of the final record, all the FLAM records have the same length. This enables optimum use to be made of the storage medium; however, it is not possible to assign the records of the original file to the records of a FLAMFILE uniquely. A FLAMFILE of this type can therefore only ever be decompressed in its entirety.
FLAM supports fixed and variable or stream record formats for the
FLAMFILE. Irrespective of the record format, the minimum permissible
size of a FLAM record is 80 bytes and the maximum size 32,760 bytes
(4095 for mode=cx7), providing a lower upper limit has not been set by
the user. This span is sufficient in practice to allow any FLAMFILE to
be transferred, even in situations where this would not necessarily be
the case with the original file, owing to restrictions imposed on the
record size or the record format by the file transfer product.
Even though many data exchange problems can be eliminated to a greater or lesser extent with the FLAM4 concept, there are some which still cannot be solved. If a file is unsuitable for uncompressed transfer with a particular file transfer product, for example on account of the binary data it contains, it will still not be possible to transfer it after it has been compressed with FLAM, even if it is compressed in the cx7 mode. The reason for this is that every bit combination which occurs in the original file also occurs in the FLAMFILE.
In addition to the compressed data, a FLAMFILE may contain other information which is stored in the FLAM file header. FLAM distinguishes between two information categories in two different types of file header:
Common information is stored in the common FLAM file header. This includes all information about the original file, such as the file organization, record format, length and position of the primary key, etc., as well as details of the FLAM version and the operating system in which the FLAMFILE was created.
User-specific information of any kind is stored in the user-specific FLAM file header. This header type can be inserted additionally if the record interface is used, for example for information which must be evaluated by the application program.
Creation of the common FLAM file header is supported by all interfaces (command, subprogram and record) by means of parameters and arguments. A user-specific FLAM file header can only be created if the record interface is used. This interface incorporates the following functions for creating and evaluating the various file header types:
FLAM file header type |
Created by |
Evaluated by |
|---|---|---|
Common |
flmphd |
flmghd |
User-specific |
flmpuh |
flmguh |
It is not mandatory to insert FLAM file headers in the FLAMFILE. If they are inserted, it is essential to observe the stipulated order when invoking the functions of the record interface.
FLAM file headers are always inserted in the FLAMFILE directly preceding the compressed version of the file to which they belong. If a FLAMFILE contains compressed versions of several files, a FLAM file header can be inserted in front of each compressed file.
When the files are decompressed, the compressed data originating from different files can be distinguished and decompressed into separate files.
Each record in a FLAMFILE is protected against manipulations and transmission errors by an appended checksum. In compliance with increased requirements regarding cryptographic security, additional security features for encrypted FLAMFILEs have been implemented which improve the protection against a greater variety of attacks. A FLAMFILE provided with these features is called "Secure FLAMFILE".
Here, "segment" denotes the compressed result of a single matrix and "member" the entirety of segments from a single original file.
In a Secure FLAMFILE, there are byte and record counts both for each
member as well as for the entire file, members are numbered
consecutively, and a timestamp marks the creation time. When encrypted
with AES, MACs (Message Authentication Codes) are attached to the
FLAMFILE. Those are checksums calculated with a secret key that allow
verifying the authenticity of the data. A MAC is calculated for each
segment. To ensure the completeness of a member, a second MAC is chained
through all segments of the same member, and a third MAC, chained
through all members of the FLAMFILE, ensures the completeness with
regard to the file. Similar features are included when FLAMFILEs are
created with mode=flamenc.
To include the information necessary for these checks, such as counts, timestamps, MACs, etc., a member header is inserted in front of each member and each member is followed by a member trailer. In addition, a file trailer is appended at the end of every Secure FLAMFILE.
Secure FLAMFILEs may not be concatenated since this would create discontinuities of numbering, counters, and MAC chaining that would make FLAM assume integrity violations and break off processing. Individual members of a FLAMFILE, however, can be extracted, since checking of the member chaining is suspended, whenever members are skipped, and only the segment chaining is then tested.
All formats and file organizations for the original file are supported for the compressed file, called FLAMFILE (PS, IS, VSAM-ESDS, -KSDS, -RRDS).
Compressed files in VSAM KSDS format can be modified record by
record using record level interface.
With parameter CRYPTOM=AES, MODE=CX8 or VR8 it is
possible to update, insert, delete original records in a compressed
and AES-encrypted VSAM-KSDS FLAMFILE.
The use of this record level interface and the facility to store compressed data in index sequential files (VSAM KSDS) allows a fast random access to compressed data. This is well suited for setting up online archives for documents and similar data.