BIFLAM
This appendix describes the line compression and decompression (BIFLAM) solution delivered with FLAM4 for the IBM z/OS environment.
Bi-/serial compression with BIFLAMK
Bi-/serial compression does not employ FL-Matrices. The compression effect is achieved by comparing the original data with a sample and/or by using serial compression techniques.
BIFLAMK works synchronously. Each call results immediately in output data. It does not need a memory-compression is done on a call-by-call or record-by-record basis. Bi-/serial compression is especially suitable for integration into other products or applications.
The compression effect is considerably lower than compression based on the FL-matrix. The advantage is the independence of each record. In many environments, this independence is a necessary requirement for integration.
Beside compression, BIFLAMK offers two more functions that become more and more important as requirements for data security and data integrity are rising. All compressed records are encrypted and protected against modification using checksums over both original and compressed file.
BIFLAMK offers several compression modes that can be selected via a function code.
First, a vanilla serial compression mode is provided that does not require a sample record. All compressed records are independent from each other and can be decompressed individually.
Second, a bi-serial mode is provided that can be adapted to different environments. This compression mode is based on a comparison of the record with a sample record byte by byte.
The result is a bitmap that denotes all positions where characters are equal and the remaining differing characters.
The first option allows to control the post-processing of the remainder. A serial compression can be applied to the remainder or the remainder is simply encrypted.
The serial compression of the remainder can be bypassed if the CPU overhead is considered too high or the compression ratio is satisfactory.
The second option controls the processing of the sample. If a dynamic sample is used, a checksum is computed for each sample record and is inserted into the compressed record. This slightly reduces the compression effect and increases CPU overhead. On the other hand, data security is improved because modifications are detected more easily. For an exact error analysis it is even possible to decide whether the compressed record or the sample has been modified. If a static sample is used, no checksum of the sample is computed. In this case, modifications in the sample can only be detected by using the checksum of the original file.
The third option allows the storage of samples within the compressed file. During decompression, these records are restored again as samples. With this method, BIFLAMK can create sequences of records that can be decompressed by BIFLAMD without additional information (samples).
One possibility would be to provide one sample first. Then all records are compressed against this sample and bi-serial compression is used to compress the remainders as well.
Another possibility is to use dynamic samples by using the processor of each record as the sample for the current record. This sequence result in good compression ratios if adjoining records are similar (reports, data entry lists). The disadvantage is that the individual records are not independent from each other. This sequence must be decompressed as a whole. However, additional information (like samples) is not necessary.
It does not make much sense, to use a different sample for each record. Sample records can only be compressed with simple serial compression. In addition the compressed data of the original data has to be stored as well.
Principle of operation BIFLAMK
BIFLAMK is used for compression of data record by record. The compressed data is always returned in the same call.
BIFLAMK is reentrant. For operation, a working storage area is needed that must be provided by the calling program. The content of the area before the call is ignored by BIFLAMK. All calls to BIFLAMK are totally independent from each other. All areas can have any alignment. The working storage area for the input record and for the compressed record must not overlap. In-place compression is not possible.
| Name | BIFLAMK |
| Parameters | |
R1 |
Address of parameter list |
R13 |
Points to save area (18 words) |
R14 |
Contains return address |
R15 |
Contains call address (entry point) |
| Parameter list | ||
|---|---|---|
| 1. FUCO | F | Function code |
0 |
Serial compression without sample |
|
8 |
Biserial compression with sample, serial post compression of the remainder and static sample |
|
9 |
Sample record for biserial compression with serial post compression |
|
10 |
Biserial compression with sample, serial post compression of the remainder and dynamic sample |
|
11 |
Sample record for biserial compression with serial post compression |
|
12 |
Biserial compression with sample, serial post compression and encryption of the remainder and static sample |
|
13 |
Sample record for biserial compression with encryption |
|
14 |
Biserial compression with sample, serial post compression and encryption of the remainder and dynamic sample |
|
15 |
Sample record for biserial compression with encryption |
|
| 2. RETCO | F | Return code |
0 |
Function executed |
|
2 |
Invalid function code |
|
3 |
Length error |
|
| 3. WORK | XLn | Working storage area. The working storage area must be at least 512 bytes in size. For biserial compression the working storage area must be at least 512 bytes + length of return area. |
| 4. WRKLEN | F | Length of working storage area in bytes |
| 5. BUFLEN | F | Length of return area or maximum length of compressed record; this size must be at least 8 bytes + 1.1 * length of original record. |
| 6. RECIN | XLn | Original record |
| 7. RECLEN | F | Record length in bytes |
| 8. COMPREC | XLn | Compressed record (length of this area = BUFLEN) |
| 9. COMPLEN | F | Length of compressed record in bytes The next two parameters are only used for biserial compression: |
| 10. SAMPREC | XLn | Sample |
| 11. SAMPLEN | F | Sample length in bytes |
Bi-/serial decompression BIFLAMD
BIFLAMD decompresses compressed records created with BIFLAMK.
For the serial decompression, no sample record (including length) is used. This implies that BIFLAMD has two fewer parameters. Therefore, it is necessary to inform BIFLAMD during the call which mode (serial or bi-serial) should be used for decompression.
For a proper decompression, it is necessary that BIFLAMD receives exactly the same compressed records and sample records as created with BIFLAMK. Modification (code translations) must not be applied to the compressed data and the sample records. If compressed data shall be transmitted between different computers, the file transmission must be transparent.
BIFLAMD automatically detects if a record was compressed serial or bi-serial and will report an error message if the compression syntax does not correspond to the call function code. Also modifications in compressed data, sample records and original data are detected via the checksums.
Principle of operation BIFLAMD
BIFLAMD is used for record by record decompression of compressed data that was created with BIFLAMK.
BIFLAMD is reentrant. For operation, a working storage area is needed that must be provided by the calling program. The content of the area before the call is ignored by BIFLAMD. All calls to BIFLAMD are totally independent from each other. All areas can have any alignment. The working storage area for the compressed record, for the sample record, and for the output record must not overlap. In-place decompression is not possible.
| Name | BIFLAMD |
| Parameters | |
R1 |
Address of parameter list |
R13 |
Points to save area (18 words) |
R14 |
Contains return address |
R15 |
Contains call address (entry point) |
| Parameter list | ||
|---|---|---|
| 1. FUCO | F | Function code |
0 |
Serial decompression without sample |
|
8 |
Biserial decompression with sample |
|
| 2. RETCO | F | Return code |
0 |
Function executed |
|
1 |
Sample record for biserial decompression returned; no original record was written (only during biserial decompression) |
|
2 |
Invalid function code or record is compressed serially (function code = 8) or record is compressed biserially (function code = 0). |
|
3 |
Length error - working storage area too small - compressed record smaller than 3 bytes - return area too small. |
|
4 |
Checksum error in compressed record |
|
5 |
Checksum error in sample record (only for dynamic samples) |
|
6 |
Checksum error in original record |
|
7 |
Other errors in compressed record |
|
8 |
Sample record is shorter as during compression (only for biserial compression) |
|
9 |
Compressed record too short |
|
| 3. WORK | XLn | Working storage area. The working storage area must be at least 512 bytes in size. For biserial compression, the working storage area must be at least 512 bytes + 1.125 * the sum of length of the return areas. |
| 4. WRKLEN | F | Length of working storage area in bytes |
| 5. BUFLEN | F | Length of return areas; maximum length of the original record or the sample record in bytes |
| 6. RECOUT | XLn | Original record (length of area = BUFLEN) |
| 7. RECLEN | F | Record length in bytes |
| 8. COMPREC | XLn | Compressed record |
| 9. COMPLEN | F | Length of compressed record in bytes The next two parameters are only needed for biserial decompression: |
| 10. SAMPREC | XLn | Sample record (length of area = BUFLEN) |
| 11. SAMPLEN | F | Length of sample record in bytes |
| 12. RETCO | F | Return code |
Compression data flow with BIFLAMK
BIFLAMK processes original records and sample records on a record by record basis.
When only serial compression is used (function code 0), BIFLAMK only processes original records and compiles them into compressed records.
If bi-serial compression with sample (function codes 8,10,12,14) is used, an original record is processed in combination with a sample record to create a compressed record. For storage of the sample records (function codes 9,11,13,15), only the sample record is processed to create a compressed record.
Compression data flow with BIFLAMD
BIFLAMD processes compressed records record by record if necessary in combination with a sample record - and creates an original record or a sample record.
During serial decompression (function code 0), the compressed record is only processed and will always create an original record. Sample records are not necessary.
During bi-serial compression (function code 8), always one compressed record is processed at a time. Depending on compression, the sample record is read in addition and the original record is created. If a sample record was compressed during compression, this sample record is reconstructed during decompression. This situation is signalled with a return code of 1.