FLAM is a method for context-free data compression. No pre-analyses of the data and no structure descriptions are required. The procedures used in FLAM-ims use the same approach.
With FLAM-ims, segments of an IMS database are compressed before writing and decompressed after reading. During compression, an checksum is calculated. The decompressed data is verified according to this checksum.
With FLAM-ims, segments can be compressed from 30% to 60% of their original size. This saves disk space and, in batch processes, I/O time and thus processing time. Without taking parallel processes into account, the processing time is reduced to about 50% of the time needed without FLAM-ims.
FLAM-ims has tamper protection. Each compressed record contains control elements that enable data to be recovered without doubt or that detect and indicate data falsification. This also applies to those records whose compression is not worthwhile.
No program changes are necessary for the use of FLAM-ims. The exit provided by IBM is used.
The exit provided by IBM for editing and compressing segments is used for compressing the database segments. The restrictions specified by IBM also apply. The exit is described in the IMS System Programming Reference Manual.
Segment types can be compressed with the access methods HDAM and HIDAM, combined with VSAM or OSAM. Segments that are described as logical children or that are in IMS/VS FAST PATH access cannot be compressed.
IBM provides 2 options for compression, with and without a key. When compressing with a key, the whole segment is compressed, without a key only the data part after the key is compressed. For faster search processes, compression without key is recommended. When compressing with a key, the segment must be decompressed before each check. For searches in dialogue, this version results in very long response times.
In FLAM-ims, only compression of the segments without key is used, because it uses the standard IMS EXIT. FLAM-ims is therefore compatible with all data dictionaries.
In the DBD (Data Base Description), the definition for compression must be entered for all segment types to be compressed. Further changes to existing database descriptions, such as the PSB (Program Specification Block), are not necessary.
FLAM-ims programs must be located as loadable modules either in the IMS library IMSVS.RESLIB or in another load library which is mapped to IMSVS.RESLIB. In the second case, the IMSVS.RESLIB must be authorised with DFSRESLB.
It should be noted that short segments cannot be optimally compressed. Depending on the character string, expansions are also possible with such segments. The minimum segment length used for compression is 24 bytes.
If a segment becomes longer during compression, a disguised original (manipulation protection) is written (no addtional CPU costs at decompression).
The FLAM-ims product contains several programs:
In order to compress segments of a database with the production version FLAMIMS, only the segment definition in the DBD (Data Base Description) has to be extended with the entry
,COMPRTN=(FLAMIMS,DATA)
(the entry DATA can also be omitted).
To check the efficiency, the entry
,COMPRTN=(FLIMSTAT,DATA,INIT)
or
,COMPRTN=(FLIMSORG,DATA,INIT)
is required. Via INIT, the counters for the statistics in the data area are set to 0 in an initialization routine in this version, and the statistics are written to a file via the DD name FLPRINT in an end routine. This file is already opened during INIT.
This statistics provides per segment in intervals of 5% the dispersion of the data records after their compression by FLAMIMS, whereby the percentage is based on the remainder in relation to the maximum or fixed record length according to the DBD segment definition.
Changes in programs and the JCL are not necessary.
Via the exit for editing and compressing the segments, FLAMIMS is called by IMS each time a segment for which compression was specified in the DBD is read or written. This call is made in user programs and in utilities.
The use of variable long segments can lead to an overload of the overflow areas in case of length changes of updates, even if one does not work with compressed data.
FLAM-ims offers various overhead procedures with which one can create a space reserve. This means: a compressed set is supplemented by a variable part.
FLAM-ims generates comprehensive statistics in an unload or reload run, with the help of which the user can decide on one of the 4 overhead procedures (see description of module FLIMSTAT).
FLAM-ims distinguishes 3 parameters: MIN, MAX and REL. REL defines a percentage overhead on the compressed data; MIN and MAX limit this overhead to fixed lengths. If one or more of these parameters are set to zero, only the others (with non-zero values) are evaluated.
FLAM-ims takes the instructions for the overhead procedure from the USER DATA AREA of the relevant segment.
This USER DATA AREA is set up after successful DBD generation by the Linkage Editor's Replace function for CSECT'S with the parameters specified by the user (see example Use of FLAM-ims overhead procedures).
There are the following 4 overhead procedures:
The following applies to all procedures: If the compressed file is already longer than Y, no extension takes place. The default value for MAX is the length before compression.
The dispersion of the segment lengths after compression gives clues as to how the parameters MIN and MAX as well as REL should be set. As a rule, one looks for suitable accumulation points for this.
The initial compression of a database is done when it is created or reorganized. In the case of a new database to be created, the compression is entered in the DBD and the database is loaded with a utility or user program.
When using the standard IMS/VS EXIT, IBM provides a reserve of 10 bytes for fixed segments. It is recommended that this 10 byte reserve is also taken into account for variable segments in the DBD.
The following work is required for a reorganization:
If no overhead procedure for FLAM-ims is used between the unloading and loading of the database, the statistics when loading the DB are identical to those when unloading.
The unload and reload utilities are:
HDAM/HIDAM Unload - DFSURGU0 HDAM/HIDAM Reload - DFSURGL0
These utilities are to be used as prescribed by IBM.
The reorganization of a database can be done with the valid DBD with entry of compression. Here, each segment is decompressed and compressed again.
The sequential unload is thus always uncompressed!
Much less time is required for the reorganization if the database is unloaded and reloaded in compressed form with a modified DBD. In this case, IMS is briefly set so that the compression is no longer known and all segments contain records of variable length.
The following changes are necessary in the DBD (see also example below):
When assembling the DBD, a recommendation is given for the control interval size of VSAM files. With this size, the clusters of the database should be set up again with the IDCAMS.
Unloading and loading of the database is performed with this DBD modified in this way, then the DBD with the compression is reinserted.
The sequential unload is compressed!
As a result, the reorganization run takes much less time (fewer I/Os) and requires less storage space on the disk.
In addition, an archiving run of these stocks has less data to save and is thus finished faster with likewise less storage requirements on tape.
The modules FLIMSTAT and FLIMSORG write statistical data per compressed segment type into a sequential file (DD name FLPRINT) during loading/ unloading of the database. This file is opened during the first call (INIT).
The statistics contain the following information:
F L A M (R) I M S D A T A C O M P R E S S I O N A N A L Y S I S
Copyright (C) by limes datentechnik gmbh
Name of Segment: SEG1
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
50 96 96 96 96 6 90 4,800 Orig.
86 33 73 8 65 0 3,644 Comp.
Total Saved 1,156
Average Data Compression 27.8%
Total Segment Reduction 24.1%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2 22 38 38 % Records
Name of Segment: SEG2
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
52 609 609 90 478 4 474 24,853 Orig.
395 33 300 4 296 0 15,614 Comp.
Total Saved 9,239
Average Data Compression 37.6%
Total Segment Reduction 37.2%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2 25 2 4 15 52 % Records
Name of Segment: SEG3
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
94 104 104 104 104 6 98 9,776 Orig.
101 38 71 8 63 0 6,673 Comp.
Total Saved 3,103
Average Data Compression 25.8%
Total Segment Reduction 31.8%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2 1 28 9 10 11 11 4 10 1 15% Records
Name of Segment: SEG4
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
47 153 153 6 75 2 73 3,527 Orig.
120 7 61 2 59 0 2,868 Comp.
Total Saved 659
Average Data Compression 19.2%
Total Segment Reduction 18.7%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
9 2 13 36 4 15 11 11 % Records
S U M M A R Y D E S C R I P T I O N
FOR DBD TESTFLAM
Original Compressed
Def ! AVG Pfx AVG Total ! AVG AVG AVG AVG Total !
Segment Records Rec ! Rec + DTA Segment ! Rec DTA DTA Pad Segment !
Len ! Len Key Len Length ! Len Len % Len Length !
+------+---------+-----+----+----+----+---------+----+----+----+----+---------+
SEG1 50 96 96 6 90 4,800 73 65 28 0 3,644
SEG2 52 609 478 4 474 24,853 300 296 38 0 15,614
SEG3 94 104 104 6 98 9,776 71 63 36 0 6,673
SEG4 47 153 75 2 73 3,527 61 59 19 0 2,868
------ ------
42,956 28,799
Total Data Base Reduction 33.0% 14,157 Bytes
Expansions can occur in segments with a very short data part (smaller than 24 bytes). With segments up to approx. 50 bytes long, expansions only occur in rare cases with character strings that are very unfavorable for compression. With longer segments, expansions can be excluded.
If a segment becomes longer during compression, a disguised original (tamper protection) is written.
FLAM-ims is regularly adapted to the IMS EXIT standard in such a way that no compatibility problems can occur if one proceeds as prescribed (see chapter 'Installation of FLAM-ims') after each service level change, i.e. reassembled and linked.
If errors occur during decompression, the cause is a change in the bit string to be decompressed which FLAM-ims cannot foresee, e.g. I/O error, external manipulation.
If errors are detected during the compression of segments, the segment is transferred uncompressed but obfuscated with checksum. During decompression, the segment is recognized as uncompressed and transferred unobfuscated.
The following ABEND FLAMIMS: messages can be issued on the console
(they are also logged in the JCL listing):
RC=004 FLAMIMS has detected a data manipulation with all theseRC=008 return codes.RC=012RC=016RC=020RC=024RC=028RC=124In addition to these error messages, the following additional information appears:
SEGMENT: name (segment name)
DBD ...: name (DBD name)
KEY ...: key (key-information for segments with key)
COMPR. : nnnnnn nn (start of the comprimate for segments without key)
EXPAND : nnnnnnnn nn (beginning of decomprimates for segments without key)
At the end of these error messages a dump is generated and the IMS-CONTROL-REGION exits to IMS-USER-ABEND 4000. Or (flag in DBD-CSECT) FLAMIMS returns only a return code and ABEND code 840.
The database is unloaded as usual, statistics for compression are not created. The corresponding DBD specification is e.g.:
* DBD for TESTDATABASE without FLAMIMS
*
*
*
DBD NAME=TESTFLAM,ACCESS=HDAM, *
RMNAME=(TRANDOM,8,63638,100)
DATASET DD1=DATASET1,DEVICE=3350,SCAN=0
*
SEGM NAME=SEG1,PARENT=0,BYTES=96,PTR=T
FIELD NAME=(S1F1,SEQ,U),BYTES=6,START=1 *
TYPE=X
FIELD NAME=S1F2,Bytes=1,START=7,TYPE=X
FIELD NAME=S1F3,Bytes=1,START=8,TYPE=X
*
SEGM NAME=SEG2,PARENT=SEG1, *
BYTES=(609,7),PTR=T
FIELD NAME=(S2F1,SEQ,U),BYTES=2, *
START=3,TYPE=P
*
SEGM NAME=SEG3,PARENT=SEG1, *
BYTES=104,PTR=T
FIELD NAME=(S3F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S3F2,Bytes=2,START=1,TYPE=P
FIELD NAME=S3F3,Bytes=2,START=3,TYPE=P
FIELD NAME=S3F4,Bytes=2,START=5,TYPE=P
FIELD NAME=S3F5,Bytes=1,START=7,TYPE=X
FIELD NAME=S3F6,Bytes=1,START=8,TYPE=X
FIELD NAME=S3F7,Bytes=1,START=20,TYPE=X
FIELD NAME=S3F8,Bytes=1,START=76,TYPE=X
*
SEGM NAME=SEG4,PARENT=SEG3 *
BYTES=(153,5),PTR=T
With this DBD and the utility DFSURGU0 the database is unloaded. The JCL details are e.g.:
//ANWULOAD JOB ... //STEP EXEC PGM=DFSRRC00, // PARM='ULU,DFSURGU0,TESTFLAM', // REGION=1024K //STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=ANW.USERLIB,DISP=SHR //DFSRESLIB DD DSN=IMSVS.RESLIB,DISP=SHR //IMS DD DSN=ANW.DBDLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD DUMMY //DFSURGU1 DD DSN=ANW.HDUNL.TESTFLAM, // DISP=(NEW,CATLG,DELETE), // UNIT=CTAPE,LABEL=RETPD=0001 //DFSVSAMP DD DSN=DBA.IMS.PARMLIB(DFSVSM01), // DISP=SHR //TESTFLAM DD DSN=ANW.DATABASE.TESTFLAM, // DISP=SHR
To do this, the compression module FLORGDB must be switched on in the corresponding DBD (without actually compressing):
* DBD for TESTDATABASE with FLORGDB
*
* for unloading without compression with statistics
*
DBD NAME=TESTFLAM,ACCESS=HDAM, *
RMNAME=(TRANDOM,8,63638,100)
DATASET DD1=DATASET1,DEVICE=3350,SCAN=0
*
SEGM NAME=SEG1,PARENT=0,BYTES=96,PTR=T, *
COMPRTN=(FLORGDB,DATA,INIT)
FIELD NAME=(S1F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S1F2,Bytes=1,START=7,TYPE=X
FIELD NAME=S1F3,Bytes=1,START=8,TYPE=X
*
SEGM NAME=SEG2,PARENT=SEG1, *
BYTES=(609,7),PTR=T, *
COMPRTN=(FLORGDB,DATA,INIT)
FIELD NAME=(S2F1,SEQ,U),BYTES=2, *
START=3,TYPE=P
*
SEGM NAME=SEG3,PARENT=SEG1, *
BYTES=104,PTR=T, *
COMPRTN=(FLORGDB,DATA,INIT)
FIELD NAME=(S3F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S3F2,Bytes=2,START=1,TYPE=P
FIELD NAME=S3F3,Bytes=2,START=3,TYPE=P
FIELD NAME=S3F4,Bytes=2,START=5,TYPE=P
FIELD NAME=S3F5,Bytes=1,START=7,TYPE=X
FIELD NAME=S3F6,Bytes=1,START=8,TYPE=X
FIELD NAME=S3F7,Bytes=1,START=20,TYPE=X
FIELD NAME=S3F8,Bytes=1,START=76,TYPE=X
*
SEGM NAME=SEG4,PARENT=SEG3 *
BYTES=(153,5),PTR=T, *
COMPRTN=(FLORGDB,DATA,INIT)
The JCL in the previous example is supplemented with the log file entry:
//FLPRINT DD SYSOUT=*
If the FLAM-ims modules are not stored in the RESLIB, the library must be concatenated with the RESLIB:
//STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=FLAM.IMSLOAD,DISP=SHR
For this, the compression module FLIMSTA must be switched on in the corresponding DBD.
During unload, all data records are decompressed, so the unloaded data is in the uncompressed (original) state.
During reload, the uncompressed unloaded stock is written into the database in compressed form.
In both cases, statistics on the efficiency of the compression are output for the respective segments.
FLIMSTA requires a compressed database:
* DBD for TESTDATABASE with FLIMSTA
*
* for loading/unloading with compression and statistics
*
DBD NAME=TESTFLAM,ACCESS=HDAM, *
RMNAME=(TRANDOM,8,63638,100)
DATASET DD1=DATASET1,DEVICE=3350,SCAN=0
*
SEGM NAME=SEG1,PARENT=0,BYTES=96,PTR=T, *
COMPRTN=(FLIMSTA,DATA,INIT)
FIELD NAME=(S1F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S1F2,Bytes=1,START=7,TYPE=X
FIELD NAME=S1F3,Bytes=1,START=8,TYPE=X
*
SEGM NAME=SEG2,PARENT=SEG1, *
BYTES=(609,7),PTR=T, *
COMPRTN=(FLIMSTA,DATA,INIT)
FIELD NAME=(S2F1,SEQ,U),BYTES=2, *
START=3,TYPE=P
*
SEGM NAME=SEG3,PARENT=SEG1, *
BYTES=104,PTR=T, *
COMPRTN=(FLIMSTA,DATA,INIT)
FIELD NAME=(S3F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S3F2,Bytes=2,START=1,TYPE=P
FIELD NAME=S3F3,Bytes=2,START=3,TYPE=P
FIELD NAME=S3F4,Bytes=2,START=5,TYPE=P
FIELD NAME=S3F5,Bytes=1,START=7,TYPE=X
FIELD NAME=S3F6,Bytes=1,START=8,TYPE=X
FIELD NAME=S3F7,Bytes=1,START=20,TYPE=X
FIELD NAME=S3F8,Bytes=1,START=76,TYPE=X
*
SEGM NAME=SEG4,PARENT=SEG3 *
BYTES=(153,5),PTR=T, *
COMPRTN=(FLIMSTA,DATA,INIT)
With this DBD and the utility DFSURGL0 the database is loaded. The JCL specifications are e.g.:
//ANWRLOAD JOB ... //STEP EXEC PGM=DFSRRC00, // PARM='ULU,DFSURGL0,TESTFLAM', // REGION=1024K //STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=ANW.USERLIB,DISP=SHR // DD DSN=FLAM.IMSLOAD,DISP=SHR //DFSRESLIB DD DSN=IMSVS.RESLIB,DISP=SHR //IMS DD DSN=ANW.DBDLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD DUMMY //DFSUINPT DD DSN=ANW.HDUNL.TESTFLAM, // DISP=SHR,UNIT=CTAPE,VOL=SER=111111 //DFSURWF1 DD DUMMY, // DCB=(RECFM=VB,LRECL=900,BLKSIZE=1008) //DFSVSAMP DD DSN=DBA.IMS.PARMLIB(DFSVSM01), // DISP=SHR //DFSURCDS DD DUMMY //TESTFLAM DD DSN=ANW.DATABASE.TESTFLAM, // DCB=(DSORG=PS), // UNIT=3350,SPACE=(CYL,(200,200)), // DISP=(NEW,CATLG,DELETE) //* FLAMIMS-STATISTIK //FLPRINT DD SYSOUT=*
Based on the statistics information, one is able to make a qualified selection of the 4 overhead procedures for the compressed segment.
If the statistics version is used, a new distribution of the segment sets is obtained after the use of the overhead procedures.
For online production, the FLAMIMS module must be used.
Listing of the DBD-CSECT'S without using FLAMIMS:
****************************************************
* SEGM COMPACTION RTN CSECT - SEGNAME CSCT *
****************************************************
SEG1 CSECT
DC CL8'SEG1' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(9*256+5) FLAG + SEQ FIELD EXEC LEN
DC AL2(0) SEQUENCE FIELD OFFSET
DC AL2(96) SEGMENT MAX DATA LENGTH
DC AL2(32) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
SEG2 CSECT
DC CL8'SEG2' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(13*256+1) FLAG + SEQ FIELD EXEC LEN
DC AL2(2) SEQUENCE FIELD OFFSET
DC AL2(609) SEGMENT MAX DATA LENGTH
DC AL2(32) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
SEG3 CSECT
DC CL8'SEG3' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(9*256+5) FLAG + SEQ FIELD EXEC LEN
DC AL2(0) SEQUENCE FIELD OFFSET
DC AL2(104) SEGMENT MAX DATA LENGTH
DC AL2(32) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
SEG4 CSECT
DC CL8'SEG4' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(5*256+0) FLAG + SEQ FIELD EXEC LEN
DC AL2(0) SEQUENCE FIELD OFFSET
DC AL2(153) SEGMENT MAX DATA LENGTH
DC AL2(32) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
These DBD-CSECT'S are to be supplemented with the details of the award procedure and edited as a member.
In our example we use the following overhead procedures:
Segment SEG1 Variant 4 Rel.Percent 10
Segment SEG2 Variant 12 Min.Value 120
Max.Value 300
Rel.Percent 10
Segment SEG3 Variant 16 Min.Value 45
Max.Value 85
Segment SEG4 Variant 8 Max.Value 100
Rel.Percent 10
Addition when using the FLAMIMS overhead method:
SEG1 CSECT
DC CL8'SEG1' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(9*256+5) FLAG + SEQ FIELD EXEC LEN
DC AL2(0) SEQUENCE FIELD OFFSET
DC AL2(96) SEGMENT MAX DATA LENGTH
DC AL2(40) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
DC XL1'00' SEGMENT NUMBER
DC AL1(04) VAR ART
DC AL2(000) MINIMAL VALUE
DC AL2(000) MAXIMAL VALUE
DC AL2(10) RELATIVE PERCENT
SEG2 CSECT
DC CL8'SEG2' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(13*256+1) FLAG + SEQ FIELD EXEC LEN
DC AL2(2) SEQUENCE FIELD OFFSET
DC AL2(609) SEGMENT MAX DATA LENGTH
DC AL2(40) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
DC XL1'00' SEGMENT NUMBER
DC AL1(12) VAR ART
DC AL2(120) MINIMAL VALUE
DC AL2(300) MAXIMAL VALUE
DC AL2(10) RELATIVE PERCENT
SEG3 CSECT
DC CL8'SEG3' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(9*256+5) FLAG + SEQ FIELD EXEC LEN
DC AL2(0) SEQUENCE FIELD OFFSET
DC AL2(104) SEGMENT MAX DATA LENGTH
DC AL2(40) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
DC XL1'00' SEGMENT NUMBER
DC AL1(16) VAR ART
DC AL2(45) MINIMAL VALUE
DC AL2(85) MAXIMAL VALUE
DC AL2(00) RELATIVE PERCENT
SEG4 CSECT
DC CL8'SEG4' SEGMENT NAME
DC CL8'FLAMIMS' COMPACTION ROUTINE NAME
DC A(0) COMPACTION RTN ENTRY POINT
DC AL2(5*256+0) FLAG + SEQ FIELD EXEC LEN
DC AL2(0) SEQUENCE FIELD OFFSET
DC AL2(153) SEGMENT MAX DATA LENGTH
DC AL2(40) COMPACTION RTN CSECT SIZE
DC A(0) RESERVED
DC XL1'00' SEGMENT NUMBER
DC AL1(08) VAR ART
DC AL2(000) MINIMAL VALUE
DC AL2(100) MAXIMAL VALUE
DC AL2(10) RELATIVE PERCENT
END
Then assemble this member and edit the DBD module with the Replace function of the Linkage Editor. The JCL specifications are e.g.:
//ASSREPLA JOB ..... //ASMXA EXEC PGM=IEV90,REGION=1024K, // PARM=(NOBATCH,NODECK,'FLAG(0)',OBJECT,NORENT) //SYSLIB DD DISP=SHR,DSN=SYS1.MACLIB,DCB=BLKSIZE=20000 // DD DISP=SHR,DSN=IMSVS.MACLIB //SYSUT1 DD DSN=&&SYSUT1,SPACE=(CYL,(1,1)),UNIT=DISK //SYSUT2 DD DSN=&&SYSUT2,SPACE=(CYL,(1,1)),UNIT=DISK //SYSUT3 DD DSN=&&SYSUT3,SPACE=(CYL,(1,1)),UNIT=DISK //SYSIN DD DISP=SHR,DSN=ANW.DBDSOURC(member) //SYSPRINT DD SYSOUT=* //SYSPUNCH DD SYSOUT=* //SYSLIN DD DSN=&&OBJSET,UNIT=DISK,SPACE(CYL,(1,1)), // DISP=(MOD,PASS) //* //LNKXA EXEC PGM=IEWL,REGION=1024K, // COND=(0,LT,ASMXA),PARM=(XREF,LET,LIST) //SYSUT1 DD DSN=&&SYSUT1, // UNIT=(DISK,SEP=(SYSLIN,SYSIN,SYSLMOD)), // SPACE=(CYL,(1,1)) //SYSPRINT DD SYSOUT=* //SYSLIB DD DISP=SHR,DSN=SYS1.LINKLIB //OLDLOAD DD DISP=SHR,DSN=ANW.DBDLIB //SYSLMOD DD DISP=SHR,DSN=ANW.DBDLIB(TESTFLAM) //SYSLIN DD DSN=&&OBJSET,DISP=(OLD,DELETE) // DD DDNAME=SYSIN //SYSIN DD * INCLUDE OLDLOAD(TESTFLAM) ENTRY TESTFLAM NAME TESTFLAM (R) /*
After the procedure described above, the database can be loaded with the utility DFSURGL0.
After the implementation of the overhead procedures, it must always be ensured that after a DBD generation, the DBD-CSECT'S are also edited, assembled and the DBD module is replaced.
If the programs FLIMSTA or FLORGDB were used for analysis purposes, the new statistics output can be used to check whether the overhead procedures used make sense with the specified values.
This statistics contains a new distribution (dispersion) of the segment records taking into account the overhead procedure:
F L A M (R) I M S D A T A C O M P R E S S I O N A N A L Y S I S
Copyright (C) by limes datentechnik gmbh
Name of Segment: SEG1
Pad Variant No. 4
Min. Value - Max. Value - Rel. Percent 10
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
50 96 96 96 96 6 90 4,800 Orig.
94 33 80 8 65 7 3,991 Comp.
Total Saved 809
Average Data Compression 27.8%
Total Pad Overhead 8.6%
Total Segment Reduction 16.9%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2 22 2 74% Records
Name of Segment: SEG2
Pad Variant No. 12
Min. Value 120 Max. Value 300 Rel. Percent 10
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
52 609 609 90 478 4 474 24,853 Orig.
395 120 306 4 296 6 15,886 Comp.
Total Saved 8,967
Average Data Compression 37.6%
Total Pad Overhead 1.7%
Total Segment Reduction 36.1%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2 25 2 4 15 52 % Records
Name of Segment: SEG3
Pad Variant No. 16
Min. Value 45 Max. Value 85 Rel. Percent -
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
94 104 104 104 104 6 98 9,776 Orig.
101 45 87 8 63 16 8,140 Comp.
Total Saved 1,636
Average Data Compression 35.8%
Total Pad Overhead 18.0%
Total Segment Reduction 16.8%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2 81 1 15% Records
Name of Segment: SEG4
Pad Variant No. 8
Min. Value - Max. Value 100 Rel. Percent 10
Prfx AVG AVG Total
No. Def Max Min AVG + Data Pad Segment
Records Len Len Len Key Len Len Length
+--------+------+------+------+------+------+------+------+----------+
47 153 153 6 75 2 73 3,527 Orig.
120 7 64 2 59 3 3,026 Comp.
Total Saved 501
Average Data Compression 19.2%
Total Pad Overhead 5.2%
Total Segment Reduction 14.3%
Procentual Distribution of Records by Procentual Def. Record Length
(100% = Def. Length)
<15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 >90% Def Len
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
9 15 6 32 13 2 11 11 % Records
S U M M A R Y D E S C R I P T I O N
FOR DBD TESTFLAM
Original Compressed
Def ! AVG Pfx AVG Total ! AVG AVG AVG AVG Total !
Segment Records Rec ! Rec + DTA Segment ! Rec DTA DTA Pad Segment !
Len ! Len Key Len Length ! Len Len % Len Length !
+------+---------+-----+----+----+----+---------+----+----+----+----+---------+
SEG1 50 96 96 6 90 4,800 80 65 28 7 3,991
SEG2 52 609 478 4 474 24,853 306 296 38 6 15,886
SEG3 94 104 104 6 98 9,776 87 63 36 16 8,140
SEG4 47 153 75 2 73 3,527 64 59 19 3 3,026
------ ------
42,956 31,043
Total Data Base Reduction 27.8% 11,913 Bytes
DBD FOR TESTDATABASE FOR FLAMIMS
*
* WITH INDICATION OF COMPRESSION FOR PRODUCTION
*
DBD NAME=TESTFLAM,ACCESS=HDAM, *
RMNAME=(TRANDOM,8,63638,100)
DATASET DD1=DATASET1,DEVICE=3350,SCAN=0
*
SEGM NAME=SEG1,PARENT=0,BYTES=96,PTR=T, *
COMPRTN=FLAMIMS
FIELD NAME=(S1F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S1F2,Bytes=1,START=7,TYPE=X
FIELD NAME=S1F3,Bytes=1,START=8,TYPE=X
*
SEGM NAME=SEG2,PARENT=SEG1, *
BYTES=(609,7),PTR=T, *
COMPRTN=FLAMIMS
FIELD NAME=(S2F1,SEQ,U),BYTES=2, *
START=3,TYPE=P
*
SEGM NAME=SEG3,PARENT=SEG1, *
BYTES=104,PTR=T *
COMPRTN=FLAMIMS
FIELD NAME=(S3F1,SEQ,U),BYTES=6, *
START=1,TYPE=X
FIELD NAME=S3F2,Bytes=2,START=1,TYPE=P
FIELD NAME=S3F3,Bytes=2,START=3,TYPE=P
FIELD NAME=S3F4,Bytes=2,START=5,TYPE=P
FIELD NAME=S3F5,Bytes=1,START=7,TYPE=X
FIELD NAME=S3F6,Bytes=1,START=8,TYPE=X
FIELD NAME=S3F7,Bytes=1,START=20,TYPE=X
FIELD NAME=S3F8,Bytes=1,START=76,TYPE=X
*
SEGM NAME=SEG4,PARENT=SEG3 *
BYTES=(153,5),PTR=T, *
COMPRTN=FLAMIMS
If an overhead procedure was used when reloading the database, it must be ensured that the replaced DBD module is in the DBDLIB before reorganization. (see previous Example: Use of the FLAM-ims overhead procedures)
With this DBD and the utility DFSURGU0 the database is loaded. The JCL specifications are e.g:
//UNLOAD JOB ... //STEP EXEC PGM=DFSRRC00, // PARM='ULU,DFSURGU0,TESTFLAM', // REGION=1024K //STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=ANW.USERLIB,DISP=SHR // DD DSN=FLAM.IMSLOAD,DISP=SHR //DFSRESLIB DD DSN=IMSVS.RESLIB,DISP=SHR //IMS DD DSN=ANW.DBDLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD DUMMY //DFSURGU1 DD DSN=ANW.HDUNL.TESTFLAM, // DISP=(NEW,CATLG,DELETE), // UNIT=CTAPE,LABEL=RETPD=0001 //DFSVSAMP DD DSN=DBA.IMS.PARMLIB(DFSVSM01), // DISP=SHR //TESTFLAM DD DSN=ANW.DATABASE.TESTFLAM, // DISP=SHR
With this DBD and the utility DFSURGL0 the database is loaded. The JCL specifications are e.g.:
//RELOAD JOB ... //STEP EXEC PGM=DFSRRC00, // PARM='ULU,DFSURGL0,TESTFLAM', // REGION=1024K //STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=ANW.USERLIB,DISP=SHR // DD DSN=FLAM.IMSLOAD,DISP=SHR //DFSRESLIB DD DSN=IMSVS.RESLIB,DISP=SHR //IMS DD DSN=ANW.DBDLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD DUMMY //DFSUINPT DD DSN=ANW.HDUNL.TESTFLAM, // DISP=SHR,UNIT=CTAPE,VOL=SER=111111 //DFSURWF1 DD DUMMY, // DCB=(RECFM=VB,LRECL=900,BLKSIZE=1008) //DFSVSAMP DD DSN=DBA.IMS.PARMLIB(DFSVSM01), // DISP=SHR //DFSURCDS DD DUMMY //TESTFLAM DD DSN=ANW.DATABASE.TESTFLAM, // DCB=(DSORG=PS), // UNIT=3350,SPACE=(CYL,(200,200)), // DISP=(NEW,CATLG,DELETE)
* DBD FOR TESTDATABASE FOR FLAMIMS
*
* CHANGE FOR REORGANISATION WITH COMPRESSED SEGMENTS
*
DBD NAME=TESTFLAM,ACCESS=HDAM, *
RMNAME=(TRANDOM,8,63638,100)
DATASET DD1=DATASET1,DEVICE=3350,SCAN=0
*
SEGM NAME=SEG1,PARENT=0,BYTES=98,10,PTR=T
* THE SPECIFICATION FOR COMPRESSION OMITTED
* SENTENCE FORMAT, SENTENCE LENGTH AND KEY-
* POSITION HAVE BEEN CHANGED
FIELD NAME=(S1F1,SEQ,U),BYTES=6, *
START=3,TYPE=X
FIELD NAME=S1F2,Bytes=1,START=9,TYPE=X
FIELD NAME=S1F3,Bytes=1,START=10,TYPE=X
*
SEGM NAME=SEG2,PARENT=SEG1, *
BYTES=(609,7),PTR=T
* THE SPECIFICATION FOR COMPRESSION OMITTED
FIELD NAME=(S2F1,SEQ,U),BYTES=2, *
START=3,TYPE=P
*
SEGM NAME=SEG3,PARENT=SEG1, *
BYTES=(106,10),PTR=T
* THE SPECIFICATION FOR COMPRESSION OMITTED
* SENTENCE FORMAT, SENTENCE LENGTH AND KEY-
* POSITION HAVE BEEN CHANGED
FIELD NAME=(S3F1,SEQ,U),BYTES=6, *
START=3,TYPE=X
FIELD NAME=S3F2,Bytes=2,START=3,TYPE=P
FIELD NAME=S3F3,Bytes=2,START=5,TYPE=P
FIELD NAME=S3F4,Bytes=2,START=7,TYPE=P
FIELD NAME=S3F5,Bytes=1,START=9,TYPE=X
FIELD NAME=S3F6,Bytes=1,START=10,TYPE=X
FIELD NAME=S3F7,Bytes=1,START=22,TYPE=X
FIELD NAME=S3F8,Bytes=1,START=78,TYPE=X
*
SEGM NAME=SEG4,PARENT=SEG3, *
BYTES=(135,5),PTR=T
* THE SPECIFICATION FOR COMPRESSION OMITTED
With this DBD and the utility DFSURGU0 the database is unloaded. The JCL details are e.g.:
//UNLOAD JOB ... //STEP EXEC PGM=DFSRRC00, // PARM='ULU,DFSURGU0,TESTFLAM', // REGION=1024K //STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=ANW.USERLIB,DISP=SHR //DFSRESLIB DD DSN=IMSVS.RESLIB,DISP=SHR //IMS DD DSN=ANW.DBDLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD DUMMY //DFSURGU1 DD DSN=ANW.HDUNL.TESTFLAM, // DISP=(NEW,CATLG,DELETE), // UNIT=CTAPE,LABEL=RETPD=0001 //DFSVSAMP DD DSN=DBA.IMS.PARMLIB(DFSVSM01), // DISP=SHR //TESTFLAM DD DSN=ANW.DATABASE.TESTFLAM, // DISP=SHR
With this DBD and the utility DFSURGL0 the database is loaded. The JCL details are e.g.:
//RELOAD JOB ... //STEP EXEC PGM=DFSRRC00, // PARM='ULU,DFSURGL0,TESTFLAM', // REGION=1024K //STEPLIB DD DSN=IMSVS.RESLIB,DISP=SHR // DD DSN=ANW.USERLIB,DISP=SHR //DFSRESLIB DD DSN=IMSVS.RESLIB,DISP=SHR //IMS DD DSN=ANW.DBDLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD DUMMY //DFSUINPT DD DSN=ANW.HDUNL.TESTFLAM, // DISP=SHR,UNIT=CTAPE,VOL=SER=111111 //DFSURWF1 DD DUMMY, // DCB=(RECFM=VB,LRECL=900,BLKSIZE=1008) //DFSVSAMP DD DSN=DBA.IMS.PARMLIB(DFSVSM01), // DISP=SHR //DFSURCDS DD DUMMY //TESTFLAM DD DSN=ANW.DATABASE.TESTFLAM, // DCB=(DSORG=PS), // UNIT=3350,SPACE=(CYL,(200,200)), // DISP=(NEW,CATLG,DELETE)
The evaluation of the used and free areas of a database on the disks can be done via IDCAMS (only with VSAM) or e.g. SMU. The evaluation with IDCAMS yields less information than with SMU (Space Management Utility).
With IDCAMS, the file names, attributes, the used disks, the number of used and free reserved areas as well as statistics on accesses are output.
With the SMU, information about the files and statistics are also output. The used and free areas on the disks are listed in detail according to various aspects:
In order to realise the advantages of compressing a database, it is particularly important to indicate the free areas on the disks.
From the free areas in the data blocks, one can see how quickly and to what extent splits are to be expected and thus how much reorganization of the database will be necessary.
To compare the disk occupancy of an uncompressed and a compressed database, an evaluation is made in each case after loading the uncompressed and the compressed database. From the comparison of the two evaluations it can be seen how much more favorable the disk occupancy is with compressed data. The results of the above tests are given as an example.
The evaluation of the disk allocation for the entire database resulted with IDCAMS and SMU:
/---------------------------------------------------------------------\ | HDAM - Reload | | ohne | mit | | Komprimierung | Komprimierung | |--------------------------------------|------------------------------| | IDCAMS | | | | | | record total 65.129 | 63.863 | | record retrieved 282.242 | 155.612 | | freespace bytes 9.711.616 | 14.897.152 | | | | |--------------------------------------|------------------------------| | SMU | | | | | | freespace bytes 110.972.888 | 166.269.526 | | blocks without | | | free space 850 | 175 | | blocks with | | | fee space 64.279 | 63.688 | | empty blocks 1.336 | 1.347 | | total blocks 66.465 | 65.213 | | blocks with space | | | for longest seg. 49.941 | 62.012 | \--------------------------------------|------------------------------/
Per segment type, the SMU evaluation resulted in:
for HDAM reload without compression:
Seg. Amount Over- Length Amt./ avera.
name Seg. flow Prfx+Data=Seg. * Root = Seg.l.
SEG1 333.191 2.886 14 96 110 1,000 110,00
SEG2 336.409 3.341 6 139v 145 1,009 146,30
SEG3 568.623 39.935 10 104 114 1,706 194,48
SEG4 80.530 12.878 6 25v 31 0,241 7,47
--------- ------ ------
1.318.753 59.040 avera. DB record length = 458,26
for HDAM-Reload with compression:
Seg. Amount Over- Length Amt./ avera.
name of Seg. flow Prfx+Data=Seg. * Root = Seg.l.
SEG1 333.191 600 14 65v 7 1,000 79,00
SEG2 336.409 891 6 76v 82 1,009 82,73
SEG3 568.623 9.510 10 53v 63 1,706 107,47
SEG4 80.530 4.512 6 25v 31 0,241 7,47
--------- ------ ------
1.318.753 15.513 avera. DB record length = 276,78
FLAM-ims uses tables from IMS for the output of error messages, e.g. to specify the names of the segment and database. Since these tables are partly version-dependent, the interface modules for the compression modules are delivered in the source code. This means that they only have to be reassembled and bound when there is a change of version of IMS, without the need for a new delivery of FLAM-ims.
The FLAMIMS, FLIMSSTA and FLIMSORG user exits are delivered as LE less assembler source in FLAM.SRCLIB. Additional to link the FLAMIMS exits the object FLAMX5, FLAMX6 and FLAMMEMO from FLAM.OBJ are required. We recommend to build an SMPE package using the current FLAM installation and the dependent IMS version to assemble and link the exits.
Additional the FLAM distribution contains a batch test program (FLIMSTST) and IMS macro definitions (FLIMSMAC) against the FLAMFIO object to build, link and test the exists without IMS in a batch environment. The corresponding build and link steps can be fount in FLAM.JOBLIB(SBUILD) and a job to run the test program can be faound in FLAM.JOBLIB(FLIMSTST).