EDC

Synopsis

HELP:   OpenSSL conform decryption of files
TYPE:   OBJECT
SYNTAX: EDC(BUFSIZ=num,METHOD=CLEAR/PASSWORD/LABEL,ALGO=AES/TDES/CAST5/CAST128/IDEA/BLOWFISH/BF/CAMELLIA,KEYLEN=num/KL064/KL128/KL192/KL256/KL08/KL16/KL24/KL32,MODE=ECB/CBC/OFB/CFB/CTR/GCM,PADDING=NONE/NOPAD/PKCS,KDF/PW2KEY=OLDSSL/PBKDF2,MD/HASH=MD5/SHA1/SHA224/SHA256/SHA384/SHA512/SHA3-224/SHA3-256/SHA3-384/SHA3-512,ITER=num,IV='bin-hex'/ZERO,SALT='bin-hex'/NONE/RANDOM,KEY='bin-hex',PASSWORD='str',LABEL='str',FKM5(),TAG='bin-hex',OFFSET=num,PARM='str'/STDOUT/STDERR,SKPFMT[=NOHEADER/SUPPORTED/REDUNDANT/SUPPREDU])

Description

The encryption/decryption converter can be used to encrypt or decrypt data streams. These streams are compatible with the openssl enc command, allowing to encrypt data with FLAM and decrypt with OpenSSL or vice versa. Crypto operations are performed in software, with hardware-acceleration (if available) or via FKM5 using a key label.

Encryption always results in binary blocks of data. The decryption result can be text or binary data. Record boundaries are lost. Please use delimiters, length fields or fixed length data if records must be exchanged.

The OpenSSL-compatible encryption output is raw encrypted data. An automatic detection of this format is not possible, except for the salt of the salted passphrase procedure, in which case a corresponding header is written.

The encrypted data can be ASCII-armored for transport/storage via/on text-based media. Compression should be done before encryption because the encrypted output looks like random data which is incompressible.

   cnv.gzp(...) cnv.edc(...) cnv.bas(... armor(...))

The component supports different methods, algorithms, key lengths, modes and paddings. Invalid or unsupported combinations will result in an error. Additional the password to key derivation function can be specified including the hashing procedure and amount of iterations.

Please have a look at the security considerations section before using this component.

Parameter side file

During encryption, a parameter side file can be generated. It contains information from the encryption process that is relevant for decryption (e.g. the GCM tag value if encrypting with AES-GCM). If decryption is done with cnv.edc() as well, the existing side file can be specified, avoiding the need to specify the relevant parameters manually. The example below uses an SSH server for the side file.

WRITE:
   cnv.edc(IV=RANDOM KEY=RANDOM parm='ssh://user@server/sidefile.txt/&utf-8')
READ:
   cnv.edc='ssh://user@server/sidefile.txt/&utf-8'
OR:
   cnv.edc(parm='ssh://user@server/sidefile.txt/&utf-8')

The parameter side file is a text file which contains the parameter values used during encryption. The same parameter values must be passed to CNVEDC for decryption.

To facilitate this, you can generate this side file during encryption by assigning a file path to the parm parameter. This same file can then be used during decryption via the same parameter, allowing you to omit most parameters (except the key) during decryption. Hence, the parameter side file is mainly useful if the data will be decrypted with FLAM. To generate random IVs or random clear keys, the keyword RANDOM can be used for the respective parameters. The generated values are then written to the side file and required for decryption.

Please be aware that if key=RANDOM was specified, a new encryption key is generated per file and the generated encryption key is written to the side file (which is required in this case) because it would be impossible to decrypt the data without the randomly generated key. Anyone who can read this side file can decrypt your data! We do not recommend to store encrypted data and side file together or transmit both over the same channel. If you do not fully understand the security implications, we recommend not to use key=RANDOM.

If you specify your own clear key via key parameter, it is not written to the parameter side file. However, you can instruct to explicitly write the key to the side file with the PRNKEY flag. The warning above also applies in this case.

The parameter side file may contain:

The side file is required if at least one of the following is true:

This corresponds to the following usage patterns:

   cnv.edc(... mode=GCM ...)
   cnv.edc(... iv=RANDOM key=RANDOM ...)
   cnv.edc(... format=NOHDR pass=a'hugo' ...)

If you use a clear key, passphrase or KFM5 key label for encryption and also specify a parameter side file, it is sufficient to specify the side file and your key/passphrase/label during decryption because all other parameters are contained in the side file:

   cnv.edc(parm='//ssh:user@server/sidefile.txt/&utf-8' pass=a'hugo')
   cnv.edc(parm='//ssh:user@server/sidefile.txt/&utf-8' key=00112233...)

Initialization vector & Salt

All cipher modes except ECB require an initialization vector. An initialization vector (IV) makes sure that the ciphertext is completely different for every encryption operation even if the same data is encrypted with the same key multiple times. Each IV value should only be used once with a given key for encryption. The same IV must be used for decryption, otherwise the output will be random data. Using the same IV more than once may allow an attacker to gather information about common prefixes in your plaintext (i.e. files that start identically). To avoid that, use a different IV for each file.

The default IV is all zeroes. You can and should either specify a (hexadecimal) IV that you generated on your own or you can generate a random IV with iv=RANDOM (recommended) which is written to the parameter side file. Alternatively, a clear key or key label can be combined with a password as additional factor. The IV is then derived from the password and a salt (with properties described below).

For password-based encryption, the key and IV are derived from the password and a salt by default. Multiple key derivation methods are available (e.g. OpenSSL, PBKDF2). Unless specified otherwise, the salt is randomly generated. The 8 bytes magic string Salted__ followed by the generated 8 bytes salt value is prepended to the encryption output, making it compatible to OpenSSL's enc command. The prepended salt is automatically detected during decryption. Writing of the salt header can be disabled with FORMAT=NOHDR. The salt is then written to the parameter side file which, in this case, is required because otherwise decryption is impossible. You can specify a custom 8-byte salt value in hexadecimal form via the parameter salt or set it to none to not use a salt, which is not recommended because the same key and IV will be derived for each encryption operation. An IV can also explicitly be specified in which case it overrides the derived IV.

Defaults

When writing with cnv.edc() using key-based encryption, the defaults below apply:

For password-based encryption (i.e. a password is supplied), these defaults apply:

These defaults may change in the future. Depending on your use case, you might want to explicitly specify all relevant parameters to avoid behavioral changes when upgrading to future FLAM versions.

AEAD modes like GCM (Authentication tag)

AEAD modes combine encryption and authentication in a single algorithm, making a separate HMAC unnecessary to ensure authenticity. In addition to protecting confidential data, AEAD ensures the authenticity and integrity of other data that is not encrypted, called associated data. The purpose is, for example, the transmission of a message header, which must be available as plaintext for processing. With AEAD, the associated data and the encrypted data become inseparable.

Currently, only GCM is supported as AEAD scheme. It was developed with the protection of network communication protocols in mind that often require header data that must be processed by involved entities without decrypting the content. Each transmitted message is typically relatively small and protocols usually re-negotiate encryption keys after a certain amount of data was transmitted. As a result, GCM has limitations that make it less suitable for file encryption. The maximum amount of data that can be encrypted with the same Key+IV combo is slightly below 64 GiB (2^36-32 = 68,719,476,704 bytes). The same key should not be used for more than 2^32 encryption operations because the IV is only 96 bits and the collision probability of randomly generated IVs rises significantly beyond that limit.

The authentication tag, which is mandatory for decrypting the data with GCM, can optionally be written to and read from a dedicated file by specifying a path for the tag parameter. If the tag parameter is not specified, the tag is written to the parameter side file. In any case, one of the two is needed during decryption to ensure authenticity.

Example for the tag parameter:

WRITE:
   cnv.edc(... tag='ssh://user@server/tagfile.txt/&utf-8' ...)
READ:
   cnv.edc(... tag=f'ssh://user@server/tagfile.txt/&utf-8' ...)

Partial encryption of files

With the parameter offset, you can start the encryption after offset amount of bytes. The data between the beginning and the offset will remain plaintext. This can be useful to keep some meta data (e.g. headers) at the beginning of the file in unencrypted form. With GCM, this unencrypted part becomes associated data and is thus part of the authentication tag, protecting it against tampering.

Use of wildcards (behavior in directory walk)

If the component is used on multiple files at once, the parameter side file will contain a concatenation of all metadata for all files processed. The original filenames are written as comment to the side file to separate the sections. However, such a parameter file cannot be used for decryption. Therefore, we don't recommend to use a side file when encrypting multiple files with a single invocation.

Considerations and security best practices

Some recommendations to safely and securely use this component:

OpenSSL compatibility

The component was implemented with compatibility to the enc command of the OpenSSL command line utility in mind. Data that was encrypted with this component can be decrypted with OpenSSL and vice versa as long as their parameterization is equivalent.

There are a few exceptions to OpenSSL compatibility:

Examples

Interoperability

Here is an example for interoperability using password-based encryption with PBKDF2 as key derivation method and a custom IV. The first two lines represent the parameters for encryption and decrytion with CNVEDC. The other two lines are the equivalent OpenSSL command line parameters.

CNVEDC encrypt:  "ALGO=AES mode=CFB keylen=KL256 method=PBKDF2 pass=a'1234'     iv=0123456789ABCDEF0123456789ABCFFF parm=STDOUT"
CNVEDC decrypt:  "ALGO=AES mode=CFB keylen=KL256 method=PBKDF2 pass=x'31323334' iv=0123456789ABCDEF0123456789ABCFFF parm=STDOUT "
OpenSSL encrypt: "-e -AES-256-CFB -pass pass:1234 -pbkdf2 -iv 0123456789ABCDEF0123456789ABCFFF -p",
OpenSSL decrypt: "-d -AES-256-CFB -pass pass:1234 -pbkdf2 -iv 0123456789ABCDEF0123456789ABCFFF -p",
File encryption

A simple example for file encryption with AES-128-CBC using a custom clear key and IV. Note that you should use a randomly generated IV per file.

 flcl xcnv "input(sav.fil(fio.blk(name=file.bin)))
            output(sav.fil(fio.blk(name=file.edc)
              cnv.edc(algo=aes keylen=kl128 mode=cbc
                iv='01234567801234567801234567801234'
                key='0123456789ABCDEFFEDCBA9876543210'
              )
            ))

Automatism

In order to be able to use the component in the detection when reading, its removal can be activated under certain circumstances via the parameter SKPFMT. On the one hand, with the selection NOHEADER (activated in the CONV command with DECRYPT.PWD()), which leads to removal if no salted header is found. The next stage is called SUPPORTED, where no decryption takes place if the data at hand corresponds to a format that is processed by other components. With REDUNDANT it can be prevented that the decryption takes place on data that show redundancies, which cannot be the case with a cryptogram (white noise). An exception to this is ECB mode, which can lead to recurring cryptograms if the data are identical. Therefore, REDUNTANT is not supported in ECB mode. With SUPPREDU, the latter two mechanisms can be requested together, which comes into play in the read string of the CONV command in DECRYPT.EDC().

Arguments