HELP: OpenSSL conform decryption of files TYPE: OBJECT SYNTAX: EDC(BUFSIZ=num,METHOD=CLEAR/PASSWORD/LABEL,ALGO=AES/TDES/CAST5/CAST128/IDEA/BLOWFISH/BF/CAMELLIA,KEYLEN=num/KL064/KL128/KL192/KL256/KL08/KL16/KL24/KL32,MODE=ECB/CBC/OFB/CFB/CTR/GCM,PADDING=NONE/NOPAD/PKCS,KDF/PW2KEY=OLDSSL/PBKDF2,MD/HASH=MD5/SHA1/SHA224/SHA256/SHA384/SHA512/SHA3-224/SHA3-256/SHA3-384/SHA3-512,ITER=num,IV='bin-hex'/ZERO,SALT='bin-hex'/NONE/RANDOM,KEY='bin-hex',PASSWORD='str',LABEL='str',FKM5(),TAG='bin-hex',OFFSET=num,PARM='str'/STDOUT/STDERR,SKPFMT[=NOHEADER/SUPPORTED/REDUNDANT/SUPPREDU])
The encryption/decryption converter can be used to encrypt or decrypt
data streams. These streams are compatible with the openssl enc
command, allowing to encrypt data with FLAM and decrypt with OpenSSL or
vice versa. Crypto operations are performed in software, with
hardware-acceleration (if available) or via FKM5 using a key label.
Encryption always results in binary blocks of data. The decryption result can be text or binary data. Record boundaries are lost. Please use delimiters, length fields or fixed length data if records must be exchanged.
The OpenSSL-compatible encryption output is raw encrypted data. An automatic detection of this format is not possible, except for the salt of the salted passphrase procedure, in which case a corresponding header is written.
The encrypted data can be ASCII-armored for transport/storage via/on text-based media. Compression should be done before encryption because the encrypted output looks like random data which is incompressible.
cnv.gzp(...) cnv.edc(...) cnv.bas(... armor(...))
The component supports different methods, algorithms, key lengths, modes and paddings. Invalid or unsupported combinations will result in an error. Additional the password to key derivation function can be specified including the hashing procedure and amount of iterations.
Please have a look at the security considerations section before using this component.
During encryption, a parameter side file can be generated. It contains
information from the encryption process that is relevant for decryption
(e.g. the GCM tag value if encrypting with AES-GCM). If decryption is
done with cnv.edc()
as well, the existing side file can be specified,
avoiding the need to specify the relevant parameters manually.
The example below uses an SSH server for the side file.
WRITE: cnv.edc(IV=RANDOM KEY=RANDOM parm='ssh://user@server/sidefile.txt/&utf-8') READ: cnv.edc='ssh://user@server/sidefile.txt/&utf-8' OR: cnv.edc(parm='ssh://user@server/sidefile.txt/&utf-8')
The parameter side file is a text file which contains the parameter values used during encryption. The same parameter values must be passed to CNVEDC for decryption.
To facilitate this, you can generate this side file during encryption by
assigning a file path to the parm
parameter. This same file can then
be used during decryption via the same parameter, allowing you to omit
most parameters (except the key) during decryption. Hence, the parameter
side file is mainly useful if the data will be decrypted with FLAM. To
generate random IVs or random clear keys, the keyword RANDOM
can be
used for the respective parameters. The generated values are then
written to the side file and required for decryption.
Please be aware that if key=RANDOM
was specified, a new
encryption key is generated per file and the generated encryption key is
written to the side file (which is required in this case) because it
would be impossible to decrypt the data without the randomly generated
key. Anyone who can read this side file can decrypt your data! We
do not recommend to store encrypted data and side file together or
transmit both over the same channel. If you do not fully understand the
security implications, we recommend not to use key=RANDOM
.
If you specify your own clear key via key
parameter, it is not written
to the parameter side file. However, you can instruct to explicitly
write the key to the side file with the PRNKEY
flag. The warning above
also applies in this case.
The parameter side file may contain:
PRNKEY
flag is set, see warning above)PRNKEY
flag is set)PRNKEY
flag is set)The side file is required if at least one of the following is true:
TAG
file is providedkey=RANDOM
or IV=RANDOM
was specifiedThis corresponds to the following usage patterns:
cnv.edc(... mode=GCM ...) cnv.edc(... iv=RANDOM key=RANDOM ...) cnv.edc(... format=NOHDR pass=a'hugo' ...)
If you use a clear key, passphrase or KFM5 key label for encryption and also specify a parameter side file, it is sufficient to specify the side file and your key/passphrase/label during decryption because all other parameters are contained in the side file:
cnv.edc(parm='//ssh:user@server/sidefile.txt/&utf-8' pass=a'hugo') cnv.edc(parm='//ssh:user@server/sidefile.txt/&utf-8' key=00112233...)
All cipher modes except ECB require an initialization vector. An initialization vector (IV) makes sure that the ciphertext is completely different for every encryption operation even if the same data is encrypted with the same key multiple times. Each IV value should only be used once with a given key for encryption. The same IV must be used for decryption, otherwise the output will be random data. Using the same IV more than once may allow an attacker to gather information about common prefixes in your plaintext (i.e. files that start identically). To avoid that, use a different IV for each file.
The default IV is all zeroes. You can and should either specify a
(hexadecimal) IV that you generated on your own or you can generate a
random IV with iv=RANDOM
(recommended) which is written to the
parameter side file. Alternatively, a clear key or key label can be
combined with a password as additional factor. The IV is then derived
from the password and a salt (with properties described below).
For password-based encryption, the key and IV are derived from the
password and a salt by default. Multiple key derivation methods are
available (e.g. OpenSSL, PBKDF2). Unless specified otherwise, the salt
is randomly generated. The 8 bytes magic string Salted__
followed by
the generated 8 bytes salt value is prepended to the encryption output,
making it compatible to OpenSSL's enc
command. The prepended salt is
automatically detected during decryption. Writing of the salt header can
be disabled with FORMAT=NOHDR
. The salt is then written to the
parameter side file which, in this case, is required because otherwise
decryption is impossible. You can specify a custom 8-byte salt value in
hexadecimal form via the parameter salt
or set it to none
to not use
a salt, which is not recommended because the same key and IV will be
derived for each encryption operation. An IV can also explicitly be
specified in which case it overrides the derived IV.
When writing with cnv.edc()
using key-based encryption, the defaults
below apply:
For password-based encryption (i.e. a password is supplied), these defaults apply:
These defaults may change in the future. Depending on your use case, you might want to explicitly specify all relevant parameters to avoid behavioral changes when upgrading to future FLAM versions.
AEAD modes combine encryption and authentication in a single algorithm, making a separate HMAC unnecessary to ensure authenticity. In addition to protecting confidential data, AEAD ensures the authenticity and integrity of other data that is not encrypted, called associated data. The purpose is, for example, the transmission of a message header, which must be available as plaintext for processing. With AEAD, the associated data and the encrypted data become inseparable.
Currently, only GCM is supported as AEAD scheme. It was developed with the protection of network communication protocols in mind that often require header data that must be processed by involved entities without decrypting the content. Each transmitted message is typically relatively small and protocols usually re-negotiate encryption keys after a certain amount of data was transmitted. As a result, GCM has limitations that make it less suitable for file encryption. The maximum amount of data that can be encrypted with the same Key+IV combo is slightly below 64 GiB (2^36-32 = 68,719,476,704 bytes). The same key should not be used for more than 2^32 encryption operations because the IV is only 96 bits and the collision probability of randomly generated IVs rises significantly beyond that limit.
The authentication tag, which is mandatory for decrypting the data with
GCM, can optionally be written to and read from a dedicated file by
specifying a path for the tag
parameter. If the tag
parameter is not
specified, the tag is written to the parameter side file. In any case,
one of the two is needed during decryption to ensure authenticity.
Example for the tag
parameter:
WRITE: cnv.edc(... tag='ssh://user@server/tagfile.txt/&utf-8' ...) READ: cnv.edc(... tag=f'ssh://user@server/tagfile.txt/&utf-8' ...)
With the parameter offset
, you can start the encryption after offset
amount of bytes. The data between the beginning and the offset will
remain plaintext. This can be useful to keep some meta data (e.g.
headers) at the beginning of the file in unencrypted form. With GCM,
this unencrypted part becomes associated data and is thus part of the
authentication tag, protecting it against tampering.
If the component is used on multiple files at once, the parameter side file will contain a concatenation of all metadata for all files processed. The original filenames are written as comment to the side file to separate the sections. However, such a parameter file cannot be used for decryption. Therefore, we don't recommend to use a side file when encrypting multiple files with a single invocation.
Some recommendations to safely and securely use this component:
iv=RANDOM
if possiblekey=RANDOM
or flag PRNKEY
)
make sure to not store it at the same location or transmit it over the
same channel as the encrypted file.The component was implemented with compatibility to the enc
command of
the OpenSSL command line utility in mind. Data that was encrypted with
this component can be decrypted with OpenSSL and vice versa as long as
their parameterization is equivalent.
There are a few exceptions to OpenSSL compatibility:
offset
parameter is used, the file cannot be decrypted with the OpenSSL command line toolHere is an example for interoperability using password-based encryption with PBKDF2 as key derivation method and a custom IV. The first two lines represent the parameters for encryption and decrytion with CNVEDC. The other two lines are the equivalent OpenSSL command line parameters.
CNVEDC encrypt: "ALGO=AES mode=CFB keylen=KL256 method=PBKDF2 pass=a'1234' iv=0123456789ABCDEF0123456789ABCFFF parm=STDOUT" CNVEDC decrypt: "ALGO=AES mode=CFB keylen=KL256 method=PBKDF2 pass=x'31323334' iv=0123456789ABCDEF0123456789ABCFFF parm=STDOUT " OpenSSL encrypt: "-e -AES-256-CFB -pass pass:1234 -pbkdf2 -iv 0123456789ABCDEF0123456789ABCFFF -p", OpenSSL decrypt: "-d -AES-256-CFB -pass pass:1234 -pbkdf2 -iv 0123456789ABCDEF0123456789ABCFFF -p",
A simple example for file encryption with AES-128-CBC using a custom clear key and IV. Note that you should use a randomly generated IV per file.
flcl xcnv "input(sav.fil(fio.blk(name=file.bin))) output(sav.fil(fio.blk(name=file.edc) cnv.edc(algo=aes keylen=kl128 mode=cbc iv='01234567801234567801234567801234' key='0123456789ABCDEFFEDCBA9876543210' ) ))
In order to be able to use the component in the detection when reading,
its removal can be activated under certain circumstances via the
parameter SKPFMT
. On the one hand, with the selection NOHEADER
(activated in the CONV command with DECRYPT.PWD()), which leads to
removal if no salted header is found. The next stage is called
SUPPORTED
, where no decryption takes place if the data at hand
corresponds to a format that is processed by other components.
With REDUNDANT
it can be prevented that the decryption takes place
on data that show redundancies, which cannot be the case with a
cryptogram (white noise). An exception to this is ECB mode, which can
lead to recurring cryptograms if the data are identical. Therefore,
REDUNTANT
is not supported in ECB mode. With SUPPREDU
, the latter
two mechanisms can be requested together, which comes into play in the
read string of the CONV command in DECRYPT.EDC().
NUMBER: BUFSIZ=num - Initial buffer size for preallocation [65536]
NUMBER: METHOD=CLEAR/PASSWORD/LABEL - Method for key determination [AUTO]
CLEAR - Clear key crypto (en/decrypt with raw unprotected key value)
PASSWORD - Password-based crypto (using key derivation function (KDF))
LABEL - Protected key crypto using label (FKM5 required)
NUMBER: ALGO=AES/TDES/CAST5/CAST128/IDEA/BLOWFISH/BF/CAMELLIA - Algorithm used for decryption [AES]
AES - Advanced encryption standard (Rijndael)
TDES - Triple DES (Data encryption standard)
CAST5 - CAST5/CAST-128 (Carlisle Adams, Stafford Tavares)
CAST128 - CAST-128/CAST5 (Carlisle Adams, Stafford Tavares)
IDEA - International Data Encryption Algorithm
BLOWFISH - Blowfish from Bruce Schneier
BF - Blowfish from Bruce Schneier
CAMELLIA - Camellia from Japan (Mitsubishi Electric and NTT)
NUMBER: KEYLEN=num/KL064/KL128/KL192/KL256/KL08/KL16/KL24/KL32 - Key length if passphrase used
KL064 - Key length 64 bits (8 bytes)
KL128 - Key length 128 bits (16 bytes)
KL192 - Key length 192 bits (24 bytes)
KL256 - Key length 256 bits (32 bytes)
KL08 - Key length 8 bytes (64 bits)
KL16 - Key length 16 bytes (128 bits)
KL24 - Key length 24 bytes (192 bits)
KL32 - Key length 32 bytes (256 bits)
NUMBER: MODE=ECB/CBC/OFB/CFB/CTR/GCM - Mode of operation [if TAG then GCM else CBC]
ECB - Electronic codebook mode
CBC - Cipher block chaining mode
OFB - Output feedback mode
CFB - Cipher feedback mode
CTR - Counter mode
GCM - Galois/Counter Mode
NUMBER: PADDING=NONE/NOPAD/PKCS - Padding for CBC or ECB mode [PKCS]
NONE - No padding (remaining data must have a multiple of the block length)
NOPAD - No padding (remaining data must have a multiple of the block length)
PKCS - OpenSSL conform PKCS padding
NUMBER: KDF/PW2KEY=OLDSSL/PBKDF2 - Password to key derivation function [OLDSSL]
OLDSSL - Legacy OpenSSL key derivation (avoid use, default: SHA-256 with 1 round, hint: OpenSSL < 1.1.0 used MD5)
PBKDF2 - PBKDF2 key derivation function (default: SHA-256 with 10000 rounds, available from OpenSSL 1.1.1)
NUMBER: MD/HASH=MD5/SHA1/SHA224/SHA256/SHA384/SHA512/SHA3-224/SHA3-256/SHA3-384/SHA3-512 - Hash algorithm for key derivation [SHA256]
MD5 - Message Digest 5 with 128 bits
SHA1 - Secure Hash Algorithm 1 with 160 bits
SHA224 - Secure Hash Algorithm 2 with 224 bits
SHA256 - Secure Hash Algorithm 2 with 256 bits
SHA384 - Secure Hash Algorithm 2 with 384 bits
SHA512 - Secure Hash Algorithm 2 with 512 bits
SHA3-224 - Secure Hash algorithm 3 with 224 bits
SHA3-256 - Secure Hash Algorithm 3 with 256 bits
SHA3-384 - Secure Hash Algorithm 3 with 384 bits
SHA3-512 - Secure Hash Algorithm 3 with 512 bits
NUMBER: ITER=num - Iteration count for PBKDF2, implies METHOD=PBKDF2 if specified [10000]
STRING: IV='bin-hex'/ZERO - Initialization vector for all modes except ECB [ZERO]
ZERO - Initialization vector of binary zeros
STRING: SALT='bin-hex'/NONE/RANDOM - Salt for salted passphrase-based decryption [AUTO]
NONE - No salt (not recommended)
RANDOM - Random generated salt
STRING: KEY='bin-hex' - Clear key value (direct entry of raw value (dangerous))
STRING: PASSWORD='str' - Passphrase with or without salting
STRING: LABEL='str' - Label to reference a protected key (FKM5 required)
STRING: TAG='bin-hex' - The tag value for verification (only for GCM mode)
NUMBER: OFFSET=num - Offset before decryption begins [0]
STRING: PARM='str'/STDOUT/STDERR - Side file with parameter used for decryption (merged if not empty else written)
STDOUT - Write output to stdout
STDERR - Write output to stderr
NUMBER: SKPFMT=NOHEADER/SUPPORTED/REDUNDANT/SUPPREDU - Skip decryption if format not valid [NO]
NOHEADER - Skip if no salted header found
SUPPORTED - Skip if format supported (PGP, GZIP, BZIP, XL, ZSTD, XML, ...)
REDUNDANT - Skip if data redundant (not a white noise)
SUPPREDU - Skip if format known or data redundant