COPY

Synopsis

HELP:   Copy an existing FLAM archive to a new FLAM archive
TYPE:   OBJECT
SYNTAX: COPY(FROM/READ(),TO/WRITE.{},LOGGING.{},MESSAGE(),NORUN)

Description

The COPY subcommand can be used to create a new or update an existing FLAM archive from an existing FLAM archive. This can be used to transfer a specific version of an archive in isolation to a new archive or to transfer archives from one storage backend to another (e.g. local => cloud).

There are four modes of operation on archives, one of which must be selected via the TO parameter:

• NEW: Writes a new archive.
• OLD: Writes a new version to an existing archive that is independent of any previous versions in the archive, i.e. the new archive starts out empty.
• MODIFY: Writes a new version to an existing archive that is based on the latest version of data in the archive, i.e. the archive is opened with the last state and modified with new data. It can be used to update data incrementally.
• DUPLICATE: Writes a new archive, like the NEW option, but does so without decrypting/decompressing the data. As the data remains encrypted with the key material of the original archive, this can be used to create a subset of an archive, which only contains data segments of relevant data. This can be used to store archives in untrusted datacenters. An archive subset can be created within this untrusted datacenter which is pre-filtered for relevant data without this third party needing access to the clear data. The subset can then be transferred to a trusted environment and decrypted using the respective data key.

You can select only certain members from the archive via member lists using wildcards and/or only select members with certain contents if the MATCH overlay is specified. For a new archive, the FLAM parameters for compression and encryption can be specified. If the archive already exists, these are adopted or the correct values can be specified for decryption. In this case, implicit re-keying can be carried out together with writing a new version if the new version is to be provided with new rights.

To save a new archive, it must not already exist. However, you can use the OVERWRITE switch to force it to be deleted beforehand. If you access an existing archive using the OLD or MODIFY parameter, you have the option to delete the previous version or all older versions after the new version was written successfully. Remember that deleting old versions only marks the data as deleted. To actually remove all deleted segment data that is no longer required the archive must be pruned (PRUNE). Depending on the storage backend, this can be an expensive operation. For file-based archives, the whole archive is rewritten.

In contrast to NEW, DUPLICATE also causes the creation of a new archive, but no clear records are transferred. Instead, the compressed and encrypted segments and thus the 'flambeed' data is transferred. It is therefore not necessary to explicitly set any parameters for 'flambeing' here. Duplication is therefore the most effective way of dealing with 'flambeed' data, whereby everything is possible here, right up to searching and/or splitting the data via EOT handling and its formats.

If you process a large number of files using directory walk via wildcards, you may not want a rollback for the entire archive if an error occurs when processing some files. In this case, you should set the NORLBK parameter to the acceptable number of errors. If this threshold is surpassed, the archive is rolled back. If you specify the parameter but do not assign a value to it, it is 2^31-1 by default. Members that are only written partially due to an error can be deleted with the REMOVE switch. If appending to an existing archive member, the entire incomplete member is deleted in the new version of the archive. If an error is encountered while opening the original file or processing its first block, nothing is written. The member remains unchanged.

The SUBSET switch causes that only the selected data segments or matched records are copied (in compressed and encrypted form) to the new archive. A subset may only contain parts of member that match the specified filter criterias. These archives are therefore identified as subsets of a complete archive. Such a subset can later be unpacked using the DECO subcommand. The advantage of a segment subset is that it can be created by the archive operator without any knowledge of the data using a pre-calculated filter. This reduced amount of data can then be transferred and read on site by the data owner. To create a copy of an archive based on encrypted data segments, the DUPLICATE method must be used.

If the originating archive is partially damaged, you can attempt to read the rest of the archive by activating the REPAIR switch. This ignores all data segments that can no longer be accessed or that can no longer be unpacked, so that all data that is still readable can be extracted.

To get syntax information, please use:

   flcl SYNTAX ARCHIVE.COPY

To get help for a parameter, please use:

   flcl HELP ARCHIVE.COPY.parameter[.parameter[...]]

To read the manual page for a parameter, please use:

   flcl MANPAGE ARCHIVE.COPY.parameter[.parameter[...]]
      or
   flcl HELP ARCHIVE.COPY.parameter[.parameter[...]] MAN

To generate the user manual for the command, please use:

   flcl GENDOCU ARCHIVE.COPY=filename

Parameters can be defined via command line (directly or by parameter file) or via properties taken from the corresponding property file.

Arguments

• SWITCH: NORUN - Don't run the command, only show parsed parameter