BACKUP Arcana, Tape Caveats, FTP and COPY and zip and unzip

BACKUP is an arcane and comparatively complex DCL command, and prone to various errors and assumptions.

Process Quotas

There are the issues around proper process quotas. Ill-configured BACKUP username process quotas can cause severe performance issues, and can potentially lead to run-time errors.

File Attributes and FTP Network Transfers

BACKUP savesets are also comparatively incompatible with FTP network transfers. In particular, BACKUP savesets transfered over FTP are almost invariably have the saveset file attributes corrupted.

The traditional FTP attribute corruptions can be recovered and the saveset processed with the Freeware tool RESET_BACKUP_SAVESET_ATTRIBUTES.COM, or by issuing the proper SET FILE /ATTRIBUTES command for the particular block size that was selected when the saveset was written.

Protecting Saveset File Attributes using Zip and Unzip

If you are seeking to transfer BACKUP savesets via FTP, you'll want to use zip “-V” to protect the attributes. You can use standard zip and unzip tools, or you can append the zipped (with “-V”) zip archive onto the available unzip-sfx self-extracting tool executable to create your own self-extracting zip archive. (The “-V” is a case-sensitive command switch; a switch which selects and enables use of the zip extensions for OpenVMS and its RMS file attributes; use of “-V” preserves the file and record attributes.)

Further complexity: zip versions prior to 3.0 and unzip prior to 6.0 (zip 2.32 and unzip 5.52 are current; zip and unzip betas for 3.0 and 6.0 are available) cannot successfully process wads of more than two to four gibibytes (GiB; base-two Gigabytes) of data.

Zip 3.0 and unzip 6.0 (currently in beta) are expected to remove this limit.

The gzip port available OpenVMS Freeware V8.0 can package larger volumes of data, but that lacks the mechanisms to protect the OpenVMS file attributes.

BACKUP itself can process far larger wads of data.

Data Compression

BACKUP /DATA_FORMAT=COMPRESS is an undocumented qualifier first present in OpenVMS V8.3. This qualifier selects and uses the zlib compression, allowing BACKUP to compress or decompress a disk-based saveset.

Being undocumented, details around known and expected incompatibility with tape devices are not mentioned; you can't write the compressed savesets to tape devices, and you can't copy the compressed savesets to tape devices. This because the output saveset format for compressed data can contain very small records, and these small records are not permitted on tape. The smallest record that can be written to tape medium is 14 bytes, and the zlib compress tool used by BACKUP /DATA_FORMAT=COMPRESS can easily create smaller records.

Various tape devices already implement compression and the rated capacities of many tape devices assume 2:1 compression. If writing savesets to tape, you will typically already have access to data compression. Within DCL, OpenVMS refers to this device-level compression as compaction; BACKUP /MEDIA_FORMAT=COMPACTION is the command that explicitly requests this. Remember to MOUNT with /MEDIA_FORMAT=COMPACTION, and to use this qualifier on each of the subsequent BACKUP commands; use the qualifier consistently through all operations associated with the target tape medium.

For quantities of data requiring compression and less than two to four gibibytes in total quantity, the use of zip “-V” and its compression can often be preferred to BACKUP, and to BACKUP V8.3 and its compression. Unfortunately for this approach, BACKUP is integrated into base OpenVMS and zip and unzip are not.

Data Encryption

BACKUP has had support for data encryption for many years, and this support is enabled through the installation and configuration of the Encryption layered product, and the registration of the Encryption license PAK.

The Encryption layered product provides DES data encryption, and various commands and mechanisms present in OpenVMS can use this encryption. Including BACKUP.

With OpenVMS V8.2, Encryption is licensed with OpenVMS, though installed separately.

With OpenVMS V8.3, Encryption is licensed with and integrated with OpenVMS; there is no separate layered product kit and no license PAK. The OpenVMS V8.3 Encryption mechanisms include DES and add FIPS-197 AES (Rijndael) data encryption.

Bootable BACKUP

OpenVMS VAX offers “Standalone BACKUP”, which is a very limited bootable environment where the only command available is BACKUP /IMAGE. No other commands, no MOUNT command (that happens in the background), and no file-level operations, are available in this environment. Standalone BACKUP can be booted from a tape device, though if you can at all manage it, you should avoid this approach. It is very, very, very slow. Use a disk-based or CD-based bootable environment. This boots far faster.

Recent OpenVMS VAX on CD and OpenVMS Alpha V6.1 and later and all OpenVMS I64 can and do provide a bootable environment. (The OpenVMS VAX mechanism is buried on the more recent of OpenVMS VAX CD distributions, but if you poke around you can find the root you need to boot.) The bootable environment allows you to boot the distribution disk (or create your own customized bootable media) and where you can perform MOUNT, DISMOUNT, DIRECTORY, COPY, BACKUP, SHOW and the other typical DCL commands.

COPY and tape devices

There are other ways to read and write tape data, including the DCL command COPY. HoffmanLabs does not recommend using the COPY command to write to tape, though such is feasible and occasionally and sometimes even necessary. There are historical cases where fixed-length records need to be delivered to or read from a target system via tape media, and COPY and MOUNT can specify fixed-size records and can provide this transfer. However, COPY does not handle tape volume switches, and COPY does not generate the data verification and recovery structures (CRC and XOR), and COPY does not deal with the variety of tape errors that BACKUP can manage.

Tape Error Recovery

You can choose to trust the tape-level data verification and error recovery mechanisms, and disable CRC (/NOCRC) and XOR (/GROUP=0) recovery data. One school of thought is that errors from DLT drives and similar are irrecoverable — while true for end-user DLT firmware, this isn't quite the case in general. Tape data recovery organizations have access to devices with firmware that can potentially read past the tape error(s), and the added BACKUP recovery data present can potentially permit reconstruction of the original data.

New to BACKUP?

The BACKUP command is complex and potentially quite confusing. Even experienced system managers can and do have occasional and various troubles with the command.

If you are just getting started with BACKUP, consider using the BACKUP$MANAGER interface. This simplifies the typical operations.

As for using the command line, HoffmanLabs recommends you study and use the example commands from the help text and the BACKUP documentation — there are an extensive set of BACKUP commands available in the examples, and examples are provided that will perform most any of the typical required operations.

vmsbackup on Unix, Mac OS X; BACKUP-related articles

Looking to access a BACKUP saveset on another platform? See the vmsbackup tool here at HoffmanLabs, and see the various related articles that are available.

Help is Available
HoffmanLabs can assist with the design and planning for data recovery and disaster recovery, and with the construction and maintenance of automated BACKUP DCL command procedures.

Caution with combined BACKUP Compression and Encryption

There's a cautionary note floating around, indicating that the BACKUP qualifier /DATA_FORMAT=COMPRESS (which is undocumented) and the qualifier /ENCRYPT (which is documented) don't play well together.

In general, an application will want to compress first, then encrypt — solidly encrypted data is inherently incompressible as encryption seeks to remove the patterns that compression searches for.

BACKUP apparently does not sequence these two operations correctly.