MM_DATA(5) Legato NetWorker 4.0 MM_DATA(5)
NAME
MM data - NetWorker media multiplexor data (tape and disk) format
DESCRIPTION
This documents the data format that the NetWorker media multiplexor
daemon, nsrmmd(8), writes to long term storage media such as tapes
and optical disks. See nsrdevice(5) and nsrmm(8) for a discussion
of supported device families and types. The format described here
applies to any fixed record device, such as raw disks, or fixed
record tape devices with file marks. NetWorker uses the eXternal
Data Representation (XDR) standard to write media which can be inter
changed among a wide variety of machines. Only the mechanism used to
multiplex save set streams onto the storage media is described here;
the formats of save set streams depend on the type of NetWorker
client, and are described in nsrdata(5).
A volume is one physical piece of media such as a tape reel or disk
cartridge. A tape volume is made up of multiple media files, and
each media file may contain several media records. These media files
and records should not be confused with a client's (for example UNIX
or DOS) user files or records; the two do not necessarily correspond.
For example, a given media file or even a single media record may
contain many small client user files. On the other hand, a single
large client file may be split across several media files, and even
across several volumes. Media files do not span volume boundaries.
Save sets may span media files and even volumes.
On most tapes, media files can be skipped very quickly by the
device's hardware or associated device driver software, and the hard
ware can detect when an end of a file has been reached. On some
tapes, records can also be quickly skipped forward. Otherwise,
access to the media is sequential.
Media records are described by the mrecord structure. These records
are a fixed size, MAXMRECSIZE bytes. NetWorker always writes, reads
and skips data in units of full-sized media records. Each mrecord
contains zero or more mchunks of data from either one or more client
save sessions, or used internally for synchronization and labels.
The XDR format of a media file's mrecords and mchunks are as follows:
const MAXMRECSIZE = 32768; /* media record size used */
const MMAXCHK = 2048; /* max number of chunks in record */
const MHNDLEN = 128; /* private area len for handlers */
typedef unsigned long ssidt; /* save set id */
typedef unsigned long ssofft; /* an offset in a save set */
typedef unsigned long volidt; /* key for the volume data base */
struct mchunk {
ssidt mcssid; /* owning save set id */
ssofft mclow; /* 1st byte, relative to save stream */
opaque mcdata<MAXMRECSIZE>; /* chunk's data */
};
Licensed material--property of copyright holder(s) 1
MM_DATA(5) Legato NetWorker 4.0 MM_DATA(5)
struct mrecord {
opaque mrhandler[MHNDLEN]; /* private to media handler */
volidt mrvolid; /* encompassing volume's id */
ulong mrfn; /* encompassing file number */
ulong mrrn; /* record number within the file */
ulong mrlen; /* record byte length */
mchunk mrchunk<MMAXCHK>; /* chunks of save streams */
};
The first field of an mrecord, mrhandler, is reserved for media-specific
data (currently it is not used by any implementation). A media record's
header fields, mrvolid, mrfn, and mrrn, are used to check the tape posi
tion and the data read from the record. The file numbers and record numbers
start at zero and increment sequentially. The record number is reset each
time the file number is incremented. On disks, file numbers are always
zero. The mrlen field is the actual number of valid bytes in this record,
as opposed to the fixed size of the device's read or write request.
If file or record skipping is unreliable, NetWorker can still recover from
isolated errors, at worst by rewinding and reading the tape from the start.
If a volume can be physically unmounted or mounted without notice to the
media management daemon, then the volume identifier in each record provides
a quick way of verifying when this happens, without the need for a full
rewind and reading of the label in most cases.
The mchunks within an mrecord contain client data from one or more save ses
sions. The mcssid and mclow values are used to reconstruct the save
streams from the chunks within the records. The mcdata field holds the
actual data of each chunk. For a given save set, mclow plus the length of
mcdata should equal the following chunk's value for mclow. Save sets may
by intermingled arbitrarily within media records.
The first chunk of the first record of the first media file on the volume
encapsulates the volume label information; subsequent data in the first file
is reserved for future expansion. The label may be duplicated in a second
file for redundancy, in case the first copy of the label gets accidentally
overwritten. The format of a volume label is described by the following XDR
data structure:
const MVOLMAGIC = 0x070460;
const NSRLENGTH = 64; /* length of several strings */
struct mvollabel {
ulong mvlmagic; /* medium volume verification number */
ulong mvlcreatetime; /* time at which volume labeled */
ulong mvlexpiretime; /* time for volume to expire */
ulong mvlrecsize; /* expected size of mrecords */
volidt mvlvolid; /* medium volume id */
string mvlvolname<NSRLENGTH>; /* medium volume name */
};
The mvlmagic field must be equal to MVOLMAGIC in order for the chunk to
represent a valid volume label. If the volume label changes in the future,
the new format will have another ``magic'' number, but the format described
Licensed material--property of copyright holder(s) 2
MM_DATA(5) Legato NetWorker 4.0 MM_DATA(5)
here must still be allowed. The mvlvolid is an internal identifier
assigned and managed by the media manager. The mvlvolname is assigned and
managed by the systems administrator, or the NSR label server. The time
fields are in UST format - the number of seconds elapsed since 00:00 GMT,
January 1, 1970. The mvlrecsize is the size of all subsequent media
records found on the tape.
Synchronization marks, called schunks, are also written periodically to the
media. Synchronization chunks are used by scanner(8) when verifying or
extracting directly from a volume. They are also used by nsrmmd when trying
to recover from media errors. The following XDR data structure describes a
synchronization chunk:
struct schunk {
opaque ssihost[NSRLENGTH]; /* save set host */
opaque ssiname[NSRLENGTH]; /* symbolic name */
ulong ssitime; /* save time */
ulong ssiexpiry; /* expiration date */
ulong ssisize; /* actual size saved */
ulong ssinfiles; /* number of files */
ssidt ssissid; /* ssid for this save set*/
ulong ssiflag; /* see following values */
volidt ssivolid; /* volid - see ssiflag */
};
#define SSISTART 1 /* start of a save set */
#define SSISYNC 2 /* synchronization point */
#define SSICONT 3 /* continued from vol `ssivolid' */
#define SSIEND 4 /* end of this save set */
#define SSILMASK 0xff000000 /* Mask to cover bits for level */
#define SSILSHIFT 24 /* Shift amount for the level */
#define SSILBIAS 0x10000000 /* make sure this bit is set */
#define SSIVALID 0X00000100 /* if set, next fields are valid */
#define SSIPURGED 0x00000200 /* currently purged */
#define SSIDEPURGED 0x00000400 /* has been de-purged */
#define SSIPLOCK 0x00000800 /* should NOT be purged */
#define SSIELIGIBLE 0x00001000 /* is over-writable */
#define SSIRESURECTED 0x00002000 /* has been scannered in */
#define SSIELOCK 0x00004000 /* is NOT to be marked eligible */
#define SSIUSED 0x00008000 /* has been used in a recover */
#define SSIINCOMPLETE 0x00010000 /* `mmsend()' not properly called */
The ssihost is the name of the index which contains this save set. Tradi
tionally this is the host name of the client where the save set originated.
The ssiname is the save set name to be presented to the user. These are
both null-terminated strings, even though the fields are fixed length. The
ssitime field contains the create time of the save set in UST. The
ssiexpiry field is the expiration date, in UST for this save set. The
ssisize and ssinfiles are the number of bytes and number of files saved so
far for this save set. The ssissid is the the save set identifier of this
save set.
The ssiflag indicates the type of this synchronization chunk and the level
Licensed material--property of copyright holder(s) 3
MM_DATA(5) Legato NetWorker 4.0 MM_DATA(5)
of the associate save set. SSISTART is used to mark the beginning of a
save set. SSISYNC marks a periodic synchronization point. SSICONT indi
cates that this is the continuation of a save set that started on a differ
ent volume. When ssiflag is SSICONT, ssivolid contains the volume iden
tifier for the save set's preceding volume. SSIEND marks the end of this
save set. The SSILMASK can be used, if the SSILBIAS bit is set, to deter
mine the level of the save set.
The SSIVALID bit is set to indicate that the following bits are valid and
may be used. The SSIPURGED bit is set when this save set has been removed
from the on-line file index. The SSIPLOCK bit is set when this save set
should not be removed from the on-line file index. The SSIELIGIBLE bit
indicates that this save set may be recycled. The SSIRESURECTED bit indi
cates that scanner put re-created this entry. The SSIELOCK bit indicates
that this save set may not be recycled. The SSIUSED bit indicates that a
recovery has been done from this save set. The SSIINCOMPLETE bit indicates
that this save set did not finish properly. This could be caused by a user
interrupting an in progress save.
SEE ALSO
nsrdevice(5), nsrdata(5), nsrmm(8), nsrmmd(8), nsrmmdbd(8), nsr(8),
scanner(8).
RFC 1014 XDR: External Data Representation Specification
Licensed material--property of copyright holder(s) 4