MM_DATA(5) Legato NetWorker 4.1.1 MM_DATA(5)
NAME
MM data - NetWorker media multiplexor data (tape and disk) format
DESCRIPTION
This documents the data format that the NetWorker media multiplexor
daemon, nsrmmd(8), writes to long term storage media such as tapes
and optical disks. See nsrdevice(5) and nsrmm(8) for a discussion
of supported device families and types. The format described here
applies to any fixed record device, such as raw disks, or fixed
record tape devices with file marks. NetWorker uses the eXternal
Data Representation (XDR) standard to write media which can be
interchanged among a wide variety of machines. Only the mechanism
used to multiplex save set streams onto the storage media is
described here; the formats of save set streams depend on the type of
NetWorker client, and are described in nsrdata(5).
A volume is one physical piece of media such as a tape reel or disk
cartridge. A tape volume is made up of multiple media files, and
each media file may contain several media records. These media files
and records should not be confused with a client's (for example UNIX
or DOS) user files or records; the two do not necessarily correspond.
For example, a given media file or even a single media record may
contain many small client user files. On the other hand, a single
large client file may be split across several media files, and even
across several volumes. Media files do not span volume boundaries.
Save sets may span media files and even volumes.
On most tapes, media files can be skipped very quickly by the
device's hardware or associated device driver software, and the
hardware can detect when an end of a file has been reached. On some
tapes, records can also be quickly skipped forward. Otherwise,
access to the media is sequential.
Media records are described by the mrecord structure. These records
are a fixed size, MAXMRECSIZE bytes. NetWorker always writes, reads
and skips data in units of full-sized media records. Each mrecord
contains zero or more mchunks of data from either one or more client
save sessions, or used internally for synchronization and labels.
The XDR format of a media file's mrecords and mchunks are as follows:
const MAXMRECSIZE = 32768; /* media record size used */
const MMAXCHK = 2048; /* max number of chunks in record */
const MHNDLEN = 128; /* private area len for handlers */
typedef unsigned long ssid_t; /* save set id */
typedef unsigned long ssoff_t; /* an offset in a save set */
typedef unsigned long volid_t; /* key for the volume data base */
struct mchunk {
ssid_t mc_ssid; /* owning save set id */
ssoff_t mc_low; /* 1st byte, relative to save stream */
opaque mc_data<MAXMRECSIZE>;/* chunk's data */
};
Licensed material--property of copyright holder(s) 1
MM_DATA(5) Legato NetWorker 4.1.1 MM_DATA(5)
struct mrecord {
opaque mr_handler[MHNDLEN]; /* private to media handler */
volid_t mr_volid; /* encompassing volume's id */
u_long mr_fn; /* encompassing file number */
u_long mr_rn; /* record number within the file */
u_long mr_len; /* record byte length */
mchunk mr_chunk<MMAXCHK>; /* chunks of save streams */
};
The first field of an mrecord, mrhandler, is reserved for media-specific
data (currently it is not used by any implementation). A media record's
header fields, mrvolid, mrfn, and mrrn, are used to check the tape
position and the data read from the record. The file numbers and record
numbers start at zero and increment sequentially. The record number is
reset each time the file number is incremented. On disks, file numbers are
always zero. The mrlen field is the actual number of valid bytes in this
record, as opposed to the fixed size of the device's read or write request.
If file or record skipping is unreliable, NetWorker can still recover from
isolated errors, at worst by rewinding and reading the tape from the start.
If a volume can be physically unmounted or mounted without notice to the
media management daemon, then the volume identifier in each record provides
a quick way of verifying when this happens, without the need for a full
rewind and reading of the label in most cases.
The mchunks within an mrecord contain client data from one or more save
sessions. The mcssid and mclow values are used to reconstruct the save
streams from the chunks within the records. The mcdata field holds the
actual data of each chunk. For a given save set, mclow plus the length of
mcdata should equal the following chunk's value for mclow. Save sets may
by intermingled arbitrarily within media records.
The first chunk of the first record of the first media file on the volume
encapsulates the volume label information; for some media, the second chunk
contains additional volume information, for example the media pool the
volume belongs to; subsequent data in the first file is reserved for future
expansion. The label may be duplicated in a second file for redundancy, in
case the first copy of the label gets accidentally overwritten. The formats
of the volume label and additional label information are described by the
following XDR data structures:
const MVOLMAGIC = 0x070460; /* volume magic number */
const NSR_LENGTH = 64; /* length of several strings */
const RAP_MAXVALLEN = 1024; /* max length of EACH value */
const RAP_MAXNAMELEN = 64; /* max length of attribute name */
struct mvollabel {
u_long mvl_magic; /* medium volume verification number */
u_long mvl_createtime; /* time at which volume labeled */
u_long mvl_expiretime; /* time for volume to expire */
u_long mvl_recsize; /* expected size of mrecords */
volid_t mvl_volid; /* medium volume id */
Licensed material--property of copyright holder(s) 2
MM_DATA(5) Legato NetWorker 4.1.1 MM_DATA(5)
string mvl_volname<NSR_LENGTH>;/* medium volume name */
};
struct vallist {
vallist *next;
string value<RAP_MAXVALLEN>;/* attribute value */
};
struct attrlist {
attrlist *next;
string name<RAP_MAXNAMELEN>;/* attribute name */
vallist *values; /* attribute values */
};
/*
* Additional information may includes the following attributes
* (listed by the name they are stored with):
* "volume pool" : the media pool
*/
struct mvolinfo {
struct attrlist *mvi_attributes;/* any other information */
};
The mvlmagic field must be equal to MVOLMAGIC in order for the chunk to
represent a valid volume label. If the volume label changes in the future,
the new format will have another ``magic'' number, but the format described
here must still be allowed. The mvlvolid is an internal identifier
assigned and managed by the media manager. The mvlvolname is the volume
name that is assigned when the media is first labeled. The time fields are
in UST format - the number of seconds elapsed since 00:00 GMT, January 1,
1970. The mvlrecsize is the size of all subsequent media records found on
the tape.
The mvppool is the pool name that is assigned when the media is first
labeled. Different media pools allow administrators to segregate their data
onto sets of volumes. Media cannot be reassigned from one media pool to
another. Pool names are a maximum of NSRLENGTH characters long.
Synchronization marks, called schunks, are also written periodically to the
media for each save set. Synchronization chunks are used by scanner(8) when
verifying or extracting directly from a volume. They are also used by
nsrmmd when trying to recover from media errors during file recovery. The
following XDR data structure describes a synchronization chunk:
struct schunk {
opaque ssi_host[NSR_LENGTH]; /* save set host */
opaque ssi_name[NSR_LENGTH]; /* symbolic name */
u_long ssi_time; /* save time */
u_long ssi_expiry; /* expiration date */
u_long ssi_size; /* actual size saved */
u_long ssi_nfiles; /* number of files */
ssid_t ssi_ssid; /* ssid for this save set */
Licensed material--property of copyright holder(s) 3
MM_DATA(5) Legato NetWorker 4.1.1 MM_DATA(5)
u_long ssi_flag; /* various flags, see below */
u_long ssi_info; /* volid or ssid, see below */
};
#define SSI_START 1 /* start of a save set */
#define SSI_SYNC 2 /* synchronization point */
#define SSI_CONT 3 /* continued from another volume */
#define SSI_END 4 /* end of this save set */
#define SSI_SSMASK 0x0000000f /* save set sync chunk type */
#define SSI_LBIAS 0x10000000 /* the level is included in the flags */
#define SSI_LMASK 0xff000000 /* mask to cover bits for level */
#define SSI_LSHIFT 24 /* shift amount for the level */
#define SSI_INCOMPLETE 0x00010000 /* not finished (aborted) */
#define SSI_CONTINUED 0x00800000 /* continued save set series */
The ssihost is the name of the index which contains this save set.
Traditionally this is the host name of the client where the save set
originated. The ssiname is the save set name to be presented to the user.
These are both null-terminated strings, even though the fields are fixed
length. The ssitime field contains the create time of the save set in UST.
The ssiexpiry field is the expiration date, in UST for this save set. This
field is zero if an explicit save set expiration time was not specified when
the save set was created. The ssisize and ssinfiles are the number of
bytes and number of files saved so far for this save set. The ssissid is
the save set identifier of this save set.
The ssiflag indicates the type of this synchronization chunk, the level of
save set, and other information about the save set. There are four basic
types of synchronization marks that can be found from examining ssiflag &
SSISSMASK. SSISTART is used to mark the beginning of a save set.
SSISYNC marks a periodic synchronization point and is only written at an
exact file boundary in the save set. SSICONT indicates that this is the
continuation of a save set that started on a different volume. When
ssiflag & SSISSMASK is SSICONT, ssiinfo contains the volume identifier
for the save set's preceding volume. These synchronization chunks are used
when a save set spans a volume boundary. SSIEND marks the end of a save
set.
If the SSILBIAS bit is set then ssiflag & SSILMASK shifted right by
SSILSHIFT specifies the level of the save set. The SSIINCOMPLETE bit
indicates that this save set did not finish properly. This could be caused
by a user interrupting an in progress save.
The SSICONTINUED bit indicates that this save set is logically continued to
or from another save set. These continued save sets are used to handle very
large save sets. If the SSICONTINUED bit is set and ssiflag & SSISSMASK
is SSISTART, then ssiinfo gives the previous save set id that this save
set was continued from. If the SSICONTINUED bit is set and ssiflag &
SSISSMASK is SSIEND, then ssiinfo gives the next save set id that this
save set is continued to.
SEE ALSO
nsrdevice(5), nsrdata(5), nsrmm(8), nsrmmd(8), nsrmmdbd(8), nsr(8),
scanner(8).
Licensed material--property of copyright holder(s) 4
MM_DATA(5) Legato NetWorker 4.1.1 MM_DATA(5)
RFC 1014 XDR: External Data Representation Specification
Licensed material--property of copyright holder(s) 5