NAME

vol − Logical Storage Manager (LSM) virtual disk devices

DESCRIPTION

There are two types of Logical Storage Manager virtual disk devices: volume devices and plex devices.  The volume devices support a virtual disk access method with disk mirroring and disk striping.  A volume is a logical entity composed of one or more plexes.  A read can be satisfied from any plex, while a write is directed to all plexes.  The virtual disk devices have a wide variety of behaviors, which are programmable through the /dev/volconfig device.  For volume devices, both block- and character-special devices are implemented. 

Each plex in the volume is a copy of the volume address space.  The plex has subdisks associated with it.  These subdisks provide backup storage for the volume address space. 

The plex virtual disk devices are implemented only as character-special devices. When a plex is associated with a volume, the vold process creates a device node for accessing the plex.  This node has the same uid, gid, and mode as the volume device it belongs to. A plex device acts like an alternate interface into the volume device. An open on a plex device opens the associated volume device.  A read on a plex device reads data through the volume, but only from the individual plex as specified by the plex device used.  A write on a plex device writes data to the volume, but only to the individual plex.  Direct use of plex devices is primarily useful for examining plexes, and for recovery from unusual conditions, where it is desirable to examine each plex of a volume. 

It is possible to create a sparse plex, which is a plex without backup storage for some of the volume address space.  The areas of a sparse plex that do not have a backing subdisk are called holes.  An attempt to read a hole in a sparse plex fails.  If there are other plexes that have backup storage for the read, then one of those plexes is read.  Otherwise, the read fails.  A write to a hole in a sparse plex is considered a success even though the data cannot be read back. 

In addition, a plex may be designated as a logging plex.  This means that a log of blocks that are in transition will be kept, which enables fast recovery after a system failure.  This feature is known as BCL, or block change logging.  The log for each plex consists of a specially designated subdisk that is not part of the normal plex address space. 

IOCTLS

The ioctl commands supported by the volume virtual disk device interface are discussed later in this section. The only ioctls supported for plex devices are GET_DAEMON and GET_VOLINFO. The format for calling each ioctl command is:

#include <sys/types.h> #include <sys/vol.h> struct tag arg; int ioctl (int fd, int cmd, struct tag ∗arg);

The value of cmd is the ioctl command code, and arg is usually a pointer to a structure containing the arguments that need to be passed to the kernel. 

The return value for all these ioctls, with some exceptions, is 0 if the command was successful, and −1 if it was rejected.  If the return value is −1, then errno is set to indicate the cause of the error. 

The following ioctl commands are supported:

GET_DAEMON
This ioctl returns the pid of the process with the volconfig device open, or 0 if the volconfig device is closed.  The value of arg is undefined and should be NULL. 

GET_VOLINFO
This command accepts a pointer to a volinfo structure as an argument.  It fills in the volinfo structure with the corresponding values from the kernel.  The members of a volinfo structure are:

long version;        /∗ 0x00 kernel version number ∗/
long voldinfo_sz;    /∗ 0x04 size of voldinfo area ∗/
major_t volbmajor;   /∗ 0x08 volume blk dev major number ∗/
major_t volcmajor;   /∗ 0x0c volume char dev major number ∗/
major_t plexmajor;   /∗ 0x10 plex device major number ∗/
long maxvol;         /∗ 0x14 max # of volumes supported ∗/
long maxplex;        /∗ 0x18 max # of associated plexes ∗/
long plexnum;        /∗ 0x1c max plexes per volume ∗/
long sdnum;          /∗ 0x20 max subdisks per plex ∗/
long max_ioctl;      /∗ 0x24 max size of ioctl data ∗/
long max_specio;     /∗ 0x28 max size of ioctl I/O op ∗/
long max_io;         /∗ 0x2c max size of I/O operation ∗/
long max_ilocksleep; /∗ 0x30 max sleeps on ilock ∗/
long dflt_iodelay;   /∗ 0x34 default I/O delay for utils ∗/
long max_freebuf;    /∗ 0x38 max number of volbufs to keep ∗/
long max_freevcb;    /∗ 0x3c max number of volcbs to keep ∗/
long max_parallelio; /∗ 0x40 max # voldios allowed ∗/

PLEX_DETACH
This command is used to force a plex to be detached from a volume.  The name of the plex is passed in as an argument.  The volume is the volume device against which the ioctl is being performed.

VOL_LOG_WRITE
This command forces a block change log to be flushed to disk.  This is used by the vold process to flush an initial log to disk before starting the volume. 

VOL_READ, VOL_WRITE
These commands provide a mechanism by which I/O can be issued to volumes larger than 2 gigabytes in length.  Current UNIX read and write system calls on 32-bit processors limit sizes to 2 gigabytes because of the signed byte-offset value used to perform the I/O. These ioctl commands provide a method of providing sector offsets to an I/O and raise the limit to one sector less than 1024 gigabytes.

The required I/O is identified to the command by the use of a vol_rdwr structure containing the following:

ulong_t vrw_flags;      /∗ flags ∗/
voff_t vrw_off;         /∗ offset in volume (sectors) ∗/
size_t vrw_size;        /∗ number of sectors to Xfer ∗/
caddr_t vrw_addr;       /∗ user address for Xfer ∗/

The vrw_flags field is currently unused; other fields are explained in the comments. 

The ATOMIC_COPY, VERIFY_READ, VERIFY_WRITE, PLEX_READ, and PLEX_WRITE ioctls, described later is this section, perform special I/O operations against the volume.  They use the vol_io structure to initiate I/O requests and receive the status information back.  The members of the vol_io structure are:

voff_t vi_offset;           /∗ 0x00 offset on plex ∗/
size_t vi_len;              /∗ 0x04 amount of data to read/write ∗/
caddr_t vi_buf;             /∗ 0x08 ptr to buffer ∗/
size_t vi_nsrcplex;         /∗ 0x0c number of source plexes ∗/
size_t vi_ndestplex;        /∗ 0x10 number of destination plexes ∗/
struct plx_ent ∗vi_plexptr; /∗ 0x14 ptr to array of plex entries ∗/
ulong_t vi_flag;            /∗ 0x18 flags associated with op ∗/

The members of the plx_ent structure are:

char pe_name[NAME_SZ]; /∗ name of plex ∗/
int pe_errno;          /∗ error number against plex ∗/

The vi_offset value specifies the sector offset of the I/O within the volume.  It must be within the address range of the volume. Also, the entire range of the I/O from vi_offset to vi_offset + vi_len must be within the address range of the volume. 

The vi_len field specifies the length of the I/O in sectors.  It must be a between 0 and 128 sectors (VOL_MAXSPECIALIO). 

The vi_buf field is a pointer to a buffer of vi_len sectors.  The  VERIFY_WRITE and PLEX_WRITE ioctls write the data stored in this buffer.  The  PLEX_READ reads data from a plex into this buffer. 

The vi_nsrcplex field is the number of source plexes available for the operation and the vi_ndestplex field is the number of destination plexes available for the operation.  The vi_nsrcplex and vi_ndestplex values must be between 0 and 8 (PLEX_NUM). 

The vi_plexptr is a pointer to an array of plx_ent structures.  The first vi_nsrcplex entries in the array are source plexes.  The pe_name contains the name of the plex.  If the name of the first source entry is the null string, then the kernel selects all plexes available for reading as part of the volume and fills in the pe_name fields. 

For  PLEX_READ and PLEX_WRITE operations, the vi_flag field may contain the  VI_LOGIO flag, indicating the I/O for that operation is from/to the log subdisk of the plex, rather than normal address space. 

After the source plexes, the next vi_ndestplex entries are the destination plexes.  If the name of the first destination entry is the null string, then the kernel selects all plexes available for writing as part of the volume and fills in the pe_name fields. 

After the I/O operations are performed, the plx_ent structures are copied back to the user.  If the kernel selected the plexes, the names of the selected plexes are in the pe_name fields.  The status of the operations on each plex are stored in the pe_errno field of the plx_ent structure for that plex. 

The pe_errno field is 0 if the operation succeeded against the plex.  If pe_errno is not 0, then the error code indicates what happened to the plex.  The possible values for pe_errno are:

ENOENT
The specified plex is not associated with the volume the ioctl was issued against.

EACCES
The specified plex is in the disabled state, so no I/O can be performed against it.

ENXIOAn error was detected in the operation, so no I/O operation was attempted to the plex.  If one plx_ent structure in a list contains a bad name, then no I/O is done.  All plx_ent structures with a valid name have their pe_errno set to ENXIO to indicate no I/O was attempted. 

EFAULT
A source plex is sparse and does not have blocks that map the entire I/O request.

EIOA read I/O error was returned against a source plex. 

EROFSA write I/O error was returned against a destination plex. 

ESRCHThe  VERIFY_READ and VERIFY_WRITE operations compare the data from different plexes against each other to verify the consistency.  If the comparison detects an error, the plex that was read first is considered correct.  An  ESRCH error is returned against the plex that was read second to indicate it contains bad data. 

For the  ATOMIC_COPY, VERIFY_READ, and VERIFY_WRITE ioctls, if the entire operation is a success, then a 0 is returned. If there is a fatal error, a −1 is returned and the external variable errno indicates the reason for failure.  If a 1 is returned, there was some sort of failure and the net results must be determined by examining the pe_errno fields of all the plexes. 

The ioctls that do volume-special I/O are:

ATOMIC_COPY
This ioctl takes a pointer to a vol_io structure as an argument.  It reads vi_len sectors of data, at offset vi_offset, from one of the plexes specified by the first vi_nsrcplex plex entries into a buffer.  Then it writes the contents of the buffer onto all the plexes specified by the next vi_ndestplex plex entries at the same offset.  This entire operation is atomic with respect to the I/O stream of the volume.  The vi_buf field is unused and should be NULL. 

If the first source plex entry has a null string for the name, the kernel selects from any plex of the volume that is enabled for read access.  The names of any selected plexes are copied into the appropriate plex entries. 

If the first destination plex entry has a null string for the name, the kernel will write to all plexes of the volume that are enabled for write access.  The names of selected plexes are copied into the appropriate plex entries. 

When the list of source plexes has been compiled, the kernel tries to read each plex in order.  A plex cannot be read if it does not have backup storage covering the entire operation.  Once a plex has been successfully read, all the destination plexes are written.  The writes can succeed even if the destination plex is sparse and does not have backup storage to cover the entire write. 

If the ioctl returns a value of −1, then some error has occurred which prevented the  ATOMIC_COPY from working.  If the return value is 0, then everything worked fine.  If the return value is 1, then the pe_errno field must be examined to determine the errors on each individual plex.  The status of the overall operation depends on these individual errors. 

PLEX_READ
This ioctl is used to read a specific plex of a volume.  The arg is a pointer to a vol_io structure.  In addition to the constraints previously mentioned, the vi_ndestplex field must be 0.  The vi_nsrcplex field must be 1 and the first plx_ent structure must contain a valid plex name.  The specified plex must be associated with the volume and not disabled. 

The ioctl reads vi_len sectors from the plex at an offset of vi_offset sectors.  The data is read into the buffer specified by the vi_buf field. 

If the VI_LOGIO flag is set in vi_flag, then the read is satisfied from the log subdisk, rather than from the normal plex address space. 

PLEX_WRITE
This ioctl is used to write a specific plex of a volume.  The arg is a pointer to a vol_io structure.  In addition to the constraints previously mentioned, the vi_nsrcplex field must be 0.  The vi_ndestplex field must be 1 and the first plx_ent structure must contain a valid plex name.  The specified plex must be associated with the volume and not disabled. 

The ioctl writes vi_len sectors to the plex at an offset of vi_offset sectors.  The data is written from the buffer specified by the vi_buf field.  Even if there is more than one plex associated with the volume, only the specified plex is written. 

If the  VI_LOGIO flag is set in vi_flag, then the write is to the log subdisk rather than to the normal plex address space. 

VERIFY_READ
This command accepts a pointer to a vol_io structure as an argument.  It reads vi_len sectors, from offset vi_offset, on the first vi_nsrcplex plexes specified by the vi_plexptr array of plex entries.  It compares the data from each plex. If any plex is different from the previous plexes read, the pe_errno value for that plex is set to ESRCH. This entire operation is atomic with respect to the I/O stream of the volume. 

The vi_ndestplex field must be 0.  If the vi_buf field is not  NULL, then the data read from the plexes not marked with ESRCH is copied into the buffer specified by vi_buf. 

As each plex is read, any data that has already been read is compared against the previous reads.  If all the data for the plex passes, then any data that was read for the first time is copied into the comparison buffer.  This allows  VERIFY_READ operations to work if the volume contains sparse plexes.  The data not represented by backup storage is not compared against anything. 

VERIFY_WRITE
This command accepts a pointer to a vol_io structure as an argument.  It writes vi_len sectors to offset vi_offset on the first vi_ndestplex plexes specified by the vi_plexptr array of plex entries.  The data to be written is stored in the buffer pointed to by vi_buf.  Then, the data is read back from each plex that was successfully written and is compared against the data written.  If the data from any plex does not match the data written, the pe_errno value for the plex is set to ESRCH.  This entire operation is atomic with respect to the I/O stream of the volume. 

The vi_nsrcplex field must be 0. 

As each plex is read, any data that does not have backup storage on that plex is filled in from the write buffer.  This allows VERIFY_WRITE operations to work if the volume contains sparse plexes.  The data not represented by backup storage will always succeed. 

VOL_LOG_WRITE
This command takes no argument, and causes the log for a volume to be written to disk immediately. This command is useful for making sure that the on-disk images of the log have been written.

This command returns −1 with errno set to  EINVAL if the specified volume does not have logging enabled. 

PLEX_DETACH
This command allows an enabled plex to be detached. The argument is the name of the plex to detach.

ERRORS

The following errors are returned by the volume and plex virtual disk device interfaces:

ENXIOA validation error occurred during an I/O operation.  If this error is returned, the driver attempted no I/O. 

EIOA physical I/O error occurred during an operation.  If this error is returned, the driver tried an I/O and it failed. 

EAGAIN
A needed kernel resource could not be obtained.

EFAULT
A pointer passed to the kernel was invalid, causing a bad memory reference.

EINVAL
Invalid data was passed to the kernel.  Some field in a vol_io structure failed a sanity check. 

EBADF
An attempt was made to write a volume that was not opened for writing or read a volume that was not open for reading.

ENOENT
A plex named in a  PLEX_READ or  PLEX_WRITE was not associated with the volume. 

ENOENT
An object named in  GET_VOL_STATS ioctl was not associated with the volume. 

ENOENT
The kernel was asked to supply a list of destination plexes for an ATOMIC_COPY or VERIFY_WRITE ioctl.  If no enabled plexes available in write mode are found, an ENOENT error is returned. 

ESRCHThe kernel was asked to supply a list of source plexes for an ATOMIC_COPY or VERIFY_READ ioctl.  If no enabled plexes available in read mode are found, an ESRCH error is returned. 

ENFILE
The kernel was asked to supply a list of source plexes for an ATOMIC_COPY or VERIFY_READ ioctl.  If more than vi_nsrcplex enabled plexes available in read mode are found, an ENFILE error is returned. 

EMFILE
The kernel was asked to supply a list of destination plexes for an ATOMIC_COPY or VERIFY_WRITE ioctl.  If more than vi_ndestplex enabled plexes available in write mode are found, an EMFILE error is returned. 

FILES

/dev/plex/∗
Raw plex device files

/dev/vol/∗
Volume block device files

/dev/rvol/∗
Volume character (raw) device files

RELATED INFORMATION

volintro(8), ioctl(2), volconfig(7), voliod(7), volevent(7).