NEWSBATCH(8) UNIX System V(7 Nov 1990) NEWSBATCH(8)
NAME
sendbatches, batchsplit - news batching to other sites
batcher, batchih, batchsm - news-batch preparation
comp, nocomp, compcun, compc7, compb - news-batch compression
c7encode, bencode - compressed-news-batch encoding
viauux, viauuxz, viauuxcun, viauuxl - news-batch transmission via uucp
viamail, viaemail, viapmail - news-batch transmission via mail
viainews, viarsh - news-batch transmission by misc. means
SYNOPSIS
/usr/dell/cnews/newsbin/batch/sendbatches [ site ... ]
.../batchsplit batchsize
.../batcher listfile
.../batchih listfile
.../batchsm listfile
.../comp
.../nocomp
.../compcun
.../compc7
.../compb
.../c7encode
.../bencode [ file ]
.../viauux site
.../viauuxz site
.../viauuxcun site
.../viauuxl group
.../viamail site
.../viaemail site
.../viapmail site
.../viainews site
.../viarsh site
DESCRIPTION
Sendbatches administers batched transmission of news to other sites. It
should be run periodically, under userid news, by cron(8) or similar
means. It prepares and sends batches of news, subject to restrictions on
available space and length of outbound queues.
Each site that can have batches sent to it needs a site directory under
/var/spool/news/out.going. If sendbatches is invoked with specific sites
given, it considers batching for those sites, only, in that order. By
default, sendbatches consults the batchparms file (see below) to
determine what to do: If there is a /default/ entry in batchparms,
sendbatches will consider batching for all sites that have directories in
/var/spool/news/out.going, in oldest-first order by modification time of
the directory. If there is no /default/ entry, sendbatches considers
batching for those sites named in batchparms, in the order named.
To use the batcher, names of files to be sent to a specific site should
be appended to a togo file in its site directory. The batcher expects
the lines in togo to have two fields, a filename (as a full pathname, or
10/89 Page 1
NEWSBATCH(8) UNIX System V(7 Nov 1990) NEWSBATCH(8)
relative to /var/spool/news) of an article and its size in bytes. A
missing size field is arbitrarily assumed to be a default average.
Sendbatches uses a number of auxiliary programs to do the real work. The
search path it uses to find them includes, in order, the site directory
for the site in question, /usr/dell/cnews/lib/bin, and
/usr/dell/cnews/newsbin/batch. This permits per-site and per-news-
database overrides of the default behaviors. Sendbatches provides all
these programs with environment variables NEWSSITE, containing the name
of the site that batches are being prepared for, and NEWSSITEDIR,
containing the full pathname of the site directory, in case these are
useful in customization.
The names of most of the auxiliary programs, and some other parameters,
are taken from the file /usr/dell/cnews/lib/batchparms, an ASCII text
file. Empty lines and lines starting with `#' are ignored. Other lines
specify the behavior for sites, one line per site. A site line is six
fields, separated by white space. The first field is the site name; a
line whose site name is /default/ specifies what parameters should be
used for sites not explicitly mentioned. (The presence or absence of
such a line also influences the behavior of sendbatches when invoked
without arguments; see above.) The second field is the size of batches
to be prepared (before compression), in bytes. The third field is the
maximum length of the output queue for transmission to that site. The
fourth, fifth, and sixth fields are the names of the programs to be used
to build, compress, and transmit (respectively) batches to that site.
For each site being considered for batches, sendbatches first determines
whether there are in fact any articles to be batched. Assuming there
are, sendbatches then finds the batchparms line for that site and invokes
queuelen (see newsaux(8)) to find out the size of the outbound queue for
the site. Sendbatches limits the number of batches prepared to the
minimum of the limits implied by queue lengths and available space.
Sendbatches uses batchsplit as necessary to slice chunks out of the togo
file, each chunk containing the togo lines for a batch limited to the
specified size. Exception: a single article bigger than the specified
size will still go out as one batch.
Each chunk is then processed through a batch preparer (typically
batcher), which assembles the articles into a batch, a batch compressor
(typically compcun), which performs compression and/or any other
auxiliary processing needed, and a batch transmitter (typically viauux),
which sends the batch on its way (e.g. enqueues it for transmission).
The preparer is run with /var/spool/news as the current directory, so
non-absolute pathnames in the chunk are valid filenames; the others are
run in the site directory for easier access to site-specific
customization files.
Batch preparers in the standard distribution are:
Page 2 10/89
NEWSBATCH(8) UNIX System V(7 Nov 1990) NEWSBATCH(8)
batcher normal batching
batchih ihave part of old ihave/sendme (not to be confused with
NNTP)
batchsm sendme part of ihave/sendme
Batchih and batchsm have to map from the phony ``site name'' given in
their batchparms line to the name of the site they should actually send
to; they do this by stripping off the last `.' and everything that
follows (usually `.ihave' or `.sendme' respectively, but on machines
which limit the size of filenames these may have to be shortened).
Batch compressors in the standard distribution are:
comp ordinary compression
compcun compression plus the silly B-news-compatible `#! cunbatch'
header
nocomp pass data straight through, no compression
compc7 compression plus 7-bit encoding using c7encode
compb compression plus very conservative 6-bit encoding using
bencode
Batch transmitters in the standard distribution are:
viauux normal transmission via UUCP
viauuxz like viauux except with -z option given to uux (for old
UUCPs where don't-report-result-on-zero-status is not
default)
viauuxcun like viauux except it invokes cunbatch rather than rnews
at the other end (for some very old news sites)
viauuxl multicast transmission using the -l option of uux (not
found on all systems) to send the same batch to all
systems listed in the file
`/usr/dell/cnews/lib/sites.group'
viamail mail the batch to site!rnews
viaemail mail the batch to site!enews
viapmail mail the batch to site!rnews, attempting to protect an
unencoded batch against the vagaries of mailers
10/89 Page 3
NEWSBATCH(8) UNIX System V(7 Nov 1990) NEWSBATCH(8)
viainews feed the batch back to inews, ignoring the site argument
(normally useful only for ihave/sendme)
viarsh use rsh to run rnews on the site via Ethernet, Internet,
etc. (the directory containing rnews must be in the
default PATH on site)
C7encode encodes 8-bit data from standard input into a 7-bit form
optimized for transmission by uucp `f' protocol. The encoding is complex
and bizarre.
Bencode takes 8-bit data from the named source file (the default is
standard input) and encodes it using only the ASCII characters ``A'' -
``Z'', ``a'' - ``z'', ``0'' - ``9'', ``+'', and ``-''. The ASCII
characters blank, newline, and ``/'' also appear in the encoded file, but
do not represent encoded bits. The encoded file is terminated with a
byte count and cyclic redundancy check for detecting corrupted files.
This ought to suffice to get data through almost any network.
Sendbatches logs some information about sites with backlogs in
/usr/dell/cnews/lib/batchlog (see also newsdaily(8)). This is intended
to help detection and diagnosis of flow problems.
FILES
/usr/dell/cnews/lib/LOCKbatchlock for sendbatches
/usr/dell/cnews/lib/LOCK overall news lock (used by batchsplit)
/usr/dell/cnews/lib/L.* lock temporaries
/var/spool/news/out.going/* batch directories
/usr/dell/cnews/lib/batchparmsparameter file
/usr/dell/cnews/lib/sites.* viauuxl multicast lists
also see text
SEE ALSO
inews(1), compress(1), uux(1), relaynews(8), rnews(8)
DIAGNOSTICS
Complaints, if any, from sendbatches and its auxiliaries are mailed to
`usenet'.
HISTORY
Written at University of Toronto as part of the C News project. A number
of the ideas came from Chris Lewis, who was then with Motorola. Bencode
written at University of Waterloo by Reg Quinton and Ken Lalonde.
BUGS
Batchsplit does not count the `#! rnews nnnn' headers inside batches when
computing batch lengths.
Ihave/sendme processing is a bit of a kludge; in particular, applying
batchsplit to a file full of article IDs just happens to work okay.
Page 4 10/89
NEWSBATCH(8) UNIX System V(7 Nov 1990) NEWSBATCH(8)
Sendbatches estimates available space without considering the effects of
compression; this is usually too conservative.
Viapmail is obsolescent; viaemail with a batch encoded with bencode is
vastly superior.
Queue-length limiting does not work for viauuxl because queuelen (see
newsaux(8)) does not know about multicast groups. Also, viauuxl has not
been tested well.
Viarsh does not incorporate a spooling subsystem, so a slow site stalls
the entire batching system and a non-responding site loses news. It is
not recommended for bulk transmission or where high reliability is
essential.
10/89 Page 5