DSEE OVERVIEW
The DOMAIN Software Engineering Environment (DSEE) is a support environment
for software development. The DSEE facility helps software engineers develop,
maintain, and manage projects; it is especially useful for large scale projects
involving a number of modules and developers. You can use the DSEE facility to
automate the following:
o Source code and documentation storage and control
o Project histories
o Audit trails
o Dependency tracking
o System building
o Release and Engineering Change Order (ECO) control.
The Issues the DSEE Facility Addresses
======================================
The DSEE facility automates many of the processes engineers use, or should use,
to handle the problems that arise in a large scale programming project.
Complexity
Large programming projects usually involve many developers working on numerous,
interrelated components such as program modules, design specs, and end-user
documentation. Developers often work concurrently on separate modules of the
same program, and sometimes wish to work concurrently on the same modules.
Often they want to keep original and modified versions of the same program. In
addition, many projects produce software for more than one target, and must
support more than one version of a system simultaneously.
Organizing all the versions of source code and text in an orderly fashion is a
complicated task, as is the maintenance of separate, consistent testing
environments for modules with numerous authors. The DSEE facility automates
these administrative functions by storing successive versions of modules and
related text in a controlled database, and by managing the binaries produced
from those modules.
The more complex the development effort, the harder it is to maintain
consistency among interdependent components. The DSEE facility allows users to
specify both inclusive and semantic dependency relationships.
Inclusive dependencies are so named because they occur when files have
relationships such as the Pascal or C "include" relationship. When a component
has an inclusive dependency on a second component, a change in the second
component logically changes the first component. For example, if an include
file used by a module changes, the module may need to be recompiled, depending
on the nature of the change.
The DSEE configuration management facility handles all inclusive dependencies
and ensures that components are rebuilt when necessary.
When a component has a semantic dependency on a second component, a change in
the second component may require a change to the first component in order to
maintain consistency between the components. A software document usually has a
semantic dependency on the program it describes: whenever a documented feature
of the program changes, the document may need to be changed too.
The DSEE monitor mechanism can automatically monitor semantic dependency
relationships entered in the project database. It can automatically notify
project members when semantic dependencies are affected by changes to source
code and documentation.
Project Communication
Projects involving more than one person always require some communication. The
amount of communication required increases sharply as the number of project
members rises. Communication is time consuming, but a lack of it can lead to
problems that are even more time consuming. For example, if a developer changes
a module which his colleagues need, but forgets to mention the changes, the
other developers could spend a long time debugging before they discover
inconsistencies introduced by the change.
The DSEE monitor and tasking mechanisms automate much of this communication.
Users who depend on particular modules for their own work can set DSEE
"monitors" on those modules, so that they'll be notified immediately when
another user makes a change. The tasking mechanism helps administrators and
users to organize large jobs into small steps and to verify their completion.
In addition, experienced project members often have to familiarize new members
with the project. Some of the background information could be written down, but
developers often don't have time to keep notes on the current wisdom about their
projects, especially if there is no organized storage and retrieval mechanism
for their notes. The DSEE facility provides an easy method for developers to
record helpful information about facets of a project, which other users can
retrieve from the database. This feature can be particularly useful when a key
project member is absent or leaves the project; it can reduce the possibility of
lost time because that person is not available to answer a question.
Project Cycles
Software development projects go through cycles. A project cycle usually
includes a planning phase, an implementation phase, a testing phase, release of
the finished product to customers, and then maintenance (bug fixes) and
enhancements to the product. The DSEE facility can handle all phases of the
software development life cycle by doing the following:
o Relating design specifications to modules
o Maintaining source code and documentation in an orderly fashion
o Providing "to do" lists that help organize work necessary to fix a bug
or add a feature
o Recording the activities actually performed as part of a bug fix or
enhancement
o Building test versions and final versions of software
o Tracking the modules used in a release
o Allowing users to incorporate their own administrative procedures and
standard forms for the various development stages.
DSEE Libraries
==============
The DSEE facility stores related components of a software development effort in
a database known as a library. Depending on its needs and scope, a development
project may have one or two libraries, or several. DSEE commands act on the
objects associated with a library. DSEE objects belong to a "protected sub-
system," that is, their access specifications ensure that only DSEE
commands can operate on them. In addition, the DSEE facility provides its own
protection mechanism for objects. Using the DSEE protection mechanism, the
project administrator sets up hierarchical categories of users, ranging from
"administrators" to "non-users," thereby restricting the use of certain DSEE
commands.
Elements
The central DSEE object is the element, a file system object tailored
specifically for the DSEE facility. The DSEE facility can create elements out
of files containing source code, design specs or other types of text, and store
them in the library database that you specify.
It makes sense to store related elements (modules of the same program, or
chapters of the same document, for instance) in the same library. A large
development effort usually has a number of libraries, each containing a set of
related elements. For example, developers could have several source libraries
for different aspects of the system or application under development, a design
library for functional specs, and a documentation library for documentation and
notes about the project.
You cannot modify elements directly: only the DSEE facility can modify
elements, at your request. When you want to modify an element, you ask the DSEE
facility to reserve the element. In response, the DSEE facility provides
you with a copy of the element in an editable file. After you've edited
this file and performed any other appropriate steps, such as compiling,
testing, formatting or printing, you ask the DSEE facility to replace the
element in the library. Replacing the element incorporates your changes to the
file as a new version, or generation, of the element.
The DSEE facility keeps the most recent version of each element and all earlier
versions in the library. The versions are numbered in succession, and stored in
a compact form so as to conserve disk space. Instead of referring to element
versions with names such as filename.old, filename.oldest, and so on, you simply
ask the DSEE facility to retrieve a particular version of the element. The DSEE
facility also allows you to associate names with various element versions
for easier reference. (For example, if version 22 of alpha.pas is part of
Software Revision 5, you might assign the name sr5 to it.)
Together, the generations of an element constitute its line of descent. The fol-
lowing figure shows an element's main line of descent.
Figure 1-1. An Element's Main Line of Descent
ALPHA.PAS -------------
| 1|
| |
------------- |
| 2 | |
| |--------|
------------ |
| 3| |
| |--------|
------------ |
| 4| |
| |-----|
| |
| |
|----------|
The DSEE facility can reconstruct a particular version of an element at any time,
and provide you with a read-only copy for compilation, printing, or other uses.
At various points in the project cycle, developers may need to create alternate
lines of descent, or branches, off an element's main line of descent. Perhaps
more than one developer wishes to modify the element at the same time, or
perhaps a developer needs to fix an old version of an element without altering
its main line of descent. Once you've created a branch, you can create
successive versions of it, merge the branch back into the main line of descent,
or end its development at a particular version. Figure 1-2 shows a main line of
descent with one branch.
Figure 1-2. Main Line of Descent with Branch
ALPHA.PAS -------------
| 1|
| |
------------- |
| 2 | |
| |--------|
------------ |
| 3| |
| |--------| Branch ECO
------------ |
| 4| | ==========> -------------
| |-----| | 1|
| | | |
| | ------------- |
------------ | 2 | |
| |--------|
| |
| |
|-----------|
When you perform certain operations, such as reserve and replace operations,
the DSEE facility prompts you for descriptive comments. The DSEE facility
uses these comments to construct a history of modifications to the element.
Various DSEE commands let you retrieve all or part of an element's history.
Tasks
A DSEE task is a structure that describes the low-level steps needed to
accomplish a high-level activity, such as fixing a bug in a program. A task
consists of the following:
o A title, which describes the high-level activity
o A list of the required low-level steps (known as active items)
o A transcript, which lists all completed steps, and records all changes
made to the element or elements related to the task.
Tasks help you perform the following:
o Plan the steps needed for a high-level activity (These steps may
involve several elements in several libraries.)
o Record the completion of each step
o Verify the completion of each step.
Tasks are useful organizing tools because they list exactly what needs to be
done to accomplish a given high-level activity.
For example, the active items in a task might list procedures or routines that
need to be fixed, documentation that needs to be rewritten, or even questions
for co-workers. An active item usually pertains to a particular element, while
the high-level activity may involve several elements.
Tasks are useful communications tools because the DSEE facility stores them in
libraries, and allows you to assign them to various people in the project.
You can also use tasks as examples for future work, thereby creating a body of
information for the project or similar projects.
The DSEE facility lets each user specify a current task. More than one user can
specify the same current task. A task's transcript shows the activities of all
developers who have that task set as their current task. In this way, the DSEE
facility records exactly what was done to accomplish a particular task,
including all elements and libraries affected. When a developer needs
to work on a different task, he simply resets his current task context.
Initially, a task consists only of the task title and any preplanned active
items recorded for the task. As developers complete and "check off" active
items, the DSEE facility moves those items from the active list to the task
transcript. The task transcript also contains automatically transcripted
items which, as we've noted, the DSEE facility records during library
operations.
Tasklists
After you create a task, you usually add a reference to it to one or more
tasklists. The DSEE facility automatically creates a personal tasklist for each
user, and an active and master tasklist for each library. In addition, the DSEE
facility lets users create "arbitrary" (nonlibrary, nonpersonal) tasklists in
any directory.
A user's personal tasklist typically lists the tasks assigned to that user.
Library tasklists usually list tasks that relate to elements in that library.
Nonlibrary tasklists usually contain tasks that relate to the project in
general. A given task may appear on more than one tasklist, since it may
involve work for several users on elements in various libraries.
A reference to a task appears on tasklists when
o A user adds the task reference to one or more tasklists when he
creates the task
o the DSEE facility adds the task reference to one or more tasklists in
response to a monitor's activation (see "Monitors," below, for more
information)
o A user copies a task reference from one tasklist to another.
A task retains active status on its tasklists until you complete the task and
explicitly delete it from all tasklists.
Monitors
The DSEE monitor mechanism is its means of automatically notifying users about
changes to elements on which their work depends.
A monitor is a kind of flag on an element that tells DSEE to do the following:
o Watch for (monitor) any modifications to the element or the deletion
of the element
o Warn users when they reserve the element
o Notify users or projects when the element changes by adding task
reference(s) to tasklists, or execute predefined Shell commands or
Shell command files (e.g., a Shell command to send mail).
You usually set a monitor on an element because you want the DSEE facility
to notify your personal tasklist (or other tasklists) when that element
changes. This use of monitors involves specifying the recipient tasklists,
and creating a task template - a prototype for the actual task you want
the DSEE facility to create when the monitor is activated. The task template
should be specific enough to describe exactly what steps you want users
to take in response to the monitor's activation.
Before the DSEE facility completes a replace, delete, or rename operation on an
element, it checks to see if the element has a monitor. If the element has a
monitor, the DSEE facility "activates" the monitor in response to the operation.
The DSEE facility also instantiates a task (makes an editable copy) from the
monitor's task template, if there is one, and posts the name of the new task on
the recipient tasklists.
If there is no specific task that needs to be done when the monitor is
activated, you can define shell commands that you want the DSEE facility to
execute instead. For example, you can create a monitor that sends mail to
certain users notifying them of the change to the element.
The monitor mechanism frequently sends task descriptions between users and
libraries on different nodes. The communications protocols guarantee that the
appropriate task will go to the correct tasklist. The communication works
properly even if the target tasklist is on a node that is not currently in the
network. Once the node is back in the network, the task description appears on
the tasklist. You can create monitors that are activated only when you change
an element, or only when someone elese changes an element.
Forms
Besides providing tasks and monitors for project communication, the DSEE
facility allows you to create forms for routine administrative procedures, and
to use them as the basis for tasks.
For instance, a project administrator might create a form that describes the
general procedures for releasing a bug fix. A project member assigned to work
on the bug fix can then instantiate a task from this standard form, edit the
task so that it contains active items specifically tailored to the job at hand,
and add it to his personal tasklist.
You can also create a form from an existing task, if you think that task, or one
like it, will be required in the future.
By using forms and tasks, you can build a guide, or body of task-oriented
information, about a project. This helps remind seasoned project members of
procedures and methods, and helps familiarize newer members with the project
quickly.
The DSEE Configuration Management Facility
==========================================
Besides using the DSEE facility to maintain source modules, you can use it to
build, rebuild, and release entire systems or individual components. A
system's components may be elements in any DSEE libraries, regular files in
any DOMAIN directories, or a combination of both. A good example of a system
whose components span libraries is a compiler that consists of front-end modules
in one library and back-end modules in another library.
The system building, or configuration management facility is uniquely suited for
building large systems. By using varying configuration threads, users can build
any system configuration from selected element versions. By conditionally
processing the system model, users can build systems for a variety of target
machines. The configuration manager maintains the resulting binaries, their
full source descriptions, and other build-related information in a database.
The configuration manager saves time by consulting the database before
rebuilding a component. Specifically, the configuration manager can detect
whether the database already contains a satisfactory binary, or whether
the component needs to be rebuilt with appropriate include files or other
sources and tools.
The configuration manager conserves disk space by allowing users who are
building different systems, or different configurations of the same system,
to share common binaries. Obsolete binaries are discarded from the database
on a least recently used basis, based on user-specified (or default) parameters.
The DSEE configuration facility can boost productivity, especially on large
projects, by helping engineers
o Avoid the common error of building an inconsistent program (by failing
to rebuild components that require rebuilding)
o Build different versions of a system without having to copy versions
of source files and binaries
o Work simultaneously on development and maintenance environments using
common source libraries and common pools of binaries
o Switch between development and maintenance activities with ease
o Gain easy access to the source modules that make up a previously built
version of a system or component as well as to precise information
about that version.
Build Settings
Before you can build a system or system component, you establish your
"build settings," or working context for the build.
The most important of these is the system directory, a special directory that
the configuration manager creates, at your request, to store the objects
required for the build. You also need a system model and a configuration
thread, which together serve as your build script.
After creating the system directory, system model and configuration thread, you
set them as the current system, current system model, and current configuration
thread. The DSEE facility remembers the current model and configuration
thread, as well as the current library, task, and tasklist (if you've set them)
on a per-system, per-user basis. This means that when you change the current
system setting, the DSEE facility automatically changes the rest of your working
context to the settings last used with that system. Thus, you can switch
from one build environment to another simply by changing the current system
setting.
When you re-enter the DSEE environment after exiting or logging off, the DSEE
facility restores the settings that were in effect before you exited.
The System Model
The system model is a kind of blueprint of the system you intend to build. It
describes the system's components, their source libraries and/or source
directories, their dependencies (that is, the source files and tools on which
they depend), and the translation rules (usually compiler or binder commands)
required to build them.
A typical system model consists of a series of nested blocks, similar to the
block structure of a program. System model blocks describe buildable components
of the system.
The system model is source-oriented. You cannot refer directly to the derived
objects generated by translations because the configuration manager keeps
such objects in its database. Instead, you refer to objects using special
symbols, which the configuration manager then interprets and resolves to the
correct objects. Except for these special symbols, the configuration
manager does not interpret translation rules; it simply passes them to the
designated translators. As a result, you can invoke any translator (any
compiler, binder, or formatter) whether or not it is supplied with the DOMAIN
system.
The Configuration Thread
The configuration thread supplements the information in the system model by
telling the configuration manager which versions of the system's constituent
elements to use. It may also specify translation options (for example, compiler
and binder switches) for the build.
A configuration thread might request explicitly named or numbered element
versions (for example, alpha.pas[15], alpha.pas[sr5]), most recent versions,
versions currently reserved in the working directory, or versions used in a
prior build. The configuration thread contains no information about components
that are regular files, since there are no versions of such files, in the DSEE
sense.
The version specifications in a configuration thread are context dependent; you
can use different versions of the same element (for example, an include file) to
build different components.
Derived Objects and Bound Configuration Threads
When the configuration manager builds a component, the result is a number of
derived objects (such as binaries and listings) with corresponding bound
configuration threads (BCTs). A derived object's BCT states exactly which
element versions, regular files, and translation options the configuration
manager used to build the object. For source files that are not stored as
elements in DSEE libraries, the BCT records characteristic information
such as their date-time-modified (DTM) stamps.
Besides creating a BCT for every individual component of the system, the
configuration manager creates an aggregate BCT for the whole system. A system's
BCT is like a system model annotated with configuration thread information;
it records the exact element versions and files that make up the system.
BCTs are important references, both for debugging and for rebuilding. The
DSEE facility provides a command that reads whatever BCT you specify, and then
creates a shell process whose source reference environment, or version context,
is set to the element versions recorded in that BCT (that is, the element
versions used in the corresponding build). This makes it easy to recapture a
build environment in order to debug an old system or its components.
You can also reference the BCT of a previous build in the configuration thread,
directing the configuration manager to use the BCT to determine version and
translation option information for a build operation. To fix a bug in a
released system, for example, you may wish to rebuild the system using the same
versions of some of the original modules, plus newer versions you've
reserved in your working directory.
Binary Pools
The configuration manager stores derived objects and their BCTs in areas known
as binary pools. When you rebuild a component using different element
versions and/or translation options, the configuration manager creates new
versions of that component's derived objects and stores them, along with
their BCTs, in the pool.
Binary pools and configuration threads enable you to build and rebuild different
versions of a system or component as your development work progresses. Since
rebuilding a system often involves changing only a few components, this approach
allows you to reuse derived objects. It also lets users building different
systems, or different versions of the same system, share common components when
possible.
The configuration manager automatically creates one default binary pool in each
system directory. You can create additional pools, however, and organize all
of a system's physical pools into logical groupings (known as logical pools) in
the system model. By defining logical pools that consist of several
physical pools, you can distribute the storage space for a system across the
network. This approach also enables a system's developers and maintainers to
separate derived objects that they don't want to share.
The configuration manager also creates a reserved pool in your working directory
for derived objects built from reserved sources. The reserved pool serves
as your own, private (unshared) pool, so that you can build, test, and rebuild
your modules before releasing them to the regular pool for general use.
The configuration manager "promotes" a derived object from the reserved pool to
the regular pool as soon as you replace all of the reserved sources used in the
derived object.
The configuration manager keeps successive versions of derived objects in each
pool up to a default or user-defined limit. When a pool exceeds its limit,
the configuration manager purges it of obsolete derived objects on a
least recently used basis. However, the configuration manager does provide a
command for "exporting" derived objects from pools to other directories,
either by copying them, or by creating links.
How the Configuration Manager Builds
Once you've established your build settings, you use the build command to build
the current system or one of its components. If you issue the command without
options, the configuration manager simply reports the number of components
that need to be built, goes ahead and builds them, and reports the date and
time of each build operation. Various command options give you control over
the build session.
One of the configuration facility's most important features is its ability to
detect when there's no need to rebuild a component because a satisfactory
derived object for it already exists. This approach not only saves time, it
conserves resources, by allowing different users and systems to share and reuse
binaries.
Before building a component, the configuration manager determines its desired
BCT. The desired BCT describes the element versions, regular DOMAIN system
files, and translation options requested for that component. The configuration
manager creates the desired BCT by examining the current configuration thread
for each component in the current system model. The configuration thread tells
the configuration manager which versions of elements and which translation
options to use.
Next, the configuration manager searches the appropriate binary pools for a
previously built, derived object whose BCT matches the desired BCT. If such
an object exists, the configuration manager uses it by default, instead of
rebuilding unnecessarily. Normally, an object with the desired BCT exists
if you or another user built the exact object desired before. build command
options let you override this approach and force DSEE to rebuild.
If there is no derived object with the desired BCT, the configuration manager
builds the object to your specifications by default. Again, you can override
this behavior and "short-circuit" the rebuilding of certain components by
instructing the configuration manager to use existing derived objects that you
know are equivalent to the desired objects.
In any case, the configuration manager always satisfies your specifications,
either by finding appropriate derived objects, or by building them.
Tracking a System
Each built component's BCT notes the date and time it was built, the person who
built it, the node from which it was built, and the version and options used in
the build. You can access a BCT by the name of the built component and the time
of the build. Although over time the configuration manager deletes derived
objects, you can export a previously built component to any directory,
either by copying its derived objects or creating links to them.
When you've finished development work on a system or system component, you
frequently want to place its derived objects in a special area where they can be
accessed for general use and distribution and will not be deleted. The
DSEE facility provides facilities for creating and adding to release
areas, directories that hold built systems.
The DSEE facility keeps track of all the release areas that you create
for a system. It can show you the contents of a release area as well
as information about its creation and modification.