[Back to Adager's IMAGE Database Page]
Date: Mon, 3 Nov 1997 09:32:27 -0800
From: Jon Bale
Subject: DDX Problem
MIME-Version: 1.0
Since there was some discussion in October on this list about a problem
with DDX, I thought I'd share what we at HP now know about this issue.
HP has received reports of a problem being experienced by seven
IMAGE/SQL users who have databases containing one or more data sets
using the Dynamic Detail Dataset Expansion (DDX) feature. The reported
problem relates to data stored in a data set which has undergone some
dynamic expansion -- that is, it has "filled up" and its capacity has
been automatically increased. The problem encountered by these users is
that some data which is subsequently written to the data set is placed
"beyond" the end-of-file (EOF), and once the database is closed and
reopened, that data is inaccessible. If a program attempts to read that
data or add more data entries in the same area, it gets an error -212,
"Database is Corrupt." The Service Request documenting this is SR
#5003-367607.
Working with some of our independent software tools vendors, we have
located a defect in the TurboIMAGE code which would cause the observed
effect. In the case we found, an erroneous expansion may take place,
which can then lead, possibly at a much later time, to data set entries
being placed beyond the real file EOF. When the file is later closed,
these entries are lost. The circumstances in which the erroneous
expansion could take place are as follows: At the same time that an
actual dynamic data set expansion is being performed within a call to
DBPUT by one process, a second process (user) accessing the same data
set for the first time causes execution of IMAGE's "open data set"
module. If the timing is just right (actually just wrong), the result
will be incorrect internal data set structural information being saved
to disk. Specifically, the recorded internal CAPACITY has been
incremented TWICE, while the EOF has been correctly incremented -- only
once. This defect has been in the code since the introduction of DDX
approximately three years ago.
We have developed a fix for TurboIMAGE that removes this defect.
Patches will be created for MPE/iX 5.0 and MPE/iX 5.5. The MPE/iX 5.0
patch is TurboIMAGE version C.06.23 and has Patch ID TIXKX11. The patch
for MPE/iX 5.5 is TurboIMAGE version C.07.07, has Patch ID TIXKX13, and
is built on the version of IMAGE released on MPE/iX 5.5 Express 3.
Customers who use DDX and think they may be susceptible to the problem
described (see below) can request either of these beta patches from the
HP Response Center.
HOW CAN YOU TELL IF YOU ALREADY HAVE THIS PROBLEM?
If you do not use Dynamic Detail Dataset Expansion, you are not
affected. If you are a heavy user of DDX with frequent data set
expansions, you might be in the "high risk group." The defect we have
identified can only affect detail data sets that are enabled for DDX on
which an expansion actually takes place. If the erroneous combination
of operations occurs, the result would be a data set whose logical EOF
(current capacity) exceeds the corresponding file's physical EOF.
You can check for this condition using the FORM SETS command of QUERY,
which gives the Current Capacity (CC) and Blocking Factor (BF) of each
data set. Calculate the MPE EOF of a DDX data set using the formula,
(CC + (BF-1))/BF. Verify this calculated EOF with the EOF given by the
MPE command :LISTF DDXSETnn,2 where "DDXSETnn" is the file name of that
DDX data set. If the two EOFs do not match, there is a problem.
If you have and use one of the IMAGE/SQL structure maintenance tools
provided by HP or an independent vendor, that tool may also have the
capability to locate instances of this problem. If you discover that
one of your data sets is so afflicted, it may be possible to correct the
problem using the same tool. (See your tool's documentation.)
HOW CAN YOU AVOID EXPERIENCING THIS PROBLEM ON YOUR IMAGE DATABASE?
Although we cannot be sure that the defect we repaired is the unique
cause of this problem, I will assume that it is, for the purpose of
addressing this question. The simplest answer, then, is: Install and
use one of the TurboIMAGE beta patches mentioned above. This is the
only option that I recommend.
However, if you are unable to acquire and install the patch immediately,
consider the following alternative: Until you install the patch, avoid
opening a data set from one process while another is expanding it! This
may be easier said than done, but here are some ideas, each one
independent of the others. 1) Access the database exclusively. 2) Open
all DDX-enabled data sets needed from each process before allowing any
process to add data to any of these sets. 3) Preallocate and pre-expand
DDX data sets by adding in exclusive mode many new dummy entries (more
than your actual processing will) and then deleting them before your
real processing begins. 4) Disable DDX on any nearly-full data set,
using an HP or third party tool in which you have confidence.
There are certainly variations on the ideas mentioned above, and perhaps
better ones.
My belief and hope is that only a few customers have actually been
affected by this particular defect. However, our goal is for no user to
lose data, and data loss IS a possibility in this instance. Since we
now have identified the likely cause and have a solution, I thought it
was worth sharing with everyone.
Jon Bale, CSY
Read the follow up post on HP3000-L from Adager's Ken Paul.
Read Jon Bale's follow-up post (January/1998).
[Back to Adager's IMAGE Database Page]