[Back to Adager's IMAGE Database Page]
Date: Mon, 3 Nov 1997 09:32:27 -0800 From: Jon BaleRead the follow up post on HP3000-L from Adager's Ken Paul.
Subject: DDX Problem MIME-Version: 1.0 Since there was some discussion in October on this list about a problem with DDX, I thought I'd share what we at HP now know about this issue. HP has received reports of a problem being experienced by seven IMAGE/SQL users who have databases containing one or more data sets using the Dynamic Detail Dataset Expansion (DDX) feature. The reported problem relates to data stored in a data set which has undergone some dynamic expansion -- that is, it has "filled up" and its capacity has been automatically increased. The problem encountered by these users is that some data which is subsequently written to the data set is placed "beyond" the end-of-file (EOF), and once the database is closed and reopened, that data is inaccessible. If a program attempts to read that data or add more data entries in the same area, it gets an error -212, "Database is Corrupt." The Service Request documenting this is SR #5003-367607. Working with some of our independent software tools vendors, we have located a defect in the TurboIMAGE code which would cause the observed effect. In the case we found, an erroneous expansion may take place, which can then lead, possibly at a much later time, to data set entries being placed beyond the real file EOF. When the file is later closed, these entries are lost. The circumstances in which the erroneous expansion could take place are as follows: At the same time that an actual dynamic data set expansion is being performed within a call to DBPUT by one process, a second process (user) accessing the same data set for the first time causes execution of IMAGE's "open data set" module. If the timing is just right (actually just wrong), the result will be incorrect internal data set structural information being saved to disk. Specifically, the recorded internal CAPACITY has been incremented TWICE, while the EOF has been correctly incremented -- only once. This defect has been in the code since the introduction of DDX approximately three years ago. We have developed a fix for TurboIMAGE that removes this defect. Patches will be created for MPE/iX 5.0 and MPE/iX 5.5. The MPE/iX 5.0 patch is TurboIMAGE version C.06.23 and has Patch ID TIXKX11. The patch for MPE/iX 5.5 is TurboIMAGE version C.07.07, has Patch ID TIXKX13, and is built on the version of IMAGE released on MPE/iX 5.5 Express 3. Customers who use DDX and think they may be susceptible to the problem described (see below) can request either of these beta patches from the HP Response Center. HOW CAN YOU TELL IF YOU ALREADY HAVE THIS PROBLEM? If you do not use Dynamic Detail Dataset Expansion, you are not affected. If you are a heavy user of DDX with frequent data set expansions, you might be in the "high risk group." The defect we have identified can only affect detail data sets that are enabled for DDX on which an expansion actually takes place. If the erroneous combination of operations occurs, the result would be a data set whose logical EOF (current capacity) exceeds the corresponding file's physical EOF. You can check for this condition using the FORM SETS command of QUERY, which gives the Current Capacity (CC) and Blocking Factor (BF) of each data set. Calculate the MPE EOF of a DDX data set using the formula, (CC + (BF-1))/BF. Verify this calculated EOF with the EOF given by the MPE command :LISTF DDXSETnn,2 where "DDXSETnn" is the file name of that DDX data set. If the two EOFs do not match, there is a problem. If you have and use one of the IMAGE/SQL structure maintenance tools provided by HP or an independent vendor, that tool may also have the capability to locate instances of this problem. If you discover that one of your data sets is so afflicted, it may be possible to correct the problem using the same tool. (See your tool's documentation.) HOW CAN YOU AVOID EXPERIENCING THIS PROBLEM ON YOUR IMAGE DATABASE? Although we cannot be sure that the defect we repaired is the unique cause of this problem, I will assume that it is, for the purpose of addressing this question. The simplest answer, then, is: Install and use one of the TurboIMAGE beta patches mentioned above. This is the only option that I recommend. However, if you are unable to acquire and install the patch immediately, consider the following alternative: Until you install the patch, avoid opening a data set from one process while another is expanding it! This may be easier said than done, but here are some ideas, each one independent of the others. 1) Access the database exclusively. 2) Open all DDX-enabled data sets needed from each process before allowing any process to add data to any of these sets. 3) Preallocate and pre-expand DDX data sets by adding in exclusive mode many new dummy entries (more than your actual processing will) and then deleting them before your real processing begins. 4) Disable DDX on any nearly-full data set, using an HP or third party tool in which you have confidence. There are certainly variations on the ideas mentioned above, and perhaps better ones. My belief and hope is that only a few customers have actually been affected by this particular defect. However, our goal is for no user to lose data, and data loss IS a possibility in this instance. Since we now have identified the likely cause and have a solution, I thought it was worth sharing with everyone. Jon Bale, CSY
Read Jon Bale's follow-up post (January/1998).
[Back to Adager's IMAGE Database Page]