Use LOGDUMP to work past a bad record

Intro

Every data shop experiences unplanned outages.  Golden Gate processing is almost always affected by an outage like this.  One of the worst cases is when a trailfile is being generated.  This can cause corrupted data in a trailfile.  Golden Gate is really good about providing us some key indicators about where we are looking in a trailfile.  Some of the information available in the LOGDUMP utility that will help us in a case like this are:

• OPEN a trailfile
• COUNTS of Transactions
• DETAIL of the image in the trailfile 
• POSition of the current RBA
• Location of the NEXT and LAST Transaction in the trailfile
• status of the current RBA (good or bad)

Information we'll need from our output of the failed replicat are:

• SEQNO of the failure
• RBA of the failure
  

Bad Data Ruins Everyone's Day

In this example we see that data has failed to insert into our target database in the first red box:

For some reason, we are getting data that doesn't fit in our target date column.  As a DBA, we know that the database nor the data format has changed on either side.  We may actually choose to do a defgen just to make sure the database has the most recent copy of the data. After we are certain of the database structure, we need to take a look at the actual data.  LOGDUMP can help us do this. 

Figure 1

First we need to start the LOGDUMP utility.  The way I do this is launch a command window for the Software location of the Golden Gate software.  This should be the same place as where ggsci runs. You can also do a file search on the executable logdump.exe to find it.   Once you find it, simply launch the logdump command line. I am going to open the trail file D:\GG\Software\dirdat\u2000050:

Figure 2

To open the trailfile, we just use the command open and the path to the trailfile on the replicat host. The command doesn't really do anything, except place you at RBA 0 in the top of the file.  Please also use the detail command so the we can look at any data that might be useful.  The detail command is just a switch that tells logdump whether or not to shop detail to the user. Now that we have the trailfile open, lets get some information about it.  

First, I like to use the COUNT command to see information about the records in a trailfile:

• The RBA where the count began
• The number of records in the file
• The total data bytes and average bytes per record
• Information about the operation types
• Information about the transactions

Now lets work our way through the trail file from the beginning.  To ensure we are at the start, use the command POS with the value 0, then the NEXT command to return back the current position.

Figure 3

Logdump 129 >pos 0
Reading forward from RBA 0
Logdump 130 >next

 We know that the file header is not our problem.  If the file header were the problem, we'd get the error for an "OGG-01028 Incompatible record".  Sometimes that can happen, too, when unplanned outages occur. But we know from Figure 1 that the REPLICAT failed on RBA 1887.  (SEQNO simply means the number 50 of the trailfile in the name of the file "D:\GG\Software\dirdat\u200005".
LOGDUMP tells us the next transaction is at RBA 1825.  Let's go to that RBA so that we can see what a good transaction looks like in LOGDUMP:

Figure 4

We don't get a whole lot of information here, but the command next does return the RBA we requested.  Now lets see what the bad RBA looks like:

When we go to the bad RBA and type next, we immediately see that there is a message about bad file compression.

Figure 5

Now that we've confirmed that there is a problem at this record, we need to find out if there are any more good records in this trail file. To do this, we make sure the DETAIL register is turned on and start using the NEXT command.  First type DETAIL ON to make sure you'll be able to see detail from the rows of the trail file (unless your data is encrypted). 

Figure 6

Keep typing the command next at the LOGDUMP prompt until you either get a record that does not indicate that it is bad OR nothing is returned.  If nothing is returned (as in the bottom of Figure 6) then type dump and how many lines to display.  In this example in Figure 6, it requests to show 122 lines.  However we get the output "EOF".  EOF of course, means "End of File".  This means our entire trailfile is bad, unfortunately.  

So now what?  

We have to keep processing, of course...the show must go on.  We know from Figure 1 that the trailfile we were working with, was D:\GG\Software\dirdat\u2000050 and that the remainder of the file is bad.  Let's pick up with the next file to process.  To sort of rollforward to the next file, we have to alter the REPLICAT to tell it to start in a new place.  We do this with the ALTER REPLICAT command.  Oracle provides this syntax:

ALTER REPLICAT group_name[threadID], {
   ADD REPLICAT option [, ...] |
   INTEGRATED | NONINTEGRATED, CHECKPOINTTABLE owner.table 
}
[, CPU number]
[, PRI number]
[, HOMETERM device_name]
[, PROCESSNAME process_name]

This documentation stinks because we actually aren't going to use any of these options.  Here's the actual statement used:

alter replicat rms_u2, extseqno 51, extrba 0

This statement, issued in the ggsci,ext (Golden Gate Command Line Interpreter) informs replicat rms_u2 to start with the trailfile D:\GG\Software\dirdat\u2000051 at the very beginning (extrba 0).

Then, we start the replicat back up.  

start rms_u2

If the replicat fails again, now you can use LOGDUMP to check out what is going on behind the covers.