ORA-00600: internal error code, arguments: [kcbgcur_9], [8388697], [27], [402653184], [134217728], On ALTER DATABASE ADD SUPPLEMENTAL LOG

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem:    ORA-00600: internal error code, arguments: [kcbgcur_9], [8388697], [27], [402653184], [134217728], On ALTER DATABASE ADD SUPPLEMENTAL LOG

Symptoms:

While setting up( downstream) streams, I am not able to enable the supplementary logging in Primary database. 
The command runs successfully. (ALTER TABLE TEST ADD SUPPLEMENTAL LOG DATA(PRIMARY KEY) COLUMNS). 
But when I try to create the capture process at downstream database, we are hitting the ORA-00600 bug in the primary database

Log files:

From alert_SID1.log
--------------------
Mon Mar 1 09:36:52 2010
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA
Mon Mar 1 09:36:52 2010
Errors in file /u01/app/oracle/product/10.2.0/admin/SID/udump/SID1_ora_29559.trc:
ORA-00600: internal error code, arguments: [kcbgcur_9], [8388697], [27], [402653184], [134217728], [], [], []

From SID1_ora_29559.trc
------------------------
*** SERVICE NAME:(SID.dubaiworld.ae) 2010-03-01 09:36:52.357
*** SESSION ID:(1565.23992) 2010-03-01 09:36:52.357
*** 2010-03-01 09:36:52.357
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [kcbgcur_9], [8388697], [27], [402653184], [134217728], [], [], []
Current SQL statement for this session:
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA

Function List (to Full stack) (to Summary stack)

ksedst <- ksedmp <- ksfdmp <- kgerinv <- kgeasnmierr
<- $cold_kcbgcur <- ktugusc <- ktucwt3 <- dbs_about_to_modify <- slog_data
<- dbsucf <- adbdrv <- opiexe <- opiosq0 <- opipls
<- opiodr <- rpidrus <- skgmstack <- rpidru <- rpiswu2
<- rpidrv <- psddr0 <- psdnal <- pevm_EXIM <- pfrinstr_EXIM
<- pfrrun_no_tool <- pfrrun <- plsql_run <- pricar <- pricbr
<- prient2 <- prient <- kkxrpc <- kporpc <- opiodr
<- ttcpip <- opitsk <- opiino <- opiodr <- opidrv
<- sou2o <- opimai_real <- main <- main_opd_entry

Current cursor: 5, pgadep: 1

Cursor#5(9fffffffbf3c1948) state=BOUND curiob=9fffffffbf3342c0
curflg=cd fl2=0 par=0000000000000000 ses=c000000238f039f8
sqltxt(c0000001f42a9940)=ALTER DATABASE ADD SUPPLEMENTAL LOG DATA

SO: c0000002389a4950, type: 2, owner: 0000000000000000, flag: INIT/-/-/0x00
(process) Oracle pid=79, calls cur/top: c0000001ffad9550/c000000223210048, flag: (0) -
int error: 0, call error: 0, sess error: 0, txn error 0
(post info) last post received: 0 0 55
last post received-location: kjata: wake up enqueue owner
last process to post me: c000000237987ae8 1 6
last post sent: 0 0 103
last post sent-location: kjmpost: post lmd
last process posted by me: c000000237987ae8 1 6
(latch info) wait_event=0 bits=2
holding (efd=18) c000000230f1cf40 Child cache buffers chains level=1 child#=105 
Location from where latch is held: kcbgcur: kslbegin: 
Context saved from call: 8388697
state=busy(exclusive) (val=0x200000000000004f) holder orapid = 79
Process Group: DEFAULT, pseudo proc: c000000238b083d0
O/S info: user: oradpp, term: UNKNOWN, ospid: 29559
OSD pid info: Unix process pid: 29559, image: oracleSID1@SIDdb03


SO: c000000238f039f8, type: 4, owner: c0000002389a4950, flag: INIT/-/-/0x00
(session) sid: 1565 trans: c00000022db26ce0, creator: c0000002389a4950, flag: (c8000041) USR/- BSY/-/-/-/-/-
DID: 0001-004F-00002D75, short-term DID: 0001-004F-00002D74
txn branch: 0000000000000000
oct: 35, prv: 0, sql: c0000001f42a9940, psql: c0000002185d9be8, user: 0/SYS
O/S info: user: GTFS.Seena, term: , ospid: 14588, machine: pdwpdb02
program: oracle@pdwpdb02 (TNS V1-V3)
application name: oracle@pdwpdb02 (TNS V1-V3), hash value=0
last wait for 'latch: ges resource hash list' blocking sess=0x0000000000000000 seq=214 wait_time=10 seconds since wait started=3
address=c00000023565e350, number=41, tries=0
Dumping Session Wait History
for 'latch: ges resource hash list' count=1 wait_time=10
address=c00000023565e350, number=41, tries=0
for 'DFS lock handle' count=1 wait_time=47588
type|mode=43490005, id1=39, id2=2
for 'DFS lock handle' count=1 wait_time=158
type|mode=43490005, id1=39, id2=3
for 'latch: ges resource hash list' count=1 wait_time=17
address=c00000023578adc0, number=41, tries=0
for 'DFS lock handle' count=1 wait_time=189
type|mode=43490005, id1=39, id2=1
for 'DFS lock handle' count=1 wait_time=333
type|mode=43490005, id1=39, id2=5
for 'control file parallel write' count=1 wait_time=4024
files=2, block#=1, requests=2


FileName
--------
alert_SID1.log & SID1_ora_29559.trc

Solution:

the most possible cause is IO saturation, Production systems can be highly active, and IO paths can get saturated at the OS level. These waits primarily occur on I/O to a single copy of the controlfile. Retrying the operation usually goes through. If the waits are significant, and bouncing the database is not possible it may be required to find out whether the waits are on particular copy of the controlfile and if so whether its I/O path is saturated. 

- Start the database instance in mount mode.

- Issue the Alter Database Add Supplemental Log Data command.

- Switch database instance from mount to open mode.

- The Alter Database Add Supplemental Log Data command should now work fine for any adding additional logs.

References:

Note.391426.1 'ORA-00600 [KCBGCUR_9] On ALTER DATABASE ADD SUPPLEMENTAL LOG'

Get Oracle Certifications for all Exams
Free Online Exams.com