Problem Description:
Database Crashes
Symptoms:
Database hangs
frequently, Database Alert Log file shows "ORA-00600: internal error code,
arguments: [17059], [0x4633429D8]"
Many trace files
generated causing shortage on storage.
Alert Log file
shows:
Mon Jan 18
08:02:21 2010
Errors in file
/orasys/ipay/bdump/ipay_j000_12065.trc:
ORA-12012: error on auto execute of
job 263
ORA-04030: out of process memory when trying to allocate 127512 bytes
(QERGH hash-agg,kllcqas:kllsltba)
ORA-06512: at
"WTR.TRANSFER_INTERNET_WEB_PAYMENTS", line 17
ORA-06512: at line 1
Mon Jan
18 08:03:32 2010
Errors in file
/orasys/ipay/bdump/ipay_j000_12065.trc:
ORA-12012: error on auto execute of
job 1
ORA-04030: out of process memory when trying to allocate bytes
(,)
ORA-06512: at "SYSMAN.EMD_MAINTENANCE", line 947
ORA-06512: at line
1
Mon Jan 18 08:03:33 2010
Errors in file
/orasys/ipay/bdump/ipay_j000_12065.trc:
ORA-00604: error occurred at
recursive SQL level 1
ORA-04030: out of process memory when trying to
allocate 72 bytes (kxs-heap-d,allocator state)
ORA-12012: error on auto
execute of job 1
ORA-04030: out of process memory when trying to allocate
bytes (,)
ORA-06512: at "SYSMAN.EMD_MAINTENANCE", line 947
ORA-06512: at
line 1
Trace files
shows :
Errors in file
/orasys/ipay/udump/ipay_ora_4868.trc:
ORA-07445: exception encountered: core
dump [kksParseChildCursor()+1180] [SIGSEGV] [Address not mapped to object]
[0xFFFFFFFF7FFF2E00] [] []
and
Errors in file
/orasys/ipay/bdump/ipay_psp0_6679.trc:
ORA-27300: OS system dependent
operation:fork failed with status: 12
ORA-27301: OS failure message: Not
enough space
ORA-27302: failure occurred at: skgpspawn3
System Error
Log shows:
Jan 19 05:58:29
Gateway2 last message repeated 22 times
Jan 19 06:09:59 Gateway2 ufs: [ID
845546 kern.notice] NOTICE: alloc: /orasys: file system full
Jan 19 06:14:45
Gateway2 last message repeated 2 times
Jan 19 06:15:59 Gateway2 genunix: [ID
470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 5420
(oracle)
Jan 19 06:16:00 Gateway2 last message repeated 1 time
Jan 19
06:16:01 Gateway2 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap
space to grow stack for pid 4363 (oracle)
Jan 19 06:16:01 Gateway2 genunix:
[ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid
5420 (oracle)
Jan 19 06:16:02 Gateway2 genunix: [ID 470503 kern.warning]
WARNING: Sorry, no swap space to grow stack for pid 4363 (oracle)
Jan 19
06:16:02 Gateway2 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap
space to grow stack for pid 5420 (oracle)
Jan 19 06:16:02 Gateway2 ufs: [ID
845546 kern.notice] NOTICE: alloc: /orasys: file system full
Jan 19 06:16:03
Gateway2 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow
stack for pid 4363 (oracle)
Jan 19 06:16:06 Gateway2 last message repeated 2
times
Jan 19 06:16:07 Gateway2 ufs: [ID 845546 kern.notice] NOTICE: alloc:
/orasys: file system full
Jan 19 06:17:41 Gateway2 last message repeated 1
time
Jan 19 06:24:42 Gateway2 ufs: [ID 845546 kern.notice] NOTICE: alloc:
/orasys: file system full
Jan 19 06:26:15 Gateway2 genunix: [ID 470503
kern.warning] WARNING: Sorry, no swap space to grow stack for pid 5486
(oracle)
Jan 19 06:26:18 Gateway2 last message repeated 3 times
Solution:
1. From the error
log, it is clear that there is no swap space, thats the reason for getting the
error. involve your system administrators to resolve this problem
2. set the stack
size to unlimited