Sunday, October 23, 2011

RACGMAIN PROCESS SPAWNS 1000+ PROCESSES THAT ARE NOT GETTING CLEANED

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com
Problem Description: CC : RACGMAIN PROCESS SPAWNS 1000+ PROCESSES THAT ARE NOT GETTING CLEANED


racgmain process spawns 1000+ processes that are not getting cleaned causing memory / swap issues. this issue is happening only on one node (second) of a 2 node RAC cluster

Alert Log file shows
2009-07-18 13:10:24.914: [ CRSEVT][139127] CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/bin/racgwrap(check) timed out for ora.PDPP.PDPP2.inst! (timeout=600)
2009-07-18 13:10:24.914: [ CRSAPP][139127] CheckResource error for ora.PDPP.PDPP2.inst error code = -2
2009-07-18 13:28:42.448: [ CRSEVT][139188] CAAMonitorHandler :: 0:Could not join /u01/app/crs/product/10.2.0/bin/racgwrap(check)
category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child


- crs_stop <resource name > -f
- start the resource again using srvctl start , then monitor for the same behavior

- If the previous steps not release this status , we will need to check the following logs

* $ORA_CRS_HOME/log/<hostname>/crsd/crsd.log
* Please also check if you can find any core file under CRS home specially under $ORA_CRS_HOME/log/<hostname>/racg/racgmain
* $ORACLE_HOME/log/<hostname>/racg/*
* as root run ocrdump and upload the generated output
- Please also upload the result of the following command list :


$crs_stat
$crs_stat -p
$crs_stat -ls
$srvctl config nodeapps -n <nodename1> -a -g -s -l
$srvctl config nodeapps -n <nodename2> -a -g -s -l
$srvctl config database -d <db_name> -a
$srvctl config service -d <db_name> -a
$srvctl config asm -n <node name>
$ocrcheck
$crsctl query css votedisk
$crsctl query crs activeversion
$crsctl query crs softwareversion
$oifcfg getif
$oifcfg iflist

Solution:

As the CRS is 10.2.0.4 , so as we agree on phone that you will need to apply either 10.2.0.4 Bundle Patch #2 ( bug 7493592 ) or 10.2.0.4 CRS Bundle Patch #4 (bug 8436582) and the both will fix bug:6196746 ,as the bundle patches are cumulative 
Get Oracle Certifications for all Exams
Free Online Exams.com

No comments: