Sunday, October 16, 2011

CRS hang ORA-27300 ORA-27301 ORA-27302

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem:   CRS hang ORA-27300  ORA-27301 ORA-27302
CRS hang on one instance and database instance got terminated with below error 
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn5
and we can see lot of process for and it consume all the memory.
/u01/app/crs/product/11g/jdk/jre//lib/rt.jar:/u01/app/crs/product/11g/jlib/cvu.jar:/u01/app/crs/product/11g/jlib/srvm.jar:/u01/app/crs/product/11g/jlib/srvmhas.jar -DCV_DESTLOC=/tmp -DCV_HOME=/u01/app/crs/product/11g oracle.ops.verification.client.CluvfyDriver comp crs -display_status -n hostname01,hostname02


Solution:


do ps -ef|grep crsd.bin and then kill -9 the same process id at OS level, this process after a kill will be respawn automatically and crs hangs will get resolved.

Eg.
$ ps -ef|grep crsd.bin
root 6029 4284 0 Jan22 ? 03:00:47 /home/oracle/product/crs/bin/crsd.bin reboot
oracle 4454 27612 0 14:25 pts/1 00:00:00 grep crsd.bin

As root, 
# kill -9 6029
exit

$ ps -ef|grep crsd.bin

This will show new process id and crs_stat -t ; will work now.

The crsd.log is full of errors like shown below:

category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
..
category: 1234, operation: scls_process_spawn, loc: read_pipe, OS error: 12, other: EOF on read pipe

--
This messages can happen if crsd.bin is hung or OS level failure caused all problems in crsd

--
Please check OS log with help from OS sysadmin to see if there are any obvious issues.

--
To avoid hitting known bugs in CRS we recommend you apply latest CRS PSU#2 Bundle patch (#9207257) available.
<!--[if !supportLineBreakNewLine]-->
<!--[endif]-->
References:
<Note:810663.1> for information about CRS Patch Bundles for 11.1 including any known issues.




Get Oracle Certifications for all Exams
Free Online Exams.com

No comments: