Tuesday, July 5, 2011

Databases shows status is down “There has been a user-initiated shutdown”

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem:    databases shows status is down “There has been a user-initiated shutdown”

Symptoms:

After Upgrading EMGC to 10.2.0.5, some databases shows status is down with "There has been a user-initiated shutdown." while databases are really UP

Log files:

emagent_perl.trc:
-------------------------
maxRowCount.pl: Mon Jun 15 15:29:04 2009: ERROR: Max Count Value not set properly in file /u01/app/oracle/product/agent10g/sysman/config/esa/database.properties line no. 68

emdctl.trc
-----------------

2008-08-04 13:49:40,163 Thread-1 ERROR main: nmectla_agentctl: Error connecting to http://jctpdb01.dubaiworld.ae:3872/emd/main/. Returning status code 1
2

2008-12-02 07:02:48,343 Thread-1 ERROR http: 6: Unable to initialize ssl connection with server, aborting connection attempt

emoms.trc:
----------------
2009-06-14 16:46:41,588 [EMUI_16_46_41_/console/targets$ctxType=Hosts] ERROR target.TargetData getTargets.958 - ORA-20001: Error in get_metrics_summary: ORA-28112: failed to execute policy function
ORA-06512: at "SYSMAN.EMD_MNTR", line 9666
ORA-06512: at line 1

java.sql.SQLException: ORA-20001: Error in get_metrics_summary: ORA-28112: failed to execute policy function
ORA-06512: at "SYSMAN.EMD_MNTR", line 9666
ORA-06512: at line 1

2009-06-15 15:20:10,871 [EMUI_15_20_10_/console/database/instance/sitemap] ERROR svlt.PageHandler handleRequest.640 - java.lang.NullPointerException
java.lang.NullPointerException

2009-06-15 15:20:10,855 [EMUI_15_20_10_/console/database/instance/sitemap] ERROR perf.sitemap logp.251 - Failed to initialize object: ORA-28112: failed to execute policy function
ORA-06512: at "SYSMAN.EMD_DATABASE", line 1166
ORA-06512: at line 1

2009-06-15 15:00:17,967 [AJPRequestHandler-ApplicationServerThread-957] ERROR eml.OMSHandshake processFailure.843 - OMSHandshake failed.(AGENT URL = https://pebpbp02.dubaiworld.ae:3872/emd/main/)(ERROR = INTERNAL_ERROR)(CAUSE =java.sql.SQLException: ORA-28112: failed to execute policy function

2009-06-15 14:43:22,151 [EMUI_14_43_22_/console/database/instance/instanceActivity] ERROR perf.instAct logp.251 - oracle.sysman.emo.perf.metric.eng.MetricCollectorTimeoutException
emagent.trc
-------------------
2009-06-15 15:29:03,246 Thread-20 ERROR : (nmecmgr.c,3207):Memory 0x0 encountered, expect struct_id=11011

Solution:

Hitting a Bug 5890966 for DB versions 10.2.0.2 and 10.2.0.3, so we need to apply one-off patch for the DB and another one for the agent

-- To implement the solution, please execute the following steps::
1.Download the needed patch:
- There are patches for Bug 5890966 for DB versions 10.2.0.2 and 10.2.0.3.
- download the one-off Patch 5890966 or upgrade to 10.2.0.4 version for the
target database.
- download Patch 7012242 for the agent.
- Check if there are any errors while accessing the job activity tab in grid
control.
2.Stop the OMS
3.stop the agent
4 .Follow the instructions the in Readme and apply the patchs
5.Restart all EM components and monitor again the error.

References:

    
Get Oracle Certifications for all Exams
Free Online Exams.com

No comments: