ORA-03113: end-of-file on communication channel

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description: Frequent Forms Processes sessions dis-connectivity


Symptoms:


on the application tier,on plsql module , database log file show the below:

Thu Feb 18 06:37:36 2010
Database adp encountered ORA-3113ORA-03113: end-of-file on communication channel
------------------------------------------------------------------------------
Thu Feb 18 06:37:42 2010
Database adp encountered ORA-3113ORA-03113: end-of-file on communication channel
------------------------------------------------------------------------------
Thu Feb 18 06:37:57 2010
Database adp encountered ORA-3113ORA-03113: end-of-file on communication channel

Some users while running sql statements from sqlplus that takes roughly >15s, they got disconnected.

Other users while working on PL/SQL application got page cannot be displayed, refresh the page sometime works and sometimes not.

Sqlnet.log shows

Fatal NI connect error 12518, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=)(PORT=1521))(CONNECT_DATA=(oracle_sid=)(CID=(PROGRAM=oracle@)(HOST=)(USER=oracle))))

VERSION INFORMATION:
TNS for Solaris: Version 10.2.0.3.0 - Production
TCP/IP NT Protocol Adapter for Solaris: Version 10.2.0.3.0 - Production
Time: 18-FEB-2010 06:14:37
Tracing not turned on.
Tns error struct:
ns main err code: 12564
TNS-12564: TNS:connection refused
ns secondary err code: 0
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0


Dbstatus.log
Thu Feb 18 06:14:19 2010
Database adp encountered ORA-3113ORA-03113: end-of-file on communication channel
------------------------------------------------------------------------------
Thu Feb 18 06:14:32 2010
Database adp encountered ORA-3113ORA-03113: end-of-file on communication channel
------------------------------------------------------------------------------
Thu Feb 18 06:14:36 2010
Database adp encountered ORA-3113ORA-03113: end-of-file on communication channel


Alert log

Thu Feb 18 08:23:35 2010
WARNING: inbound connection timed out (ORA-3136)
Thu Feb 18 08:25:35 2010
WARNING: inbound connection timed out (ORA-3136)


Solution:


1. During the time of error ORA-03113 please check is there is any trace files got generated under udump/bdump/cdump location


 If there exist a core dump file under location cdump then please extract the call stack functions from that core file
using the steps as mentioned in the below article :

Article-ID: Note 1812.1
Title: TECH: Getting a Stack Trace from a CORE file


2. Follow Article-ID: Note 120498.1 Title: How To Resolve Most ORA-3113 Errors


3.Check if your server administrator has any scripts which abort long running or CPU intensive processes. 
An ORA-3113 process can occur if someone kills your Oracle shadow process at O/S level (Eg: kill -9 on Unix).


     

Get Oracle Certifications for all Exams
Free Online Exams.com

Increase Heap Resource for Oracle Forms Processes more than default

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

In Case Oracle Forms Processes requires more heap resources, then the appropriate heap resource can be
modified in the Registry.

1. Modify the appropriate variable in the registry. This should not be done
unless:

- A backup has been made of the registry. Any error in modification of the
registry can render the system unusable.
- Solution 1 has been tested and has increased the number of processes
that will run successfully.


The following information applies to Windows NT, Windows 2000.
From the HKEY_LOCAL_MACHINE sub tree, go to the following key:

\System\CurrentControlSet\Control\Session Manager\SubSystems

Select the Windows value.From the Edit menu, choose String. Increase the
SharedSection parameter. SharedSection specifies the system and desktop
heaps using the followin g format:
SharedSection=xxxx,yyyy,zzzz

The default values are 1024,3072,512

Example :

Set the value to something like 1024,2048,2048

All the values are in kilobytes (KB).

xxxx = System-wide Heapsize. There is no need to modify this value and
it is ignored for the rest of this discussion.

yyyy = IO Desktop Heapsize. This is the heap for memory objects in the
IO Desktop.

zzzz = Non-IO Destop Heapsize. This is the heap for memory objects in
the Non-IO Desktop.

If you change these values, you must reboot the system.

Programs that get run as a service or which are spawned from a service fall
into the Non-IO Desktop, and therefore consume memory from that heap. All
other programs, such as those run from an icon or from a command line, or get
spawned from those, consume memory from the IO Desktop heap.

For server-style applications, such as a database, or Forms, a service has
the advantage that a user can logout and the program continues to run.

If you can't run any more processes, then you should increase the relevent
heap. Using the Forms examples in this bug, if you start Apache / Oracle
Process Manager from the command line, then it consumes memory from the IO Desktop heap.
Then for each user, there is a Forms runtime process spawned, which also
consumes memory form the IO Desktop heap.In the case of this bug, the memory
ran out after 483 users. The solution would be to increase the relevant heap.
As stated earlier in this bug, when it was changed to 5112 KB (from the default
of 3072 KB), more than 700 users were able to run concurrently. Note that
changing the Non-IO Desktop heap (the 512 KB) would achieve nothing in this
case.

The converse is also true. If the Apache / Oracle Process Manager process were
started as a service, then the third value, Non-IO Desktop heap, would have to
have been increased.

The values are recommended to be a multiple of 512, but this isn't necessary.
There is a hard limit of 48 Mb, which is the total heap size for Windows. But
it seems unlikely that you will need to go this high.

If you increase the Non -IO Desktop heap and your server is dedicated to Forms
or some other server process , then you can safely decrease the IO Desktop heap
since that is just wasted. This isn't necessary, though.

In summary, if you are hitting this problem where your server can not create
any more processes, you should increase the heap size of the either the IO
Desktop or the Non-IO Desktop. The correct one depends on how you run your
application, whether from a service, or from an icon or command line.

You have to restart the all services from Oracle AS, after saving the changes.

There are several parameters that need to be set inorder to tune the application.

So please folloe the below Action Plan also further improve the performance the application.

1. Check Are the users logging off without closing the sessions properly.

2. Take the process id and check which IP address is using the process and then
talk to the person with the IP address what they were running.
memory leak

3. Set forms_timeout and heartbeat and make sure the sessions are killed with an x
mins of inactivity

*****

1) Please note that for the session time out there is three parameters, which need to handle. Those are

FORMS_TIMEOUT
heartBeat
session_timeout

1) Please specify the forms_timeout=5 parameter in you env file and specify the heartBeat=10
parameter in your basejini.htm (incase if you are using JRE specify the parameter in basejpi.htm) file.

2) Please note that you need to set the Session_Timeout in web.xml file.

set in the web.xml file in OracleAS --> j2EE --> OC4J_BI_Forms --> applications -->
forms90app or formsapp -->forms90web or formsweb --> Web-inf

The purpose of the parameters to set forms_timeout and heartbeat parameters, is if the users keep
the session ideal for some x time, then to kill the sessions automatically in that x time.




    

Get Oracle Certifications for all Exams
Free Online Exams.com

java.io.IOException: FRM-93000: No HTTP headers received from runform

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description: Forms session disconnects continuously



Forms Error FRM-92102 occurs after 50 to 55 users connected to application.
This prevents further users logging in and prevents use of current users who a
reconnected.


Symptoms:


Application Log file shows:


Forms session <93> exception stack trace:
java.io.IOException: FRM-93000: ÎØà ÏÇÎáí ÛíÑ ãÊæÞÚ.
ÇáÊÝÇÕíá : No HTTP headers received from runform
at oracle.forms.servlet.ListenerServlet.forwardResponseFromRunform(Unknown Source)
at oracle.forms.servlet.ListenerServlet.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:824)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:330)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:830)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.AJPRequestHandler.run(AJPRequestHandler.java:224)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.AJPRequestHandler.run(AJPRequestHandler.java:133)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:186)
at java.lang.Thread.run(Thread.java:534)
10/02/01 14:27:17 formsweb: Êã ÅíÞÇÝ ÌáÓÉ Úãá Forms <102>: ÛíÑ ÞÇÏÑ Úáì ÇáÇÊÕÇá ÈãÚÇáÌÉ æÞÊ ÇáÊÔÛíá.
10/02/01 14:27:17 formsweb: Forms session <102> exception stack trace:
java.io.IOException: FRM-93000: ÎØà ÏÇÎáí ÛíÑ ãÊæÞÚ.
ÇáÊÝÇÕíá : No HTTP headers received from runform
at oracle.forms.servlet.ListenerServlet.forwardResponseFromRunform(Unknown Source)
at oracle.forms.servlet.ListenerServlet.doPost(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:824)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:330)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:830)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.AJPRequestHandler.run(AJPRequestHandler.java:224)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].server.http.AJPRequestHandler.run(AJPRequestHandler.java:133)
at com.evermind[Oracle Application Server Containers for J2EE 10g (10.1.2.0.2)].util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:186)
at java.lang.Thread.run(Thread.java:534)


Explanation:


This problem can occur due to the scalability issue that the system is not able
to share the resources among the concurrent users and Windows has exceeded the
capacity of the operating systems Non-IO Desktop heap resource.




Windows has limitation on the heap alocation and usage..
It is the expected behaviour when too many users are trying to access the
application.


Solution:


Change the logon-property from the responsible service to
"[x] Allow service to interact with desktop"
(using all the defaults of the registry!).


This forces the Apache Service (Forms 6i) or the Oracle Process Manager Service 
(Forms 9i and above) to use the IO Desktop heap which is larger than
the Non-IO Desktop heap by default. The term "Desktop", here, is not to be
confused with the normal Windows desktop, which holds your icons and your
background, etc. In this context, it is Microsoft terminology for an area of
memory.

If more heap resource is needed, then the appropriate heap resource can be modified in the Registry

1. Modify the appropriate variable in the registry. This should not be done
unless:

- A backup has been made of the registry. Any error in modification of the
registry can render the system unusable.
- Solution 1 has been tested and has increased the number of processes
that will run successfully.


The following information applies to Windows NT, Windows 2000.
From the HKEY_LOCAL_MACHINE sub tree, go to the following key:

\System\CurrentControlSet\Control\Session Manager\SubSystems

Select the Windows value.From the Edit menu, choose String. Increase the
SharedSection parameter. SharedSection specifies the system and desktop
heaps using the followin g format:
SharedSection=xxxx,yyyy,zzzz

The default values are 1024,3072,512

Example :

Set the value to something like 1024,2048,2048

All the values are in kilobytes (KB).

xxxx = System-wide Heapsize. There is no need to modify this value and
it is ignored for the rest of this discussion.

yyyy = IO Desktop Heapsize. This is the heap for memory objects in the
IO Desktop.

zzzz = Non-IO Destop Heapsize. This is the heap for memory objects in
the Non-IO Desktop.

If you change these values, you must reboot the system.

Programs that get run as a service or which are spawned from a service fall
into the Non-IO Desktop, and therefore consume memory from that heap. All
other programs, such as those run from an icon or from a command line, or get
spawned from those, consume memory from the IO Desktop heap.

For server-style applications, such as a database, or Forms, a service has
the advantage that a user can logout and the program continues to run.

If you can't run any more processes, then you should increase the relevent
heap. Using the Forms examples in this bug, if you start Apache / Oracle
Process Manager from the command line, then it consumes memory from the IO Desktop heap.
Then for each user, there is a Forms runtime process spawned, which also
consumes memory form the IO Desktop heap.In the case of this bug, the memory
ran out after 483 users. The solution would be to increase the relevant heap.
As stated earlier in this bug, when it was changed to 5112 KB (from the default
of 3072 KB), more than 700 users were able to run concurrently. Note that
changing the Non-IO Desktop heap (the 512 KB) would achieve nothing in this
case.

The converse is also true. If the Apache / Oracle Process Manager process were
started as a service, then the third value, Non-IO Desktop heap, would have to
have been increased.

The values are recommended to be a multiple of 512, but this isn't necessary.
There is a hard limit of 48 Mb, which is the total heap size for Windows. But
it seems unlikely that you will need to go this high.

If you increase the Non -IO Desktop heap and your server is dedicated to Forms
or some other server process , then you can safely decrease the IO Desktop heap
since that is just wasted. This isn't necessary, though.

In summary, if you are hitting this problem where your server can not create
any more processes, you should increase the heap size of the either the IO
Desktop or the Non-IO Desktop. The correct one depends on how you run your
application, whether from a service, or from an icon or command line.

You have to restart the all services from Oracle AS, after saving the changes.

There are several parameters that need to be set inorder to tune the application.

So please folloe the below Action Plan also further improve the performance the application.

1. Check Are the users logging off without closing the sessions properly.

2. Take the process id and check which IP address is using the process and then
talk to the person with the IP address what they were running.
memory leak

3. Set forms_timeout and heartbeat and make sure the sessions are killed with an x mins of inactivity

Also what can be done:

1) Please note that for the session time out there is three parameters, which need to handle. Those are

FORMS_TIMEOUT
heartBeat
session_timeout

1) Please specify the forms_timeout=5 parameter in you env file and specify the heartBeat=10
parameter in your basejini.htm (incase if you are using JRE specify the parameter in basejpi.htm) file.

2) Please note that you need to set the Session_Timeout in web.xml file.

set in the web.xml file in OracleAS --> j2EE --> OC4J_BI_Forms --> applications -->
forms90app or formsapp -->forms90web or formsweb --> Web-inf

The purpose of the parameters to set forms_timeout and heartbeat parameters, is if the users keep
the session ideal for some x time, then to kill the sessions automatically in that x time.


References:


Note 549735.1 Description List For Parameters Affect Timeout In Webforms




    

Get Oracle Certifications for all Exams
Free Online Exams.com

ORA-20100: Error: FND_FILE failure

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description: OAM Applications Dashboard Collection concurrent request is failing


SQL> exec fnd_file.put(fnd_file.log,'TESTING');
BEGIN fnd_file.put(fnd_file.log,'TESTING'); END;

*
ERROR at line 1:
ORA-20100: Error: FND_FILE failure. Unable to create file, o0018180.tmp in the
directory, /applogs/admin/conc/appltmp.
You will find more information in the request log.
ORA-06512: at "APPS.FND_FILE", line 417
ORA-06512: at "APPS.FND_FILE", line 456
ORA-06512: at line 1




After this see if the issue has been fixed.


Work Around:


Modifying APPLPTMP to /usr/tmp and make /usr/tmp first entry on the utl_file_dir did solve the issue.

1 - Be sure to have a valid Backup
2 - Execute autoconfig.
3 - Bounce the database
4 - Stop all Application Services
5 - Run autoconfig at the Application Tier
6 - Start all Services
7 - See if the issue has been resolved


Solution:


Review and Apply: Note 749491.1 - ORA-20100: Error: FND_FILE Failure, Unable To Create File , be sure it willbe done at both nodes.


and


test the procedure as described in Note 74268.1 - Using utl_file, how file permissions are determined




    

Get Oracle Certifications for all Exams
Free Online Exams.com

OPatch failed with error code 73

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Explanation:


Apply Database patches using OPatch failed with " failed with OPatch failed with error code 73 "


Solution:


Missing operating System Patch


applying SUN patch 123908-01 (or later) and relink oracle binaries




Reference:


Note 353150.1 - OPatch Failing Validation Phase When Archiving Really Did Not Fail


     

Get Oracle Certifications for all Exams
Free Online Exams.com

ORA-30926: unable to get a stable set of rows in the source

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description: 


Alert Log files show the the following:


ORA-12012: error on auto execute of job 101491
ORA-30926: unable to get a stable set of rows in the source tables
ORA-06512: at "Procedure", line 15


Trace File shows:


/oradg2/orasys/admin/adp2/udump/adp_ora_19344.trc

Comments
--------
*** ACTION NAME:() 2010-01-05 08:15:38.214
*** MODULE NAME:(TOAD 8.5.0.50) 2010-01-05 08:15:38.214
*** SERVICE NAME:(SYS$USERS) 2010-01-05 08:15:38.214
*** SESSION ID:(1509.9559) 2010-01-05 08:15:38.214
*** 2010-01-05 08:15:38.214
ksedmp: internal or fatal error
ORA-30926: unable to get a stable set of rows in the source tables
Current SQL statement for this session:
MERGE INTO HR_IMAGE_IDCARDS D USING (SELECT * FROM ADP_TO_HR_V ) S ON (D.EMP_ID = S.EMP_ID) WHEN MATCHED THEN UPDATE SET D.EMP_PHOTO = S.EMP_PHOTO , D.PERS_ID = S.PERS_ID , D.OBSOLFLAG = S.OBSOLFLAG WHEN NOT MATCHED THEN INSERT (D.EMP_ID, D.EMP_PHOTO , D.PERS_ID, D.OBSOLFLAG) VALUES (S.EMP_ID, S.EMP_PHOTO , S.PERS_ID, S.OBSOLFLAG)
----- PL/SQL Call Stack -----
object line object
handle number name
3cd8b43a0 3 procedure HR.UPDATE_EMP_CARD_PHOTO
3ccaff628 1 anonymous block


Explanation:


You can not update LOB objects remotely throw Database Links
ORA-22992: cannot use LOB locators selected from remote tables


Solution:


Change Business Logic




   

Get Oracle Certifications for all Exams
Free Online Exams.com

TNS-12545-12560-00515, Solaris Error : 126: Cannot assign requested address

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem:


Starting Database Listener  for and ACTIVE/PASSIVE cluster setup throw the following errors:


TNS-12545: Connect failed because target host or object does not exist

TNS-12560: TNS:protocol adapter error

TNS-00515: Connect failed because target host or object does not exist

Solaris Error: 126: Cannot assign requested address




Explanation:


We have active-passive 10g database under Sun Solaris 10 operating system cluster.

We tried to switch the database to work on the passive node.

when we tried to start the passive node, we got the following errors:
..................................................................
TNSLSNR for Solaris: Version 10.2.0.3.0 - Production

System parameter file is /oracle/aderp/10.2.0/network/admin/aderp_erp-lh/listener.ora

Log messages written to /oracle/aderp/10.2.0/network/admin/aderp.log

Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=erp-lh.auh.police)(PORT=1521)))

TNS-12545: Connect failed because target host or object does not exist

TNS-12560: TNS:protocol adapter error

TNS-00515: Connect failed because target host or object does not exist

Solaris Error: 126: Cannot assign requested address


Solution:


Active/Passive databases and listeners under operating system cluster setup should NOT be stopped or started manually, it should be throw the cluster.


    

Get Oracle Certifications for all Exams
Free Online Exams.com