Step by Step Oracle 11gR2 RAC Installation on Linux

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Steps by step RAC Installation

I want to divide total installation work into four parts.

A. Preinstallation Configuration.

B. Installing Oracle Grid Infrastructure.

C. Installing the Oracle Database Software.

D. Creating Oracle Database.

Before I go through each individual steps let's discuss about some RAC concepts so that you can easily understand the documents.

Concepts & New things related to Oracle 11gR2

Starting with Oracle Database 11g Release 2, Oracle Clusterware and Oracle ASM are installed into a single home directory, which is called the Grid home. Oracle grid infrastructure refers to the installation of the combined products. Oracle Clusterware and Oracle ASM are still individual products, and are referred to by those names.

Oracle Clusterware enables servers, referred to as hosts or nodes, to operate together as if they are one server, commonly referred to as a cluster. Although the servers are standalone servers, each server has additional processes that communicate with other servers. In this way the separate servers appear as if they are one server to applications and end users. Oracle Clusterware provides the infrastructure necessary to run Oracle RAC.

An Oracle Database database has a one-to-one relationship between data files and the instance. An Oracle RAC database, however, has a one-to-many relationship between data files and instances. In an Oracle RAC database, multiple instances access a single set of database files. That's why datafiles must be kept in a shared storage so that every instances can access. Instances indicate its own memory structures and background processes.

Oracle Automatic Storage Management (ASM) is an integrated, high-performance volume manager and file system. With Oracle Database 11g Release 2, Oracle ASM adds support for storing the Oracle Clusterware OCR and voting disk files. OCR and voting disk files are two component of Oracle clusterware.

To install and configuration Oracle RAC there are several tools that RAC provides. Let's know the name of those tools.

i)                    Oracle Universal Installer (OUI) – GUI tool which installs the Oracle grid infrastructure software (which consists of Oracle Clusterware and Oracle ASM)

ii)                  Cluster Verification Utility (CVU) - a command-line tool that you can use to verify your environment. It is used for both preinstallation as well as postinstallation checks of your cluster environment.

iii)                Oracle Enterprise Manager(OEM) – GUI tool for managing single- instance and Oracle RAC environments.

iv)                SQL*Plus – Command line interface that enables you to perform database management operations for a database.

v)                  Server Control (SRVCTL) - A command-line interface that you can use to manage the resources defined in the Oracle Cluster Registry (OCR). 

vi)                Cluster Ready Services Control (CRSCTL)—A command-line tool that you can use to manage Oracle Clusterware daemons. These daemons include Cluster Synchronization Services (CSS), Cluster-Ready Services (CRS), and Event Manager (EVM).

vii)              Database Configuration Assistant (DBCA)—A GUI utility that is used to create and configure Oracle Databases.

viii)            Oracle Automatic Storage Management Configuration Assistant (ASMCA)—ASMCA is a utility that supports installing and configuring Oracle ASM instances, disk groups, volumes. It has both a GUI and a non-GUI interface.

ix)                Oracle Automatic Storage Management Command Line utility (ASMCMD)—A command-line utility that you can use to manage Oracle ASM instances, Oracle ASM disk groups, file access control for disk groups, files and directories within Oracle ASM disk groups, templates for disk groups, and Oracle ASM volumes.

x)                  Listener Control (LSNRCTL)—A utility is a command-line interface that you use to administer listeners.

If you use ASMCMD, srvctl, sqlplus, or lnsrctl to manage Oracle ASM or its listener, then use the binaries located in the Grid home, not the binaries located in the Oracle Database home, and set ORACLE_HOME environment variable to the location of the Grid home.

If you use srvctl, sqlplus, or lnsrctl to manage a database instance or its listener, then use the binaries located in the Oracle home where the database instance or listener is running, and set the ORACLE_HOME environment variable to the location of that Oracle home

OUI no longer supports installation of Oracle Clusterware files on block or raw devices.

A. Preinstallation Requirements.

1. Hardware Requirements.
2. Network Hardware Requirements.
3. IP Address Requirements.
4. OS and software Requirements.
5. Preparing the server to install Grid Infrastructure.

1. Hardware Requirements: 

The minimum required RAM is 1.5 GB for grid infrastructure for a cluster, or 2.5 GB for grid infrastructure for a cluster and Oracle RAC. To check your RAM issue,

# grep MemTotal /proc/meminfo

The minimum required swap space is 1.5 GB. Oracle recommends that you set swap space to

- 1.5 times the amount of RAM for systems with 2 GB of RAM or less.

- Systems with 2 GB to 16 GB RAM, use swap space equal to RAM.

- Systems with more than 16 GB RAM, use 16 GB of RAM for swap space.

To check swap space issue,

# grep SwapTotal /proc/meminfo

At least you need to have 1 GB of temp space in /tmp. However if you have more it will not hurt any.

To check issue you temp space issue,

# df -h /tmp

You will need at least 4.5 GB of available disk space for the Grid home directory, which includes both the binary files for Oracle Clusterware and Oracle Automatic Storage Management (Oracle ASM) and their associated log files, and at least 4 GB of available disk space for the Oracle Database home directory.

To check space in the OS partition issue,

# df –h

2. Network Hardware Requirements:

Each node must have at least two network interface cards (NIC), or network adapters. One adapter is for the public network interface and the other adapter is for the private network interface (the interconnect).

You need to install additional network adapters on a node if that node does not have at least two network adapters or has two network interface cards but is using network attached storage (NAS). You should have a separate network adapter for NAS.

Public interface names must be the same for all nodes. If the public interface on one node uses the network adapter eth0, then you must configure eth0 as the public interface on all nodes.

You should configure the same private interface names for all nodes as well. If eth1 is the private interface name for the first node, then eth1 should be the private interface name for your second node.

The private network adapters must support the user datagram protocol (UDP) using high-speed network adapters and a network switch that supports TCP/IP (Gigabit Ethernet or better). Oracle recommends that you use a dedicated network switch.

3. IP Address Requirements.

You must have a DNS server in order to make SCAN listener work. So, before you proceed installation prepare you DNS server. You must give the following entry manually in your DNS server.

i)  A public IP address for each node

ii) A virtual IP address for each node

ii) Three single client access name (SCAN) addresses for the cluster

During installation a SCAN for the cluster is configured, which is a domain name that resolves to all the SCAN addresses allocated for the cluster. The IP addresses used for the SCAN addresses must be on the same subnet as the VIP addresses. The SCAN must be unique within your network. The SCAN addresses should not respond to ping commands before installation.

4. OS and software Requirements.

To determine which distribution and version of Linux is installed as root user issue,

# cat /proc/version

Be sure your linux version is supported by Oracle dataabase 11gR2.

To determine which chip architecture each server is using and which version of the software you should install, as the root user

issue,

# uname -m

This command displays the processor type. For a 64-bit architecture, the output would be "x86_64".

To determine if the required errata level is installed, as the root user issue,

# uname -r

2.6.9-55.0.0.0.2.ELsmp

From the output  kernel version is 2.6.9, and the errata level (EL) is 55.0.0.0.2.ELsmp.

To determine whether the required packages are installed, enter commands similar to the following:

# rpm -q package_name

Without cluster verification utility as well as by running OUI you can determine whether you have missed any packages that is required to install Grid Infrastructure. If you get any package missing you can install it by,

# rpm -Uvh package_name

5. Preparing the server to install Grid Infrastructure.

i)                    Synchronize the time between each RAC nodes:

Oracle Clusterware 11g release 2 (11.2) requires time synchronization across all nodes within a cluster when Oracle RAC is deployed. The linux

# dateconfig

Command provides you a GUI through which you can set same timing across all nodes. But for accurate time time synchronization across the nodes you have two options: an operating system configured network time protocol (NTP), or Oracle Cluster Time Synchronization Service(octss).

I recommend to use oracle cluster time synchronization service because it can synchronize time among cluster members without contacting an external time server.

Note that If you use NTP, then the Oracle Cluster Time Synchronization daemon (ctssd) starts up in observer mode. If you do not have NTP daemons, then ctssd starts up in active mode.

If you have NTP daemons on your server but you cannot configure them to synchronize time with a time server, and you want to use Cluster Time Synchronization Service to provide synchronization service in the cluster, then deactivate and deinstall the Network Time Protocol (NTP).

To deactive do the following things:

# /sbin/service ntpd stop

# chkconfig ntpd off

# rm /etc/ntp.conf

or, mv /etc/ntp.conf to /etc/ntp.conf.org.

Also remove the following file:

/var/run/ntpd.pid

ii)                  Create the required OS users and groups.

# groupadd -g 1000 oinstall

# groupadd -g 1200 dba

# useradd -u 1100 -g oinstall -G dba oracle

# mkdir -p  /u01/app/11.2.0/grid

# mkdir -p /u01/app/oracle

# chown -R oracle:oinstall /u01

# chmod -R 775 /u01/

# passwd oracle

iii)                Modify the linux kernel parameters.

Open the /etc/sysctl.conf file and change the value like below.

#vi /etc/sysctl.conf

kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=262144
net.core.wmem_max=262144

Note that the GUI does not give any warning about kernel.sem parameter setting and if you don't set this parameter manually then later you may get unforeseen error.

iv)                Configure the network.

Determine the cluster name. We set the cluster name as dc-db-cluster

Determine the public, private and virtual host name for each node in the cluster.

It is determined as,

For host dc-db-01 public host name as dc-db-01

For host dc-db-02 public host name as dc-db-02

For host db-db-01 private host name as dc-db-01-priv

For host dc-db-02 private host name as dc-db-02-priv

For host dc-db-01 virtual host name as dc-db-01-vip

For host dc-db-02 virtual host name as dc-db-02-vip

Identify the interface names and associated IP addresses for all network adapters by executing the following command on each node:

# /sbin/ifconfig

On each node in the cluster, assign a public IP address with an associated network name to one network adapter. The public name for each node should be registered with your domain name system (DNS).

Also configure private IP addresses for cluster member nodes in a different subnet in each node.

Also determine the virtual IP addresses for each nodes in the cluster. These addresses and name should be registered in your DNS server. The virtual IP address must be on the same subnet as your public IP address.

Note that you do not need to configure these private, public, virtual addresses manually in the /etc/hosts file.

You can test whether or not an interconnect interface is reachable using a ping command.

Define a SCAN that resolves to three IP addresses in your DNS.

My full IP Address assignment table is as following.

Identity	Host Node	Name	Type	Address	Address static or dynamic	Resolved by
Node 1 Public	dc-db-01	dc-db-01	Public	192.168.100.101	Static	DNS
Node 1 virtual	Selected by oracle clusterware	dc-db-01-vip	Virtual	192.168.100.103	Static	DNS and/ or hosts file
Node 1 private	dc-db-01	dc-db-01-priv	Private	192.168.200.101	Static	DNS, hosts file, or none
Node 2 Public	dc-db-02	dc-db-02	Public	192.168.100.102	Static	DNS
Node 2 virtual	Selected by oracle clusterware	dc-db-02-vip	Virtual	192.168.100.104	Static	DNS and/ or hosts file
Node 2 private	dc-db-02	dc-db-02-priv	Private	192.168.200.102	Static	DNS, hosts file, or none
SCAN vip 1	Select by oracle clusterware	dc-db-cluster	Virtual	192.168.100.105	Static	DNS
SCAN vip 2	Selected by oracle clusterware	dc-db-cluster	Virtual	192.168.100.106	Static	DNS
SCAN vip 3	Selected by oracle clusterware	dc-db-cluster	Virtual	192.168.100.107	Static	DNS

In your /etc/resolve.conf file entry your DNS nameserver address.

# vi /etc/resolve.conf

192.168.100.1

Verify the network configuration by using the ping command to test the connection from each node in your cluster to all the

other nodes.

$ ping -c 3 dc-db-01

$ ping -c 3 dc-db-02

v) Configure shared storages

a. Oracle RAC is a shared everything database. All datafiles, clusterware files, database files must share a common space. Oracle strongly recommends to use ASM type of shared storage.  

b. When using Oracle ASM for either the Oracle Clusterware files or Oracle Database files, Oracle creates one Oracle ASM instance on each node in the cluster, regardless of the number of databases.

c. You need to prapare the storage for Oracle Automatic Storage Management(ASM). This storage preparatinon is necessary When you reboot the server, unless you have configured special files for device persistence, a disk that appeared as /dev/sdg before the system shutdown can appear as /dev/sdh as well as permission on the device is changed after the system is restarted.

d. Install the Linux ASMLIB RPMs is the simpliest solution to storage administration. ASMLIB provides persistent paths and permissions for storage devices used with Oracle ASM, eliminating the need for updating udev or devlabel files with storage device paths and permissions.

e. You can download the ASMLIB RPMs browsing http://www.oracle.com/technetwork/topics/linux/downloads/index.html, select downloads

tabs and click on "Linux Drivers for Automatic Storage Management". You will see ASMLIB RMPs for various operating systems like SuSE Linux Enterprise Server 11, SuSE Linux Enterprise Server 10, Red Hat Enterprise Linux 5 AS, Red Hat Enterprise Linux 4 AS, SuSE Linux Enterprise Server 9, Red Hat Enterprise Linux 3 AS, SuSE Linux Enterprise Server 8 SP3,  Red Hat Advanced Server 2.1.

Select your OS from the list, Download the oracleasmlib and oracleasm-support packages for your version of Linux. Then you must download the appropriate package for the kernel you are running. Use the uname -r command to determine the version of the kernel

on your server. For example, if your kernel version is 2.6.18-194.8.1.el5 then you need to download oracleasm drivers for kernel 2.6.18-194.8.1.el5.

ASMLib 2.0 is delivered as a set of three Linux packages:

i) oracleasmlib-2.0 - the Oracle ASM libraries

ii) oracleasm-support-2.0 - utilities needed to administer ASMLib

iii)oracleasm - a kernel module for the Oracle ASM library

f. As a root user, install these three packages.

# rpm -Uvh oracleasm-support-2.1.3-1.el4.x86_64.rpm

# rpm -Uvh oracleasmlib-2.0.4-1.el4.x86_64.rpm

# rpm -Uvh oracleasm-2.6.9-55.0.12.ELsmp-2.0.3-1.x86_64.rpm

g. To configure ASMLIB issue,

# oracleasm configure -i

If you enter the command oracleasm configure without the -i flag, then you are shown the current configuration.

After you issue oracleasm configure –i you will be prompted to provide

Default user to own the driver interface (example: oracle),

Default group to own the driver interface (example: dba),

Start Oracle Automatic Storage Management Library driver on boot (y/n): (provide: y), Fix permissions of Oracle ASM disks on boot? (y/n): (provide: y)

After it is run, it

Creates the /etc/sysconfig/oracleasm configuration file

Creates the /dev/oracleasm mount point

Mounts the ASMLIB driver file system

Enter the following command to load the oracleasm kernel module:

# /usr/sbin/oracleasm init

Repeat above steps in all nodes.

h. To mark a disk for use by Oracle ASM, enter the following command syntax, where ASM_DISK_NAME is the name of the Oracle ASM

disk group, and candidate_disk is the name of the disk device that you want to assign to that disk group:

# oracleasm createdisk ASM_DISK_NAME candidate_disk

In other words,

#  /usr/sbin/oracleasm createdisk ASM_DISK_NAME device_partition_name

For example,

# oracleasm createdisk data1 /dev/sdf

Note that For Oracle Enterprise Linux and Red Hat Enterprise Linux version 5, when scanning, the kernel sees the devices as /dev/mapper/XXX entries. By default, the 2.6 kernel device file naming scheme udev creates the /dev/mapper/XXX names for human readability. Any configuration using ORACLEASM_SCANORDER should use the/dev/mapper/XXX entries.

So your command would look like,

# oracleasm createdisk data1 /dev/mapper/mpath1

For each disk you need to issue createdisk statment that will be used for Oracle ASM.

If you need to unmark a disk that was used in a createdisk command use,

# /usr/sbin/oracleasm deletedisk disk_name

After you have created all the ASM disks for your cluster, use the listdisks command to verify their availability:

# /usr/sbin/oracleasm listdisks

Note that you need to create the ASM disks only on one node. On all the other nodes in the cluster, use the scandisks command to

view the newly created ASM disks.

# /usr/sbin/oracleasm scandisks

After scanning for ASM disks, display the available ASM disks on each node to verify their availability:

# /usr/sbin/oracleasm listdisks

i. If you use ASMLIB, then you do not need to ensure permissions and device path persistency in udev. If you do not

use ASMLib, then you must create a custom rules file in the path /etc/udev/rules.d/.

Preinstallation configuration is done at this stage. Now we will move to install Oracle Grid Infrastructure.

B. Installing Oracle Grid Infrastructure.

As Oracle run the runInstaller from the Oracle Grid Infrastructure CD room.

$ ./runInstaller

If you don't have CD then download the software from fromhttp://download.oracle.com/otn/linux/oracle11g/R2/linux_11gR2_grid.zip, unzip and then run the runInstaller.

Get Oracle Certifications for all Exams
Free Online Exams.com

Index on a partitioned table waits on 'row cache lock' in RAC environment

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description
Creating Index on a partitioned table fails with Oracle error ORA-04021. This is a 2-node Oracle RAC environment and in the table there is large number of partitions and sub-partitions. Index creation waits on 'row cache lock'. The same is true while dropping a tablespace. Drop Tablespace hangs on Row Cache Enqueue.

Problem Investigation
From the trace file we get following information.

PROCESS 24:      SO: 0xb9e505908, type: 4, owner: 0xb9f351708, flag: INIT/-/-/0x00     (session) sid: 477 trans: 0xb7ce39798, creator: 0xb9f351708, flag: (41) USR/- BSY/-/-/-/-/-               DID: 0001-0018-00000018, short-term DID: 0001-0018-00000019               txn branch: (nil)               oct: 9, prv: 0, sql: 0xacafc75e8, psql: 0xb0266cc18, user: 55/TA     O/S info: user: oracle, term: pts/1, ospid: 16322, machine: rac2               program: sqlplus@rac2 (TNS V1-V3)     application name: SQL*Plus, hash value=3669949024     waiting for 'row cache lock' blocking sess=0x(nil) seq=4750 wait_time=0 seconds since wait started=18                 cache id=2, mode=0, request=5  

Process 24 is executing sql: 0xacafc75e8 which is:

LIBRARY OBJECT HANDLE: handle=acafc75e8 mtx=0xacafc7718(1) cdp=1       name=CREATE INDEX USER_ACTIVITY_IDX ON PROD.USER_ACTIVITY(IMSI, CALLING_IMSI) TABLESPACE HISTORY_IDX_01  LOCAL PARALLEL 32 UNUSABLE 

It seems this process is waiting for exclusive enqueue lock mode on dc_segments:

----------------------------------------         SO: 0xafb5989f0, type: 50, owner: 0xb7ce39798, flag: INIT/-/-/0x00         row cache enqueue: count=1 session=0xb9e505908 object=0xb004f3690, request=X         savepoint=0x25a3749         row cache parent object: address=0xb004f3690 cid=2(dc_segments)         hash=2997173a typ=11 transaction=(nil) flags=00000000         own=0xb004f3760[0xb004f3760,0xb004f3760] wat=0xb004f3770[0xafb598a20,0xafb598a20] mode=N         status=-/-/-/-/-/-/-/-/-         request=X release=FALSE flags=2         instance lock id=QC 1cd04d4c 3ab28c03         data=         0000006a 000001da 0039918b 00000000 00000000 00000000 00000000 00000000         00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000         00000000 00000000         ---------------------------------------- 

From the trace file it is not clear who is holding this row cache enqueue.

Cause of the Problem
Index creation waits on 'row cache lock' in RAC environment occurred due to oracle bug.
Oracle named this bug as 'Bug 6321551' in version 10.2.0.2
'Bug 8417354' in version 10.2.0.4
'Bug 6004916' in version 11.0.0.0

Solution of the Problem
The bug is Oracle internal unpublished bug.
- This bug is fixed in Oracle version 10.2.0.5. So apply Oracle 10.2.0.5 patchset if you are running Oracle version 10g.

- If your Oracle database version is 10.2.0.2 or 10.2.0.3 you can apply one-off patch 6004916.

- Another workaround is to create the index with only one node being started (single instance).
i) Stop all RAC instance but one.
ii) Run the index creation script on one node.
iii) After process is completed startup rest of the instances.

Get Oracle Certifications for all Exams
Free Online Exams.com

Primary DB freezed with waited too long for a row cache enqueue lock

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description
On Oracle database 10.2.0.3.0 Data Guard with Broker configuration whenever there is an attempting to restart the standby in read only, the following error occurs:

ERROR: WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK
ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
ORA-12170: TNS:Connect timeout occurred
PING[ARC6]: Error 3113 when pinging standby
ARC6: Attempting destination LOG_ARCHIVE_DEST_2 network reconnect (1089)

The errors occurred in standby database and primary database hangs!

Cause of the Problem
The above error is caused by LGWR process in Oracle Data Guard having RAC environment and it is in fact Oracle bug. The bug number is 7487408.
The bug fires whenever Data Guard is managed by Broker and standby database is shutdown or opened in read only mode. The Primary still tries to ship/send redo streams to standby in SYNC mode and eventually hangs. 

Solution of the Problem
- Download the optach 7487408 from oracle support/metalink in order to solve the bug.
- The above bug is fixed in Oracle patchset 10.2.0.5. So you need to upgrade your oracle database to 10.2.0.5.

Get Oracle Certifications for all Exams
Free Online Exams.com

Step by step Create an Oracle 11gR2 Physical Standby Database

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

In this article I will show step by step procedure to create an Oracle 11gR2 physical standby database. This example environment is as follows:

- Both primary and standby database are on RAC environment. Both are 2 nodes RAC.
- The distance between primary and standby database is 300 kilometers and they are within Bangladesh.
- There is 10 Mbps dedicated bandwidth between primary and standby database. 
- All 4 nodes are using Red Hat Linux Enterprise version 5 operating system and same hardware structure. 
- Oracle 11gR2 is installed both in primary and physical standby database. 
- Oracle datafiles and clusterware are on ASM file system.

Though this example is based on RAC and ASM, you can use this document to setup any type of physical standby database. I will tell explicitly what to do based on whether your environment is non-RAC or not, whether you use ASM or not.

In this example the following values are used:
- Database name is bddipdc
- Primary database unique name as well as oracle net service name is bddipdc
- Standby database unique name as well as oracle net service name is bddipdrs
- Primary database hostname is DC-DB-01 and standby database hostname is DRS-DB-01
- Primary database IP Address is 192.168.100.101 and standby database IP Address is 192.168.105.101

Step 01: Prepare the Primary and physical standby database environment: (Both bddipdc and bddipdrs)
i) After you have physically placed hardware in both site install operating system in both end. The site in which primary database reside called as Data Center (DC) and the site in which standby database reside called as Disaster Recovery Site (DRS) and that is I choose the unique name as bddipdc and bddipdrs correspondingly. 

ii) Install Oracle software as well as oracle database both in primary and standby database site. Though database creation in standby site is not mandatory but you can create database specially if you are in RAC environment. Because in that case you don't need to do many manual tasks like register database to cluster registry etc. Note that both in primary and standby site, while creating database keep the database name same that is bddipdc. 

Step 02: Prepare the Primary Database for Standby Database Creation: (Only bddipdc)
i) Enable force logging on primary database.
SQL> ALTER DATABASE FORCE LOGGING;

ii) Set the database initialization parameters. 

#Primary database role initialization parameters. DB_NAME=bddipdc DB_UNIQUE_NAME=bddipdc LOG_ARCHIVE_CONFIG='DG_CONFIG=(bddipdc,bddipdrs)' LOG_ARCHIVE_DEST_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=bddipdc' LOG_ARCHIVE_DEST_2='SERVICE=bddipdrs SYNC AFFIRM VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=bddipdrs' LOG_ARCHIVE_DEST_STATE_1=ENABLE LOG_ARCHIVE_DEST_STATE_2=ENABLE REMOTE_LOGIN_PASSWORDFILE=EXCLUSIVE LOG_ARCHIVE_FORMAT=%t_%s_%r.arc LOG_ARCHIVE_MAX_PROCESSES=30  # Standby role initialization parameters. FAL_SERVER=bddipdrs DB_FILE_NAME_CONVERT='bddipdrs','bddipdc' LOG_FILE_NAME_CONVERT='bddipdrs','bddipdc' STANDBY_FILE_MANAGEMENT=AUTO 

As primary database is using spfile so run the following commands on primary database.

alter system set DB_UNIQUE_NAME=bddipdc scope=spfile sid='*'; alter system set LOG_ARCHIVE_CONFIG='DG_CONFIG=(bddipdc,bddipdrs)' scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=bddipdc' scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_2='SERVICE=bddipdrs SYNC AFFIRM VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=bddipdrs' scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_STATE_1=ENABLE scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_STATE_2=ENABLE scope=both sid='*'; alter system set LOG_ARCHIVE_MAX_PROCESSES=30 scope=both sid='*'; alter system set STANDBY_FILE_MANAGEMENT=AUTO scope=both sid='*'; alter system set FAL_SERVER=bddipdrs scope=both sid='*'; alter system set FAL_CLIENT=bddipdc scope=both sid='*'; 

We have not set any DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT parameters because in standby database we want to keep same file structure as it is in primary database.

iii) Enable archivelog in primary database.
If the primary database is not in archivelog mode then enable archivelog in primary database. If your database is in archivelog mode then you can simply forward to next phase.

SQL> SHUTDOWN IMMEDIATE; SQL> STARTUP MOUNT; SQL> ALTER DATABASE ARCHIVELOG; SQL> ALTER DATABASE OPEN; 

If your database is in RAC environment then you can follow Enable archivelog for RAC. If you are in non RAC environment then you can follow Change archivelog mode .

Step 03: Determine datafiles, controlfiles, password file, initialization parameter files to be copied from primary to standby database (bddipdc):
You can create standby database using RMAN. But in this post I want to show all steps manually because RMAN has several bug while creating another copy of database over the network.

i)Connect to one Oracle instance and determine the location of datafiles and tempfiles. Save the file location in a notepad.
Determine the location of datafile,

SQL> select name from v$datafile; NAME ---------------------------------------------------- +DATA/bddipdc/datafile/system.267.728507607 +DATA/bddipdc/datafile/sysaux.259.728507607 +DATA/bddipdc/datafile/undotbs1.258.728507609  +DATA/bddipdc/datafile/users.257.728507609 +DATA/bddipdc/datafile/undotbs2.270.728507783  

Determine the location of oracle tempfile.
SQL> select name from v$tempfile;
NAME
----------------------------------
+DATA/bddipdc/tempfile/temp.271.728507701

ii)Create the pfile from spfile of primary database.
SQL> conn / as sysdba
SQL> create pfile='/tmp/bddipdc/initbddipdc.ora' from spfile;

iii) Shutdown the primary database. If you are in RAC environment, then shut down all instances.
SQL> shut immediate;

Using srvctl,
$ srvctl stop database -d bddipdc

iv) This step is only applicable if your database files are in ASM. As our datafiles and tempfiles are in ASM file system so we need to copy them under /tmp/bddipdc in order to move the datafiles and tempfiles into another computer.

Ensure that your ORACLE_SID and ORACLE_HOME points to ASM home and ASM SID.
Invoke asmcmd.

$asmcmd ASMCMD> cp +DATA/bddipdc/datafile/system.267.728507607 /tmp/bddipdc/system ASMCMD> cp +DATA/bddipdc/datafile/sysaux.259.728507607 /tmp/bddipdc/sysaux ASMCMD> cp +DATA/bddipdc/datafile/undotbs1.258.728507609 /tmp/bddipdc/undotbs1 ASMCMD> cp +DATA/bddipdc/datafile/users.257.728507609 /tmp/bddipdc/users ASMCMD> cp +DATA/bddipdc/datafile/undotbs2.270.728507783 /tmp/bddipdc/undotbs2 ASMCMD> cp +DATA/bddipdc/tempfile/temp.271.728507701 /tmp/bddipdc/temp 

v) Start one instance of primary database in mount state. If you are in non RAC environment then just start database in mount state.
SQL> startup mount;

Create the standby control file.
SQL> ALTER DATABASE CREATE STANDBY CONTROLFILE AS '/tmp/bddipdc/control01.ora';

Step 04: Copy password file, datafile, standby controlfile, pfile from primary database to standby database.
Up to now we have datafile, tempfile, standby controlfile, pfile under one directory /tmp/bddipdc in the primary database. Now also we need to move password file into /tmp/bddipdc directory so that we can simply scp to remote location and with one command all files will be copied to standby database.
$ scp -r /tmp/bddipdc oracle@drs-db-01:/tmp/bddipdc

Copy node 1 of primary database password file across all nodes in standby database (This one is only applicable for RAC)
The password file of node 1 of primary database need to be copied across all instances in standby database as well as all other nodes in primary database as well.

Step 05: Start the Standby database and set initialization parameters. (bddipdrs)
At this stage we have all required files under location /tmp/bddipdc to setup physical standby database. We have already one database created in standby site. So we can use that spfile for our standby database creation. Though we have pfile in the location /tmp/bddipdc (which we copied from primary database) we will use current spfile of existing standby database in order minimize the steps.

i) Shut down standby database (in RAC shut down all instances) and start in nomount state (in RAC start one instance in nomount stage).
SQL> shut immediate;
SQL> startup nomount;

Set the initialization parameters.

SQL> alter system set DB_UNIQUE_NAME=bddipdrs scope=spfile sid='*'; alter system set LOG_ARCHIVE_CONFIG='DG_CONFIG=(bddipdc,bddipdrs)' scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=bddipdrs' scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_2='SERVICE=bddipdc SYNC AFFIRM VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=bddipdc' scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_STATE_1=ENABLE scope=both sid='*'; alter system set LOG_ARCHIVE_DEST_STATE_2=ENABLE scope=both sid='*'; alter system set LOG_ARCHIVE_MAX_PROCESSES=30 scope=both sid='*'; alter system set STANDBY_FILE_MANAGEMENT=AUTO scope=both sid='*'; alter system set FAL_SERVER=bddipdc scope=both sid='*'; alter system set FAL_CLIENT=bddipdrs scope=both sid='*'; 

We also set the location of control_files. We will copy standby control file into this location.
alter system set control_files='+DATA/bddipdc/controlfile/control01.ctl' scope=spfile sid='*';

iii) shutdown the standby database.
SQL> shut immediate;

iv) Delete all control files, datafiles, tempfiles of current standby database and copy these standby control file, datafile, tempfile into ASM file system. I am keeping the path of these files same as primary database. Note that if you are not in ASM file system then instead of +DATA use your desired location.

$ asmcmd ASMCMD> cp /tmp/bddipdc/control01.ora +DATA/bddipdc/controlfile/control01.ctl ASMCMD> cp /tmp/bddipdc/system +DATA/bddipdc/datafile/system ASMCMD> cp /tmp/bddipdc/sysaux +DATA/bddipdc/datafile/sysaux ASMCMD> cp /tmp/bddipdc/undotbs1 +DATA/bddipdc/datafile/undotbs1 ASMCMD> cp /tmp/bddipdc/undotbs2 +DATA/bddipdc/datafile/undotbs2 ASMCMD> cp /tmp/bddipdc/users +DATA/bddipdc/datafile/users ASMCMD> cp /tmp/bddipdc/temp /tmp/bddipdc/temp 

v) Start the standby database in mount state. 
SQL> startup mount

Set the undo_management parameter manual.
SQL> alter system set undo_management=MANUAL scope=both sid='*';

Now rename datafiles and tempfiles in order to use the correct location as we copied to ASM file system.

SQL> alter database rename file '+DATA/bddipdc/datafile/system.267.728507607' to '+DATA/bddipdc/datafile/system'; SQL> alter database rename file '+DATA/bddipdc/datafile/sysaux.259.728507607' to '+DATA/bddipdc/datafile/sysaux'; SQL> alter database rename file '+DATA/bddipdc/datafile/undotbs1.258.728507609' to '+DATA/bddipdc/datafile/undotbs1'; SQL> alter database rename file '+DATA/bddipdc/datafile/users.257.728507609' to '+DATA/bddipdc/datafile/users'; SQL> alter database rename file '+DATA/bddipdc/datafile/undotbs2.270.728507783' to '+DATA/bddipdc/datafile/undotbs2'; SQL> alter database rename file '+DATA/bddipdc/tempfile/temp.271.728507701' to  '+DATA/bddipdc/datafile/temp'; 

Step 06: Configure TNS entry and listener across all nodes. (All nodes)
In all nodes configure TNS entry so that you can reach from primary to standby database and vice versa. 
Within the file $ORACLE_HOME/network/admin/tnsnames.ora add the following two entries.

BDDIPDRS =   (DESCRIPTION =     (ADDRESS = (PROTOCOL = TCP)(HOST = DRS-DB-01)(PORT = 1521))     (CONNECT_DATA =       (SERVER = DEDICATED)       (SERVICE_NAME = bddipdc.world)     )   )  BDDIPDC =   (DESCRIPTION =     (ADDRESS = (PROTOCOL = TCP)(HOST = DC-DB-01)(PORT = 1521))     (CONNECT_DATA =       (SERVER = DEDICATED)       (SERVICE_NAME = bddipdc.world)     )   ) 

Before configuring TNS entry be sure about your host name, port number and service name. You can easily check that by issuing,
$ lsnrctl status

You should put an static listener entry within $GRID_HOME/network/admin/listener.ora if you are in RAC 
and you should put an static listener entry within $ORACLE_HOME/network/admin/listener.ora if you are in non RAC system.

Following is an static entry for node 1 from primary database.

SID_LIST_LISTENER =   (SID_LIST =     (SID_DESC =       (GLOBAL_DBNAME = bddipdc.world)       (ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)       (SID_NAME = bddipdc1)     )    ) LISTENER =   (DESCRIPTION_LIST =     (DESCRIPTION =       (ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER))       (ADDRESS = (PROTOCOL = TCP)(HOST =DC-DB-01)(PORT = 1521))     )   ) 

Note: If you are in non RAC system then instead of SID bddipdc1 you must use Database name that is bddipdc.

Following is an static listener entry for node 2 in primary database.

SID_LIST_LISTENER =   (SID_LIST =     (SID_DESC =       (GLOBAL_DBNAME = bddipdc.world)       (ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)       (SID_NAME = bddipdc2)     )    ) LISTENER =   (DESCRIPTION_LIST =     (DESCRIPTION =       (ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER))       (ADDRESS = (PROTOCOL = TCP)(HOST =DC-DB-02)(PORT = 1521))     )   ) 

Note: Just host name and Oracle SID name is change for node 2.

Following is the listener entry for the node 1 in standby database.

SID_LIST_LISTENER =   (SID_LIST =     (SID_DESC =       (GLOBAL_DBNAME = bddipdc.world)       (ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)       (SID_NAME = bddipdc1)     )    ) LISTENER =   (DESCRIPTION_LIST =     (DESCRIPTION =       (ADDRESS = (PROTOCOL = IPC)(KEY = LISTENER))       (ADDRESS = (PROTOCOL = TCP)(HOST =DRS-DB-01)(PORT = 1521))     )   ) 

Note: Just I have changed the hostname. If you are non RAC system then instead of instance SID bddipdc1 use database name bddipddc instead.

Step 7: Verify that TNS names and password file across all the instances are working properly. (All nodes):
Both in primary and standby database across all nodes issue,

$ tnsping bddipdc $ tnsping bddipdrs $ sqlplus sys/password@bddipdc as sysdba $ sqlplus sys/password@bddipdrs as sysdba 

If you can connect to both primary and standby instances using TNS entry from all nodes through sqlplus then your password file as well as TNS entry is perfect.


Step 8: Create standby redo log and start redo apply on standby database (bddipdrs):
Create standby redo logfile on standby database bddipdrs by following command,

alter database add standby logfile thread 1 size 50M; alter database add standby logfile thread 2 size 50M; alter database add standby logfile thread 1 size 50M; alter database add standby logfile thread 2 size 50M; alter database add standby logfile thread 1 size 50M; alter database add standby logfile thread 2 size 50M; 

Or, simply you can run following command if you are in non RAC environment.

alter database add standby logfile size 50M; 

Set the undo_management parameter to AUTO.

SQL> alter system set undo_management=AUTO scope=both sid='*'; 

Start Redo Apply,

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION;

Step 09: Verify that standby database is performing properly.
i)Identify the existing archived redo log files. (bddipdrs -standby)
On the standby database, query the V$ARCHIVED_LOG view to identify existing files in the archived redo log. For example:

SQL> SELECT SEQUENCE#, FIRST_TIME, NEXT_TIME FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#;

ii)Force a log switch to archive the current online redo log file.
On the primary database, issue the ALTER SYSTEM SWITCH LOGFILE statement to force a log switch and archive the current online redo log file group:

SQL> ALTER SYSTEM SWITCH LOGFILE;

iii) Verify the new redo data was archived on the standby database.
On the standby database, query the V$ARCHIVED_LOG view to verify the redo data was received and archived on the standby database:

SQL> SELECT SEQUENCE#, FIRST_TIME, NEXT_TIME FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#;
In the output you should see one extra row than previous query.

iv)Verify that received redo has been applied.
On the standby database, query the V$ARCHIVED_LOG view to verify that received redo has been applied:

SQL> SELECT SEQUENCE#,APPLIED FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#;

Get Oracle Certifications for all Exams
Free Online Exams.com

INS-06001: Failed to perform operation due to internal driver error INS-06006: Passwordless SSH connectivity

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description
While installing Oracle 11gR2 Grid Infrastructure on HP-UX or AIX machin OUI giving error message whenever it is clicked on setup of configuring ssh.
INS-06006: Passwordless SSH connectivity not set up between the following node(s).

You might also face following on your AIX machine after clicking ssh setup.
INS-06001: Failed to perform operation due to internal driver error.

Cause of the Problem
1. The problem happened as the grid user or oracle user (the user you used to run runInstaller) does not have password on OS level. 

2. If OUI is able to setup passwordless SSH connectivity but still shows the error, likely "ping" command is inaccessible - grid or oracle user should be able to execute ping command. 

3. OS command ssh or scp are not in expected location.

4. May be $HOME/.ssh or files in it are owned by different user. 

Solution of the Problem
1. Test the ssh.

Issue "passwd" command to set password in both machines. Note that if you don't set the password in the RAC nodes then these error will fire.

Also, please test ssh command in both nodes of your RAC.

on node1 execute:
ssh node1 date
ssh node2 date

and on node 2:
ssh node1 date
ssh node2 date

2. If OS command ssh or scp are not in expected location, the workaround is to create symbolic link on all nodes:

ln -s /usr/local/bin/ssh /usr/bin/ssh
ln -s /usr/local/bin/scp /usr/bin/scp

3. Check the permission on file $HOME/.ssh
$ ls -l $HOME/.ssh

Get Oracle Certifications for all Exams
Free Online Exams.com

Data Pump Export Fails With ORA-39125 ORA-19502 ORA-27072 Error: 11: Resource temporarily unavailable

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

Problem Description
In a RAC database Data Pump export or import operations are intermittently failing with:

ORA-31693: Table data object "owner"."table name" failed to load/unload and is being skipped due to error:  ORA-31644: unable to position to block number 182645 in dump file  "dump file name"  ORA-19502: write error on file "dump file name", blockno 184437 (blocksize=4096)  ORA-27072: File I/O error  Linux-x86_64 Error: 11: Resource temporarily unavailable  Additional information: 4  Additional information  

In our case data pump directory is placed on OCFS2 volume.

Cause of the Problem
The mount options for the OCFS2 volume do not include the "datavolume" and "nointr".

The "datavolume" option ensures that the Oracle processes opens these files with the o_direct flag. 
The "nointr" option ensures that the ios are not interrupted by signals. 

These two attributes needed in order to use for Oracle.

Solution of the Problem
Mount the OCFS2 volume with the "datavolume" and "nointr" options. 

For example:

# mount -o datavolume,nointr -t ocfs2 /dev/sda1 /dev/mount

Get Oracle Certifications for all Exams
Free Online Exams.com

Mount Options for Oracle files when used with NAS devices

Visit the Below Website to access unlimited exam questions for all IT vendors and Get Oracle Certifications for FREE
http://www.free-online-exams.com

In case of RAC

Operating System	Mount options for Binaries	Mount options for Oracle Datafiles	Mount options for CRS Voting Disk and OCR
Sun Solaris *	rw,bg,hard,nointr,rsize=32768, wsize=32768,proto=tcp,noac, vers=3,suid	rw,bg,hard,nointr,rsize=32768, wsize=32768,proto=tcp,noac, forcedirectio, vers=3,suid	rw,bg,hard,nointr,rsize=32768, wsize=32768,proto=tcp,vers=3, noac,forcedirectio
AIX (5L) **	rw,bg,hard,nointr,rsize=32768, wsize=32768,proto=tcp, vers=3,timeo=600	cio,rw,bg,hard,nointr,rsize=32768, wsize=32768,proto=tcp,noac, vers=3,timeo=600	cio,rw,bg,hard,intr,rsize=32768, wsize=32768,tcp,noac, vers=3,timeo=600
HPUX 11.23 ***	rw,bg,vers=3,proto=tcp,noac, hard,nointr,timeo=600, rsize=32768,wsize=32768,suid	rw,bg,vers=3,proto=tcp,noac, forcedirectio,hard,nointr,timeo=600, rsize=32768,wsize=32768,suid	rw,bg,vers=3,proto=tcp,noac, forcedirectio,hard,nointr,timeo=600 ,rsize=32768,wsize=32768,suid
Windows	Not Supported	Not Supported	Not Supported
Linux x86 ****	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp, vers=3, timeo=600, actimeo=0	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,actimeo=0, vers=3,timeo=600	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,actimeo=0, vers=3,timeo=600
Linux x86-64 ****	rw,bg,hard,nointr,rsize=32768,  wsize=32768,tcp,vers=3, timeo=600, actimeo=0	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,actimeo=0, vers=3,timeo=600	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,vers=3, timeo=600,actimeo=0
Linux - Itanium	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,vers=3, timeo=600, actimeo=0	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,actimeo=0, vers=3,timeo=600	rw,bg,hard,nointr,rsize=32768, wsize=32768,tcp,vers=3, timeo=600,actimeo=0

* NFS mount option "forcedirectio" is required on Solaris platforms when mounting the OCR/CRS files when using Oracle 10.1.0.4 or 10.2.0.2 or later (Oracle unpublished bug 4466428) 

** AIX is only supported with NAS on AIX 5.3 TL04 and higher with Oracle 10.2.0.1 and later (NetApp) 

*** NAS devices are only supported with HPUX 11.23 or higher ONLY 

**** These mount options are for Linux kernels 2.6 and above for older kernels

Note that,dDue to Unpublished bug 5856342, it is necessary to use the following pfile/spfile initialization parameter when using NAS with all versions of RAC on Linux (x86 & X86-64 platforms) until 10.2.0.4. This bug is fixed and included in 10.2.0.4 patchset.
The parameter is,
filesystemio_options = DIRECTIO

In case of Single Instance

Operating System	Mount options for Binaries	Mount options for Oracle Datafiles
Sun Solaris * (8, 9, 10)	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,proto=tcp,suid	rw,bg,hard,rsize=32768,wsize=32768,vers=3,[forcedirectio or llock],nointr,proto=tcp,suid
AIX (5L) **	rw,bg,hard,rsize=32768,wsize=32768,vers=3,intr,timeo=600,proto=tcp	rw,bg,hard,rsize=32768,wsize=32768,vers=3,cio,intr,timeo=600,proto=tcp
HPUX 11.23 ***	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,proto=tcp,suid	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,proto=tcp, suid, forcedirectio
Windows	Not Supported	Not Supported
Linux x86 ****	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,tcp	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,tcp,actime=0*
Linux x86-64 ****	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,tcp	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,tcp,actime=0*
Linux - Itanium	rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,tcp

Get Oracle Certifications for all Exams
Free Online Exams.com