Tuesday, 25 April 2017

ORA-16024: parameter log_archive_dest_1 cannot be parsed

ORA-16024 Referencing LOG_ARCHIVE_DEST_n Parameter


SYMPTOMS


You are using the log_archive_dest_n parameter in your pfile:

log_archive_dest_1 = 'location = /u01/oracle/arc' 

and are receiving an:
ORA-16024: parameter log_archive_dest_1 cannot be parsed

SOLUTION


Ensure that /u01/oracle/arc is a valid directory and that Oracle has write permissions there.

Remove the spaces on either side of the '=' sign.
You cannot have as space on either side on the '=' sign after 'location'.
Note that the word 'location' can be in either upper or lower case.
log_archive_dest_1 = 'location=/u01/oracle/arc'

Make sure that there is no trailing /.

Make sure that LOG_ARCHIVE_DEST is not set in the init.ora.
These 2 parameters are not compatible with each other.

NOTE

The error ORA-16024 could also be encountered when an SPFILE is used, for instance when upgrading a database.

To prevent that, you can remove the log_archive_dest from spfile by issuing

SQL> alter system reset log_archive_dest scope=spfile;

If still not working set the LOG_ARCHIVE_DEST_n parameter to a null value:

LOG_ARCHIVE_DEST_n=''

------------------------------------------------------------------------------------------------

Sunday, 23 April 2017

How to change EBS R12 Database and Application Port Number

Execute the below in Application Tier:

Run adpreclone on Application Tier
   $ perl ./adpreclone.pl appsTier

Execute the following on the Database Tier:

Logon as oracle database user and source the env file for R12 RDBMS ORACLE_HOME

Start the database instance for which you want to change R12 port

Backup contextfile

Change as below in the contextfile

        dbport  : Change database port from 1521 to 1531
        cmanport : 1521 to 1531

Run adautocfg.sh

Execute the following in Application Tier:

Logon as application user and source the env for R12

Backup contextfile

Run adcfgclone and choose new port pool
   $ cd $COMMON_TOP/clone/bin
   $ perl ./adcfgclone.pl appsTier

Check new application URL

SQL> select home_url from icx_parameters;

HOME_URL
------------------------------------------------------------
http://<hostname.domainname>:8010/OA_HTML/AppsLogin

Run adpreclone on both apps and database tier

----------------------------------------------------------------------------------------------------

Wednesday, 19 April 2017

weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT

The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT

I examined $EBS_DOMAIN_HOME/servers/AdminServer/logs/AdminServer.log and found this error:

BEA-280060 - The persistent store "WLS_DIAGNOSTICS" encountered a fatal error, and it must be shut down: weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT.
weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT.
        at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:128)
        at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
        at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
        at weblogic.store.xa.PersistentStoreManagerXA.createFileStore(PersistentStoreManagerXA.java:117)
        at weblogic.diagnostics.archive.DiagnosticStoreRepository.getStore(DiagnosticStoreRepository.java:91)
        at weblogic.diagnostics.lifecycle.ArchiveLifecycleImpl.initialize(ArchiveLifecycleImpl.java:94)
        at weblogic.diagnostics.lifecycle.DiagnosticFoundationService.start(DiagnosticFoundationService.java:108)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: java.io.IOException: Error from fcntl() for file locking, Resource temporarily unavailable, errno=11
        at weblogic.store.io.file.direct.DirectIONative.openConsiderLock(Native Method)
        at weblogic.store.io.file.direct.DirectFileChannel.(DirectFileChannel.java:54)
        at weblogic.store.io.file.direct.DirectIOManager.open(DirectIOManager.java:179)
        at weblogic.store.io.file.StoreFile.openInternal(StoreFile.java:138)
        at weblogic.store.io.file.StoreFile.open(StoreFile.java:161)
        at weblogic.store.io.file.Heap.openStoreFile(Heap.java:401)
        at weblogic.store.io.file.Heap.open(Heap.java:325)
        at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:117)
        at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
        at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
        at weblogic.store.xa.PersistentStoreManagerXA.createFileStore(PersistentStoreManagerXA.java:117)
        at weblogic.diagnostics.archive.DiagnosticStoreRepository.getStore(DiagnosticStoreRepository.java:91)
        at weblogic.diagnostics.lifecycle.ArchiveLifecycleImpl.initialize(ArchiveLifecycleImpl.java:94)
        at weblogic.diagnostics.lifecycle.DiagnosticFoundationService.start(DiagnosticFoundationService.java:108)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

A search on support.oracle.com for keywords resulted in this hit:

WebLogic Fails to Start with Error: “The persistent store "WLS_DIAGNOSTICS" encountered a fatal error, and it must be shut down: weblogic.store.PersistentStoreFatalException" (Doc ID 859622.1)

WORKAROUNDS:

This is a recurrent workaround. This workaround has to be applied every time the server(s) will be restarted. 

Go to DOMAIN_HOME/servers/<SERVERNAME>/data/ldap/ldapfiles and remove the EmbeddedLDAP.lok lock file
Delete the *.dat file under  DOMAIN_HOME/servers/<SERVERNAME>/data /store/default and DOMAIN_HOME/servers/<SERVERNAME>/data /store/diagnostics/
Remove the contents under  DOMAIN_HOME/servers/<SERVERNAME>/ tmp and DOMAIN_HOME/servers/<SERVERNAME>/ cache

Now start the server
./startWebLogic.sh

Monday, 20 March 2017

How to modify resource dependencies in CRS


I wanted to change the diskgroup used for flash recovery area. So I created a new diskgroup and updated the parameters

SQL> show parameter db_recovery_file_dest 

db_recovery_file_dest string +RACFRA

SQL> alter system set db_recovery_file_dest=’+FRA’

System altered.

SQL> show parameter db_recovery_file_dest

db_recovery_file_dest string +FRA

Also removed the old diskgroup

SQL> drop diskgroup RACFRA

Diskgroup dropped.

After that shutdown the database for some maintenance. But when I tried starting it back, it started throwing
following error
oragrid@node-20> srvctl start database -d mydb
PRCR-1079 : Failed to start resource ora.mydb.db
CRS-2640: Required resource ‘ora.RACFRA.dg’ is missing.
I was not sure, what exactly happened. As I already dropped this diskgroup, database & ASM diskgroup
dependency should have been removed. But error message was indicating something else
So my first check, was to see if the resource was still present


oragrid@node-20> crsctl stat res -t
——————————————————————————–
NAME TARGET STATE SERVER STATE_DETAILS
——————————————————————————–
Local Resources
——————————————————————————–


ora.FRA.dg
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.LISTENER.lsnr
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.DG.dg
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.DG2.dg
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.DG3.dg
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.DG4.dg
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.asm
ONLINE ONLINE node-20 Started
ONLINE ONLINE node-21 Started
ora.eons
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.gsd
OFFLINE OFFLINE node-20
OFFLINE OFFLINE node-21
ora.net1.network
ONLINE ONLINE node-20
ONLINE ONLINE node-21
ora.ons
ONLINE ONLINE node-20
ONLINE ONLINE node-21
——————————————————————————–
Cluster Resources
——————————————————————————–
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE node-21
ora.oc4j
1 OFFLINE OFFLINE
ora.node-20.vip
1 ONLINE ONLINE node-20
ora.node-21.vip
1 ONLINE ONLINE node-21
ora.scan1.vip
1 ONLINE ONLINE node-21
ora.mydb.db
1 OFFLINE OFFLINE Instance Shutdown
2 OFFLINE OFFLINE Instance Shutdown


crsctl output did not show this diskgroup. So checked the dependency for the database resource
# crsctl status resource ora.mydb.db -f
Where I found, that START_DEPENDENCIES still have reference to old diskgroup. So removed, re-added the


database & associated instances
oragrid@node-21> srvctl remove database -d mydb
oragrid@node-21> srvctl add database -d mydb -o /oracle/11gr2
ora11gr2@node-21> srvctl add instance -d mydb -i mydb1 -n node-20
ora11gr2@node-21> srvctl add instance -d mydb -i mydb2 -n node-21
Again checked the dependency for the database resource
# crsctl status resource ora.mydb.db -f
This time START_DEPENDENCIES was having correct set of ASM diskgroups & the old DG reference was
removed.

Now started the database, all was fine.
oragrid@node-21> srvctl start database -d mydb
So to avoid this problem, we should be using srvctl remove diskgroup, instead of using sqlplus.



Wednesday, 15 March 2017

How to Calculate Network Bandwidth for Oracle Dataguard - Disaster Recovery (DR)

The formula used (assuming a conservative TCP/IP network overhead of 30%) for calculating the network bandwidth is :

Required bandwidth = ((Redo rate bytes per sec. / 0.75) * 8) / 1,000,000 = bandwidth in Mbps

Measuring the Peak Redo RateUse the Oracle Statspack utility for an accurate measurement of the redo rate. Based on your business you should have a good idea as to what your peak periods of normal business activity are. For example, you may be running an online store which historically sees the peak activity for 4 hours every Monday between 10:00 am - 2:00 pm. Or, you may be running a merchandising database which batch-loads a new catalog every Thursday for 2 hours between 1 am - 3 am. Note that we say "normal" business activity - this means that in certain days of the year you may witness much heavier business volume than usual, e.g. the 2-3 days before Mother's Day or Valentine's Day for an online florist business. Just for those days, perhaps you may allocate higher bandwidth than usual, and you may not consider those as "normal" business activity.However, if such periodic surges of traffic are regularly expected as part of your business operations, you must consider them in your redo rate calculation. 

During the peak duration of your business, run a Statspack snapshot at periodic intervals. For example, you may run it three times during your peak hours, each time for a five-minute duration. The Statspack snapshot report will include a "Redo size" line under the "Load Profile" section near the beginning of the report. This line includes the "Per Second" and "Per Transaction" measurements for the redo size in bytes during the snapshot interval. Make a note of the "Per Second" value. Take the highest "Redo size" "Per Second" value of these three snapshots, and that is your peak redo generation rate. 

Note that if your primary database is a RAC database, you must run the Statspack snapshot on every RAC instance. Then, for each Statspack snapshot, sum the "Redo Size Per Second" value of each instance, to obtain the net peak redo generation rate for the primary database. Remember that for a RAC primary database, each node generates its own redo and independently sends that redo to the standby database - hence the reason to sum up the redo rates for each RAC node, to obtain the net peak redo rate for the database.

As an Alternative you can also get the 'Redo rate bytes per sec.' from V$SYSMETRIC_HISTORY, eg.

SQL> select * from v$sysmetric_history where metric_name = 'Redo Generated Per Sec';

or in a RDA-Output:

Performance - AWR Report - Statistic: "redo size" 

Example:

Let us assume the redo rate is a 500 KB/sec.

Required bandwidth = ((Redo rate bytes per sec. / 0.75) * 8) / 1,000,000 = bandwidth in Mbps  Required bandwidth = ((512000/0.75) * 8) /1,000,000 Required bandwidth = 5.46 Mbps

Also you can find the Minimum and Maximum Bandwidth limit using the below query,

SELECT DT,SUM(RB*8/3600000000*1.3) TOTAL_Mbps_REQ_FOR_A_DAY,MIN(RB*8/3600000000*1.3) MIN_Mbps_REQ_FOR_AN_HOUR,MAX(RB*8/3600000000*1.3) MAX_Mbps_REQ_FOR_AN_HOUR ,AVG(RB*8/3600000000*1.3) AVG_Mbps_REQ_FOR_AN_HOURFROM(SELECT TRUNC (COMPLETION_TIME) DT,TO_CHAR (COMPLETION_TIME,'HH24') HH,SUM(BLOCKS*BLOCK_SIZE) RBFROMV$ARCHIVED_LOGWHERE COMPLETION_TIME > SYSDATE-5AND DEST_ID=1GROUP BY TRUNC(COMPLETION_TIME),TO_CHAR (COMPLETION_TIME, 'HH24'))GROUP BY DT;

Reference:


How To Calculate The Required Network Bandwidth Transfer Of Redo In Data Guard Environments (Doc ID 736755.1)

Friday, 10 March 2017

AutoPatch error: Missing file format id in file applcust.txt



APPLIES TO:

Oracle Order Management - Version 12.1.2 and later
Information in this document applies to any platform.

SYMPTOMS

On :  12.1.2 version, Install / Upgrade

When attempting to apply OM Patch 10087292:R12.ONT.B per Note 1195450.1 Error In Sales Order Form OEXOEORD: Please Choose An Existing Combination, the following error occurs.

NOTE: this error can happen with any patch or product and is not specific to OM.

ERROR
-----------------------
Getting error when applying the patch:
STRT_TASK: [Display customized files] [] [Fri Oct 01 2010 19:06:38]

Reading customized files list (if any)...

AutoPatch error:
:
Missing file format id in file
/u01/appldev/DEV/apps/apps_st/appl/admin/applcust.txt

This version of AutoPatch requires a file format id
on the first line of the file.

Error reading customized files file
Freeing includes hash table
Freeing fixes hash table
Freeing basedons hash table
Freeing entities hash table

You should check the file
/u01/appldev/DEV/apps/apps_st/appl/admin/DEV/log/u10087292.log
for errors.,

NOTE: the log file will vary per customer and instance.  This is only an example shown above.

STEPS
-----------------------
The issue can be reproduced at will with the following steps:
1. Apply patch with AutoPatch
2. Receive error:
Missing file format id in file applcust.txt
This version of AutoPatch requires a file format id on the first line
of the file.


BUSINESS IMPACT
-----------------------
The issue has the following business impact:
Due to this issue, users cannot apply the patch without error.

CAUSE

The applcust.txt file is not used in R12i, and has been replaced by registering
any customized files in the Register Flagged Files tool in Oracle Applications
Manager (OAM).

SOLUTION

To implement the solution, please execute the following steps:

1. Rename the applcust.txt file :

  mv applcust.txt  applcust_BAK.txt

2. If there are no custom files:
  a) Try to re-apply the patch at this time, and report any errors/logs back to us.

3. If there are custom files:
 a) refer to Note 459156.1 "Oracle Applications Patching FAQ for Release 12" FAQ section "How can I track my customizations? What happens to my customizations during patching?"

This will direct you to follow the procedure outlined in the document titled:
"Oracle Applications Patching Procedures Release 12.1 part No E12148-02"
      section "Register Flagged Files".

 Note: Registering customized files does not prevent the object or the patch from being applied. It only makes them available to AutoPatch for review.

  b) Try to re-apply the patch at this time, and report any errors/logs back to us.


4. Retest the issue.

5. Migrate the solution as appropriate to other environments.

MOS Doc ID 1227113.1

Monday, 16 January 2017

ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired


Caused By: Error executing SQL CREATE INDEX ****.claimno ON ****.BPS_STATIC_PAY_DED (CLAMIN_NO) LOGGING NOPARALLEL         

Caused By: ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired

Execute the below command to check if there is any lock occurred,

SELECT O.OBJECT_NAME, S.SID, S.SERIAL#, P.SPID, S.PROGRAM,SQ.SQL_FULLTEXT, S.LOGON_TIME FROM V$LOCKED_OBJECT L, DBA_OBJECTS O, V$SESSION S, V$PROCESS P, V$SQL SQ WHERE L.OBJECT_ID = O.OBJECT_ID AND L.SESSION_ID = S.SID AND S.PADDR = P.ADDR AND S.SQL_ADDRESS = SQ.ADDRESS;

Using the Output of the above query, please execute the below.

Killing the DB session

To kill the DB session execute the  sql

alter system kill session 'sid,serial#'

In my case it will be
         alter system kill session '953,40807'