Tuesday, 11 March 2014

Oracle User lock on Standby - ADG on 11g

On oracle ADG instances upto 11.2.0.3, if an oracle user is locked it wont get unlocked despite after reaching "PASSWORD_LOCK_TIME" time which we have mentioned in user profile.

Reason you can refer the doc "Doc ID 1600401.1" from metalink

Upto 11gr3, we need to explictly login to the Standby/ADG database and unlock this user.

But from 11.2.0.4 ADG this issue has been fixed, user will be unlocked automatically after "PASSWORD_LOCK_TIME" time and after successful login with the correct password for the user.

We no need to explictly login with SYSDBA unlock that user on Standby or ADG instances. 

Locked user info can been seen from view "v$RO_USER_ACCOUNT" until its been unlocked on ADG instances.

After successful login with the correct password, info from v$RO_USER_ACCOUNT will be cleared.

Query to Check Oracle Locked User Details on ADG:

select a.USERNAME, a.USER_ID,  b.PASSW_LOCKED, b.FAILED_LOGINS, b.PASSW_LOCK_TIME from dba_users a, v$RO_USER_ACCOUNT b

where a.USERNAME='TEST' and a.USER_ID=b.USERID;

prstat info

We had situation where oracle database allocated sga is 1.5 GB but at the OS level each oracle process has consumed more memory approximately 32 GB of memory.

How its feasible ? if a total sga is 1.5 GB but at OS level oracle has consumed 32 GB of memory.

Below is the example and cause:


With “prstat –t”  (Report total usage summary for each  user) on server1 shows only 1.8 GB of memory been used by the oracle processes.

server1 - (TEST1)/ash/test> prstat -t
NPROC USERNAME  SWAP   RSS MEMORY      TIME  CPU
   133 oracle   1749M 1854M   5.7%  58:35:25 0.9%
     5 gmread     20M   32M   0.1%  29:39:46 0.8%
     1 noaccess  134M  121M   0.4%  11:40:03 0.0%
    49 root      467M  428M   1.3%  24:01:12 0.0%
     1 afrsreas  360K 2528K   0.0%   0:43:17 0.0%
     8 webse      16M 8992K   0.0%   0:00:21 0.0%
     1 lp       1144K 3576K   0.0%   0:00:00 0.0%
     1 knbsp    1432K 6544K   0.0%   0:01:56 0.0%
     6 daemon     22M   25M   0.1% 145:42:38 0.0%


Whereas the command “prstat -s size” (Sort by size of process image) indicates oracle processes has consumed more memory for each process.

$ prstat -s size
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
23048 oracle   1411M 1273M sleep   48    0   0:06:55 0.0% oracle/15
23067 oracle   1411M 1270M sleep   48    0   0:21:14 0.0% oracle/14
22350 oracle   1411M 1258M sleep   59    0   4:03:58 0.0% oracle/11
22275 oracle   1406M 1253M sleep   59    0   1:22:48 0.0% oracle/241
22312 oracle   1406M 1249M sleep   59    0   1:19:20 0.0% oracle/235
22294 oracle   1406M 1249M sleep   59    0   1:15:16 0.0% oracle/231
22331 oracle   1405M 1249M sleep   59    0   1:14:17 0.0% oracle/224
12971 oracle   1402M 1311M sleep   59    0   0:00:05 0.0% oracle/11

Reason:

Example let us check top process id “23048”
We can not ISM shmid has consumed memory

server1 - (TEST1)/ash/test> pmap -x 23048 | grep -v lib
23048:  ora_arc0_TW5P1
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
0000000100000000      95648      94976          -          - r-x--  oracle
0000000105E66000        648        616        256          - rwx--  oracle
0000000105F08000         24         24          8          - rwx--  oracle
0000000105F0E000          8          8          8          - rwx--    [ heap ]
0000000105F10000       3712       2752        384          - rwx--    [ heap ]
0000000380000000    1228808    1228808          -    1228808 rwxsR    [ ism shmid=0x4 ]
FFFFFFFF7860E000          8          8          -          - rw--R    [ stack tid=15 ]
FFFFFFFF7870E000          8          8          8          - rw--R    [ stack tid=14 ]
FFFFFFFF787A0000       1088       1088        576          - rw---    [ anon ]
FFFFFFFF788B0000       1088       1088        832          - rw---    [ anon ]
FFFFFFFF78C0E000          8          8          8          - rw--R    [ stack tid=13 ]
FFFFFFFF78D0E000          8          8          8          - rw--R    [ stack tid=12 ]
FFFFFFFF78E0E000          8          8          -          - rw--R    [ stack tid=11 ]
FFFFFFFF78F0E000          8          8          -          - rw--R    [ stack tid=10 ]
FFFFFFFF78FA0000       1088       1088        448          - rw---    [ anon ]
FFFFFFFF790B0000       1088       1088        832          - rw---    [ anon ]
FFFFFFFF791C0000       1088       1088        512          - rw---    [ anon ]
FFFFFFFF7940E000          8          8          -          - rw--R    [ stack tid=9 ]
FFFFFFFF7950E000          8          8          -          - rw--R    [ stack tid=8 ]
FFFFFFFF7960E000          8          8          8          - rw--R    [ stack tid=7 ]
FFFFFFFF7970E000          8          8          -          - rw--R    [ stack tid=6 ]
FFFFFFFF797A0000       1088       1088        640          - rw---    [ anon ]
FFFFFFFF79A0E000          8          8          -          - rw--R    [ stack tid=5 ]
FFFFFFFF79AA0000       1088       1088        640          - rw---    [ anon ]
FFFFFFFF79D0E000          8          8          -          - rw--R    [ stack tid=4 ]
FFFFFFFF79D90000       1088       1088        576          - rw---    [ anon ]
FFFFFFFF79F0E000          8          8          -          - rw--R    [ stack tid=3 ]
FFFFFFFF79F80000       1088       1088        576          - rw---    [ anon ]
FFFFFFFF7A10E000          8          8          -          - rw--R    [ stack tid=2 ]
FFFFFFFF7A170000       1088       1088        704          - rw---    [ anon ]
FFFFFFFF7A300000         64         64          -          - rwx--    [ anon ]
FFFFFFFF7A400000         64         64         64          - rw---    [ anon ]
FFFFFFFF7A410000         64         64          -          - rw---    [ anon ]
FFFFFFFF7A420000         64         64         64          - rw---    [ anon ]
FFFFFFFF7A430000         64         64          -          - rw---    [ anon ]
FFFFFFFF7A440000         64         64          -          - rw---    [ anon ]
FFFFFFFF7A450000        192        128        128          - rw---    [ anon ]
FFFFFFFF7A800000         64         64          -          - rw---    [ anon ]
FFFFFFFF7AF00000         64         64         64          - rw---    [ anon ]
FFFFFFFF7B500000          8          8          -          - rwx--    [ anon ]
FFFFFFFF7BD00000         64         64         64          - rwx--    [ anon ]
FFFFFFFF7BF00000         24         16          8          - rwx--    [ anon ]
FFFFFFFF7CA00000          8          8          -          - rw-s-  dev:85,106999 ino:277774
FFFFFFFF7CD00000          8          8          8          - rwx--    [ anon ]
FFFFFFFF7D000000          8          8          8          - rwx--    [ anon ]
FFFFFFFF7D300000          8          8          8          - rwx--    [ anon ]
FFFFFFFF7D600000          8          8          8          - rwx--    [ anon ]
FFFFFFFF7DB00000          8          8          8          - rwx--    [ anon ]
FFFFFFFF7DD00000          8          8          8          - rwx--    [ anon ]
FFFFFFFF7DE00000        216        216          -          - r-x--  ld.so.1
FFFFFFFF7DF36000         16         16          8          - rwx--  ld.so.1
FFFFFFFF7DF3A000          8          8          8          - rwx--  ld.so.1
FFFFFFFF7DF92000          8          8          -          - rwxs-    [ anon ]
FFFFFFFF7FFF0000         64         64         64          - rw---    [ stack ]
---------------- ---------- ---------- ---------- ----------
        total Kb    1363416    1353280       7600    1228808



The ISM is a shared memory segment that consists of large locked memory pages. The ISM number of locked pages remains constant or unchanged. Dynamic ISM (DISM) is pageable ISM shared memory, where the number of locked pages is variable or changeable. Therefore, the DISM supports releasing or adding more physical memory to the system during dynamic reconfiguration. The size of the DISM can span available physical memory plus disk swap.