MERHABA

Uzun zamandır düşünüpte hayata geçiremediğim blog dünyasına bu yazı ile merhaba diyorum.  Tasarlarken, düşünürken faydalı,güzel yazılar yazarak, karşılaştığım durumları paylaşarak,  özellikle Türkçe kaynak arayanlara faydalı olmayı planlıyordum. umarım düşündüğüm şeyleri burada hayata geçirebilirim.

Faydalı,sürekli yeni bilgilerin paylaşıldığı bir blog olması dileğiyle,

Merhaba Dünya 🙂

Advertisements

Unlug PDB and plug into another CDB

unplug a PDB and plug into another CDB
unplug from CDB1 and plugin to CDB2

on CDB1

SQL> select CON_ID, DBID, CON_UID, GUID,NAME from v$pdbs

CON_ID DBID CON_UID GUID NAME
---------- ---------- ---------- -------------------------------- --------------------------------------------
2 2743990362 2743990362 6C77C27828EC08A2E05303E1060A2939 PDB$SEED
3 2688943372 2688943372 71E7C773DC8E7659E05303E1060A61C4 PDB3
4 2848179871 2848179871 6D052E1C34FE1EC3E05303E1060A72DF PDB1
5 4140312712 4140312712 6DCD095FEFDC2693E05303E1060A9333 PDB2

SQL> show pdbs

CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
4 PDB1 READ WRITE NO
5 PDB2 READ WRITE NO
8 PDB3 READ WRITE NO
SQL> set lines 100
SQL> select name from v$datafile where con_id=8

NAME
----------------------------------------------------------------------------------------------------
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/system.374.982858999
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/sysaux.373.982858999
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/undotbs1.378.982858999
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/undo_2.371.982859021
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/users.370.982859025
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/PDB3data.269.982859213
+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751/DATAFILE/PDB3index.267.982859213

7 rows selected.
SQL>alter pluggable database PDB3 close instances=all;
Pluggable database altered.

SQL>alter pluggable database PDB3 unplug into '/dblog/PDB3.xml';
Pluggable database altered.

SQL> drop pluggable database PDB3 keep datafiles;
Pluggable database dropped.

on CDB2
–check compability of the unplugged PDB with destination CDB


set serveroutput on

DECLARE
compatible BOOLEAN := FALSE;
BEGIN
compatible := DBMS_PDB.CHECK_PLUG_COMPATIBILITY(
pdb_descr_file => '/dblog/PDB3.xml');
if compatible then
DBMS_OUTPUT.PUT_LINE('Is pluggable PDB3 compatible? YES');
else DBMS_OUTPUT.PUT_LINE('Is pluggable PDB3 compatible? NO');
end if;
END;
/

Is pluggable PDB3 compatible? YES

PL/SQL procedure successfully completed.
SQL> create pluggable database PDB3 using '/dblog/PDB3.xml' COPY FILE_NAME_CONVERT=('+DATA/CDBCONS/72385D6869CF333FE05303E1060AB751','+UDATA');

Pluggable database created.
SQL> select pdb_name, status from cdb_pdbs where pdb_name='PDB3';

PDB_NAME STATUS
------------------------------------------------------------- -----------------------------------
PDB3 NEW

SQL> select open_mode from v$pdbs where name='PDB3';

OPEN_MODE
----------
MOUNTED

SQL> show pdbs

CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
4 PDB3 MOUNTED
SQL> select name from v$datafile where con_id=4

NAME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/system.286.988222753
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/sysaux.287.988222753
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/undotbs1.284.988222753
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/undo_2.285.988222753
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/users.281.988222753
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/PDB3data.282.988222753
+UDATA/TEK/72385D6869CF333FE05303E1060AB751/DATAFILE/PDB3index.283.988222753

7 rows selected.

SQL>alter pluggable database PDB3 open instances=all;
Pluggable database altered.

SQL> show pdbs;

CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
4 PDB3 READ WRITE NO
[oracle@server01 ~]$ sqlplus sys/@10.5.5.5:1521/PDB3 AS SYSDBA

SQL*Plus: Release 12.2.0.1.0 Production on Sun Sep 30 18:04:16 2018

Copyright (c) 1982, 2016, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> show con_name

CON_NAME
------------------------------
PDB3

 

Linux Kermel parameters HugePages

In Linux sytems Oracle recommends using Huge Pages.
Because memory allocation for sga can cause updating page tables.

Without HugePages, the operating system keeps each 4 KB of memory as a page. When it allocates pages to the database System Global Area (SGA), the operating system kernel must continually update its page table with the page lifecycle (dirty, free, mapped to a process, and so on) for each 4 KB page allocated to the SGA.

With HugePages, the operating system page table (virtual memory to physical memory mapping) is smaller, because each page table entry is pointing to pages from 2 MB to 256 MB.

Also, the kernel has fewer pages whose lifecycle must be monitored. For example, if you use HugePages with 64-bit hardware, and you want to map 256 MB of memory, you may need one page table entry (PTE). If you do not use HugePages, and you want to map 256 MB of memory, then you must have 256 MB * 1024 KB/4 KB = 65536 PTEs.

HugePages provides the following advantages:

  • Increased performance through increased TLB hits
  • Pages are locked in memory and never swapped out, which provides RAM for shared memory structures such as SGA
  • Contiguous pages are preallocated and cannot be used for anything else but for System V shared memory (for example, SGA)
  • Less bookkeeping work for the kernel for that part of virtual memory because of larger page sizes

check Huge Pages:

$ grep Huge /proc/meminfo| grep "HugePages_Total"

add folloving line in /etc/sysctl.conf to set Huge Pages as 200GB. because our hugepagesize is 2MB.

vm.nr_hugepages = 102400

check hugepagesize:

[root@server01 ~]# more /proc/meminfo | grep Hugepagesize
Hugepagesize: 2048 kB

change can be seted like:

[root@server01 ~]# sysctl -p

new Huge Page value can set with

$ grep Huge /proc/meminfo| grep "HugePages_Total"

Linux Kernel parameters SHMMAX SHMMNI SHMALL

SHMMAX:
Oracle recommeds Half the size of physical memory in bytes for 64 bit systems.

See My Oracle Support Note 567506.1 for additional information about configuring shmmax.

This parameter defines the maximum size in bytes of a single shared memory segment that a Linux process can allocate in its virtual address space.
when tying to allocate SGA size bigger than SHMMAX the error below occur.
“SGA size con not be greater than maximum shared memory segment size (shmmax value)”

We neend to change shmamx value as oracle recommendations.

check current size of

 
[root@server01 ~]# cat /proc/sys/kernel/shmmax
68719476736

my total memory is 512 GB and hmmax would be 250GB.
change kernel.shmmax line as 268435456000 in /etc/sysctl.conf file.
and applying changes with command

sysctl -p

then check it with

 [root@server01 ~]# cat /proc/sys/kernel/shmmax
268435456000

SHMMNI: parameter recommended as 4096.

SHMALL: parameter recommended as %40 size of physical memory in pages.

Note: If the server supports multiple databases, or uses a large SGA, then set this parameter to a value that is equal to the total amount of shared memory, in 4K pages, that the system can use at one time.

check default page size:

[root@toryum01 ~]# getconf PAGE_SIZE
4096

Create AWR for PDB on 12.2

step : set awr_pdb_autoflush_enabled value as “true”.
The default value of AWR_PDB_AUTOFLUSH_ENABLED is false. Thus, by default, automatic AWR snapshots are disabled for all the PDBs in a CDB.

When you change the value of AWR_PDB_AUTOFLUSH_ENABLED in the CDB root, the new value takes effect in all the PDBs in the CDB.

Therefore, if you change the value of AWR_PDB_AUTOFLUSH_ENABLED in the CDB root to true, the value of AWR_PDB_AUTOFLUSH_ENABLED is also changed to true in all of the PDBs, so that automatic AWR snapshots are enabled for all the PDBs.

You can also change the value of AWR_PDB_AUTOFLUSH_ENABLED in any of the individual PDBs in a CDB, and the value that is set for each individual PDB will be honored. This enables you to enable or disable automatic AWR snapshots for individual PDBs.

When a new PDB is created, or a PDB from a previous database release is upgraded to the current database release, automatic AWR snapshots are enabled or disabled for the PDB based on the current value of AWR_PDB_AUTOFLUSH_ENABLED in the root.

for specified PDB

alter session set container=PDBtest;
alter system set awr_pdb_autoflush_enabled=true;

for all pdbs and CDB

alter session set container=CDB$ROOT;
alter system set awr_pdb_autoflush_enabled=true;

step: set interval and retention period by PDB or CDB$ROOT level. for all PDBSs an dCDB we have to run script separately. for 45 days retention and 30minutes interval script can be below.

alter session set container=CDB$ROOT;
execute dbms_workload_repository.modify_snapshot_settings(interval => 30, retention=>64800);	
alter session set container=PDB1;                                 
execute dbms_workload_repository.modify_snapshot_settings(interval => 30, retention=>64800);
alter session set container=PDB2;                                 
execute dbms_workload_repository.modify_snapshot_settings(interval => 30, retention=>64800);	
alter session set container=PDB3;                                 
execute dbms_workload_repository.modify_snapshot_settings(interval => 30, retention=>64800);	
alter session set container=PDB4;                                 

step:if you have many instances or PDBS, you can set AWR_SNAPSHOT_TIME_OFFSET parameter bigger than 0. default value is 0. to avoid CPU problems this parameter can be set in CDB.

AWR snapshots normally start at the top of the hour (12:00, 1:00, 2:00, and so on). This parameter allows DBAs to specify an offset for the AWR snapshot start time.

This is a useful parameter to avoid CPU spikes from multiple instances all starting their AWR snapshots at the same time. If you have a large system with many instances on it (like many Exadata installations), and you are experiencing such CPU spikes, this parameter can be very useful.

The parameter is specified in seconds. Normally, you set it to a value less than 3600. If you set the special value 1000000 (1,000,000), you get an automatic mode, in which the offset is based on the database name.

The automatic mode is an effective way of getting a reasonable distribution of offset times when you have a very large number of instances running on the same node.

alter session set container=CDB$ROOT;
alter system set AWR_SNAPSHOT_TIME_OFFSET=3600 scope=both;

step: create snapshot for PDB

alter session set container=PDB1;
SQL> exec dbms_workload_repository.create_snapshot();
SQL> exec dbms_workload_repository.create_snapshot();
SQL> exec dbms_workload_repository.create_snapshot();
SQL>select * from awr_pdb_snapshot;

step:create awr report.

SQL>@?/rdbms/admin/awrrpt

Specify the location of AWR Data
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
AWR_ROOT – Use AWR data from root (default)
AWR_PDB – Use AWR data from PDB

...

RMAN-06091: no channel allocated for maintenance (of an appropriate type)

When backing up archive logs, RMAN script ended with below errors. actually backup finished successfully. but when delete backed up acrchive logs, RMAN ended with error because of obsolete backups.

RMAN-03002: failure of delete command at 09/04/2018 14:15:18
RMAN-06091: no channel allocated for maintenance (of an appropriate type)

delete obsolete backups can resolve problem.

solution for tape backups:

RMAN> allocate channel for maintenance device type 'sbt_tape' PARMS 'ENV=(NSR_SERVER=, NSR_CLIENT=)';

RMAN> delete obsolete;

RMAN> crosscheck archivelog all;

RMAN> delete archivelog all backed up 2 times to sba_tape;

RMAN> crosscheck archivelog all;

solution for disk backups:

RMAN> allocate channel for maintenance device type disk;

RMAN> delete obsolete device type disk;

RMAN> crosscheck archivelog all;

RMAN> delete archivelog all backed up 2 times to sba_tape;

RMAN> crosscheck archivelog all;

Encountered RMAN-03002 and RMAN-06091 when Deleting Obsolete Backups (Doc ID 567555.1

Automatically open pdbs after OS reboot or db restart

PDB’s state can be mounted after reboot

SQL> show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  	READ ONLY  NO
	 3 PDB1			  		READ WRITE NO
	 4 PDB2			  		MOUNTED NO
	 5 PDB3			  		MOUNTED NO
	 

SQL> select con_name,state,instance_name from dba_pdb_saved_states;

CON_NAME				 STATE		INSTANCE_NAME
---------------------------------------- -------------- ----------------------------------------
PDB1				 OPEN		CDBCONS1
PDB1				 OPEN		CDBCONS2

2 rows selected.

we can configure to open automatically with “save state” direction.
run “alter pluggable database all save state” comand on all nodes.
firstly, open all mounted PDBS.

SQL> alter pluggable database PDB2 open instances=all;

Pluggable database altered.

SQL> alter pluggable database PDB2 open instances=all;

Pluggable database altered.


SQL>  alter pluggable database all save state;

Pluggable database altered.

SQL> select CON_NAME,STATE,INSTANCE_NAME from dba_pdb_saved_states;

CON_NAME				 STATE		INSTANCE_NAME
---------------------------------------- -------------- ----------------------------------------
PDB1				 OPEN		CDBCONS1
PDB2 				 OPEN		CDBCONS1
PDB3				 OPEN		CDBCONS1
PDB1				 OPEN		CDBCONS2
PDB2 				 OPEN		CDBCONS2
PDB3				 OPEN		CDBCONS2


6 rows selected.

after reboot, all PDBs will start with open state.

SQL> show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  	READ ONLY  NO
	 3 PDB1			  		READ WRITE NO
	 4 PDB2			  		READ WRITE NO
	 5 PDB3			  		READ WRITE NO

Restore pluggable database(PDB) with Point in time Recovery(PITR) after a table drop from RMAN full CDB backup

scenario : restore pluggable database(PDB) with Point in time Recovery(PITR) after a table drop from RMAN full CDB backup

SQL> show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 PDB5 			  MOUNTED
	 4 PDB3 			  READ WRITE NO
	 5 PDB1 			  READ WRITE NO

	 
SQL> select name, con_id from v$active_services order by 1;
	 NAME								     CON_ID
------------------------------------------ ----------
ORCLP9									  		1
ORCLP9XDB								  		1
SYS$BACKGROUND								  	1
SYS$USERS								  		1
pdb1									  		5
pdb3									  		4
pdb5									  		3

Create a table on PDB1. We will test on PDB1.

SQL> alter session set container=PDB1;

Session altered.

SQL> create table t1 (id number);

Table created.

SQL> insert into t1 values (1);

1 row created.

SQL> commit;

Commit complete.

SQL> select * from t1;

	ID
----------
	 1

switch logfiles 2-3 times.

SQL> alter session set  container=CDB$ROOT;

Session altered.

SQL> select GROUP#, THREAD#, SEQUENCE#, ARCHIVED,STATUS, FIRST_CHANGE#, NEXT_CHANGE#, CON_ID from v$log;

    GROUP#    THREAD#  SEQUENCE# ARC STATUS	      FIRST_CHANGE# NEXT_CHANGE#     CON_ID
---------- ---------- ---------- --- ---------------- ------------- ------------ ----------
	 1	    1	      61 YES INACTIVE		    4754097	 4755650	  0
	 2	    1	      62 NO  CURRENT		    4755650   1.8447E+19	  0
	 3	    1	      60 YES INACTIVE		    4754032	 4754097	  0

SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;

System altered.

SQL> select GROUP#, THREAD#, SEQUENCE#, ARCHIVED,STATUS, FIRST_CHANGE#, NEXT_CHANGE#, CON_ID from v$log;

    GROUP#    THREAD#  SEQUENCE# ARC STATUS	      FIRST_CHANGE# NEXT_CHANGE#     CON_ID
---------- ---------- ---------- --- ---------------- ------------- ------------ ----------
	 1	    1	      61 YES INACTIVE		    4754097	 4755650	  0
	 2	    1	      62 YES ACTIVE		    4755650	 4771864	  0
	 3	    1	      63 NO  CURRENT		    4771864   1.8447E+19	  0

SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;

System altered.

SQL> select GROUP#, THREAD#, SEQUENCE#, ARCHIVED,STATUS, FIRST_CHANGE#, NEXT_CHANGE#, CON_ID from v$log;

    GROUP#    THREAD#  SEQUENCE# ARC STATUS	      FIRST_CHANGE# NEXT_CHANGE#     CON_ID
---------- ---------- ---------- --- ---------------- ------------- ------------ ----------
	 1	    1	      64 NO  CURRENT		    4771908   1.8447E+19	  0
	 2	    1	      62 YES ACTIVE		    4755650	 4771864	  0
	 3	    1	      63 YES ACTIVE		    4771864	 4771908	  0

SQL>

before test, I deleted all backups and then create a full+archivelog backup.

rman target /;

RMAN> report need backup;

RMAN retention policy will be applied to the command
RMAN retention policy is set to redundancy 1
Report of files with less than 1 redundant backups
File #bkps Name
---- ----- -----------------------------------------------------
1    0     +DATA/ORCLP9/DATAFILE/system.309.980703899
3    0     +DATA/ORCLP9/DATAFILE/sysaux.310.980703965
4    0     +DATA/ORCLP9/DATAFILE/undotbs1.311.980703999
5    0     +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/system.319.980704091
6    0     +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/sysaux.318.980704091
7    0     +DATA/ORCLP9/DATAFILE/users.312.980704001
8    0     +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/undotbs1.320.980704091
17   0     +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/system.325.980790451
18   0     +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/sysaux.323.980790451
19   0     +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/undotbs1.324.980790451
20   0     /postgres/oradata/users01.dbf
28   0     +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/system.329.981991625
29   0     +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/sysaux.330.981991625
30   0     +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/undotbs1.328.981991625
31   0     +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/system.333.982596975
32   0     +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/sysaux.334.982596975
33   0     +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/undotbs1.332.982596975
34   0     +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/users.336.982596993

RMAN> list backup;

specification does not match any backup in the repository

RMAN> report schema;

Report of database schema for database with db_unique_name ORCLP9

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    840      SYSTEM               YES     +DATA/ORCLP9/DATAFILE/system.309.980703899
3    1120     SYSAUX               NO      +DATA/ORCLP9/DATAFILE/sysaux.310.980703965
4    90       UNDOTBS1             YES     +DATA/ORCLP9/DATAFILE/undotbs1.311.980703999
5    250      PDB$SEED:SYSTEM      NO      +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/system.319.980704091
6    330      PDB$SEED:SYSAUX      NO      +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/sysaux.318.980704091
7    5        USERS                NO      +DATA/ORCLP9/DATAFILE/users.312.980704001
8    100      PDB$SEED:UNDOTBS1    NO      +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/undotbs1.320.980704091
17   420      PDB5:SYSTEM          NO      +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/system.325.980790451
18   600      PDB5:SYSAUX          NO      +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/sysaux.323.980790451
19   150      PDB5:UNDOTBS1        NO      +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/undotbs1.324.980790451
20   28       PDB5:USERS           NO      /postgres/oradata/users01.dbf
28   260      PDB3:SYSTEM          YES     +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/system.329.981991625
29   410      PDB3:SYSAUX          NO      +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/sysaux.330.981991625
30   100      PDB3:UNDOTBS1        YES     +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/undotbs1.328.981991625
31   250      PDB1:SYSTEM          YES     +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/system.333.982596975
32   340      PDB1:SYSAUX          NO      +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/sysaux.334.982596975
33   100      PDB1:UNDOTBS1        YES     +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/undotbs1.332.982596975
34   5        PDB1:USERS           NO      +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/users.336.982596993

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    20       TEMP                 32767       +DATA/ORCLP9/TEMPFILE/temp.317.980704083
2    64       PDB$SEED:TEMP        32767       +DATA/ORCLP9/7042A561D8551781E055025056AA0FA0/TEMPFILE/temp.321.980704133
3    129      PDB5:TEMP            32767       +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/TEMPFILE/temp.326.980790453
4    131      PDB3:TEMP            32767       +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/TEMPFILE/temp.331.981991627
5    64       PDB1:TEMP            32767       +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/TEMPFILE/temp.335.982596979


RMAN> backup database plus archivelog format '/postgres/oradata/%d_full+arc_%T_%U.bak';


Starting backup at 27-JUL-18
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=62 RECID=62 STAMP=982597966
input archived log thread=1 sequence=63 RECID=63 STAMP=982598060
input archived log thread=1 sequence=64 RECID=64 STAMP=982598527
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/ORCLP9_full+arc_20180727_1pt92gs0_1_1.bak tag=TAG20180727T160207 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 27-JUL-18

Starting backup at 27-JUL-18
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00003 name=+DATA/ORCLP9/DATAFILE/sysaux.310.980703965
input datafile file number=00001 name=+DATA/ORCLP9/DATAFILE/system.309.980703899
input datafile file number=00004 name=+DATA/ORCLP9/DATAFILE/undotbs1.311.980703999
input datafile file number=00007 name=+DATA/ORCLP9/DATAFILE/users.312.980704001
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/1qt92gs3_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00029 name=+DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/sysaux.330.981991625
input datafile file number=00028 name=+DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/system.329.981991625
input datafile file number=00030 name=+DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/undotbs1.328.981991625
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/1rt92gsi_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00032 name=+DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/sysaux.334.982596975
input datafile file number=00031 name=+DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/system.333.982596975
input datafile file number=00033 name=+DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/undotbs1.332.982596975
input datafile file number=00034 name=+DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/users.336.982596993
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/1st92gt2_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00006 name=+DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/sysaux.318.980704091
input datafile file number=00005 name=+DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/system.319.980704091
input datafile file number=00008 name=+DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/undotbs1.320.980704091
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/1tt92gth_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:15
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00018 name=+DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/sysaux.323.980790451
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/1ut92gu0_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00017 name=+DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/system.325.980790451
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/1vt92gu8_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00020 name=/postgres/oradata/users01.dbf
input datafile file number=00019 name=+DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/undotbs1.324.980790451
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/20t92guf_1_1 tag=TAG20180727T160210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
Finished backup at 27-JUL-18

Starting backup at 27-JUL-18
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=65 RECID=65 STAMP=982598615
channel ORA_DISK_1: starting piece 1 at 27-JUL-18
channel ORA_DISK_1: finished piece 1 at 27-JUL-18
piece handle=/postgres/oradata/ORCLP9_full+arc_20180727_21t92guo_1_1.bak tag=TAG20180727T160335 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 27-JUL-18

Starting Control File and SPFILE Autobackup at 27-JUL-18
piece handle=/postgres/oradata/cnt_bck_c-3719526396-20180727-05 comment=NONE
Finished Control File and SPFILE Autobackup at 27-JUL-18

before drop table we can check date and time. then drop the table T1.

SQL> alter session set nls_Date_format='dd.mm.yyyy HH24:MI:SS';

Session altered.

SQL> select  sysdate from dual;

SYSDATE
------------------
27.07.2018 16:11:08

SQL> show con_id con_name

CON_ID
------------------------------
5

CON_NAME
------------------------------
PDB1

SQL> drop table t1;

Table dropped.

SQL> select  * from t1;
select	* from t1
               *
ERROR at line 1:
ORA-00942: table or view does not exist

the table was deleted. before restore we neet to close PDB.

SQL> alter session set container=cdb$root;

Session altered.

SQL> ALTER PLUGGABLE DATABASE PDB1 CLOSE;

Pluggable database altered.

in “list backup” output we can see that if PDB1 has backupslike “Container ID: 5, PDB Name: PDB1”

RMAN> list backup;


List of Backup Sets
===================


BS Key  Size       Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
56      34.41M     DISK        00:00:00     27-JUL-18      
        BP Key: 59   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160207
        Piece Name: /postgres/oradata/ORCLP9_full+arc_20180727_1pt92gs0_1_1.bak

  List of Archived Logs in backup set 56
  Thrd Seq     Low SCN    Low Time  Next SCN   Next Time
  ---- ------- ---------- --------- ---------- ---------
  1    62      4755650    27-JUL-18 4771864    27-JUL-18
  1    63      4771864    27-JUL-18 4771908    27-JUL-18
  1    64      4771908    27-JUL-18 4772508    27-JUL-18

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
57      Full    1.63G      DISK        00:00:10     27-JUL-18      
        BP Key: 60   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/1qt92gs3_1_1
  List of Datafiles in backup set 57
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  1       Full 4772515    27-JUL-18              NO    +DATA/ORCLP9/DATAFILE/system.309.980703899
  3       Full 4772515    27-JUL-18              NO    +DATA/ORCLP9/DATAFILE/sysaux.310.980703965
  4       Full 4772515    27-JUL-18              NO    +DATA/ORCLP9/DATAFILE/undotbs1.311.980703999
  7       Full 4772515    27-JUL-18              NO    +DATA/ORCLP9/DATAFILE/users.312.980704001

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
58      Full    526.71M    DISK        00:00:08     27-JUL-18      
        BP Key: 61   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/1rt92gsi_1_1
  List of Datafiles in backup set 58
  Container ID: 4, PDB Name: PDB3
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  28      Full 4772523    27-JUL-18              NO    +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/system.329.981991625
  29      Full 4772523    27-JUL-18              NO    +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/sysaux.330.981991625
  30      Full 4772523    27-JUL-18              NO    +DATA/ORCLP9/715A4B588571D86FE055025056AA0FA0/DATAFILE/undotbs1.328.981991625

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59      Full    457.20M    DISK        00:00:08     27-JUL-18      
        BP Key: 62   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/1st92gt2_1_1
  List of Datafiles in backup set 59
  Container ID: 5, PDB Name: PDB1
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  31      Full 4772538    27-JUL-18              NO    +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/system.333.982596975
  32      Full 4772538    27-JUL-18              NO    +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/sysaux.334.982596975
  33      Full 4772538    27-JUL-18              NO    +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/undotbs1.332.982596975
  34      Full 4772538    27-JUL-18              NO    +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/users.336.982596993

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
60      Full    473.54M    DISK        00:00:07     27-JUL-18      
        BP Key: 63   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/1tt92gth_1_1
  List of Datafiles in backup set 60
  Container ID: 2, PDB Name: PDB$SEED
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  5       Full 1420301    05-JUL-18              NO    +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/system.319.980704091
  6       Full 1420301    05-JUL-18              NO    +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/sysaux.318.980704091
  8       Full 1420301    05-JUL-18              NO    +DATA/ORCLP9/4700A987085B3DFAE05387E5E50A8C7B/DATAFILE/undotbs1.320.980704091

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
61      Full    463.36M    DISK        00:00:05     27-JUL-18      
        BP Key: 64   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/1ut92gu0_1_1
  List of Datafiles in backup set 61
  Container ID: 3, PDB Name: PDB5
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  18      Full 4755463    27-JUL-18              NO    +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/sysaux.323.980790451

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
62      Full    327.40M    DISK        00:00:06     27-JUL-18      
        BP Key: 65   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/1vt92gu8_1_1
  List of Datafiles in backup set 62
  Container ID: 3, PDB Name: PDB5
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  17      Full 4755489    27-JUL-18              NO    +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/system.325.980790451

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
63      Full    123.61M    DISK        00:00:07     27-JUL-18      
        BP Key: 66   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160210
        Piece Name: /postgres/oradata/20t92guf_1_1
  List of Datafiles in backup set 63
  Container ID: 3, PDB Name: PDB5
  File LV Type Ckp SCN    Ckp Time  Abs Fuz SCN Sparse Name
  ---- -- ---- ---------- --------- ----------- ------ ----
  19      Full 4755501    27-JUL-18              NO    +DATA/ORCLP9/7056BE3D0229AD39E055025056AA0FA0/DATAFILE/undotbs1.324.980790451
  20      Full 4755501    27-JUL-18              NO    /postgres/oradata/users01.dbf

BS Key  Size       Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
64      31.00K     DISK        00:00:00     27-JUL-18      
        BP Key: 67   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160335
        Piece Name: /postgres/oradata/ORCLP9_full+arc_20180727_21t92guo_1_1.bak

  List of Archived Logs in backup set 64
  Thrd Seq     Low SCN    Low Time  Next SCN   Next Time
  ---- ------- ---------- --------- ---------- ---------
  1    65      4772508    27-JUL-18 4772562    27-JUL-18

BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
65      Full    17.94M     DISK        00:00:01     27-JUL-18      
        BP Key: 68   Status: AVAILABLE  Compressed: NO  Tag: TAG20180727T160337
        Piece Name: /postgres/oradata/cnt_bck_c-3719526396-20180727-05
  SPFILE Included: Modification time: 27-JUL-18
  SPFILE db_unique_name: ORCLP9
  Control File Included: Ckp SCN: 4772583      Ckp time: 27-JUL-18
  
  

now we can start to point in time recovery.
If you do not use a fast recovery area, then you must specify the temporary location of the auxiliary set files by using the AUXILIARY DESTINATION clause.

“RECOVER pluggable database pdb1 auxiliary destination=’/oradata/PDB1/reco’;”

 RMAN>  run
 {
  SET UNTIL TIME "TO_DATE('27.07.2018 16:11:08','DD.MM.YYYY HH24:MI:SS')";
  RESTORE pluggable database pdb1;
  RECOVER pluggable database pdb1;
 }

executing command: SET until clause

Starting restore at 27-JUL-18
using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00031 to +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/system.333.982596975
channel ORA_DISK_1: restoring datafile 00032 to +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/sysaux.334.982596975
channel ORA_DISK_1: restoring datafile 00033 to +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/undotbs1.332.982596975
channel ORA_DISK_1: restoring datafile 00034 to +DATA/ORCLP9/71FB5B776FCE8704E055025056AA0FA0/DATAFILE/users.336.982596993
channel ORA_DISK_1: reading from backup piece /postgres/oradata/1st92gt2_1_1
channel ORA_DISK_1: piece handle=/postgres/oradata/1st92gt2_1_1 tag=TAG20180727T160210
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:03
Finished restore at 27-JUL-18

Starting recover at 27-JUL-18
current log archived
using channel ORA_DISK_1


starting media recovery
media recovery complete, elapsed time: 00:00:01

Finished recover at 27-JUL-18


RMAN> ALTER PLUGGABLE DATABASE pdb1 OPEN RESETLOGS;

Statement processed

Is our table restored?

SQL> alter session set container=PDB1;

Session altered.


SQL> select  * from t1;

	ID
----------
	 1

Bingo! Table T1 has recovered

we can see new incarnation because of restlog.

SQL> select * from V$PDB_INCARNATION;

DB_INCARNATION# PDB_INCARNATION# STATUS  INCARNATION_SCN INCARNATI BEGIN_RESETLOGS_SCN BEGIN_RES END_RESETLOGS_SCN END_RESET PRIOR_DB_INCARNATION# PRIOR_PDB_INCARNATION#		    FLA     CON_ID
--------------- ---------------- ------- --------------- --------- ------------------- --------- ----------------- --------- --------------------- ---------------------------------------- --- ----------
	      2 	       0 CURRENT	 1408558 05-JUL-18	       1408558 05-JUL-18	   1408558 05-JUL-18			 1 0					    YES 	 1
	      1 	       0 PARENT 	       1 26-JAN-17		     1 26-JAN-17		 1 26-JAN-17			 0					    YES 	 1
	      2 	       0 CURRENT	 1408558 05-JUL-18	       1408558 05-JUL-18	   1408558 05-JUL-18			 1 0					    YES 	 2
	      1 	       0 PARENT 	       1 26-JAN-17		     1 26-JAN-17		 1 26-JAN-17			 0					    YES 	 2
	      2 	       0 CURRENT	 1408558 05-JUL-18	       1408558 05-JUL-18	   1408558 05-JUL-18			 1 0					    YES 	 3
	      1 	       0 PARENT 	       1 26-JAN-17		     1 26-JAN-17		 1 26-JAN-17			 0					    YES 	 3
	      2 	       0 CURRENT	 1408558 05-JUL-18	       1408558 05-JUL-18	   1408558 05-JUL-18			 1 0					    YES 	 4
	      1 	       0 PARENT 	       1 26-JAN-17		     1 26-JAN-17		 1 26-JAN-17			 0					    YES 	 4
	      2 	       1 CURRENT	 4773185 27-JUL-18	       4773822 27-JUL-18	   4773822 27-JUL-18			 2 0					    YES 	 5
	      2 	       0 PARENT 	 1408558 05-JUL-18	       1408558 05-JUL-18	   1408558 05-JUL-18			 0					    YES 	 5

10 rows selected.

Linux disk wait time

Linux disk latency

–when disk red-write latency problems occurs, we need to understand the problem side. It depends on os site, network side or storage side.
–in this blog I will show you how to check latency for an ASM disk, currently and historicaly.

–get name of the disk which we interested.

	SQL> column path format a40
	SQL> select GROUP_NUMBER, DISK_NUMBER, MOUNT_STATUS, HEADER_STATUS, STATE, PATH,label from v$asm_disk where group_number=9

	GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE    PATH					
	------------ ----------- ------- ------------ -------- ---------------------------------------- 
		   9	       0 CACHED  MEMBER       NORMAL   /dev/mapper/asmdata76

–for historical monitor, we need major-minur numbers.

[oracle@db_host sa]$ ll /dev/mapper/asmdata76
lrwxrwxrwx. 1 root root 8 Mar  2 12:01 /dev/mapper/asmdata76 -> ../dm-86

[oracle@toryum01 sa]$ ll /dev/dm-86
brw-rw----. 1 oracle dba 253, 86 Mar  2 12:02 /dev/dm-86

–major-minor number can be seen in ll /dev/dm-86 output. 253-86 is major-minor numbers for ASM disk refer to /dev/mapper/asmdata76

–now we can monitor the disk with sar comamnd’s historical logs.
–in /var/log/sa, logs createt daily for sar command.

[oracle@toryum01 sa]$  sar -d -f sa01 | egrep "DEV|253-86" | more
12:00:01 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
12:10:02 AM dev253-86     17.83   4926.50     60.83    279.64      0.02      1.03      0.33      0.59
12:20:01 AM dev253-86      7.70    193.56     41.60     30.54      0.01      0.70      0.67      0.51
12:30:01 AM dev253-86      7.71    157.58     41.37     25.81      0.00      0.59      0.56      0.43
12:40:01 AM dev253-86     16.92   4935.31     40.52    294.11      0.02      1.10      0.35      0.59
12:50:01 AM dev253-86      7.46    190.12     39.54     30.77      0.00      0.50      0.49      0.37
01:00:01 AM dev253-86      7.19    143.56     39.72     25.50      0.00      0.41      0.40      0.29
01:10:01 AM dev253-86     18.10   4949.05     62.60    276.84      0.02      1.18      0.47      0.84
01:20:02 AM dev253-86      7.48    188.09     41.25     30.64      0.00      0.62      0.61      0.46
01:30:01 AM dev253-86      7.72    160.01     40.08     25.91      0.00      0.30      0.30      0.23


[oracle@toryum01 sa]$  sar -d -f sa02 | egrep "DEV|253-86" | more
12:00:01 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
12:10:01 AM dev253-86     17.60   4903.26     58.58    281.87      0.02      1.22      0.43      0.76
12:20:02 AM dev253-86      7.50    189.63     38.61     30.45      0.00      0.39      0.37      0.28
12:30:01 AM dev253-86      7.80    160.11     41.46     25.85      0.00      0.51      0.49      0.38
12:40:01 AM dev253-86     16.85   4923.18     41.11    294.59      0.02      1.10      0.39      0.65
12:50:02 AM dev253-86      7.55    192.72     39.25     30.72      0.00      0.42      0.41      0.31
01:00:01 AM dev253-86      7.16    144.09     39.04     25.59      0.00      0.63      0.61      0.43
01:10:01 AM dev253-86     17.89   4957.48     62.35    280.54      0.02      1.06      0.35      0.63
01:20:01 AM dev253-86      7.45    191.03     38.16     30.75      0.00      0.59      0.51      0.38
01:30:01 AM dev253-86      7.59    157.50     39.93     26.01      0.00      0.59      0.59      0.44

–for real time monitoring

[oracle@db_host sa]$ iostat -x 2 dm-86
Linux 2.6.32-696.3.1.el6.x86_64 (toryum01) 	03/02/2018 	_x86_64_	(32 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.94    0.00    2.98    0.51    0.00   92.57

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
dm-86             0.03     0.00    6.79    2.97  1276.77    45.23   135.45     0.01    0.75    0.89    0.42   0.39   0.38

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           6.00    0.00    2.68    0.14    0.00   91.17

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
dm-86             0.00     0.00    5.50    4.00   176.00    40.50    22.79     0.00    0.16    0.09    0.25   0.11   0.10

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           5.67    0.00    2.02    0.06    0.00   92.25

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
dm-86             0.00     0.00    4.00    1.50   128.00    32.50    29.18     0.00    0.18    0.12    0.33   0.18   0.10

–await is system+storage service time
–svctm is storage service time

Try to recover ASM disk

Because of some errors ASM diskgroup con not mounted. errors are in ASM alert.log:

WARNING: Disk Group DATA containing spfile for this instance is not mounted
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "20" is missing from group number "2"
ERROR: ALTER DISKGROUP ALL MOUNT /* asm agent call crs *//* {0:0:2} */
SQL> ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:0:2} */
SUCCESS: ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:0:2} */
NOTE: diskgroup resource ora.ARCH.dg is online
NOTE: diskgroup resource ora.REDO1.dg is online
NOTE: diskgroup resource ora.REDO2.dg is online

–diskgroup and db resources can not get online.

[grid@sigmafulltdb ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ARCH.dg
               ONLINE  ONLINE       sigmafulltdb
ora.DATA.dg
               ONLINE  OFFLINE      sigmafulltdb
ora.LISTENER.lsnr
               ONLINE  ONLINE       sigmafulltdb
ora.REDO1.dg
               ONLINE  ONLINE       sigmafulltdb
ora.REDO2.dg
               ONLINE  ONLINE       sigmafulltdb
ora.asm
               ONLINE  ONLINE       sigmafulltdb             Started
ora.ons
               OFFLINE OFFLINE      sigmafulltdb
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       sigmafulltdb
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       sigmafulltdb
ora.test.db
      1        ONLINE  OFFLINE
ora.synotest.db
      1        ONLINE  OFFLINE
ora.maptest.db
      1        ONLINE  OFFLINE

it wouldn’t be possible to mount diskgroup either mount normal or mount force.

SQL> alter diskgroup data mount;
alter diskgroup data mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete

SQL> alter diskgroup data mount  force;
alter diskgroup data mount  force
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "20" is missing from group number "2"

Essentually, the disk 20 is member of ‘DATA’ diskgroup. but it seen as ‘candidate’

GROUP_NUMBER NAME                           DISK_NUMBER MOUNT_S HEADER_STATU STATE    NAME                           PATH
------------ ------------------------------ ----------- ------- ------------ -------- ------------------------------ ----------------------------------------
           0 DATA                                     0 CLOSED  MEMBER       NORMAL                                  /dev/sdaj
           0 DATA                                     1 CLOSED  MEMBER       NORMAL                                  /dev/sdao
           0 DATA                                    28 CLOSED  MEMBER       NORMAL                                  /dev/sdd
           0 DATA                                    27 CLOSED  MEMBER       NORMAL                                  /dev/sdf
           0 DATA                                    26 CLOSED  MEMBER       NORMAL                                  /dev/sde
           0 DATA                                    25 CLOSED  MEMBER       NORMAL                                  /dev/sdg
           0 DATA                                    24 CLOSED  CANDIDATE    NORMAL                                  /dev/sdp
           0 DATA                                    23 CLOSED  MEMBER       NORMAL                                  /dev/sdt
           0 DATA                                    22 CLOSED  MEMBER       NORMAL                                  /dev/sdu
           0 DATA                                    21 CLOSED  MEMBER       NORMAL                                  /dev/sdw
           0 DATA                                    20 CLOSED  MEMBER       NORMAL                                  /dev/sdv
           0 DATA                                    19 CLOSED  CANDIDATE    NORMAL                                  /dev/sdx
           0 DATA                                     2 CLOSED  MEMBER       NORMAL                                  /dev/sdad
           0 DATA                                     3 CLOSED  MEMBER       NORMAL                                  /dev/sdaq
           0 DATA                                     4 CLOSED  MEMBER       NORMAL                                  /dev/sdak
           0 DATA                                     5 CLOSED  MEMBER       NORMAL                                  /dev/sdap
           0 DATA                                     6 CLOSED  MEMBER       NORMAL                                  /dev/sdan
           0 DATA                                     7 CLOSED  MEMBER       NORMAL                                  /dev/sdab
           0 DATA                                     8 CLOSED  MEMBER       NORMAL                                  /dev/sdam
           0 DATA                                     9 CLOSED  MEMBER       NORMAL                                  /dev/sdal
           0 DATA                                    10 CLOSED  MEMBER       NORMAL                                  /dev/sdai
           0 DATA                                    11 CLOSED  MEMBER       NORMAL                                  /dev/sdaf
           0 DATA                                    12 CLOSED  MEMBER       NORMAL                                  /dev/sdag
           0 DATA                                    13 CLOSED  MEMBER       NORMAL                                  /dev/sdah
           0 DATA                                    14 CLOSED  MEMBER       NORMAL                                  /dev/sdae
           0 DATA                                    15 CLOSED  CANDIDATE    NORMAL                                  /dev/sdy
           0 DATA                                    16 CLOSED  MEMBER       NORMAL                                  /dev/sdac
           0 DATA                                    17 CLOSED  CANDIDATE    NORMAL                                  /dev/sdaa
           0 DATA                                    18 CLOSED  CANDIDATE    NORMAL                                  /dev/sdz

–disk header is destroyed as I look into disk header.

[grid@sigmafulltdb ~]$ kfed read /dev/sdx
kfbh.endian:                          0 ; 0x000: 0x00
kfbh.hard:                            0 ; 0x001: 0x00
kfbh.type:                            0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt:                          0 ; 0x003: 0x00
kfbh.block.blk:                       0 ; 0x004: blk=0
kfbh.block.obj:                       0 ; 0x008: file=0
kfbh.check:                           0 ; 0x00c: 0x00000000
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
7EFF2AB01400 00000000 00000000 00000000 00000000  [................]
  Repeat 255 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]

"kfed repair" can not repair the disk.
[grid@sigmafulltdb ~]$ kfed repair /dev/sdx
KFED-00320: Invalid block num1 = [0], num2 = [1], error = [endian_kfbh]

The disks that have broken headers can be seen with kfod;

[grid@sigmafulltdb ~]$ kfod status=TRUE asm_diskstring='/dev/sd*' disk=all dscvgroup=TRUE
--------------------------------------------------------------------------------
 Disk          Size Header    Path                                     Disk Group   User     Group
================================================================================
   1:     204800 Mb CANDIDATE /dev/sdaa                                #            oracle   dba
   2:     204800 Mb MEMBER    /dev/sdab                                DATA         oracle   dba
   3:     204800 Mb MEMBER    /dev/sdac                                DATA         oracle   dba
   4:     204800 Mb MEMBER    /dev/sdad                                DATA         oracle   dba
   5:     204800 Mb MEMBER    /dev/sdae                                DATA         oracle   dba
   6:     204800 Mb MEMBER    /dev/sdaf                                DATA         oracle   dba
   7:     204800 Mb MEMBER    /dev/sdag                                DATA         oracle   dba
   8:     204800 Mb MEMBER    /dev/sdah                                DATA         oracle   dba
   9:     204800 Mb MEMBER    /dev/sdai                                DATA         oracle   dba
  10:     204800 Mb MEMBER    /dev/sdaj                                DATA         oracle   dba
  11:     204800 Mb MEMBER    /dev/sdak                                DATA         oracle   dba
  12:     204800 Mb MEMBER    /dev/sdal                                DATA         oracle   dba
  13:     204800 Mb MEMBER    /dev/sdam                                DATA         oracle   dba
  14:     204800 Mb MEMBER    /dev/sdan                                DATA         oracle   dba
  15:     204800 Mb MEMBER    /dev/sdao                                DATA         oracle   dba
  16:     204800 Mb MEMBER    /dev/sdap                                DATA         oracle   dba
  17:     204800 Mb MEMBER    /dev/sdaq                                DATA         oracle   dba
  18:     204800 Mb MEMBER    /dev/sdd                                 DATA         oracle   dba
  19:     204800 Mb MEMBER    /dev/sde                                 DATA         oracle   dba
  20:     204800 Mb MEMBER    /dev/sdf                                 DATA         oracle   dba
  21:     204800 Mb MEMBER    /dev/sdg                                 DATA         oracle   dba
  22:     204800 Mb MEMBER    /dev/sdh                                 ARCH         oracle   dba
  23:       4096 Mb MEMBER    /dev/sdi                                 REDO1        oracle   dba
  24:       4096 Mb MEMBER    /dev/sdj                                 REDO1        oracle   dba
  25:       4096 Mb MEMBER    /dev/sdk                                 REDO1        oracle   dba
  26:       4096 Mb MEMBER    /dev/sdl                                 REDO1        oracle   dba
  27:       4096 Mb MEMBER    /dev/sdm                                 REDO1        oracle   dba
  28:       4096 Mb MEMBER    /dev/sdn                                 REDO2        oracle   dba
  29:       4096 Mb MEMBER    /dev/sdo                                 REDO2        oracle   dba
  30:      65536 Mb CANDIDATE /dev/sdp                                 #            oracle   dba
  31:       4096 Mb MEMBER    /dev/sdq                                 REDO2        oracle   dba
  32:       4096 Mb MEMBER    /dev/sdr                                 REDO2        oracle   dba
  33:       4096 Mb MEMBER    /dev/sds                                 REDO2        oracle   dba
  34:     204800 Mb MEMBER    /dev/sdt                                 DATA         oracle   dba
  35:     204800 Mb MEMBER    /dev/sdu                                 DATA         oracle   dba
  36:     204800 Mb MEMBER    /dev/sdv                                 DATA         oracle   dba
  37:     204800 Mb MEMBER    /dev/sdw                                 DATA         oracle   dba
  38:     204800 Mb CANDIDATE /dev/sdx                                 #            oracle   dba
  39:     204800 Mb CANDIDATE /dev/sdy                                 #            oracle   dba
  40:     204800 Mb CANDIDATE /dev/sdz                                 #            oracle   dba
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
================================================================================
      +ASM /grid/11.2.0/grid

we can unerstand the problem with reading blocks from a normal disk and a broken disk.

[grid@sigmafulltdb ~]$ dd if=/dev/sdx of=/tmp/sdx.txt bs=8192 count=3
3+0 records in
3+0 records out
24576 bytes (25 kB) copied, 0.000737703 s, 33.3 MB/s
[grid@sigmafulltdb ~]$ dd if=/dev/sdw of=/tmp/sdw.txt bs=8192 count=3
3+0 records in
3+0 records out
24576 bytes (25 kB) copied, 0.00126345 s, 19.5 MB/s
[grid@sigmafulltdb ~]$
[grid@sigmafulltdb ~]$ od -c /tmp/sdw.txt | head -10
0000000 001 202 001 001  \0  \0  \0  \0 030  \0  \0 200   @   f   I 256
0000020 234 033 001  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000040   O   R   C   L   D   I   S   K  \0  \0  \0  \0  \0  \0  \0  \0
0000060  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000100  \0  \0      \v 030  \0 001 003   D   A   T   A   _   0   0   2
0000120   4  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000140  \0  \0  \0  \0  \0  \0  \0  \0   D   A   T   A  \0  \0  \0  \0
0000160  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000200  \0  \0  \0  \0  \0  \0  \0  \0   D   A   T   A   _   0   0   2
0000220   4  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
[grid@sigmafulltdb ~]$
[grid@sigmafulltdb ~]$
[grid@sigmafulltdb ~]$
[grid@sigmafulltdb ~]$ od -c /tmp/sdx.txt | head -10
0000000  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0060000

The header block do not have any data. There is 2 cases
1- /dev/sdx is really a new disk
2- the disk has been deleted, so disk can not be repaired. because header block deleted completely.

In our case head blocks destroyed completely, so disk couldn’t repaired.

create partiton,volume group, logical volume and mount point

RHEL 7 create new partition on one disk

[root@seyhgalip ~]# fdisk /dev/sdr
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xc79b3c02.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-62914559, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-62914559, default 62914559): 
Using default value 62914559
Partition 1 of type Linux and of size 30 GiB is set

Command (m for help): p

Disk /dev/sdr: 32.2 GB, 32212254720 bytes, 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xc79b3c02

   Device Boot      Start         End      Blocks   Id  System
/dev/sdr1            2048    62914559    31456256   83  Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris        
 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx         
 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data    
 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility   
 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt         
 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access     
 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O        
 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor      
 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs        
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT            
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor      
12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor      
14  Hidden FAT16 ❤ 61  SpeedStor       ab  Darwin boot     f2  DOS secondary  
16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS    
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE 
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep        
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT            
1e  Hidden W95 FAT1 80  Old Minix      
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sdr: 32.2 GB, 32212254720 bytes, 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xc79b3c02

   Device Boot      Start         End      Blocks   Id  System
/dev/sdr1            2048    62914559    31456256   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

we created parititon from disk. now we can create parititon volumne

[root@seyhgalip ~]# pvcreate /dev/sdr1
  Physical volume "/dev/sdr1" successfully created.
[root@seyhgalip ~]# pvdisplay

  --- NEW Physical volume ---
  PV Name               /dev/sdr1
  VG Name               
  PV Size               <30.00 GiB
  Allocatable           NO
  PE Size               0   
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               hXa5Bf-7uFI-3yMQ-67dB-oQOY-znXy-7E2qYa

next step is create volume group

[root@seyhgalip ~]# vgcreate vg_test /dev/sdr1
  Volume group "vg_test" successfully created

[root@seyhgalip ~]# vgdisplay 

  --- Volume group ---
  VG Name               vg_test
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <30.00 GiB
  PE Size               4.00 MiB
  Total PE              7679
  Alloc PE / Size       0 / 0   
  Free  PE / Size       7679 / <30.00 GiB
  VG UUID               d7eF3q-JCsZ-T1PC-aceJ-6Uwl-eJe5-a41oqI

next step is create logical volume.

[root@seyhgalip ~]# lvcreate --name lv_test -l 100%FREE vg_test
  Logical volume "lv_test" created.
[root@seyhgalip ~]# lvdisplay
   
  --- Logical volume ---
  LV Path                /dev/vg_test/lv_test
  LV Name                lv_test
  VG Name                vg_test
  LV UUID                wvQy39-MxeZ-HFl1-UWcK-PEPf-MpPz-NykTs7
  LV Write Access        read/write
  LV Creation host, time seyhgalip, 2018-02-16 10:20:03 +0300
  LV Status              available
  # open                 0
  LV Size                <30.00 GiB
  Current LE             7679
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     8192
  Block device           253:4

next step is formating logical volume for file system.

[root@seyhgalip ~]# mkfs.ext4 /dev/vg_test/lv_test 
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1966080 inodes, 7863296 blocks
393164 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2155872256
240 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done 

last step is create mount point and mount the LVM.

[root@seyhgalip ~]# mkdir -p /test
[root@seyhgalip ~]# 
[root@seyhgalip ~]# mount /dev/vg_
vg_default/  vg_oracle/   vg_postgres/ vg_test/     
[root@seyhgalip ~]# mount /dev/vg_test/lv_test /test
[root@seyhgalip ~]# 
[root@seyhgalip ~]# df -h
Filesystem                           Size  Used Avail Use% Mounted on
/dev/mapper/vg_default-root           52G  5.3G   47G  11% /
devtmpfs                              12G     0   12G   0% /dev
tmpfs                                 12G     0   12G   0% /dev/shm
tmpfs                                 12G   18M   12G   1% /run
tmpfs                                 12G     0   12G   0% /sys/fs/cgroup
/dev/sda1                            497M  199M  298M  41% /boot
tmpfs                                2.4G   12K  2.4G   1% /run/user/42
tmpfs                                2.4G     0  2.4G   0% /run/user/1001
tmpfs                                2.4G     0  2.4G   0% /run/user/0
/dev/mapper/vg_oracle-lv_oracle       59G   52M   57G   1% /oracle
/dev/mapper/vg_postgres-lv_postgres   30G   45M   28G   1% /postgres
/dev/mapper/vg_test-lv_test           30G   45M   28G   1% /test

in terms;
disk :/dev/sdr
parititon :/dev/sdr1
Physical volume: /dev/sdr1
volume group : vg_test
logical volume:lv_test
mount point:/test