Oracle Database Recommendations 19c

Email from Oracle Support - no doubt one every year from now on, ad infinitum (should not complain really keeps us DBAs in a job).




Dear Oracle Database Customer,

Oracle Database 19c is now available on all major platforms. Oracle will be supporting Oracle Database 19c (Long Term Support Release) with Oracle Premier Support through March 2023 and with Extended Support through March 2026.

In order to stay current, minimize future upgrades and take advantage of new features, we encourage you to upgrade to Oracle Database 19c without delay.

We recommend the following upgrade options:

If you are running Oracle Database 11.2.0.4 or 12.1.0.2, you need to maintain an Unlimited License Agreement that includes Extended Support, or pay for Extended Support fees and upgrade to Oracle Database 19c before patching ends on these releases.

--Oracle Database 11.2.0.4 (Terminal patch set release for 11.2, currently in Extended Support through December 2020)

--Oracle Database 12.1.0.2 (Terminal patch set release for 12.1, currently in Extended Support through July 2021 with a fee waiver available through July 2019)

If you are running Oracle Database 12.2.0.1 or 18c, you will need to upgrade to Oracle Database 19c before the error correction end dates are reached.

--Oracle Database 12.2.0.1 (Annual release with error correction ending July 2020)

--Oracle Database 18c (Annual release with error correction ending June 2021)

The purpose of these recommendations is to help you stay within the guidelines of Lifetime Support and Error Correction Policies (allowing you to avoid costly support exceptions), and to encourage you to take advantage of the new features that are available in the latest product releases.

For questions related to certification combinations and Lifetime Support agreements with other Oracle products like Oracle E-Business Suite, Oracle JD Edwards, Oracle PeopleSoft, and Oracle Siebel, please contact your Oracle Account Representative.

For questions about the Oracle Database 19c upgrade process, refer to the following MOS Community topic:

--Planning on or Currently Upgrading to 19c?

For additional information, refer to:

Swamped by audit files


Seeing massive numbers of .aud files on many of the server I am working on.

Uwe has beaten me to it :

https://uhesse.com/2015/07/24/audit_sys_operations-defaults-to-true-in-oracle-12c/

Many of the audit files relate to the dbms_stats job. Wondering if there is a way to somehow filter what oracle decides to audit; seems to be all or nothing.

Going to investigate purging using the link below, rather than manually deleting the files at the o/s level.

https://aprakash.wordpress.com/2012/01/20/use-dbms_audit_mgmt-for-purging-audit-files-at-os-level/

See also :

https://www.durham.computer/search?q=clearing

Cloning a pluggable database

Not going to reinvent the wheel, all the information you need here :

https://oracle-base.com/articles/12c/multitenant-hot-clone-remote-pdb-or-non-cdb-12cr2

Some quick notes:

SQL> select con_id, name, open_mode, RESTRICTED from v$pdbs where NAME like 'MDS5FA%';

CON_ID NAME OPEN_MODE RES
---------- ------------------------------ ---------- ---
34 MDS5FA1 READ WRITE NO
39 MDS5FA1_20190626 MOUNTED
47 MDS5FA1_2JUN019 MOUNTED

SQL> alter session set container = MDS5FA1;

Session altered.

SQL> show pdbs;

CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
34 MDS5FA1 READ WRITE NO

SQL> shutdown immediate;
Pluggable Database closed.

SQL> alter session set container = CDB$ROOT;
Session altered.

SQL> select con_id, name, open_mode, RESTRICTED from v$pdbs where NAME like 'MDS5FA%';

CON_ID NAME OPEN_MODE RES
---------- ------------------------------ ---------- ---
34 MDS5FA1 MOUNTED
39 MDS5FA1_20190626 MOUNTED
47 MDS5FA1_2JUN2019 MOUNTED

-- Use read only so no worries about date changes.

SQL> alter pluggable database MDS5FA1 open read only force;

Pluggable database altered.

SQL> create pluggable database MDS5FA1_10JUN2019 from MDS5FA1;


RAC Crib Sheet

Some quick notes on RAC cluster commands.

Also see: http://www.dba-oracle.com/real_application_clusters_rac_grid/srvctl.htm

[grid@PRSODB01 ~]$ crsctl check cluster -all
**************************************************************
prsodb01:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
prsodb02:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************


Example of RAC Cluster


[grid@PRSODB01 ~]$ crsctl status resource -t
--------------------------------------------------------------------------------
Name Target State Server State details
-------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE prsodb01 STABLE
ONLINE ONLINE prsodb02 STABLE
ora.FRA.dg
ONLINE ONLINE prsodb01 STABLE
ONLINE ONLINE prsodb02 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE prsodb01 STABLE
ONLINE ONLINE prsodb02 STABLE
ora.asm
ONLINE ONLINE prsodb01 Started,STABLE
ONLINE ONLINE prsodb02 Started,STABLE
ora.net1.network
ONLINE ONLINE prsodb01 STABLE
ONLINE ONLINE prsodb02 STABLE
ora.ons
ONLINE ONLINE prsodb01 STABLE
ONLINE ONLINE prsodb02 STABLE
--------------------------------------------------------------------------------

Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE prsodb01 STABLE
ora.prsodb01.vip
1 ONLINE ONLINE prsodb01 STABLE
ora.prsodb02.vip
1 ONLINE ONLINE prsodb02 STABLE
ora.cvu
1 ONLINE ONLINE prsodb02 STABLE
ora.oc4j
1 OFFLINE OFFLINE STABLE
ora.patches.db
1 OFFLINE OFFLINE Instance Shutdown,ST
ABLE
2 OFFLINE OFFLINE Instance Shutdown,ST
ABLE
ora.preprod.db
1 ONLINE ONLINE prsodb01 Open,STABLE
2 ONLINE ONLINE prsodb02 Open,STABLE
ora.preprod.preprod_lmsstp.scc.local.svc
1 ONLINE ONLINE prsodb01 STABLE
2 ONLINE ONLINE prsodb02 STABLE
ora.rpap_gdc2.db
1 OFFLINE OFFLINE Instance Shutdown,ST
ABLE
2 OFFLINE OFFLINE Instance Shutdown,ST
ABLE
ora.scan1.vip
1 ONLINE ONLINE prsodb01 STABLE
ora.sit.db
1 ONLINE ONLINE prsodb01 Open,STABLE
2 ONLINE OFFLINE STABLE
ora.sit.osmm.scc.local.svc
1 ONLINE ONLINE prsodb01 STABLE
--------------------------------------------------------------------------------

[grid@PRSODB01 ~]$ crsctl status server
NAME=prsodb01
STATE=ONLINE
NAME=prsodb02
STATE=ONLINE

[grid@PRSODB01 ~]$ crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

[grid@PRSODB01 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

[grid@PRSODB01 ~]$ crsctl query crs softwareversion
Oracle Clusterware version on node [prsodb01] is [12.1.0.1.0]

[oracle@PRSODB01 ~]$ srvctl status database -db SIT
Instance sit1 is running on node prsodb01
Instance sit2 is not running on node prsodb02

[oracle@PRSODB01 ~]$ srvctl status database -db PREPROD
Instance preprod1 is running on node prsodb01
Instance preprod2 is running on node prsodb02

[oracle@PRSODB01 ~]$ srvctl config nodeapps
Network 1 exists
Subnet IPv4: 192.168.98.0/255.255.255.192/eth0, static
Subnet IPv6:
VIP exists: network number 1, hosting node prsodb01
VIP Name: prsodb01-vip.scc.local
VIP IPv4 Address: 192.168.98.31
VIP IPv6 Address:
VIP exists: network number 1, hosting node prsodb02
VIP Name: prsodb02-vip.scc.local
VIP IPv4 Address: 192.168.98.32
VIP IPv6 Address:
ONS exists: Local port 6100, remote port 6200, EM port 2016

[oracle@PRSODB01 ~]$ srvctl config listener
Name: LISTENER
Network: 1, Owner: grid
Home:
End points: TCP:1800

[oracle@PRSODB01 ~]$ srvctl config asm
ASM home: /u01/app/12.1.0.1/grid
Password file: +DATA/orapwASM
ASM listener: LISTENER

[oracle@PRSODB01 ~]$ srvctl status asm
ASM is running on prsodb01,prsodb02

[oracle@PRSODB01 ~]$ srvctl –help

 -output removed for brevity but for reference included the command.

Check RAC related service status.
srvctl status -d DATABASE_UNIQUE_NAME
srvctl status nodeapps
srvctl status diskgroup -g DISKGROUP_NAME -a
srvctl status listener
srvctl status asm
Use config option can configure the related service. However, if we don’t provide any parameters, it would show the detail for the service.
srvctl config -d DATABASE_UNIQUE_NAME
srvctl config nodeapps
srvctl config listener
srvctl config asm
crsctl status resource -t
crsctl status server
crsctl check cluster
crsctl check crs
crsctl query css votedisk
crsctl querry crs activeversion
crsctl query crs releaseversion
crsctl query crs softwareversion
Read more at :



MobaXTerm download feature

Although I mainly use Putty as a terminal emulator (very small footprint), MobaXTerm is  very handy: https://mobaxterm.mobatek.net/download.html

One thing that can be very useful is the download option; saves using the likes of WinSCP or Filezilla. Thought I would stick this on here as easy to forget about this feature.


Just right click on the file you need and download; you can cut & paste the directory you require into the small box (i.e replace /home/oracle with whatever you need).

Rebuild indexes / partitions current user

--rebuild any indexes or index partitions for the current user
set serveroutput on
declare
  procedure ddl (str in varchar2) is
  begin
    execute immediate (str);
    dbms_output.put_line (str);
  end ddl;
begin
  for t in (select table_name from user_tables order by table_name) loop 
    for i in (
        select index_name, partition_name, 'partition' ddl_type
        from user_ind_partitions
        where (index_name) in
           ( select index_name
             from   user_indexes
             where  table_name  = t.table_name
           )
        and status = 'UNUSABLE'
        union all
        select index_name, subpartition_name, 'subpartition' ddl_type
        from user_ind_subpartitions
        where (index_name) in
           ( select index_name
             from   user_indexes
             where  table_name  = t.table_name
           )
        and status = 'UNUSABLE'
        union all
        select index_name, null, null
        from user_indexes
        where table_name  = t.table_name
        and status = 'UNUSABLE'
    )
    loop
      if i.ddl_type is null then
        ddl('alter index '||i.index_name||' rebuild');
      else
        ddl('alter index '||i.index_name||' rebuild '||i.ddl_type||' '||i.partition_name);
      end if;
    end loop;
  end loop;
end;
/

Patch Set Update 12.1.0.2 190416

Notes for the Oracle PSU for 12.1 (check carefully - very easy to download the 12.2 version). This is for a RAC one node server, please check the patch info when using RAC / Dataguard, as this process will differ.

See ➊ at bottom of post when using replication, will expand later.

Make sure you are using the latest Opatch, i.e. p6880880). See here :

Unzip the p6880880 patch (check latest patch downloaded) - ensure you have a valid backup of the Opatch directory :
ORA01:+ASM:gridhome>unzip p6880880_122010_Linux-x86-64.zip

This will create a new OPatch directory with new versions of OPatchauto and OPatch:

Do this on both the GI and rdbms home otherwise you will hit opatch mis-match errors when running combo patches.

Check the version is as expected - i.e. run $./opatch version

Now run opatch with the analyze option to confirm everything ok :

OPatch>./opatchauto apply /u99/media/29176115 -analyze
If the analyze completes ok against the GI and DB home then apply the patch :

Initially, I ran against the GI and then the DB Home but running against both homes is the better option.

Check you don't have any apps running. We had to shutdown Oracle Golden Gate and the EM Agent on this server; otherwise you will hit errors if processes are connection to Oracle and this will stop the patch in it's tracks. 

++++++++++++++

ORA01::OPatch>./opatchauto apply /u99/media/29176115
OPatchauto session is initiated at Thu Jun  6 01:05:19 2019

 System initialization log file is /u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/opatchautodb/systemconfig2019-06-06_01-05-21AM.log.
 Session log file is apr*

/u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/opatchauto/opatchauto2019-06-06_01-05-33AM.log
The id for this session is M521

 Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.1.0.2/dbhome
Patch applicability verified successfully on home /u01/app/oracle/product/12.1.0.2/dbhome

Verifying SQL patch applicability on home /u01/app/oracle/product/12.1.0.2/dbhome

SQL patch applicability verified successfully on home /u01/app/oracle/product/12.1.0.2/dbhome
 Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/grid/12.1.0.2/gridhome

Patch applicability verified successfully on home /u01/app/oracle/grid/12.1.0.2/gridhome
Preparing to bring down database service on home /u01/app/oracle/product/12.1.0.2/dbhome

Successfully prepared home /u01/app/oracle/product/12.1.0.2/dbhome to bring down database service
Bringing down database service on home /u01/app/oracle/product/12.1.0.2/dbhome

Following database has been stopped and will be restarted later during the session: oradb1,oradb2
Database service successfully brought down on home /u01/app/oracle/product/12.1.0.2/dbhome

 Bringing down CRS service on home /u01/app/oracle/grid/12.1.0.2/gridhome
Prepatch operation log file location: /u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/crsconfig/hapatch_2019-06-06_01-08-58AM.log

CRS service brought down successfully on home /u01/app/oracle/grid/12.1.0.2/gridhome
 Start applying binary patch on home /u01/app/oracle/product/12.1.0.2/dbhome

Binary patch applied successfully on home /u01/app/oracle/product/12.1.0.2/dbhome
 Start applying binary patch on home /u01/app/oracle/grid/12.1.0.2/gridhome

Binary patch applied successfully on home /u01/app/oracle/grid/12.1.0.2/gridhome
 Starting CRS service on home /u01/app/oracle/grid/12.1.0.2/gridhome

Postpatch operation log file location: /u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/crsconfig/hapatch_2019-06-06_01-17-34AM.log
CRS service started successfully on home /u01/app/oracle/grid/12.1.0.2/gridhome

Starting database service on home /u01/app/oracle/product/12.1.0.2/dbhome
Database service successfully started on home /u01/app/oracle/product/12.1.0.2/dbhome

Preparing home /u01/app/oracle/product/12.1.0.2/dbhome after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/12.1.0.2/dbhome
SQL patch applied successfully on home /u01/app/oracle/product/12.1.0.2/dbhome

 OPatchAuto successful.
--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:ora01

SIDB Home:/u01/app/oracle/product/12.1.0.2/dbhome
Version:12.1.0.2.0

Summary:
 ==Following patches were SKIPPED:

 Patch: /u99/media/29176115/26983807
Reason: This patch is not applicable to this specified target type - "oracle_database"

 Patch: /u99/media/29176115/29217637

Reason: This patch is not applicable to this specified target type - "oracle_database"
 ==Following patches were SUCCESSFULLY applied:

 Patch: /u99/media/29176115/29141015
Log:/u01/app/oracle/product/12.1.0.2/dbhome/cfgtoollogs/opatchauto/core/opatch/opatch2019-06-06_01-09-21AM_1.log

 Patch: /u99/media/apr2019_patches/29176115/29141168
Log:/u01/app/oracle/product/12.1.0.2/dbhome/cfgtoollogs/opatchauto/core/opatch/opatch2019-06-06_01-09-21AM_1.log

 Host:ora01
SIHA Home:/u01/app/oracle/grid/12.1.0.2/gridhome

Version:12.1.0.2.0
Summary:

 ==Following patches were SKIPPED:
 Patch: /u99/media/29176115/26983807

Reason: This patch is already been applied, so not going to apply again.
 ==Following patches were SUCCESSFULLY applied:

 Patch: /u99/media/29176115/29141015
Log:/u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/opatchauto/core/opatch/opatch2019-06-06_01-11-21AM_1.log

 Patch: /u99/media/29176115/29141168
Log:/u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/opatchauto/core/opatch/opatch2019-06-06_01-11-21AM_1.log

 Patch: /u99/media/29176115/29217637
Log:/u01/app/oracle/grid/12.1.0.2/gridhome/cfgtoollogs/opatchauto/core/opatch/opatch2019-06-06_01-11-21AM_1.log

OPatchauto session completed at Thu Jun  6 01:25:25 2019
Time taken to complete the session 20 minutes, 6 seconds
+++++++++++++++++++

Ran this sql to confirm the patch had completed :

select patch_id, status, action_time, description from dba_registry_sqlpatch

PATCH_ID STATUS ACTION_TIME DESCRIPTION

29141015 SUCCESS 06-JUN-19 01.20.44.735844 AM DATABASE PATCH SET UPDATE 12.1.0.2.190416

So far so good, next ran the patch 29251241:  OJVM PSU Patch

Run from the 29251241 patch directory.

+++++++++++++++++++

ORA01:29251241>$ORACLE_HOME/OPatch/opatch apply
Oracle Interim Patch Installer version 12.2.0.1.17
Copyright (c) 2019, Oracle Corporation.  All rights reserved.
Oracle Home       : /u01/app/oracle/product/12.1.0.2/dbhome
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.1.0.2/dbhome/oraInst.loc
OPatch version    : 12.2.0.1.17
OUI version       : 12.1.0.2.0
Log file location : /u01/app/oracle/product/12.1.0.2/dbhome/cfgtoollogs/opatch/opatch2019-06-06_01-47-07AM_1.log
Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   29251241
Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/12.1.0.2/dbhome')
Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '29251241' to OH '/u01/app/oracle/product/12.1.0.2/dbhome'
ApplySession: Optional component(s) [ oracle.sqlj, 12.1.0.2.0 ]  not present in the Oracle Home or a higher version is found.
Patching component oracle.javavm.server, 12.1.0.2.0...
Patching component oracle.javavm.server.core, 12.1.0.2.0...
Patching component oracle.rdbms.dbscripts, 12.1.0.2.0...
Patching component oracle.rdbms, 12.1.0.2.0...
Patching component oracle.javavm.client, 12.1.0.2.0...
Patching component oracle.dbjava.jdbc, 12.1.0.2.0...
Patching component oracle.dbjava.ic, 12.1.0.2.0...
Patch 29251241 successfully applied.
Sub-set patch [27475603] has become inactive due to the application of a super-set patch [29251241].
Please refer to Doc ID 2161861.1 for any possible further required actions.
Log file location: /u01/app/oracle/product/12.1.0.2/dbhome/cfgtoollogs/opatch/opatch2019-06-06_01-47-07AM_1.log
OPatch succeeded.
+++++++++++++++++++
Checked the dba_registry_sqlpatch table (see above) but nothing was reported for the OJVM patch. 
However, ran $./opatch lsinventory and the patch was listed - needed further investigation.
COL DBMS_JAVA.LONGNAME('OK') FORMAT a9

SELECT dbms_java.longname('OK') FROM dual;

-- OK Example Output

DBMS_JAVA
---------
OK

-- Not OK Example Output

ORA-29548: Java system class reported: release of Java system classes in the database
(12.1.0.2.181016 1.6) does not match that of the oracle executable (12.1.0.2.180717 1.6)

We hit the ORA-29548 issue - if you search on the web you will see instruction on rebuilding java and lots of other stuff - in this situation ignore!
The patch had worked on other environments without issue so knew it worked.


The oracle executable on the o/s is different to that on the database - as in the error.

So we ran the a ./datapatch verbose command from the OPatch directory for each of the databases in that home and et voilla!!

The java in the database was ok - SELECT dbms_java.longname('OK') FROM dual;

DBMS_JAVA.LONGNAME('OK')

OK✔

and the dba_registry_sqlpatch was reporting correctly :

PATCH_ID STATUS          ACTION_TIME                    DESCRIPTION

29141015 SUCCESS         06-JUN-19 01.20.44.735844 AM   DATABASE PATCH SET UPDATE 12.1.0.2.190416
29251241 SUCCESS         06-JUN-19 03.59.27.168461 AM   Database PSU 12.1.0.2.190416, Oracle JavaVM Component (APR2019)


There is a good note here :

https://mikedietrichde.com/2018/04/19/do-you-have-to-execute-datapatch-when-you-create-a-new-database/

So it looks like if you running 12.2 you probably don't need to run the datapatch verbose but for the patching on 12.1 above it looks like it is necessary.

We ran a full "shake down" and integrity test on the server and it is 100% but please. please, please check patches on a test / "sand box" / QA  environment before running on Production.
For standby first patching (see section1.25 in the link below) :






Also:


So you stop replication.
Patch the standby binaries but not the data patch.
Patch the primary , but not the data patch.
Start replication.
Apply the data patch to the primary , and this will get replicated to the standby.
++++++++++++++