Thursday, November 6, 2025

RMAN Pipe Interface & Concepts

RMAN Pipe Interface & Concepts

Alireza Kamrani

Using the RMAN Pipe Interface
The RMAN pipe interface is an alternative method for issuing commands to RMAN and receiving the output from those commands. Using this interface, it is possible to write a portable programmatic interface to RMAN.
With the pipe interface, RMAN obtains commands and sends output by using the DBMS_PIPE PL/SQL package instead of the operating system shell. The pipe interface is invoked by using the PIPE command-line parameter for the RMAN client. RMAN uses two private pipes: one for receiving commands and the other for sending output. The names of the pipes are derived from the value of the PIPE parameter. For example, you can invoke RMAN with the following command:

% rman PIPE abc TARGET /

RMAN opens the two pipes in the target database: ORA$RMAN_ABC_IN, which RMAN uses to receive user commands, and ORA$RMAN_ABC_OUT, which RMAN uses to send all output back to RMAN.
All messages on both the input and output pipes are of type VARCHAR2.

RMAN does not permit the pipe interface to be used with public pipes, because they are a potential security problem. With a public pipe, any user who knows the name of the pipe can send commands to RMAN and intercept its output.
If the pipes are not initialized, then RMAN creates them as private pipes. If you want to put commands on the input pipe before starting RMAN, you must first create the pipe by calling DBMS_PIPE.CREATE_PIPE. Whenever a pipe is not explicitly created as a private pipe, the first access to the pipe automatically creates it as a public pipe, and RMAN returns an error if it is told to use a public pipe.

Note:
If multiple RMAN sessions can run against the target database, then you must use unique pipe names for each RMAN session. The DBMS_PIPE.UNIQUE_SESSION_NAME function is one method that you can use to generate unique pipe names.

Possible used for the pipes interface are:
• Applications coded to automatically invoke one or more backup and recovery functions to achieve their design objectives
• Third party tools which whish to provide their own interface to RMAN
• DBA tools that wish to incorporate some RMAN functionality

Executing Multiple RMAN Commands in Succession Through a Pipe: Example

This example assumes that the application controlling RMAN wants to run multiple commands in succession. After each command is sent down the pipe and executed and the output returned, RMAN pauses and waits for the next command.
To execute RMAN commands through a pipe:
• Start RMAN by connecting to a target database (required) and specifying the PIPE option. For example, enter:

% rman PIPE abc TARGET /
You can also specify the TIMEOUT option, which forces RMAN to exit automatically if it does not receive any input from the input pipe in the specified number of seconds. For example, enter:

% rman PIPE abc TARGET / TIMEOUT 60
• Connect to the target database and put the desired commands on the input pipe by using DBMS_PIPE.PACK_MESSAGE and DBMS_PIPE.SEND_MESSAGE. In pipe mode, RMAN issues message RMAN-00572 when it is ready to accept input instead of displaying the standard RMAN prompt.
• Read the RMAN output from the output pipe by using DBMS_PIPE.RECEIVE_MESSAGE and DBMS_PIPE.UNPACK_MESSAGE.
• Repeat Steps 2 and 3 to execute further commands with the same RMAN instance that was started in Step 1.
• If you used the TIMEOUT option when starting RMAN, then RMAN terminates automatically after not receiving any input for the specified length of time. To force RMAN to terminate immediately, send the EXIT command.

Executing RMAN Commands in a Single Job Through a Pipe: Example

This example assumes that the application controlling RMAN wants to run one or more commands as a single job. After running the commands that are on the pipe, RMAN exits.
To execute RMAN commands in a single job through a pipe:
• After connecting to the target database, create a pipe (if it does not already exist under the name ORA$RMAN_pipe_IN).
• Put the desired commands on the input pipe. In pipe mode, RMAN issues message RMAN-00572 when it is ready to accept input instead of displaying the standard RMAN prompt.
• Start RMAN with the PIPE option, and specify TIMEOUT 0. For example, enter:

% rman PIPE abc TARGET / TIMEOUT 0

• RMAN reads the commands that were put on the pipe and executes them by using DBMS_PIPE.PACK_MESSAGE and DBMS_PIPE.SEND_MESSAGE. When it has exhausted the input pipe, RMAN exits immediately.
• Read RMAN output from the output pipe by using DBMS_PIPE.RECEIVE_MESSAGE and DBMS_PIPE.UNPACK_MESSAGE.


A Practical Scenario

The RMAN Pipe Interface allows Oracle sessions to send RMAN commands to a running RMAN process using Oracle’s internal DBMS_PIPE mechanism.

It’s mainly used for controlling RMAN from inside PL/SQL rather than through OS scripts.

So instead of typing commands in the RMAN CLI, you:
• Start RMAN with PIPE <name>
• Send commands into that pipe using DBMS_PIPE.PACK_MESSAGE and SEND_MESSAGE
• RMAN executes those commands
• Read results via DBMS_PIPE.RECEIVE_MESSAGE

Step 1: Start RMAN in Pipe Mode
From the OS:

$rman PIPE DEMO_PIPE  TARGET / TIMEOUT 600

This launches RMAN and creates two private Oracle pipes:

• ORA$RMAN_DEMO_PIPE_IN
• ORA$RMAN_DEMO_PIPE_OUT

RMAN will wait for incoming PL/SQL commands for up to 600 seconds.

You can confirm it’s waiting via:

SELECT sid, event FROM v$session_wait WHERE event LIKE '%pipe%';

You’ll see RMAN waiting on "pipe get handle" or similar events.

Step 2: Create the PL/SQL Procedure RMAN_CMD

CREATE OR REPLACE PROCEDURE rman_cmd(cmd VARCHAR2) AS
  in_pipe_name  VARCHAR2(2000) := 'ORA$RMAN_DEMO_PIPE_IN';
  out_pipe_name VARCHAR2(2000) := 'ORA$RMAN_DEMO_PIPE_OUT';
  v_info        VARCHAR2(255);
  v_status      INTEGER;
BEGIN
  DBMS_OUTPUT.PUT_LINE('Begin RMAN command: ' || cmd);

  -- Clean old pipe data
  v_status := DBMS_PIPE.RECEIVE_MESSAGE(out_pipe_name, 0);
  WHILE v_status = 0 LOOP
    DBMS_PIPE.UNPACK_MESSAGE(v_info);
    v_status := DBMS_PIPE.RECEIVE_MESSAGE(out_pipe_name, 10);
  END LOOP;

  -- Send the RMAN command
  DBMS_PIPE.PACK_MESSAGE(cmd);
  v_status := DBMS_PIPE.SEND_MESSAGE(in_pipe_name);

  -- Wait for RMAN output
  v_status := 0;
  WHILE v_status = 0 OR v_status = 1 LOOP
    v_status := DBMS_PIPE.RECEIVE_MESSAGE(out_pipe_name, 30);
    IF v_status = 0 THEN
      DBMS_PIPE.UNPACK_MESSAGE(v_info);
      DBMS_OUTPUT.PUT_LINE(v_info);
      IF v_info LIKE '%RMAN-00572%' OR v_info LIKE '%Recovery Manager complete%' THEN
        EXIT;
      END IF;
    END IF;
  END LOOP;
END;
/

This procedure:
• Cleans any stale messages from previous runs.
• Sends your command to RMAN.
• Reads back RMAN’s textual output line-by-line.
• Displays or logs it.

Step 3: Execute Real RMAN Commands from SQL*Plus

Now, in the same SQL*Plus session, you can send actual RMAN commands to the RMAN process:

EXEC rman_cmd('SHOW ALL;');
EXEC rman_cmd('CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 3 DAYS;');
EXEC rman_cmd('BACKUP SPFILE;');
EXEC rman_cmd('LIST BACKUP OF TABLESPACE SYSTEM;');
EXEC rman_cmd('LIST BACKUP SUMMARY;');
EXEC rman_cmd('REPORT NEED BACKUP;');
EXEC rman_cmd('CONFIGURE RETENTION POLICY CLEAR;');
EXEC rman_cmd('EXIT;');

Step 4: What Happens in the Background

Each call:
• Sends the text (e.g. "BACKUP SPFILE;") through the pipe to RMAN.
• RMAN executes the command.
• RMAN writes back the results (progress, logs, errors) to the OUT pipe.
• Your PL/SQL procedure prints these results in real time.

A Real-World Use Case
To automate backups or monitoring from PL/SQL (without relying on external OS scripts).

You should follow these steps:

1- Schedule RMAN to start with a private pipe (as a background process or job):

$nohup rman PIPE AUTO_BACKUP TARGET / TIMEOUT 1800 &

2- Inside Oracle, define your procedure (like rman_cmd) to send commands.

3-Create a scheduler job:

BEGIN
  DBMS_SCHEDULER.CREATE_JOB (
    job_name => 'DB_AUTO_BACKUP',
    job_type => 'PLSQL_BLOCK',
    job_action => q'[
      BEGIN
        rman_cmd('BACKUP DATABASE PLUS ARCHIVELOG DELETE INPUT;');
      END;
    ]',
    start_date => SYSTIMESTAMP,
    repeat_interval => 'FREQ=DAILY;BYHOUR=1',
    enabled => TRUE
  );
END;
/

• Store and analyze RMAN output in a log table for reporting or alerts.

This approach keeps backup orchestration inside Oracle, using RMAN’s own engine but controlled via PL/SQL.

Monitoring and Troubleshooting

To check current RMAN pipe sessions:

SELECT name, type, pipe_size
FROM v$db_pipes
WHERE name LIKE '%RMAN%';

Important Notes :

Privileges: Grant EXECUTE ON DBMS_PIPE only to DBA/specific service account.
Security: Use private pipes only to increase security. Public pipes are blocked by default in 21c.
RMAN lifecycle: The RMAN process must be running before you send commands.
Timeout: If RMAN doesn’t receive any new command within the TIMEOUT window, it will exit automatically.
Parallel RMANs: You can run multiple RMAN instances with different pipe names, e.g., RMAN21_DEV_PIPE, RMAN21_PRD_PIPE.


#############################

Tuesday, November 4, 2025

Restrictions When Connected to a PDB

Restrictions When Connected to a PDB

Certain restrictions apply when you connect directly to a pluggable database (PDB):
The following operations are not available when you connect as target directly to a PDB:

• Back up archived logs
• Delete archived logs
• Delete archived log backups
• Restore archived logs (RMAN does restore archived logs when required during media recovery.)
• Point-in-time recovery (PITR) when using shared undo mode
• TSPITR when using shared undo mode
• Table recovery when using shared undo mode
• Duplicate database when using shared undo mode
• Flashback operations when using shared undo mode
• Running Data Recovery Advisor
• Report/delete obsolete
• Register database
• Import catalog
• Reset database
• Configuring the RMAN environment (using the CONFIGURE command)

Note:
Starting in Oracle Database 19c, the Data Recovery Advisor (DRA) feature is deprecated.
The deprecation of DRA includes deprecation of the following Oracle Recovery Manager (RMAN) commands: LIST FAILURE, ADVISE FAILURE, REPAIR FAILURE, and CHANGE FAILURE. Database administrators will no longer have access to these commands. There is no replacement feature for DRA.






Sunday, November 2, 2025

Optimizing Composite Indexes in Oracle

Optimizing Composite Indexes in Oracle: Balancing Selectivity and Index Skip Scan


How to Decide the Leading Column Order in Oracle Composite Indexes? (Using Index Skip Scan Effectively)

As a Developer or Application DBA you may have challenges to setting fist column in composite indexes as a Leading Column based on many SQL statements in your environment, in this topic I demonstrate all aspects of these situations and finally present a formula to handling these type of concerns.

In the previous post, I have a comparison between Oracle and PostgreSQL in Skip Scan concept, it it better to fist visit mentioned post to get a deeply detailed guide.

https://lnkd.in/dP6f3hU2

🔹 1. General rule: leading column should be highly selective

In most cases, Oracle recommends that the most selective column should come first in a composite index.

Reason:
• The optimizer can use the index efficiently when filtering by the first column.
• The fewer rows per key value, the faster it can find matching entries.

Example:

CREATE INDEX idx_t_color_machine ON t (machine_id, color);

If machine_id has many distinct values (high selectivity), this is good when most queries filter on machine_id first.

🔹 2. Problem: what if your queries filter by different combinations?
Let’s say you have queries like:

SELECT * FROM t WHERE color = 'BLUE';
SELECT * FROM t WHERE color = 'RED' AND machine_id = 123;
SELECT * FROM t WHERE machine_id = 123;

Now you have a dilemma:
• If you make (color, machine_id) → good for the first two, bad for the third.
• If you make (machine_id, color) → good for the last two, bad for the first.

So what’s best?

🔹 3. Enter: Index Skip Scan

Oracle can skip scan a composite index when the leading column is not specified in the WHERE clause — but only if the leading column has low cardinality (few distinct values).

So if color has few values like 'RED', 'BLUE', 'GREEN', and machine_id is very selective, then:

CREATE INDEX idx_t_color_machine ON t (color, machine_id);

is actually useful for all these:
• WHERE color = 'BLUE' → normal index range scan
• WHERE color = 'RED' AND machine_id = 123 → normal index range scan
• WHERE machine_id = 123 → index skip scan may kick in
Oracle internally scans index partitions per color value and then looks for machine_id=123.
This is efficient only if the number of distinct color values is small.

🔹 4. Summary table

▶️ If most queries filter on one specific column (and it’s highly selective):
→ Put that selective column first in the index.
Reason: Oracle can use it directly for fast lookups.

▶️ If most queries filter on both columns together:
→ Put the most selective column first.
Reason: Classic composite index design — minimizes scanned rows.

▶️ If some queries omit the first column, and that column has low cardinality (few distinct values):
→ Put that low-cardinality column first.
Reason: Enables Index Skip Scan, allowing Oracle to still use the index efficiently even when the first column isn’t in the WHERE clause.

▶️ If some queries omit the first column, and that column has high cardinality (many distinct values):
→ Don’t rely on skip scans — instead, create separate indexes for different query patterns.
Reason: Skip scans become inefficient when the leading column has too many distinct values.

🔹 5. example
Index created on  (color, machine_id)
And queries are:

1. SELECT * FROM table WHERE color = 'blue';
2. SELECT * FROM table WHERE color = 'red' AND machine_id = 123;
3. SELECT * FROM table WHERE machine_id = 123;

If:
• color = low cardinality (say 5–10 colors)
• machine_id = high cardinality (many machines)

Then  (color, machine_id) is actually best:
• #1 and #2 → normal index scan
• #3 → index skip scan (because color has few distinct values)

But if:
• color = high cardinality (hundreds of distinct values)
• machine_id = low cardinality
Then skip scan will be expensive → better to use (machine_id, color) or two separate indexes.

🟪 Final Recommendation

IF (leading column used in most queries)
    THEN make it leading
ELSE IF (leading column has low cardinality AND others have high)
    THEN put low-cardinality column first (for skip scan)
ELSE
    use separate indexes;

Saturday, November 1, 2025

Database 26ai Installation Guide

 Database 26ai Installation Guide


In this topic, I explain two new and important steps when installing Oracle new versions as 26ai:


Setting Clock Source for VMs on Linux x86-64

Oracle recommends that you set the clock source to tsc for better performance in virtual

environments (VM) on Linux x86-64.


As the root user, check if the tsc clock source is available on your system.

# cat /sys/devices/system/clocksource/clocksource0/available_clocksource

kvm-clock tsc acpi_pm

If the tsc clock source is available, then set tsc as the current clock source.

# echo "tsc">/sys/devices/system/clocksource/clocksource0/

current_clocksource

Verify that the current clock source is set to tsc.

# cat /sys/devices/system/clocksource/clocksource0/current_clocksource

tsc

Using any text editor, append the clocksource directive to the GRUB_CMDLINE_LINUX line in

the /etc/default/grub file to retain this clock source setting even after a reboot.

GRUB_CMDLINE_LINUX="rd.lvm.lv=ol/root rd.lvm.lv=ol/swap rhgb quiet

numa=off transparent_hugepage=never clocksource=tsc"

Run the following command to update GRUB and ensure that these settings are retained

even after a reboot:

# grub2-mkconfig -o /boot/grub2/grub.cfg


Setting Transparent HugePages to madvise

Transparent HugePages memory is enabled by default with Oracle Linux. 


📕However, for optimal

performance, Oracle recommends that you set Transparent HugePages to madvise on all

Oracle Database servers UEK7 and later kernels and not disable Transparent HugePages as

was recommended in prior releases.👈


If you install Oracle Database Preinstallation RPM, then it sets Transparent HugePages to

madvise on all Oracle Database servers UEK7 and later kernels.


To set madvise on Transparent HugePages:


For Oracle Linux and Red Hat Enterprise Linux, add or modify the

transparent_hugepage=madvise parameter in the /etc/default/grub file:

GRUB_TIMEOUT=5

GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"

GRUB_DEFAULT=saved

GRUB_DISABLE_SUBMENU=true

GRUB_TERMINAL_OUTPUT="console"

GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet

transparent_hugepage=madvise"

GRUB_DISABLE_RECOVERY="true"

Note:

The file name may vary depending on your operating system and hardware. For

example, on Oracle Cloud Infrastructure (OCI) the grub.cfg file is located at /boot/efi/EFI/redhat/

Check your operating system documentation for the exact file name.


Run the grub2–mkconfig command to regenerate the grub.cfg file.

On Oracle Linux 8

# grub2-mkconfig -o /boot/grub2/grub.cfg

On Oracle Linux 9

grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline

Restart the system to make the changes permanent. If you do not want to restart your

system, then enter the following command:

# echo madvise >

/sys/kernel/mm/transparent_hugepage/enabled

To check if Transparent HugePages is enabled and configured with madvise, run one of

the following commands as the root user:

Oracle Linux kernels:

# cat /sys/kernel/mm/transparent_hugepage/enabled

Red Hat Enterprise Linux kernels:

# cat /sys/kernel/mm/redhat_transparent_hugepage/enabled

The following is a sample output that shows Transparent HugePages is being used and set

to madvise.

always [madvise] never


What is THP:

Transparent HugePages (THP) is a Linux kernel feature that automates the use of large memory pages (typically 2MB instead of the standard 4KB) to improve memory management efficiency and application performance. Here’s a breakdown of what it is and why Oracle 26ai recommends setting it to `madvise`:


What Are Transparent HugePages?

Purpose: THP reduces overhead in memory management by using larger memory pages, which means fewer pages to track and manage.

Benefit: This can lead to better performance for applications with large memory footprints, like Oracle databases.

Modes:• `always`: Kernel tries to use hugepages for all memory allocations.

• `madvise`: Kernel uses hugepages only when applications explicitly request them via the `madvise()` system call.

• `never`: THP is disabled entirely.


Why Oracle 26ai Recommends `madvise` on UEK7+

Oracle previously advised disabling THP due to performance unpredictability and latency spikes. However, with improvements in UEK7 (Unbreakable Enterprise Kernel version 7) and later:


Improved THP behavior: Kernel enhancements have made THP more stable and predictable.


Selective usage: Setting THP to `madvise` allows Oracle to use hugepages only where beneficial, avoiding the downsides of `always` mode.


Oracle-aware optimization: Oracle Database components can now better manage memory and selectively request hugepages, leading to more consistent performance gains.


🧠 HugePages vs Transparent HugePages

Feature

HugePages (Static)

Transparent HugePages (THP)

Configuration

Manually configured by the system admin

Automatically managed by the kernel

Usage

Reserved explicitly for applications

Kernel tries to use them for anonymous memory

Control

Applications must be configured to use them

Kernel decides when to use them

Swapping

Not swappable (memory stays resident)

Can be swapped unless pinned

Performance

Predictable, stable for large memory apps

May cause latency spikes due to defragmentation

Best for

Oracle SGA, databases, performance-critical apps

General workloads, less admin overhead

Page Allocation

Static allocation at boot time

Dynamic allocation during runtime


What is `clocksource=tsc`?

The `clocksource=tsc` kernel parameter forces Linux to use the Time Stamp Counter (TSC) as the system clock source. Here’s what that means:


• TSC (Time Stamp Counter) is a CPU register that counts cycles since reset. It’s fast and low-overhead.


By default, Linux chooses the most stable and precise clock source available. TSC is often preferred for performance, but only if it’s invariant (i.e., unaffected by CPU frequency scaling or power-saving modes).


Using `clocksource=tsc` can improve performance in latency-sensitive environments like databases or real-time systems, especially when:

•The hardware supports invariant TSC.

• You want consistent timing across CPUs in multi-core systems.

• You’re running Oracle or other performance-critical workloads.


💡 Why use it? It minimizes overhead and improves timestamp precision, which is crucial for Oracle diagnostics and wait event analysis.


 Why install Oracle 26ai?

Oracle Database 26ai is Oracle’s AI-first long-term support release, packed with over 300 new features. Here’s why it’s worth installing:


Key Benefits:


• AI Integration at the Core: Native support for AI vector search, bringing AI directly to your data without external systems.

• Annotations & Metadata Enhancements: Easier documentation and tracking of database objects.

• Interval Aggregation: New support for `SUM` and `AVG` on `INTERVAL` datatypes.

• Autonomous Options: Can be deployed via Autonomous DB containers or free-tier cloud options.

• Future-Proofing: It’s the next long-term release after 19c, meaning extended support and stability for years.


Ideal Use Cases:

• AI-powered analytics

• High-performance transactional systems

• Cloud-native or hybrid deployments

• Modernizing legacy Oracle environments



Alireza Kamrani 

RMAN Pipe Interface & Concepts

RMAN Pipe Interface & Concepts Alireza Kamrani Using the RMAN Pipe Interface The RMAN pipe interface is an alternative method for is...