Technical

jeremy.schneider's picture

Create Tablespace and ORA-28374

One of the projects I worked on last week was improving our processes around encryption in Oracle. I spent a lot of time becoming intimate friends with the database wallet. Late in the week while attempting to create an encrypted tablespace on an 11.2.0.3 (PSU6) system, we mysteriously ran into ORA-28374. Oddly enough we were building two identical databases in parallel using a scripted approach but only one of the databases ran into this problem. And no matter how many times I rolled back and re-ran the wallet setup script, I kept getting this same error! There are a handful of informative Oracle Support notes related to this error and one of them had the solution to my problem. However it wasn’t immediately obvious why, so I thought it would be worthwhile to write a description based on my experience.

jeremy.schneider's picture

OSP #1: The Foundation

This is the third of twelve articles in a series called Operationally Scalable Practices. The first article gives an introduction and the second article contains a general overview. In short, this series suggests a comprehensive and cogent blueprint to best position organizations and DBAs for growth.

As a starting point for our discussion of scalable practices, it makes sense to talk about three fundamentals. These will provide a strong foundation for everything else we discuss. First, change history; second, checklists; and third, a few server basics.

Change History

Change History is the single most important concept in Operationally Scalable Practices. Read that sentence again.

The basic idea is simple:

jeremy.schneider's picture

OSP #1: The Foundation

This is the third of twelve articles in a series called Operationally Scalable Practices. The first article gives an introduction and the second article contains a general overview. In short, this series suggests a comprehensive and cogent blueprint to best position organizations and DBAs for growth.

As a starting point for our discussion of scalable practices, it makes sense to talk about three fundamentals. These will provide a strong foundation for everything else we discuss. First, change history; second, checklists; and third, a few server basics.

Change History

Change History is the single most important concept in Operationally Scalable Practices. Read that sentence again.

The basic idea is simple:

jeremy.schneider's picture

Listener Error from addNode.sh with Second Network

Recently I ran into an problem with 11.2.0.3 RAC. I observed this on a system patched to PSU6 and it looks like a bug to me. But the interesting part isn’t the problem – it’s an impressive and creative workaround that my colleague found over the weekend. I should add that this teammate doesn’t have much background with Oracle RAC though he does have lots of experience with other technologies. His email this weekend surprised me and also gave me a good laugh – hope you find it equally useful and enjoyable!

The problem originated with a requirement I was given when designing this particular cluster system: I was asked to run Data Guard traffic over the backup network instead of the public network. This sounds simple enough if you haven’t worked with RAC. But if you’ve worked with Oracle clusters you realize that nothing is simple anymore. (A big reason I often encourage people to wait on moving to RAC, especially if the main driver is high availability…)

jeremy.schneider's picture

Listener Error from addNode.sh with Second Network

Recently I ran into an problem with 11.2.0.3 RAC. I observed this on a system patched to PSU6 and it looks like a bug to me. But the interesting part isn’t the problem – it’s an impressive and creative workaround that my colleague found over the weekend. I should add that this teammate doesn’t have much background with Oracle RAC though he does have lots of experience with other technologies. His email this weekend surprised me and also gave me a good laugh – hope you find it equally useful and enjoyable!

The problem originated with a requirement I was given when designing this particular cluster system: I was asked to run Data Guard traffic over the backup network instead of the public network. This sounds simple enough if you haven’t worked with RAC. But if you’ve worked with Oracle clusters you realize that nothing is simple anymore. (A big reason I often encourage people to wait on moving to RAC, especially if the main driver is high availability…)

jeremy.schneider's picture

OSP: Overview

This is the second of twelve articles in a series called Operationally Scalable Practices.  You can read the introduction in the first article. In short, this series offers helpful suggestions for younger organizations and newer DBAs to best position them for very large-scale growth.

Before getting into specifics, we will lay out a general overview of the content. I expect this overview to be revised the most as the series is refined over time – so check  periodically to see if there have been updates!

jeremy.schneider's picture

OSP: Overview

This is the second of twelve articles in a series called Operationally Scalable Practices.  You can read the introduction in the first article. In short, this series offers helpful suggestions for younger organizations and newer DBAs to best position them for very large-scale growth.

Before getting into specifics, we will lay out a general overview of the content. I expect this overview to be revised the most as the series is refined over time – so check  periodically to see if there have been updates!

jeremy.schneider's picture

Operationally Scalable Practices

I really enjoy being a technical guy. So far in my career I’ve made development choices favoring a technical path over other options. It’s been a great ride – I’ve worked in small teams and large teams; consulting roles and in-house roles; architecture/engineering roles and operations roles; big databases and little databases; environments with a few databases and environments with thousands of databases.

I’ve never been anywhere which had everything right. Also, my own ideas about what’s “right” are still evolving today. I have a habit of trying to be around people who are smarter than me… over the course of my career, my own knowledge and experience with Oracle have grown exponentially and yet I’ve never had trouble continuing to feel like a junior DBA. (In particular, my invitation to the Oak Table made this very easy!)

jeremy.schneider's picture

Operationally Scalable Practices

I really enjoy being a technical guy. So far in my career I’ve made development choices favoring a technical path over other options. It’s been a great ride – I’ve worked in small teams and large teams; consulting roles and in-house roles; architecture/engineering roles and operations roles; big databases and little databases; environments with a few databases and environments with thousands of databases.

I’ve never been anywhere which had everything right. Also, my own ideas about what’s “right” are still evolving today. I have a habit of trying to be around people who are smarter than me… over the course of my career, my own knowledge and experience with Oracle have grown exponentially and yet I’ve never had trouble continuing to feel like a junior DBA. (In particular, my invitation to the Oak Table made this very easy!)

jeremy.schneider's picture

Voting Disk Lies (CRS-4000)

Add this to the category of annoyingly unhelpful error messages.

I’m working on a mostly-automated process to create a new cluster by cloning another existing cluster. After running OUI (Oracle Universal Installer – called by config.sh to just run config assistants) there is a single ASM diskgroup which contains both the OCR and Voting Disk; however I wanted to switch the voting disks over to some different physical devices.

Upon which I received this errors:


(root)# /oracle/11203/grid/bin/crsctl replace votedisk +CLST3_VOTING
Failed to create voting files on disk group CLST3_VOTING.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.

Lovely… so very informative. And just to be clear, it didn’t complete with errors, it completely failed. Thanks Oracle.

Syndicate content