Exadata X6

Blink and you might have missed it, but the Exadata X6 was officially announced today

As has become the norm, Oracle have doubled-down on the specs compared to the X5:

  • 2x disk capacity
  • 2x Flash capacity
  • 2x faster Flash
  • 25% faster CPUs
  • 13% faster DRAM

With the X6-2 machine, you still have Infiniband running at 40Gb/sec, but the compute nodes and the storage servers now have the following:

X-6 Compute Node

  • 2x 22-core Broadwell CPUs
  • 256Gb of DDR4 DRAM (expandable to 768Gb)
  • 4x 600Gb 10,000 RPM disks for local storage (expandable to 6)

High Capacity Storage Server

  • 2x 10-core Broadwell CPUs
  • 128Gb of DDR4 DRAM
  • 12x 8Tb 7,000 RPM Helium SAS3 disks
  • 4x 3.2Tb NVMe PCIe 3.0 Flash cards

Extreme Flash Storage Server

  • 2-socket, 10-core Broadwell CPUs
  • 128Gb of DDR4 DRAM
  • 8x 3.2Tb NVMe PCIe 3.0 Flash cards

What does all of that give you when it comes down to it?

Well, remember that the eighth-rack is the same as a quarter-rack, but you have access to half the cores and half the storage across the board (you still have two compute nodes and three storage servers):

High Capacity Eighth-Rack

  • 44-core compute nodes
  • 30-core storage servers
  • 144Tb raw usable disk storage
  • 19.2Tb Flash storage

Extreme Flash Eighth-Rack

  • 44-core compute nodes
  • 30-core storage servers
  • 38.4Tb Flash storage

Minimum licensing requirements is 16 cores for the eighth-rack and 28 cores for the quarter-rack.

I’m sure you can read through the sales stuff yourself, but aside from the UUUUGE increase in hardware, two new features of the X6 really pop out for me.

Exadata now has the ability to preserve storage indexes through a storage cell reboot. Anyone who had to support an older Exadata machine will remember quite how much of a big deal that used to be: the wait for the storage index to be rebuilt would take hours and often require some major understanding on the part of user population and management to get through the first day or so after some maintenance.

Probably the biggest thing is that Oracle have introduced high availability quorum disks for the quarter-rack and eighth-rack machines. I blogged about this before as I thought it had the potential to be a real “gotcha” if you were expecting to run high redundancy diskgroups on anything less than a half-rack.

No longer.

Now, a copy of the quorum disk is stored locally on each database node, allowing you to lose a storage cell and still be able to maintain your high redundancy.

This is a particularly useful development when you remember that Oracle have doubled the size of the high-capacity disks from 4Tb to 8Tb. Why? Well, because re-balancing a bunch of 8Tb disks is going to take longer than re-balancing the same number of 4Tb disks.

I’ll be going to Collaborate IOUG 2016 next week and I’m looking forward to hearing more about the new kit there.


Tagged , , ,


It’s been a very busy summer for yours truly and the rest of the database world.  Some interesting nouvelles (I thought so, at least) in case you missed them:


The UK Government has ordered its agencies to “get rid of Oracle“. While Oracle have been shooting themselves in the foot spectacularly of late with their bedside manner, I have personal experience of the last time that the UKG wanted to replace them.

It didn’t go well. At all.

Nor did it go cheaply, which is all that anyone is caring about, of course.

Despite the horror stories in the media about how difficult it is to deal with Oracle’s support, sales and auditing teams, it’s still the best database out there by a country mile.


Ask … Someone Other Than Tom
It’s no longer possible to Ask Tom. Mr. Kyte has decided to take a very well-deserved sabbatical and has handed over Ask Tom duties to … someone who isn’t called Tom.

What a crazy world we live in.

I’m not going to lie – I definitely was dazzled by his stardom on more than one occasion. At a NEOUG conference in Cleveland, I managed to get him to sign a copy of his book and one of the most memorable moments of my DBA career was when I asked him a “great question” on a webinar many years ago. It helped me win an important argument at work, so I will always be thankful to him for that 🙂


Don’t Believe The Hype
Talking of which, not even Gartner thinks Big Data is worthy of the hype now. Instead of moving to their “Slope of Enlightenment” or “Plateau of Productivity”, it fell off their Hype Cycle for Emerging Technologies completely.



Maybe part of the reason is that the industry has realized that a lot of NoSQL databases just plain suck. Don’t forget that the “big” Hadoop story of 2015 is “Hadoop-on-SQL”, which would have been QUITE the juxtaposition eighteen months ago.

Still think NoSQL databases will replace relational databases? Then read this beauty and try and say that “relational databases are outdated” with a straight face.

While they found over a PETABYTE of unsecured data without too much trouble, probably the most noteworthy finding is that they found 347 different MongoDB databases called “DELETED_BECAUSE_YOU_DIDNT_PASSWORD_PROTECT_YOUR_MONGODB”.



Tagged , , , , ,

Today’s Nugget: Oracle OpenWorld 2015 … Or Not

Alas, my submission for this year’s Oracle OpenWorld was turned down by Oracle a little while ago.

Maybe I shouldn’t have installed this browser extension?

Tee-hee 🙂

I’m nothing if not persistent(ly annoying) – so I submitted a similar abstract to the 2016 RMOUG Training Days.

Tagged , , , ,

Standard Edition, Standard Edition One or Standard Edition 2?

You know how people complain that Oracle licensing can be very complicated?

Well, Oracle Standard Edition 2 has been released after being announced earlier in the summer.  Great, but what about Standard Edition and Standard Edition One?

  • Oracle 12c Database Standard Edition will only be available as a release.
  • Oracle 12c Database Standard Edition One 12c will only be available as a release.
  • Oracle 12c Database Standard Edition 2 12c will only be available as a release.

A bit confused?  I know I was.

Basically, SE and SEOne (SE1?) are available options if you’re running a database. However, if you like living your life in the fast lane (as well as making use of some really cool new features) and you’re running, both SE and SE1 editions are replaced by SE2.

The licensing restrictions are as follows.  The bold emphasis is mine:

“Oracle Database Standard Edition 2 may only be licensed on servers that have a maximum capacity of 2 sockets. When used with Oracle Real Application Clusters, Oracle Database Standard Edition 2 may only be licensed on a maximum of 2 one-socket servers. In addition, notwithstanding any provision in Your Oracle license agreement to the contrary, each Oracle Database Standard Edition 2 database may use a maximum of 16 CPU threads at any time. When used with Oracle Real Application Clusters, each Oracle Database Standard Edition 2 database may use a maximum of 8 CPU threads per instance at any time. The minimums when licensing by Named User Plus (NUP) metric are 10 NUP licenses per server.”

By the way, SE2 does not support multi-tenant.  Don’t forget, though, Oracle have deprecated non-“CDB / PDB” architecture from onwards, so you should install SE2 as a single-tenant pluggable database with a container database to follow Oracle’s recommended path.

One wonders whether the “SE2” nomenclature will persist. Will Oracle only offer “Standard Edition 2” and “Enterprise Edition” for Database 13.1?

“What happened to Standard Edition 1?
Why don’t they just call it ‘Standard Edition’?”

I do not know, dear reader.  I do not know.

Tagged , , , , , ,

248 Days

This week, a client encountered a particularly nasty bug – 10194190 – which caused their ASM instance processes to “spin” and cause a bunch of errors, essentially leading to a instance crash on both of their RAC nodes.

In the ASM instance:

ORA-00490: PSP process terminated with error
PMON (ospid: 12345): terminating the instance due to error 490

In the database instance, we either saw an ORA-00240 or ORA-03113 error and an instance crash.

ORA-00240: control file enqueue held for more than 120 seconds

ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel

What triggered this was that the uptime of each node was greater than 248 days. On Solaris SPARC systems, there is a bug in the compiler which can cause either a database or an ASM instance crash if the server has been up for more than 248 days.

There are bug fixes available for versions through, but there is no fix for any 12c database at the moment.


Larry’s Secret Number?
“248 days”, you ask? Curious, n’est pas? Indeed, especially when you learn that there are another couple of Oracle bugs out there which really seem to fixate on that number.

In a previous life, I encountered bug 4612267 while running the Client for an EDW batch server.

It is looping on the times() function.

In addition to sqlplus, it has been reported that the netca and dbca tools also hang.
This may happen with a new installation of Instant Client or Oracle on UNIX platform, or it can just occur after some period of time with no other changes.

This is a known, unpublished bug.


This machine was critical to the enterprise and had to be rock solid, so once we got things stable, the project team were loathe to mess with it, even though it was running

“Not messing with it” also involved maintaining server uptime, because we had a bunch of NFS mounts attached to it, which seemed to somehow cause the server to take a very long time to reboot.

Unfortunately, the platform was a bit too “solid” and we hit bug 4612267 during a particularly busy afternoon of batch processing. As with such things, it took a while to find the culprit, but it ended up being new sqlplus sessions started after 1pm that day on the EDW batch server. The server had been up 231 days.

We looked and we could see the sessions had started, but they had got no CPU time whatsoever. In typical IT fashion, we collected as much diagnostic info as we could and bounced the server. Naturally, the problem went away.

After much head-scratching and seeing the same issue once or twice more whenever the server uptime ranged between 60 and 248 days, we realized we were hitting the bug. As we were using a client install, we couldn’t upgrade to and applying the patch failed for some obscure reason, despite assistance from Oracle Support. We only “fixed” the bug when we upgraded the client to 11g, which was deemed to be too much “messing about” until after our busiest period of the year was over.

Our workaround was to schedule a server reboot, which was a pain, though it was better than the alternative.

I always did wonder why someone would code something which would “spin” if the server uptime was between 60 and 248 days. What does that matter to SQL*Plus? Besides, once you went beyond 248 days of uptime, you were in the clear. This is why it took three years before it was noticed.

I wonder what significance 248 days has with Oracle?  Maybe it’s a code which is somehow used to obtain privileged access somewhere in Redwood City? 🙂



Tagged , ,

Today’s Nugget: Materialized Views

As I’m still trying to get my feet under the door at my new gig, here’s another nugget in place of a real blog entry. While troubleshooting for a client, I noted that there were three types of materialized view. I knew this before, sort of, but this troubleshooting exercise was a great aide memoire.

The three types of materialized views:

  1. Read-only
  2. Updateable
  3. Writeable

The read-only MV omits the FOR UPDATE clause in its DDL during creation and does not permit DML.

The updateable MV includes the FOR UPDATE clause in its DDL and is included in a materialized view group.  This allows changes made to the MV to be pushed back to the ‘master’ during a refresh.

The writeable MV includes the FOR UPDATE clause in its DDL, but does not belong to a materialized view group.  All changes made to the MV are lost during a refresh.


Tagged , , , ,

Today’s Nugget: Oracle GoldenGate on E-Business Suite Databases

Throughout my work week, I often learn something new and unexpected about my travels. Instead of writing a full blog post about the subject, I figure I’ll occasionally post these as “nuggets” – bite-size chunks which are easily digested!

Here is today’s “nugget”:

Oracle GoldenGate is certified to capture data from E-Business Suite databases, but it cannot be used to apply data to E-Business Suite databases.

This includes writing back to the source database in an active-active configuration.



Tagged , , , , ,

Dude … Where’s My GUI?

Are you running Exadata Storage Server

Have you tried to get a graphical tool, such as DBCA, DBUA or even VNC to run on your Exadata lately?

If, like me, you had quite the struggle until a friendly sysadmin installed a bunch of packages for you, you might be interested in reading this MOS note:

1969308.1 – Unable to run graphical tools (gui) on Exadata or later.

I understand that Oracle have been increasingly hardening Exadata since the birth of their X3-2 machines, but you’d think that you wouldn’t need to add extra packages to a system that’s meant to be “ready to use” once your choice of consultant has finished with the Deployment Assistant.

After all, aren’t DBCA / DBUA Oracle’s tools of choice? Do they really want DBAs to spend their time creating response files and running these tools from the command line?


Tagged , , , , , ,

Hamlet – Prince of DBAs

While searching through my exhaustive collection of notes, scribbles and documentation, I found this particular GEM of mine that I wrote to Oracle Support after they had ignored my Priority Service Request for a month.  After finding that various degrees of pleading didn’t work, I went on the offensive and THREATENED them!

*I* found it very funny, anyway …

A last-ditch attempt to resolve this peacefully from ME:

I’m not kidding – the next update you’re going to be getting from me is poem. And I’m a very bad poet.

You have been warned.

They gave me no choice by ignoring me again.  I did what I had to do.  I’m not proud of it, but I’m not sorry either:

You were warned. You’re not only getting a poem, but you’re getting a Shakespeare soliloquy. And a TERRIBLE one at that.

If there’s no update tomorrow, I’m going to write and perform a song. Trust me when I say I can’t even speak in tune, never mind sing…

I hope you recognize this fabulous piece of literature…

To reply to the SR or not to reply to the SR, that is the question:
Whether there are nobler items in your in-queue
So to deal with other, less annoying customers
Or to at least *look* at this priority handled SR.
To update, to ignore
No more; and by ignoring, to say
The ignorance of a month-old priority SR
That a tortured DBA is heir to?
‘Tis a consummation devoutly to be wished,
To escalate or wait
To update, perchance to persuade
Aye, there’s the rub
For in that persuading, what solutions may come,
When you have stopped ignoring this request,
Must give us pause. There’s the respect
That makes calamity of ‘PRIORITY SR’
For who would bear the whips and scorns of having to deal with Metalink
The analysts’ wrong, the DBA’s contumely,
The pangs of paying for this, no response and delay,
The insolence of writing you a poem
To get your attention after too many winter nights
When he himself might his answer make
By putting this down to a freak of Exadata
That will be fixed in a future release.
Be all my sins remembered.

Seriously – is there any sign of an update – I don’t want to call in and ask why a Priority SR has been ignored for over a month. At least tell me you’ve not forgotten it…


Acknowledging your last update..
I will make sure this SR gets adequate attention and is progressed
Let me review the SR and get back to you…
(BTW, your poem is good- but I am sorry you had to write it)

Tagged , ,

New Exadata Critical Issues – EX23 and EX24

Over the weekend, Oracle announced two new Critical Issues for Exadata Storage Server (EX23 and EX24), both impacting version

Both the new Critical Issues can be patched by either applying the one-off patch 21251493 or by upgrading the Exadata Storage Server software to


Critical Issue EX23
Affects Exadata Storage Server and

Bug 21174310 – wrong results, ORA-1438 errors or other internal errors (ORA-00600 and ORA-07445) are possible from smart scan offloaded queries against HCC or OLTP compressed tables stored on Exadata storage if:

  • Exadata Storage Server is version or AND
  • Oracle Database was upgraded from 11.2 to 12.1 AND
  • A smart scan offloaded query is issued against an OLTP compressed table or an HCC table containing OLTP compressed blocks

The workaround is to recreate the table.

The recommended action is to upgrade to Exadata Storage Server software version (or higher).

Alternatively, apply patch 21251493 to Exadata Storage Servers running version  Note that patch 21251493 contains additional fixes required to resolve other critical issues.

MOS 2032464.1 has additional details.


Critical Issue EX24
Affects Exadata Storage Server

After replacing a failed system disk (disk 0 or disk 1), the new disk is not correctly configured leaving the system vulnerable to the other system disk failing. The likelihood of occurrence is high when running Exadata version and a failed system disk is replaced.

The workaround is to follow the instructions in MOS 2003674.1.

The recommended action is to upgrade to Exadata Storage Server software version (or higher).

Alternatively, apply Patch 21251493 to Exadata Storage Servers running version Note that patch 21251493 contains additional fixes required to resolve other critical issues.

MOS 2032402.1 has additional details.



  • Bug 21174310 – Wrong results or ORA-1438 errors possible from smart scan offloaded queries against HCC or OLTP compressed tables stored on Exadata storage (Doc ID 2032464.1)
  • Important Fixes Required for System Disk Replacement on Exadata Storage Servers Running Version (Doc ID 2032402.1)
  • Exadata Storage Software and System Disk Replacement Issues (Doc ID 2003674.1)
  • Exadata Critical Issues (Doc ID 1270094.1)
  • Patch 21251493
Tagged , , , , , , , , , , ,
%d bloggers like this: