My OpenWorld Session

From An Expert's Guide to Oracle Technology

I am at Oracle HQ this week for the Oracle ACE Director briefing. So far there has been an amazing amount of information about upcoming hardware, software and solutions. Unfortunately, I am under an NDA to not disclose the specifics until next week. I promise you, there will be an announcement that you find interesting.

Gauging Online Oracle Training Interest

From An Expert's Guide to Oracle Technology

I wanted to get some feedback from the reades on this blog. I'd like feedback from regular readers as well as drive by readers - beginner, intermediate and expert.

I am thinking about setting up some online training. I don't mean just a blog or some downloadable papers. I mean a fullscale training si

Security Alert for CVE-2011-3192 Released

Hi, this is Eric Maurice.

Oracle just released a Security Alert for CVE-2011-3192, also known as “Apache Killer.”  This vulnerability was recently discovered in the Apache HTTP Server and publicly disclosed.  It affects a number of Oracle products through their implementation of Apache, including Oracle Application Server and Oracle Fusion Middleware (a list of the affected products is provided in the Security Alert Advisory). 

The National Vulnerability Database has reported a CVSS Base Score for this vulnerability of 7.8 indicating a complete Operating System denial of service (DOS); however a complete Operating System denial of service is not possible on any platform supported by Oracle, and as a result, Oracle has given the vulnerability a CVSS Base Score of 5.0 indicating a complete DOS of the Oracle HTTP Server but not the Operating System. 

This vulnerability allows a malicious attacker to hang the Oracle HTTP Server product via an easy-to-deploy, unauthenticated network attack.  Due to the criticality of this vulnerability and particularly its ease of exploitation, Oracle decided to release fixes for the affected and supported products as soon as the testing for these fixes was completed, before the release of the next scheduled Critical Patch Update on October 18th 2011. 

Oracle recommends that all affected customers apply the fixes provided by this Security Alert as soon as possible in order to maintain their “security in depth” posture, even if they have applied some of the workarounds that have been published on the Internet, as  Oracle has found that many of these workarounds can cause regression issues across the stack. 

This recommendation is also applicable to other vendors’ products which may contain this vulnerability as a result of their implementation of the Apache HTTP Server.  Organizations should quickly determine which of their systems is vulnerable, obtain the necessary patches from their respective suppliers, and plan to quickly apply these patches, especially in external facing systems.

For More Information:

• Security Alert CVE-2011-3192 is located at http://www.oracle.com/technetwork/topics/security/alert-cve-2011-3192-485304.html.
• More information about Oracle Software Security Assurance is located at http://www.oracle.com/us/support/assurance/index.html

Security Alert for CVE-2011-3192 Released

Hi, this is Eric Maurice.

Oracle just released a Security Alert for CVE-2011-3192, also known as “Apache Killer.”  This vulnerability was recently discovered in the Apache HTTP Server and publicly disclosed.  It affects a number of Oracle products through their implementation of Apache, including Oracle Application Server and Oracle Fusion Middleware (a list of the affected products is provided in the Security Alert Advisory). 

The National Vulnerability Database has reported a CVSS Base Score for this vulnerability of 7.8 indicating a complete Operating System denial of service (DOS); however a complete Operating System denial of service is not possible on any platform supported by Oracle, and as a result, Oracle has given the vulnerability a CVSS Base Score of 5.0 indicating a complete DOS of the Oracle HTTP Server but not the Operating System. 

This vulnerability allows a malicious attacker to hang the Oracle HTTP Server product via an easy-to-deploy, unauthenticated network attack.  Due to the criticality of this vulnerability and particularly its ease of exploitation, Oracle decided to release fixes for the affected and supported products as soon as the testing for these fixes was completed, before the release of the next scheduled Critical Patch Update on October 18th 2011. 

Oracle recommends that all affected customers apply the fixes provided by this Security Alert as soon as possible in order to maintain their “security in depth” posture, even if they have applied some of the workarounds that have been published on the Internet, as  Oracle has found that many of these workarounds can cause regression issues across the stack. 

This recommendation is also applicable to other vendors’ products which may contain this vulnerability as a result of their implementation of the Apache HTTP Server.  Organizations should quickly determine which of their systems is vulnerable, obtain the necessary patches from their respective suppliers, and plan to quickly apply these patches, especially in external facing systems.

For More Information:

• Security Alert CVE-2011-3192 is located at http://www.oracle.com/technetwork/topics/security/alert-cve-2011-3192-485304.html.
• More information about Oracle Software Security Assurance is located at http://www.oracle.com/us/support/assurance/index.html

Oracle 11g R1/R2 Real Application Clusters Essentials Book (Review – Sort Of)

From An Expert's Guide to Oracle Technology

So, I got an email from Packt asking if I wanted to review an Oracle RAC book. I only have a couple of those in my library so I thought sure, that might be an interest

Oracle XE 11gR2 General Availability

From An Expert's Guide to Oracle Technology

Yeah, I know. I'm late with the news. Still, just in case you haven't already seen it blogged, tweeted or linkedin'ed, Oracle XE 11gR2 has moved out of beta and into production.

Why Apple Will Survive

Do bind variables matter?

From An Expert's Guide to Oracle Technology

That title is somewhat facetious. Even without a performance benefit using bind variables is more secure and easier to maintain but do they really make a difference in performance?

I still hear, on occassion, that using bind varaibles is somehow slower. I think mostly it's just makes the coding slowe

Those Who Can’t Do, Audit

I am often asked what the biggest change is that I’ve seen in my years working in information security. (Most of those who ask are too polite to ask how many years I’ve worked in information security. I used to say I joined Oracle when I was 8. Now, I tell everyone I started at Oracle when I was 3. Pretty soon, I’ll be a prenatal employee.)

There are too many changes to pick just one, but on a personal level, I seem to spend a lot more time on government affairs (relative to the rest of my job) that I used to: working with others in industry to discuss cybersecurity-related public policy, meeting with staffers on the Hill, commenting on draft legislation, commenting on others’ comments on draft legislation, and so on. On the one hand, it’s a natural result of the amount of dependence that everyone has on cybersecurity and the amount of highly publicized breaches, hacks, and so on we continue to see. You can hardly open the paper – or open an email with a link to the daily electronic paper – without reading about another data breach, “outing” of sensitive information, new thing you didn’t even know was IP-accessible being broken into (e.g., starting cars remotely), and so on.

Legislators often want to Do Something when they see a problem – that’s why we elected them, presumably. I’ve opined in previous blogs on the importance of defining what problem you want to solve, specifying what “it” is that you want to legislate, understanding costs – especially those pertaining to unintended consequences - and so on. I spend more time on govie affairs now because more legislators are proposing legislation that they hope will improve cybersecurity.

Most people who help frame public policy discussions about cybersecurity are well intended and they want to minimize or resolve the problem. However, there are also a whole lotta entities that want to use those discussions to get handed a nice, big, fat “public policy printing press”: ideally, legislative mandates where their businesses receive a direct and lucrative benefit, ka-ching. Their idea of public policy begins and ends at the cash register.

Some businesses would love to be in a position where they could write/draft/influence legislation about how no commercial company can be trusted with security and therefore needs someone (presumably people like themselves) to bless their systems and all their business practices – the more, the merrier. This includes legislative mandates on suppliers – who, as we all know – are busy throwing crappy code over the wall with malice aforethought. Those evil suppliers simply cannot be trusted, and thus the entirety of supplier business practices in security must be validated by Outside Experts* if not the entirety of their code scanned. (Nice work if you can get it.) (Aside: boy, with all the energy we suppliers expend on deliberately producing rotten code and putting our customers’ systems at risk – not to mention our own businesses since we run on our own stuff - it’s really hard to find time for other malicious acts like starving baby rabbits and kitties.**)

At the same time, these businesses are less than forthcoming when asked how they will be vetted: what do they find, how well do they find it and at what cost, since everybody who has worked with code scanning tools know they need to be tuned. (Having to plow through 1000 alleged vulnerabilities to find the three that are legitimate is way too expensive for any company to contemplate doing it.) Now is a perfect time for a disclosure: I am on an advisory board to a company in this sector.

One company in particular is beginning to get under my skin on a lot of levels pertaining to “creating a market for themselves.” Let’s call them SASO (Static Analysis Service Offering) that – ta da! - do static analysis as a service offering. More specifically, they analyze the binaries to do static analysis – an important point. When SASO got started, I thought they had a useful value proposition in that a lot of small software companies don’t have security expertise in-house – one reason why Oracle has strong “business alignment” processes when we acquire a company that include alignment with our assurance practices, which the acquired entities are by-and-large really happy to do. The small company wants to increase their ability to sell into core markets and to do so they may need to convince prospective customers that their code is appropriately secure. They hire SASO to analyze their code, and SASO produces a summary report that tells the customer whether there are significant potentially unfixed vulnerabilities and also gives the small company the details of those vulnerabilities so they can validate actual vulnerabilities and then fix them. Fair enough, and a good service offering, to a point, more on which later. The real benefit to the small company is to get actionable intelligence on problems in their code earlier because for one, it’s almost always easier and cheaper to fix these before products ship, and/or remediate them in some severity order. Also, it’s not always easy to use static analysis tools, yourself: the tools require knowledgeable people to run them and analyze the results.

It’s not, in other words, the “gold star” from SASO they get that has value; it’s the SASO expertise they get to fix issues earlier until they get that expertise in-house and up-to-speed. And they really do need that expertise in-house because security is a core element of software development and you cannot outsource it -- or you will have no security. (Among other things Oracle does in terms of secure development, we have our ethical hacking team conduct product assessments. They are as good as anybody in industry – or better – and they find problems that code analysis tools do not find and never will find. Their expertise is often codified in our own tools that we use in addition to commercially available static and dynamic analysis tools.)

Of course, the market for “helping small inexperienced companies improve the security-worthiness of their code” isn’t big enough, so SASO has been trying to expand the size of their market by means of two understandably - yet obnoxiously - self-promoting activities. One of them is by convincing entire market sectors that they cannot trust their vendors, and their suppliers’ code needs to be “tested” by SASO. You do want to ask, why would anybody trust them? I mean, more and more of SASO’s potential market size is based on FUD. Whereas, the suppliers’ market is based on their customers’ ability to run their entire businesses on the software or hardware. That is, customers bet the digital farm on their suppliers, and the suppliers understand this. If you lose your customer’s confidence for whatever reason – security among them – your customers will switch suppliers faster than you can say “don’t bother with the code scan.” And thus, suppliers are out of business if they screw it up, because their competitors will be ruthless. Competitors are ruthless.

Whom do you think is more trustworthy? Who has a greater incentive to do the job right – someone who builds something, or someone who builds FUD around what others build? Did I mention that most large hardware and software companies run their own businesses on their own products so if there’s a problem, they – or rather, we – are the first ones to suffer? Can SASO say that? I thought not.

Being approached by more and more companies who want our code SASOed has led me to the “enough, already” phase of dealing with them. For the record, we do not and will not allow our code to be SASOed; in fact, we do not allow any third parties access to source code for purposes of security analysis (with one exception noted below). I’ve also produced a canned response that explains to customers why SASO will never darken our code base. It covers the following points, though admittedly when I express them to customers they are less glib and more professional:

1) We have source code and we do our own static analysis. Ergo, I don’t need SASO or anybody else to analyze our binaries (to get at the source code). Moreover, we have not only one but two static analysis tools we use in-house, one of which we built (Parfait, which is built by Sun Labs, a fine outfit we got via the Sun acquisition). Also, as I’ve opined at length in past blog entries, these tools are not “plug and play” – if they were, everybody would run down to Radio Shack, or wait for late night television to see advertisements for Code Scannerama (“finds all security vulnerabilities and malware in 28 seconds -- with No False Positives! But wait, there’s more: order now, and get a free turnip twaddler!”). For the amount of time we’d expend to work with SASO and get useful, accurate and actionable output – we could just get on with it and use the tools we’ve already licensed or already have. SASOing our code hurts us and doesn’t help us.

2) Security != testing. I am all for the use of automated tools to find vulnerabilities in software – it’s a code hygiene thing – but it is not the totality of assurance. Oracle has an extensive assurance program for our products that includes the use of static analysis tools. (In fact, Oracle requires the use of multiple tools, some of which we have developed in-house because no single tool “does it all.”) We also track compliance with our assurance program and report it to our senior management, all the way up to the CEO. In short, we do a far more comprehensive job in security than SASO can do for us. (I also note that – hype to the contrary – these tools will not find much if any malware in code. I am a blond and a bad programmer and it took me about ten minutes to think of ways to put bad stuff in code in a way that would not be detectable by automated code analysis.) “Code analysis” may be a necessary element for security-worthy code but it sure ain’t sufficient.

3) Precedent. It’s a really, really bad precedent to hand your source code to a third party for purposes of “security analysis” because gee, lots of governments have asked for the same privilege. And some governments who ask (I won’t name them, but we all know who they are) engage in state-sponsored industrial espionage so you might as well just kiss your intellectual property good bye as hand your code to those guys (“Wanna start a new software/hardware company? Here’s our source code, O Official Government of Foobaria IP Theft Department!”). Some governments also want to easily do security analysis they then exploit for national security purposes – e.g., scan source code to find problems that they hand to their intelligence agencies to . Companies can’t really say, “we will allow SASO to scan our code if customers nudge us enough” and then “just say no” to governments who want exactly the same privilege. Also, does anybody think it is a good idea for any third party to amass a database of unfixed vulnerabilities in a bunch of products? How are they going to protect that? Does anybody think that would be a really nice, fat, juicy hacker target? You betcha.

(As a matter of fact, Oracle does think that “bug repositories” are potentially nice, fat, juicy hacker targets. Security bug information is highly sensitive information, which is why security bugs are not published to customers and our bug database enforces “need to know” using our own row level access control technology. The security analysts who work for me have security bug access so they can analyze, triage, ensure complete fixes, and so on, but I don’t, because we enforce “need to know,” and I don’t.) I don’t trust any third party to secure detailed bug information and it is bad public policy to allow a third party to amass it in the first place.

4) Fixability. Only Oracle can fix vulnerabilities: SASO cannot. We have the code: they don’t.

5) Equality as public policy. Favoring a single customer or group of customers over others in terms of security vulnerability information is bad public policy. Frankly, all customers' data is precious to them and who would not want to be on the "insider" list? When we do have vulnerabilities in products, all customers get the same information at the same time and the same level of information. It’s basic equity and fairness. It’s also a lot better policy than playing favorites.

6) Global practices for global markets. The industry standard for "security evaluations" is an ISO standard called the Common Criteria (ISO 15408) that looks at the totality of security functionality needed to address specific threats as well as "assurance." A product with no exploitable security defects - that has no authentication, auditing or access control – and isn’t designed to protect against any threats - is absolutely, utterly useless. SASOing your code does not provide any information about the presence or absence of security functionality that meets particular threats. It’s like having someone do a termite inspection on the house you are about to buy but not looking at structural soundness: you know you don’t have bugs – pun intended – but you have not a clue about whether the house will fall down. Common Criteria evaluations do include vulnerability analysis but do not require static analysis. However, many vendors who Common Criteria-evaluate their product also do static analysis, whether or not they get any “credit” for doing it, because it is good practice to do so. Our Common Criteria labs do in some cases get source code access because of the type of analysis they have to do at some assurance levels. But the labs are not “governments,” they are licensed (so there is some quality control around their skill set), access to our source code is incredibly restricted, and we have highly enforceable liability provisions regarding protection of that source code.

7) All tools are not created equal. The only real utility in static analysis is in using the tools in development and fixing what you find in some priority order. In particular, requiring static analysis – e.g., by third parties - will not lead to significant improvement except possibly in the very short term and in the medium to long term will increase consumer costs for no benefit. Specifically: · The state of the art in vulnerability discovery tools is changing very rapidly. Mandating use of a particular tool - or service - will almost guarantee that the mandated tool/service will be obsolete quickly, resulting in added cost for incremental benefit at best. Obsolescence is especially guaranteed if a particular tool or service is mandated because the provider, having been granted a monopoly, has no incentive to improve their service. · The best types of tools used to discover vulnerabilities differ depending on the type of product or service that is being analyzed. Mandating a particular tool is thus suboptimal at best, and an expensive mistake at worst.

I said at the outset that SASO has a useful model for smaller companies that perhaps do not have security expertise in house - yet. But having said that, I offer a cautionary story. A company Oracle acquired some time ago had agreed to have their code SASOed at the request of a customer - before we acquired them. The first problem that created was that the team, by agreeing to a customer request – and continuing to agree to “outside code auditing” until we stopped it – set the market expectation that “gosh, we don’t know what we are doing in security and you shouldn’t trust us,” which is bad enough. What is far, far, worse: I believe this led to a mentality that “those (SASO or other) testing people will find security issues, so I don’t need to worry about security.” I told the product group that they absolutely, positively, needed in-house security expertise, that “outsourcing testing” would create an “outsourcing security” mentality that is unacceptable. Oracle cannot – does not – outsource security.

By way of contrast, consider another company that does static analysis as a service. Let’s call them HuiMaika‘i (Hawaiian for “good group”). HuiMaika‘i provides a service where customers can send in a particular type of code (i.e., based on a particular development framework) and get a report back from HuiMaika‘i that details suspected security vulnerabilities in it. Recently, a HuiMaika‘i customer sent them code which, when analyzed, was found to contain multiple vulnerabilities. HuiMaika‘i somehow determined that this code was actually Oracle code (that is, from an Oracle site) and, because it is their policy not to provide vulnerability reports to customers who submit code not owned by the customer, they returned the vulnerability report to Oracle and not the paying customer. (How cool is that?)

I think their policy is a good one, for multiple reasons. The first is that running such vulnerability-detecting software against code not owned by the customer may violate terms-of-use licenses (possibly resulting in liability for both the customer and the vendor of such vulnerability detecting software). The second reason is that it would be best all around if the vendor of the software being scanned be informed about possible vulnerabilities before public disclosure so that the vendor has a chance to provide fixes in the software. Having said that, you can bet that Oracle had some takeaways from this exercise. First, we pulled the code down wikiwiki (Hawaiian for “really fast”) until we could fix it. And we fixed it wikiwiki. Second, my team is reminding development groups - in words of one syllable - that our coding standards require that any code someone gets from Oracle – even if “just an example” - be held to the same development standards as products are. Code is code. Besides, customers have a habit of taking demo code that does Useful Stuff and using it in production. Ergo, you shouldn’t write crappy code and post it as “an example for customers.” Duh.

Our third takeaway is that we are looking at using HuiMaika‘i onsite in that particular development area. It wasn’t, by the way, just because of what they found - it’s the way they conducted themselves: ethically, professionally, and they didn’t play “vendor gotcha.” Thanks very much, Hui. Small company – big integrity.

Returning to SASO, the other way – besides marketing “you can’t trust suppliers but you can trust us” - in which they are trying to expand their market is a well-loved and unfortunately, often-attempted technique – get the government to do your market expansion work for you. I recently heard that SASO has hired a lobbyist. (I did fact check with them and they stated that, while they had hired a lobbyist, they weren’t “seriously pursuing that angle” – yet.) I have to wonder, what are they going to lobby for? Is it a coincidence that they hired a lobbyist just at the time we have so much draft cybersecurity legislation, e.g., that would have the Department of Homeland Security (DHS) design a supply chain risk management strategy – including third party testing of code - and then use that as part of Federal Acquisition Regulations (FAR) to regulate the supply chain of what the government buys? (Joy.)

I suspect that SASO would love the government to mandate third party code scans – it’s like winning the legislative lottery! As to assurance, many of my arguments above about why we “just say No, to S-A-S-O!” are still equally – if not more – relevant in a legislative context. Secondly, and related to “supply chain” concerns, you really cannot find malware in code by means of static analysis. You can find some coding errors that lead to certain types of security vulnerabilities - exploitable or not. You might be able to find known patterns of “commercially available” malware (e.g., someone downloaded a library and incorporated it into their product, and did not bother to scan it first before doing so. Quelle surprise! The library you got from CluelessRandomWebSite.com is infected with a worm). That is not the same thing as a deliberately introduced back door, and code scans will not find those.

In my opinion, neither SASO - nor any other requirement for third party security testing - has any place in a supply chain discussion. If the concern is assurance, the answer is to work within existing international assurance standards, not create a new one. Particularly not a new, US-only requirement to “hand a big fat market win by regulatory fiat to any vendor who lobbied for a provision that expanded their markets.” Ka-ching.

The bigger concern I have is the increasing amount of attempts by many companies, not to innovate, but to convince legislators that people who actually do stuff and build things can’t be trusted (how about asking themselves the same question?). I recently attending a meeting in which one of the leading presenters was a tenured professor of law whose area of focus is government regulation, and who admitted having no expertise in software. Said individual stated that we should license software developers because “we need accountability.” (I note that as a tenured professor, that person has little accountability since she is “unfireable” regardless of how poor a job she does, or whether there is any market demand for what she wants to teach.)

I felt compelled to interject: “Accountable to whom? Because industry hires many or most of these grads. We can enforce accountability – we fire people who don’t perform. If your concern is our accountability to our customers – we suppliers absolutely are accountable. Our customers can fire us – move to a competitor – if we don’t do our jobs well.”

In some cases, there are valid public policy arguments for regulation. The to- and fro- that many of us have in discussions with well-meaning legislators is what the problem really is, whether there is a market failure and - frankly -whether regulation will work, and at what cost. I can agree to disagree with others over where a regulatory line is drawn. But having said that, what I find most distasteful in all the above, is the example of a company with a useful service - “hiring guns to defend yourself, until you get your own guns” - deciding that instead of innovating their way to a larger market, they are FUDing their way to a larger market and potentially “regulating” their way to a larger market.

America has been in the past, a proud nation of innovators, several of whom I have been privileged to know and call “friends.” I fear that we are becoming nation of auditors, where nobody will be able to create, breathe, live, without asking someone else, who has never built anything but has been handed power they did not earn, “Mother, May I?”

Live free, or die.  

* Aside from everything else, I draw the line at someone who knows less than I do about my job tell me how to do it. Unless, of course, there is some reciprocity. I’d like to head the Nuclear Regulatory Commission. Of course, I don’t know a darn thing about nuclear energy, but I’m smart and I mean well. Of course, many people given authority to do things nobody asked them to do - and wouldn’t ask them to do - create havoc because, despite being smart and meaning well, they don’t have a clue. Like the line goes from The Music Man, “you gotta know the territory.”

** Of course we don’t do that – it was a joke. I have a pair of bunnies who live at the end of my street and I’ve become quite attached to them. I always slow down at the end of the street to make sure I don’t hit them. I also hope no predator gets them because they are quite charming. Of course, I might not find them so charming if they were noshing on my arugula, but that’s why you schlep on over to Moss Garden Center and buy anti-deer and anti-bunny spray for your plants. I like kitties, too, for the record.

Book of the Month

Bats at the Library by Brian Lies 
This is a children’s book but it is utterly charming, to the point where I bought a copy just to have one. I highly recommend it for the future little reader in your life. Bats see that a library window has been left open, and they enter the library to enjoy all its delights, including all the books (and of course, playing with the copy machine and the water fountain). The illustrations are wonderful, especially all the riffs on classic children’s stories (e.g.,Winnie the Pooh as a bat, Peter Rabbit with bat wings, the headless horseman with bat wings, etc.). There are two other books in the series: Bats at the Beach and Bats at the Ballgame. I haven’t read them, but I bet they are good, too.

Shattered Sword: The Untold Story of the Battle of Midway by Jonathan Parshall and Anthony Tully
I can’t believe I bought and read, much less I am recommending yet another book on the battle of Midway, but this is a must read. The authors went back to primary Japanese sources and discuss the battle from the Japanese point of view. It’s particularly interesting since so many Western discussions of the battle use as a primary source the account of the battle by Mitsuo Fuchida (who led the air strike on Pearl Harbor and who was at Midway on ADM Nagumo’s flagship), which was apparently largely – oh dear – fabricated to help various participants “save face.” For example, you find out how important little details are such as the US forces regularly drained aviation fuel lines (to avoid catastrophic fires during battles – which the Japanese did not do) and that the Japanese – unlike the US - had dedicated damage control teams (if the damage control team was killed, a Japanese ship could not do even basic damage control). It’s an amazing and fascinating book, especially since the authors are not professional historians but regular folks who have had a lifelong interest in the battle. Terrific read.

Neptune’s Inferno: The US Navy at Guadalcanal by James Hornfischer 
I might as well just recommend every book James Hornfischer has ever written since he is a wonderful naval historian, a compelling storyteller, and incorporates oral history so broadly and vividly into his work. You see, hear, and feel -- to the extent you can without having been there -- what individuals who were in the battle see, heard and felt. Despite having read multiple works on Guadalcanal (this one focuses on the sea battles, not the land battle), I came away with a much better understanding of the “who” and “why.” I was so convinced it was going to be wonderful and I’d want to read it again that I bought it in hardcover (instead of waiting for paperback and/or getting it at the library). Terrif.

Cutting for Stone by Abraham Verghese 
I don’t generally read much contemporary fiction since so much of it is postmodern dreary drivel. However, a friend bought this for me - and what a lovely gift! This is a compelling story (twin brothers born in Ethiopia to a surgeon farther and a nun mother) about what constitutes family - and forgiveness. It has been a best seller (not that that always means much) and highly acclaimed (which in this case does mean much). The story draws you in and the characters are richly drawn. I am not all that interested in surgery but the author makes it interesting, particularly in describing the life and work of surgeons in destitute areas.

NVL versus Coalesce

From An Expert's Guide to Oracle Technology

I'm doing some PL/SQL tuning and I'm examining several existing stored procedures to see how I can speed them up. While I won't share my clients structures or code, I will talk about some of the interesting things I'm doing.

While not the biggest time sink, a particular piece of code is using NVL ver

Those Who Can’t Do, Audit

I am often asked what the biggest change is that I’ve seen in my years working in information security. (Most of those who ask are too polite to ask how many years I’ve worked in information security. I used to say I joined Oracle when I was 8. Now, I tell everyone I started at Oracle when I was 3. Pretty soon, I’ll be a prenatal employee.)

There are too many changes to pick just one, but on a personal level, I seem to spend a lot more time on government affairs (relative to the rest of my job) that I used to: working with others in industry to discuss cybersecurity-related public policy, meeting with staffers on the Hill, commenting on draft legislation, commenting on others’ comments on draft legislation, and so on. On the one hand, it’s a natural result of the amount of dependence that everyone has on cybersecurity and the amount of highly publicized breaches, hacks, and so on we continue to see. You can hardly open the paper – or open an email with a link to the daily electronic paper – without reading about another data breach, “outing” of sensitive information, new thing you didn’t even know was IP-accessible being broken into (e.g., starting cars remotely), and so on.

Legislators often want to Do Something when they see a problem – that’s why we elected them, presumably. I’ve opined in previous blogs on the importance of defining what problem you want to solve, specifying what “it” is that you want to legislate, understanding costs – especially those pertaining to unintended consequences - and so on. I spend more time on govie affairs now because more legislators are proposing legislation that they hope will improve cybersecurity.

Most people who help frame public policy discussions about cybersecurity are well intended and they want to minimize or resolve the problem. However, there are also a whole lotta entities that want to use those discussions to get handed a nice, big, fat “public policy printing press”: ideally, legislative mandates where their businesses receive a direct and lucrative benefit, ka-ching. Their idea of public policy begins and ends at the cash register.

Some businesses would love to be in a position where they could write/draft/influence legislation about how no commercial company can be trusted with security and therefore needs someone (presumably people like themselves) to bless their systems and all their business practices – the more, the merrier. This includes legislative mandates on suppliers – who, as we all know – are busy throwing crappy code over the wall with malice aforethought. Those evil suppliers simply cannot be trusted, and thus the entirety of supplier business practices in security must be validated by Outside Experts* if not the entirety of their code scanned. (Nice work if you can get it.) (Aside: boy, with all the energy we suppliers expend on deliberately producing rotten code and putting our customers’ systems at risk – not to mention our own businesses since we run on our own stuff - it’s really hard to find time for other malicious acts like starving baby rabbits and kitties.**)

At the same time, these businesses are less than forthcoming when asked how they will be vetted: what do they find, how well do they find it and at what cost, since everybody who has worked with code scanning tools know they need to be tuned. (Having to plow through 1000 alleged vulnerabilities to find the three that are legitimate is way too expensive for any company to contemplate doing it.) Now is a perfect time for a disclosure: I am on an advisory board to a company in this sector.

One company in particular is beginning to get under my skin on a lot of levels pertaining to “creating a market for themselves.” Let’s call them SASO (Static Analysis Service Offering) that – ta da! - do static analysis as a service offering. More specifically, they analyze the binaries to do static analysis – an important point. When SASO got started, I thought they had a useful value proposition in that a lot of small software companies don’t have security expertise in-house – one reason why Oracle has strong “business alignment” processes when we acquire a company that include alignment with our assurance practices, which the acquired entities are by-and-large really happy to do. The small company wants to increase their ability to sell into core markets and to do so they may need to convince prospective customers that their code is appropriately secure. They hire SASO to analyze their code, and SASO produces a summary report that tells the customer whether there are significant potentially unfixed vulnerabilities and also gives the small company the details of those vulnerabilities so they can validate actual vulnerabilities and then fix them. Fair enough, and a good service offering, to a point, more on which later. The real benefit to the small company is to get actionable intelligence on problems in their code earlier because for one, it’s almost always easier and cheaper to fix these before products ship, and/or remediate them in some severity order. Also, it’s not always easy to use static analysis tools, yourself: the tools require knowledgeable people to run them and analyze the results.

It’s not, in other words, the “gold star” from SASO they get that has value; it’s the SASO expertise they get to fix issues earlier until they get that expertise in-house and up-to-speed. And they really do need that expertise in-house because security is a core element of software development and you cannot outsource it -- or you will have no security. (Among other things Oracle does in terms of secure development, we have our ethical hacking team conduct product assessments. They are as good as anybody in industry – or better – and they find problems that code analysis tools do not find and never will find. Their expertise is often codified in our own tools that we use in addition to commercially available static and dynamic analysis tools.)

Of course, the market for “helping small inexperienced companies improve the security-worthiness of their code” isn’t big enough, so SASO has been trying to expand the size of their market by means of two understandably - yet obnoxiously - self-promoting activities. One of them is by convincing entire market sectors that they cannot trust their vendors, and their suppliers’ code needs to be “tested” by SASO. You do want to ask, why would anybody trust them? I mean, more and more of SASO’s potential market size is based on FUD. Whereas, the suppliers’ market is based on their customers’ ability to run their entire businesses on the software or hardware. That is, customers bet the digital farm on their suppliers, and the suppliers understand this. If you lose your customer’s confidence for whatever reason – security among them – your customers will switch suppliers faster than you can say “don’t bother with the code scan.” And thus, suppliers are out of business if they screw it up, because their competitors will be ruthless. Competitors are ruthless.

Whom do you think is more trustworthy? Who has a greater incentive to do the job right – someone who builds something, or someone who builds FUD around what others build? Did I mention that most large hardware and software companies run their own businesses on their own products so if there’s a problem, they – or rather, we – are the first ones to suffer? Can SASO say that? I thought not.

Being approached by more and more companies who want our code SASOed has led me to the “enough, already” phase of dealing with them. For the record, we do not and will not allow our code to be SASOed; in fact, we do not allow any third parties access to source code for purposes of security analysis (with one exception noted below). I’ve also produced a canned response that explains to customers why SASO will never darken our code base. It covers the following points, though admittedly when I express them to customers they are less glib and more professional:

1) We have source code and we do our own static analysis. Ergo, I don’t need SASO or anybody else to analyze our binaries (to get at the source code). Moreover, we have not only one but two static analysis tools we use in-house, one of which we built (Parfait, which is built by Sun Labs, a fine outfit we got via the Sun acquisition). Also, as I’ve opined at length in past blog entries, these tools are not “plug and play” – if they were, everybody would run down to Radio Shack, or wait for late night television to see advertisements for Code Scannerama (“finds all security vulnerabilities and malware in 28 seconds -- with No False Positives! But wait, there’s more: order now, and get a free turnip twaddler!”). For the amount of time we’d expend to work with SASO and get useful, accurate and actionable output – we could just get on with it and use the tools we’ve already licensed or already have. SASOing our code hurts us and doesn’t help us.

2) Security != testing. I am all for the use of automated tools to find vulnerabilities in software – it’s a code hygiene thing – but it is not the totality of assurance. Oracle has an extensive assurance program for our products that includes the use of static analysis tools. (In fact, Oracle requires the use of multiple tools, some of which we have developed in-house because no single tool “does it all.”) We also track compliance with our assurance program and report it to our senior management, all the way up to the CEO. In short, we do a far more comprehensive job in security than SASO can do for us. (I also note that – hype to the contrary – these tools will not find much if any malware in code. I am a blond and a bad programmer and it took me about ten minutes to think of ways to put bad stuff in code in a way that would not be detectable by automated code analysis.) “Code analysis” may be a necessary element for security-worthy code but it sure ain’t sufficient.

3) Precedent. It’s a really, really bad precedent to hand your source code to a third party for purposes of “security analysis” because gee, lots of governments have asked for the same privilege. And some governments who ask (I won’t name them, but we all know who they are) engage in state-sponsored industrial espionage so you might as well just kiss your intellectual property good bye as hand your code to those guys (“Wanna start a new software/hardware company? Here’s our source code, O Official Government of Foobaria IP Theft Department!”). Some governments also want to easily do security analysis they then exploit for national security purposes – e.g., scan source code to find problems that they hand to their intelligence agencies to . Companies can’t really say, “we will allow SASO to scan our code if customers nudge us enough” and then “just say no” to governments who want exactly the same privilege. Also, does anybody think it is a good idea for any third party to amass a database of unfixed vulnerabilities in a bunch of products? How are they going to protect that? Does anybody think that would be a really nice, fat, juicy hacker target? You betcha.

(As a matter of fact, Oracle does think that “bug repositories” are potentially nice, fat, juicy hacker targets. Security bug information is highly sensitive information, which is why security bugs are not published to customers and our bug database enforces “need to know” using our own row level access control technology. The security analysts who work for me have security bug access so they can analyze, triage, ensure complete fixes, and so on, but I don’t, because we enforce “need to know,” and I don’t.) I don’t trust any third party to secure detailed bug information and it is bad public policy to allow a third party to amass it in the first place.

4) Fixability. Only Oracle can fix vulnerabilities: SASO cannot. We have the code: they don’t.

5) Equality as public policy. Favoring a single customer or group of customers over others in terms of security vulnerability information is bad public policy. Frankly, all customers' data is precious to them and who would not want to be on the "insider" list? When we do have vulnerabilities in products, all customers get the same information at the same time and the same level of information. It’s basic equity and fairness. It’s also a lot better policy than playing favorites.

6) Global practices for global markets. The industry standard for "security evaluations" is an ISO standard called the Common Criteria (ISO 15408) that looks at the totality of security functionality needed to address specific threats as well as "assurance." A product with no exploitable security defects - that has no authentication, auditing or access control – and isn’t designed to protect against any threats - is absolutely, utterly useless. SASOing your code does not provide any information about the presence or absence of security functionality that meets particular threats. It’s like having someone do a termite inspection on the house you are about to buy but not looking at structural soundness: you know you don’t have bugs – pun intended – but you have not a clue about whether the house will fall down. Common Criteria evaluations do include vulnerability analysis but do not require static analysis. However, many vendors who Common Criteria-evaluate their product also do static analysis, whether or not they get any “credit” for doing it, because it is good practice to do so. Our Common Criteria labs do in some cases get source code access because of the type of analysis they have to do at some assurance levels. But the labs are not “governments,” they are licensed (so there is some quality control around their skill set), access to our source code is incredibly restricted, and we have highly enforceable liability provisions regarding protection of that source code.

7) All tools are not created equal. The only real utility in static analysis is in using the tools in development and fixing what you find in some priority order. In particular, requiring static analysis – e.g., by third parties - will not lead to significant improvement except possibly in the very short term and in the medium to long term will increase consumer costs for no benefit. Specifically: · The state of the art in vulnerability discovery tools is changing very rapidly. Mandating use of a particular tool - or service - will almost guarantee that the mandated tool/service will be obsolete quickly, resulting in added cost for incremental benefit at best. Obsolescence is especially guaranteed if a particular tool or service is mandated because the provider, having been granted a monopoly, has no incentive to improve their service. · The best types of tools used to discover vulnerabilities differ depending on the type of product or service that is being analyzed. Mandating a particular tool is thus suboptimal at best, and an expensive mistake at worst.

I said at the outset that SASO has a useful model for smaller companies that perhaps do not have security expertise in house - yet. But having said that, I offer a cautionary story. A company Oracle acquired some time ago had agreed to have their code SASOed at the request of a customer - before we acquired them. The first problem that created was that the team, by agreeing to a customer request – and continuing to agree to “outside code auditing” until we stopped it – set the market expectation that “gosh, we don’t know what we are doing in security and you shouldn’t trust us,” which is bad enough. What is far, far, worse: I believe this led to a mentality that “those (SASO or other) testing people will find security issues, so I don’t need to worry about security.” I told the product group that they absolutely, positively, needed in-house security expertise, that “outsourcing testing” would create an “outsourcing security” mentality that is unacceptable. Oracle cannot – does not – outsource security.

By way of contrast, consider another company that does static analysis as a service. Let’s call them HuiMaika‘i (Hawaiian for “good group”). HuiMaika‘i provides a service where customers can send in a particular type of code (i.e., based on a particular development framework) and get a report back from HuiMaika‘i that details suspected security vulnerabilities in it. Recently, a HuiMaika‘i customer sent them code which, when analyzed, was found to contain multiple vulnerabilities. HuiMaika‘i somehow determined that this code was actually Oracle code (that is, from an Oracle site) and, because it is their policy not to provide vulnerability reports to customers who submit code not owned by the customer, they returned the vulnerability report to Oracle and not the paying customer. (How cool is that?)

I think their policy is a good one, for multiple reasons. The first is that running such vulnerability-detecting software against code not owned by the customer may violate terms-of-use licenses (possibly resulting in liability for both the customer and the vendor of such vulnerability detecting software). The second reason is that it would be best all around if the vendor of the software being scanned be informed about possible vulnerabilities before public disclosure so that the vendor has a chance to provide fixes in the software. Having said that, you can bet that Oracle had some takeaways from this exercise. First, we pulled the code down wikiwiki (Hawaiian for “really fast”) until we could fix it. And we fixed it wikiwiki. Second, my team is reminding development groups - in words of one syllable - that our coding standards require that any code someone gets from Oracle – even if “just an example” - be held to the same development standards as products are. Code is code. Besides, customers have a habit of taking demo code that does Useful Stuff and using it in production. Ergo, you shouldn’t write crappy code and post it as “an example for customers.” Duh.

Our third takeaway is that we are looking at using HuiMaika‘i onsite in that particular development area. It wasn’t, by the way, just because of what they found - it’s the way they conducted themselves: ethically, professionally, and they didn’t play “vendor gotcha.” Thanks very much, Hui. Small company – big integrity.

Returning to SASO, the other way – besides marketing “you can’t trust suppliers but you can trust us” - in which they are trying to expand their market is a well-loved and unfortunately, often-attempted technique – get the government to do your market expansion work for you. I recently heard that SASO has hired a lobbyist. (I did fact check with them and they stated that, while they had hired a lobbyist, they weren’t “seriously pursuing that angle” – yet.) I have to wonder, what are they going to lobby for? Is it a coincidence that they hired a lobbyist just at the time we have so much draft cybersecurity legislation, e.g., that would have the Department of Homeland Security (DHS) design a supply chain risk management strategy – including third party testing of code - and then use that as part of Federal Acquisition Regulations (FAR) to regulate the supply chain of what the government buys? (Joy.)

I suspect that SASO would love the government to mandate third party code scans – it’s like winning the legislative lottery! As to assurance, many of my arguments above about why we “just say No, to S-A-S-O!” are still equally – if not more – relevant in a legislative context. Secondly, and related to “supply chain” concerns, you really cannot find malware in code by means of static analysis. You can find some coding errors that lead to certain types of security vulnerabilities - exploitable or not. You might be able to find known patterns of “commercially available” malware (e.g., someone downloaded a library and incorporated it into their product, and did not bother to scan it first before doing so. Quelle surprise! The library you got from CluelessRandomWebSite.com is infected with a worm). That is not the same thing as a deliberately introduced back door, and code scans will not find those.

In my opinion, neither SASO - nor any other requirement for third party security testing - has any place in a supply chain discussion. If the concern is assurance, the answer is to work within existing international assurance standards, not create a new one. Particularly not a new, US-only requirement to “hand a big fat market win by regulatory fiat to any vendor who lobbied for a provision that expanded their markets.” Ka-ching.

The bigger concern I have is the increasing amount of attempts by many companies, not to innovate, but to convince legislators that people who actually do stuff and build things can’t be trusted (how about asking themselves the same question?). I recently attending a meeting in which one of the leading presenters was a tenured professor of law whose area of focus is government regulation, and who admitted having no expertise in software. Said individual stated that we should license software developers because “we need accountability.” (I note that as a tenured professor, that person has little accountability since she is “unfireable” regardless of how poor a job she does, or whether there is any market demand for what she wants to teach.)

I felt compelled to interject: “Accountable to whom? Because industry hires many or most of these grads. We can enforce accountability – we fire people who don’t perform. If your concern is our accountability to our customers – we suppliers absolutely are accountable. Our customers can fire us – move to a competitor – if we don’t do our jobs well.”

In some cases, there are valid public policy arguments for regulation. The to- and fro- that many of us have in discussions with well-meaning legislators is what the problem really is, whether there is a market failure and - frankly -whether regulation will work, and at what cost. I can agree to disagree with others over where a regulatory line is drawn. But having said that, what I find most distasteful in all the above, is the example of a company with a useful service - “hiring guns to defend yourself, until you get your own guns” - deciding that instead of innovating their way to a larger market, they are FUDing their way to a larger market and potentially “regulating” their way to a larger market.

America has been in the past, a proud nation of innovators, several of whom I have been privileged to know and call “friends.” I fear that we are becoming nation of auditors, where nobody will be able to create, breathe, live, without asking someone else, who has never built anything but has been handed power they did not earn, “Mother, May I?”

Live free, or die.  

* Aside from everything else, I draw the line at someone who knows less than I do about my job tell me how to do it. Unless, of course, there is some reciprocity. I’d like to head the Nuclear Regulatory Commission. Of course, I don’t know a darn thing about nuclear energy, but I’m smart and I mean well. Of course, many people given authority to do things nobody asked them to do - and wouldn’t ask them to do - create havoc because, despite being smart and meaning well, they don’t have a clue. Like the line goes from The Music Man, “you gotta know the territory.”

** Of course we don’t do that – it was a joke. I have a pair of bunnies who live at the end of my street and I’ve become quite attached to them. I always slow down at the end of the street to make sure I don’t hit them. I also hope no predator gets them because they are quite charming. Of course, I might not find them so charming if they were noshing on my arugula, but that’s why you schlep on over to Moss Garden Center and buy anti-deer and anti-bunny spray for your plants. I like kitties, too, for the record.

Book of the Month

Bats at the Library by Brian Lies 
This is a children’s book but it is utterly charming, to the point where I bought a copy just to have one. I highly recommend it for the future little reader in your life. Bats see that a library window has been left open, and they enter the library to enjoy all its delights, including all the books (and of course, playing with the copy machine and the water fountain). The illustrations are wonderful, especially all the riffs on classic children’s stories (e.g.,Winnie the Pooh as a bat, Peter Rabbit with bat wings, the headless horseman with bat wings, etc.). There are two other books in the series: Bats at the Beach and Bats at the Ballgame. I haven’t read them, but I bet they are good, too.

Shattered Sword: The Untold Story of the Battle of Midway by Jonathan Parshall and Anthony Tully
I can’t believe I bought and read, much less I am recommending yet another book on the battle of Midway, but this is a must read. The authors went back to primary Japanese sources and discuss the battle from the Japanese point of view. It’s particularly interesting since so many Western discussions of the battle use as a primary source the account of the battle by Mitsuo Fuchida (who led the air strike on Pearl Harbor and who was at Midway on ADM Nagumo’s flagship), which was apparently largely – oh dear – fabricated to help various participants “save face.” For example, you find out how important little details are such as the US forces regularly drained aviation fuel lines (to avoid catastrophic fires during battles – which the Japanese did not do) and that the Japanese – unlike the US - had dedicated damage control teams (if the damage control team was killed, a Japanese ship could not do even basic damage control). It’s an amazing and fascinating book, especially since the authors are not professional historians but regular folks who have had a lifelong interest in the battle. Terrific read.

Neptune’s Inferno: The US Navy at Guadalcanal by James Hornfischer 
I might as well just recommend every book James Hornfischer has ever written since he is a wonderful naval historian, a compelling storyteller, and incorporates oral history so broadly and vividly into his work. You see, hear, and feel -- to the extent you can without having been there -- what individuals who were in the battle see, heard and felt. Despite having read multiple works on Guadalcanal (this one focuses on the sea battles, not the land battle), I came away with a much better understanding of the “who” and “why.” I was so convinced it was going to be wonderful and I’d want to read it again that I bought it in hardcover (instead of waiting for paperback and/or getting it at the library). Terrif.

Cutting for Stone by Abraham Verghese 
I don’t generally read much contemporary fiction since so much of it is postmodern dreary drivel. However, a friend bought this for me - and what a lovely gift! This is a compelling story (twin brothers born in Ethiopia to a surgeon farther and a nun mother) about what constitutes family - and forgiveness. It has been a best seller (not that that always means much) and highly acclaimed (which in this case does mean much). The story draws you in and the characters are richly drawn. I am not all that interested in surgery but the author makes it interesting, particularly in describing the life and work of surgeons in destitute areas.

PL/SQL Expert Practices from Apress

A simple example of a secure context

From An Expert's Guide to Oracle Technology

This simple example isn't much different that one I did quite a while back but it's more complete. If you have your own database, you can run this and get a working example of using a secure context.

I was asked for an example and I was blocked from my blog from work so I had to code a new one. I tho