Security testing at Oracle includes both functional and non-functional activities for verification of products' features and quality. Although these types of tests often target overlapping product features, they have orthogonal goals and are carried out by different teams. Functional and non-functional security tests complement each other to ensure comprehensive security coverage of Oracle products.
Functional Security Testing
Functional security testing is typically executed by regular product QA teams as part of normal product testing cycle. During this testing QA engineers verify conformance of implemented security features to what had been previously agreed upon in the functional specifications during the architectural and checklist reviews process.
Security Assurance Analysis and Testing
Security assurance analysis and testing verify security qualities of Oracle products against various types of attacks. There are two broad categories of tests employed for testing Oracle products: static and dynamic analysis, which are further described in the sections below. These tests fit differently in the product development lifecycle and tend to find different categories of issues, so they are used together by Oracle product teams.
Static security analysis of source code is the initial line of defense used during the product development cycle. Oracle uses a static code analyzer from Fortify Software, an HP company, as well a variety of internally developed tools, to catch problems while code is being written. Products developed in most modern programming languages (such as C/C++, Java, C#) and platforms (J2EE, .NET) are scanned to identify possible security issues. This type of checking works very well for identifying buffer overflows and memory leaks in C/C++ code, resource handling issues in J2EE and .NET, finding improper credentials handling, various injections, incorrect system configurations, and so on. One of the drawbacks of this type of analysis is high level of false positive reports – i.e., quite a few of the reported items do not constitute a real issue. Typically, analysis of these scan reports involves senior engineers from the product teams, who are well-familiar with the product code, can sort out false positives from real issues, and who are familiar with Fortify analysis and rules writing to reduce the number of false positives.
Dynamic analysis activity always takes place during latter phases of product development – at the very least, the product or component should be able to run. Although this may vary among Oracle organizations, typically this activity is handled by Security QA team (or a similar dedicated group), which may be shared by multiple product teams. Dynamic analysis is aimed at externally visible product interfaces and APIs and frequently relies on specialized tools for testing. Both manual and automatic tools are used for testing within Oracle. Automatic tools employ fuzzing technique to test network-accessible product interfaces and protocols, while manual tools require making the modifications by hand, but allow for much greater accuracy and precision.