As Published In
Oracle Magazine
November/December 2009

FEATURE


Consolidate. Compress. Control.

By David Baum

Improve quality of service and reduce costs with Oracle Database 11g Release 2.

Demands on information systems are greater than ever. DBAs must manage more information and deliver that information to users with a high quality of service. They must reduce costs while deriving greater value out of existing investments. And they must execute all that while mitigating risk.

“There are three major things that organizations are faced with today,” says Andy Mendelsohn, senior vice president, Database Server Technologies, at Oracle. “Number one, IT needs to deliver the best information to the business, so managers can use that information to make the best business decisions. Number two, the business wants this to work 24/7. Number three, every budget is at best flat, but IT is being asked to do even more. So IT needs to get more efficient.”

Oracle Database 11g Release 2 provides a solid foundation that allows companies to address these challenges and control and manage their information more effectively. With Oracle Database 11g Release 2, businesses can increase application performance while consolidating their processing and storage infrastructures and using advanced data compression techniques to further reduce storage requirements.

Mendelsohn adds that the consolidation and compression features of Oracle Database 11g Release 2 can help organizations do more than deliver high quality of service. “We can help lower the cost of the servers underneath the databases by leveraging our grid technologies,” he says, “and we can help lower the cost of storage by leveraging our compression and partitioning technologies.”

Other new functionalities in Oracle Database 11g Release 2 simplify the addition, removal, and consolidation of clustered servers and eliminate idle redundancy in the data center. “Our maximum availability architecture allows customers to get the most return on their hardware investment by using redundant servers and storage not only to provide protection from failures but also to improve the performance of production systems,” Mendelsohn says.

In addition to simplifying clustering and high-availability technologies, Oracle Database 11g Release 2 delivers new technologies for DBAs. “We’re providing DBAs with a lot of automation technologies to make them much more productive,” Mendelsohn says.

Several companies have already seen the benefits of using Oracle Database 11g Release 2. Here’s how.

Banking on Reducing Costs

Like most large retail banks, Postbank Systems AG is finding innovative ways to reduce the cost of doing business. For Jens-Christian Pokolm and other employees in Postbank Systems’ IT department, cost-cutting measures include consolidating their database infrastructure, deploying standardized installations of Oracle Real Application Clusters (Oracle RAC), and simplifying routine database administration and maintenance chores.

“The cheapest CPU is the CPU you don’t have to buy,” says Pokolm, a database architect at Postbank Systems, which has more than 100 terabytes of data in Oracle databases. “With Oracle Database 11g Release 2, we’re seeing faster I/O for the storage system and quicker processing for large queries. This means we don’t need as many CPUs to perform large processing tasks.”

Postbank Systems is just one of many companies that beta-tested Oracle Database 11g Release 2. The bank was especially interested in discovering how Oracle Database 11g Release 2 might improve on its query-processing capabilities.

Postbank Systems requires powerful query-processing capabilities to identify likely customers for new bank products and services—commonly called segment marketing. Marketing personnel typically target customers based on their bank balance, income, family size, and whether they rent or own their primary residence, among other variables. Identifying these market segments entails drilling down to extract the correct records from the bank’s data warehouse.

“Out of 14 million customers, we might target 60,000,” says Pokolm. “This is a huge processing task that is now up to 30 percent faster with Oracle Database 11g Release 2.”

Postbank attributes these performance gains, in part, to automatic in-memory parallel execution of queries. Oracle Database 11g Release 2 can determine the optimal degree of parallelization for a query based on the available resources. The database also automatically partitions a large compressed table (or a smaller noncompressed table) into the available memory across all the servers in an Oracle grid. The database then localizes parallel query processing to the data in memory on the individual nodes.

Best of all, says Pokolm, there’s no need to monitor these database functions, because the parallelism is controlled by the database kernel. “Based on the query that is submitted, the database uses all available CPUs to make sure you get the best results,” he says.

Server and Storage Consolidation

As database efficiencies improve, Postbank Systems can simplify its Oracle RAC configurations by reducing the number of servers and storage devices it needs. For example, previously Postbank Systems had to install three storage subsystems behind every high-availability cluster. With the Oracle Automatic Storage Management functionality of Oracle Database 11g Release 2, the bank is reducing storage installation and administration time by 20 to 30 percent.

Oracle Automatic Storage Management automates the striping and mirroring of a database, eliminating the need to purchase third-party volume-management software. As data volumes increase, additional disks can be added, and Oracle Automatic Storage Management will automatically restripe and rebalance the data across available disks to ensure optimal performance. Similarly, disks that report errors can be removed from the disk array, and Oracle Automatic Storage Management will readjust data accordingly. Now the Oracle Automatic Storage Management Cluster File System, a new feature of Oracle Database 11g Release 2, enables Oracle Automatic Storage Management to be used as a volume manager and automatically manages all operating system files (previously it managed database files only).

In Oracle Automatic Storage Management, configuration agents help control disk groups graphically, and Pokolm finds this method much easier to control. “These tasks were formerly left to our DBAs, but now the UNIX administrators can do them as well,” he says.

In addition to the changes in Oracle Automatic Storage Management, Pokolm likes the Oracle Database 11g Release 2 improvements in Oracle Scheduler, which can automatically alert administrators to a fault situation; Automatic Block Repair; and Policy-Based Cluster and Capacity Management, which allocates and reassigns capacity based on policies that the user defines. Policy-Based Cluster and Capacity Management enables dynamic capacity assignment and faster resource failover.

“You can configure baseline performance such that the clusterware will automatically start a new instance on another box to maintain service levels and ensure quality of service for end users,” he says. “Once you define your baseline, this is automatic.”

Advanced Compression Capabilities

Grupo Elektra, one of Latin America’s leading specialty retailers and financial services companies, is looking to Oracle Database 11g Release 2 to help solve the challenges of managing and using a large data warehouse. The company owns more than 1,000 stores in Mexico, Guatemala, Honduras, Peru, El Salvador, Panama, Argentina, and Brazil, from which it processes about 15 gigabytes of point-of-sale data on a daily basis. Grupo Elektra’s multiterabyte Oracle data warehouse supports many applications that business users rely on.

“Each day we acquire 50 to 60 million rows of XML data, as well as data from other operational databases on different platforms including Oracle, AS/400, and Microsoft SQL Server—in all, about seven sources of information—translate it to relational format, and update our Oracle data warehouse,” says Fernando Hernandez, DBA and manager of Grupo Elektra’s production data warehouse environment. “The volume of information that our company generates is expected to grow steadily for the foreseeable future.”

To help Grupo Elektra manage these extract, transform, and load (ETL) operations as well as to streamline query and reporting activities for more than 36,000 employees in multiple time zones, Hernandez is enthusiastic about the compression capabilities of Oracle Database 11g Release 2.

“Oracle Database 11g Release 2 lets us combine various compression options according to historical partitions, representing cold data versus hot data,” says Hernandez, adding that cold data refers to archival information, and hot data refers to frequently accessed production information. 

Snapshots



 Postbank Systems AG

www.postbank.de
Location: Bonn, Germany
Industry: Banking
Employees: 1,400
Oracle products: Oracle Database, Oracle Real Application Clusters, Oracle WebLogic Server, Oracle Grid Control
 

 Grupo Elektra

www.grupoelektra.com.mx
Location: Mexico City, Mexico
Industry: Retail and financial services
Employees: 36,518
Oracle products: Oracle Database 11g, Oracle Real Application Clusters, Oracle Application Server, Oracle Automatic Storage Management
 

 XIUS-bcgi

www.xius-bcgi.com
Location: Bedford, Massachusetts
Industry: Telecommunications
Employees: More than 500
Oracle products: Oracle Database 11g, Oracle Real Application Clusters, Oracle Partitioning, Oracle Application Server, Oracle business intelligence solutions, Oracle Automatic Storage Management, Oracle Data Guard
 

 McKesson

www.mckesson.com
Location: San Francisco, California
Industry: Healthcare
Employees: 32,000
Oracle products: Oracle Database 11g, Oracle Real Application Clusters, Oracle Application Server, Oracle Automatic Storage Management

The data warehouse is running on a three-node Sun Solaris cluster with Oracle Database, Oracle RAC, and Oracle Automatic Storage Management. By using the compression and partitioning technologies of Oracle Database 11g Release 2 in this environment, Grupo Elektra has dramatically improved performance for update statements, aggregate queries, and other essential database operations. Hernandez gives an example: “Multitable joins with complex group-by operations that used to take 45 seconds now take less than 10 seconds,” he says.

In part, Hernandez says that this performance improvement has been achieved with online transaction processing (OLTP) table compression. A feature of Oracle Advanced Compression, OLTP table compression allows data to be compressed during all types of data manipulation operations. The savings cascade through the enterprise, because backups and copies of the databases also use less storage.

“Obviously I like the storage savings,” Hernandez says, but he sees other possibilities for the compression technologies, as well. “With compressed data, we can reduce the size of the database, which saves time doing backups,” he says. “And we can perform a lot of activities in less time—testing and development, for example—for which we need to export and import a lot of information.”

Hernandez also reports seeing consequent memory savings, because each block on the disk drive can deliver more data to memory during I/O operations. And because Grupo Elektra plans to migrate its massive database, Hernandez says compressing the information beforehand will significantly reduce the migration time and therefore the downtime for the data warehouse.

All of these improvements put Grupo Elektra further on the road to success, Hernandez says. “Oracle gives us the performance, stability, and the response times that we need,” says Hernandez. “When we moved to Oracle Database, we started a history of success.”

Controlling Demand and Capacity

XIUS-bcgi, a division of Megasoft Limited, wants to improve Busy Hour Call Attempts (BHCA), a telecommunications metric that refers to the number of telephone calls attempted at the busiest hour of the day. This measurement is important to mobile operators because it helps them determine how much capacity they need on their voice and data networks.

“The higher the BHCA, the higher the stress on the network processors and on our mobile telephone clients,” says Satyendra Kumar, chief database architect at XIUS-bcgi’s Global Delivery Headquarters in Hyderabad, India. “We continually strive to improve the BHCA capacity of our platforms,” says Kumar, who is also the cofounder of the All India Oracle Users Group and serves as its vice president. “Thanks to Oracle Database 11g Release 2 features such as Zero Downtime Patching for Oracle RAC and Automated Application Failover in Oracle Data Guard configurations, we can provide unmatched availability to our customers.”

XIUS-bcgi relies on Oracle RAC, Oracle Automatic Storage Management, and Oracle Data Guard to increase the scalability and performance of its products. “With application failover in Oracle Data Guard configurations, our applications can transparently fail over to a new primary database, and Oracle Active Data Guard will help us offload major queries to a standby database and reduce the load on the primary database,” he says.

When it comes to managing telecommunications solutions, Kumar finds that the Oracle Database 11g Release 2 enhancements to Oracle Enterprise Manager capabilities for clusterware resource management have made a big difference.

“The enhancements to Oracle Enterprise Manager reduce the downtime required for maintenance, ease daily management, and improve the performance of the overall Oracle RAC environment,” Kumar says.

Oracle Enterprise Manager also provides a graphical user interface (GUI) to manage the Oracle Automatic Storage Management Dynamic Volume Manager and Oracle Automatic Storage Management Cluster File System as part of Oracle Automatic Storage Management. The GUI makes it easier to manage the environment, whether it is a standalone server or a cluster deployment of Oracle Automatic Storage Management. The centralized console provides a consistent interface for managing volumes, database files, and file systems as well as Oracle databases.

Easier to Use

There are also significant ease-of-management improvements in Oracle RAC for Oracle Database 11g Release 2. Industry analyst Carl Olofson, a research vice president at IDC responsible for information management, data integration, and software research, says that Oracle has made steady progress in enhancing Oracle RAC so that it is easier to set up and administer. “You no longer need special knowledge in building database clusters, cluster file systems, and so forth,” he says. 

Oracle Real Application Clusters One Node


Oracle Real Application Clusters (Oracle RAC) One Node is a single-node version of Oracle RAC. A new enterprise option to Oracle Database 11g Release 2, Oracle RAC One Node offers greater benefits than traditional server virtualization to single-instance databases running on a hardware cluster.
 

Oracle RAC lets companies cluster servers and build an infrastructure that’s flexible, scalable, and highly available. Oracle RAC One Node provides the fault tolerance and flexibility of Oracle RAC but runs databases on just a single server. Oracle RAC One Node capabilities include support for cluster failover, rolling upgrades of hardware and software, and online movement of a database between servers in the grid.
 

Mark Townsend, Oracle’s vice president of product management for database technology, says that companies use Oracle RAC One Node not only to gain the high-availability characteristics of a cluster but also as a way to consolidate line-of-business applications onto cost-effective platforms.
 

“Large IT shops often have hundreds or even thousands of smaller databases,” Townsend says. “With Oracle Database 11g Release 2, Oracle offers Oracle RAC One Node, which allows customers to take advantage of clustered servers without having to step into a full Oracle RAC-enabled deployment.”

XIUS-bcgi’s Kumar agrees. “The new features for Oracle Clusterware upgrades, management, and recovery have become easy and require a lower degree of DBA expertise,” he says. “Moreover, the ease of management of the entire stack of database and clusters helps us to reduce costs for operation personnel.”

What Kumar values most in the improvements to Oracle Database 11g Release 2, though, is how key new functionality—from Zero Downtime Patching for Oracle Clusterware and Oracle RAC to Oracle Enterprise Manager-based clusterware resource management—will improve customer service. “Above all, it helps us to proactively monitor and troubleshoot issues for our customers,” he says.

All of these features help XIUS-bcgi get the job done. “We chose Oracle because of its unmatched performance and Oracle RAC technology,” Kumar says. “Implementing Oracle RAC and Oracle Automatic Storage Management addresses all our business challenges, including scalability, performance, and manageability.”

Consolidation Yields Benefits

Like XIUS-bcgi, McKesson is looking to Oracle Database 11g Release 2 not only to improve its internal operations but also to improve the Oracle technology-based solutions it provides to customers. McKesson Provider Technologies (MPT), a division of McKesson Corporation, has almost 700 hospital clients running earlier versions of Oracle Database as part of its clinical, patient care, financial, supply chain, and strategic management software solutions. MPT has been using Oracle solutions internally, as well as supplying them to its customers since Oracle released Oracle7.

Due to federal regulations such as the Health Insurance Portability and Accountability Act (HIPAA), McKesson must manage an immense volume of data that needs to be stored securely for long periods of time while finding ways to minimize rising costs.

“We use Oracle technologies to secure, store, and access this information,” says Jay Caviness, a senior systems engineer and Oracle DBA for MPT. “Almost all of our new clients and most of our existing clients are adopting commodity x86 hardware running Linux. We’ve taken full advantage of Oracle RAC and Oracle Automatic Storage Management to create scalable, high-availability solutions for these clients.”

Caviness describes MPT’s newest offering, the McKesson Horizon Appliance, as a self-contained data center in a blade server chassis. “All of our Oracle applications, databases, application servers, and storage devices are hosted within the appliance with common, standardized interfaces,” he says. 

Next Steps



READ more about
 Oracle Database 11g Release 2
Oracle RAC One Node

 VIEW Webcast on Oracle Database 11g Release 2 features

DOWNLOAD
Oracle Database 11g Release 2
Oracle Database 11g Release 2 documentation

Oracle Database 11g Release 2’s new storage capabilities will improve the McKesson Horizon Appliance even further. Caviness says the new Oracle Automatic Storage Management capabilities in Oracle Database 11g Release 2—including moving Oracle Cluster Registry and voting disks into Oracle Automatic Storage Management and Oracle Automatic Storage Management Cluster File System—will simplify configuration and management by reducing costs and minimizing maintenance chores.

“Oracle Automatic Storage Management Cluster File System is a huge feature,” Caviness says.

New capabilities in Oracle Database 11g Release 2 also simplify cluster management. Policy-Based Cluster and Capacity Management enables dynamic capacity assignment when needed to provide server capacity in accordance with the priorities set in policies.

“Because of the reduction in overhead provided by policy-managed databases and server pools in Oracle Database 11g Release 2, we can reduce total cost of ownership for these appliances by more than 33 percent,” Caviness says.

McKesson’s typical database implementation includes two or three clusters with two to four nodes per cluster. Caviness believes that Oracle RAC 11g will allow customers to consolidate these Oracle RAC installations into a single cluster with fewer nodes.

“Based on our extensive testing, Oracle Database 11g Release 2 simplifies the addition, removal, and provisioning of nodes a great deal,” he says. “The time that we formerly spent changing connection information for client software has been eliminated completely. Additionally, the new Grid Plug and Play features in Oracle Database 11g Release 2 simplify the addition and deletion of nodes in a cluster and make it easier to provision capacity to accommodate greater consolidation and future business growth.” 

Oracle 11g Database New Features


The many new features of and enhancements to Oracle Database 11g Release 2 include upgrades to application development, availability, business intelligence and data warehousing, clustering, performance, security, server manageability, and unstructured-data management. For a complete list and description of the new features, see the Oracle Database New Features Guide 11 g Release 2 , available with the Oracle Database 11g Release 2 documentation .
 

Application Development

Application developers can benefit from new application development features of Oracle Database 11g Release 2. Oracle Application Express includes several enhancements:

 

  • Application date format

  • Custom themes

  • Declarative binary large object (BLOB) support

  • Documented JavaScript libraries

  • Report printing

  • Forms conversion

  • Security

  • Interactive reporting region

  • Runtime-only installation



Availability

To support high availability, Oracle Database 11g Release 2 includes enhanced backup-and-recovery features:

 

  • Automatic Block Repair

  • Backup to Amazon Simple Storage Service (S3) using Oracle Secure Backup cloud computing

  • DUPLICATE without connection to target database

  • Tablespace point-in-time recovery

  • DUPLICATE options



New features in Oracle Database 11g Release 2 make online application maintenance and upgrades easier:

 

  • Edition-Based Redefinition

  • CREATE or REPLACE TYPE to allow FORCE

  • Fine-grained dependencies for triggers

  • IGNORE_ROW_ON_DUPKEY_INDEX hint for INSERT statement



Edition-Based Redefinition makes it possible to change an application’s database objects without interrupting the application’s availability, by making the changes in a new edition.

High availability in Oracle Database 11g Release 2 is also enhanced by new Oracle Data Guard features:

 

  • Compressed table support in logical standby databases

  • Configurable real-time query apply lag limit

  • Integrated support for application failover in an Oracle Data Guard configuration

  • Support for as many as 30 standby databases



Business Intelligence and Data Warehousing

Oracle Database 11g Release 2 improves many business intelligence and data warehouse capabilities, including analytics:

 

  • Analytic Functions 2.0

  • Recursive WITH clause



Data loading has been made simpler:

 

  • EXECUTE privilege for DIRECTORY objects

  • Preprocessing data for ORACLE_LOADER access driver in external tables



Partitioning in Oracle Database 11g Release 2 has been improved in several ways:

 

  • Virtual columns in the primary key or foreign key for reference partitioning

  • System-managed indexes for list partitioning



Features that enhance performance and scalability have been added to Oracle Database 11g Release 2:

 

  • In-memory parallel execution

  • Minimal-effort parallel execution—auto degree of parallelism and queuing

  • DBMS_PARALLEL_EXECUTE package



Many improvements have been made to Oracle Warehouse Builder:

 

     

  • Find support in mapping editor
  • Business intelligence tool integration

  • Copy and paste of operators and attributes in mapping editor

  • Current configuration list in Design Center toolbar

  • Support for flat-file imports

  • Table function support

  • Experts available in Editor menu

  • Expression editing in Operator Edit dialog box

  • Grouping and spotlighting of objects in mapping editor

  • Management of locations registered in multiple control centers

  • User interface for managing locations

  • Key lookup operator enhancements

  • Mapping debugger enhancements

  • New Oracle JDeveloper-style user interface

  • Operator references included in generated PL/SQL code

  • Quick mapper

  • Repository Browser changes

  • Simplified Oracle Warehouse Builder repository upgrades

  • Support for extracting data from tables containing LONG datatype and subqueries in JOIN operator



Clustering

Oracle Real Application Clusters (Oracle RAC) now provides many features for simplifying installation and management of clusters and Oracle RAC databases:

 

  • Configuration Assistants, which support new Oracle RAC features

  • Cluster Verification Utility

  • Integration of Cluster Verification Utility and Oracle Universal Installer

  • Cluster Time Service

  • Oracle Cluster Registry enhancements

  • Grid Plug and Play

  • Oracle Restart

  • Policy-Based Cluster and Capacity Management

  • Oracle Clusterware resource modeling

  • Role-separated management

  • Agent development framework

  • Zero Downtime Patching for Oracle Clusterware and Oracle RAC

  • Oracle Enterprise Manager-based Oracle Clusterware resource management

  • Oracle Enterprise Manager provisioning for Oracle Clusterware and Oracle RAC

  • Oracle Enterprise Manager support for Grid Plug and Play and Oracle Restart

  • Oracle Universal Installer support for removing Oracle RAC installations

  • Improved deinstallation support with Oracle Universal Installer

  • Scripts that support DBControl database downgrades

  • Oracle Restart integration with Oracle Universal Installer

  • Support for out-of-place Oracle Clusterware upgrade

  • Server Control Utility enhancements

  • Oracle Universal Connection Pool integrations



These improvements include faster relocation of services on node failure, support for as many as five copies of Oracle Cluster Registry for improved availability of the cluster, and storage of Oracle Cluster Registry in Oracle Automatic Storage Management.
 

Performance

Server performance capabilities are improved by new features in Oracle Database 11g Release 2:

 

  • Stored outlines migration to SQL plan management

  • Client result cache using table annotations support

  • Support for 4K-sector disk drives security



New auditing and encryption features of Oracle Database 11g Release 2 make deployments more secure:

 

  • Audit Trail Cleanup

  • Tablespace master key



Audit Trail Cleanup provides for automatic file completion; the new encryption key management functionality enables complete integration with hardware security modules and increased performance for transparent data encryption.
 

Server Manageability

Oracle Database 11g Release 2 is easier to use than its predecessors, because of features that enhance manageability:

 

  • Oracle Automatic Storage Management Cluster File System

  • Oracle Automatic Storage Management Dynamic Volume Manager

  • Oracle Automatic Storage Management FS Snapshot

  • Oracle Cluster Registry and voting disk on Oracle Automatic Storage Management

  • Oracle Automatic Storage Management intelligent data placement

  • Oracle Automatic Storage Management Configuration Assistant

  • Oracle Automatic Storage Management file access control

  • ASMCMD command extensions

  • Oracle Enterprise Manager support for Oracle Automatic Storage Management Cluster File System

  • Oracle Enterprise Manager integration for Oracle Automatic Storage Management file access control



Oracle Automatic Storage Management now supports all types of data, including database files, Oracle Clusterware files, and file system data such as Oracle homes and binaries.

Oracle Automatic Storage Management Cluster File System extends Oracle Automatic Storage Management by providing a file system for Oracle binaries, report files, trace files, alert logs, and other application data files.

Database management features complement storage management features for general ease of use:

 

  • Oracle RAC configuration for Oracle Enterprise Manager

  • Patch application with DBControl

  • Automatic patching of time stamp with time zone data

  • Local time zone datatype

  • Segment creation on demand

  • Zero-size unusable indexes and index partitions

  • Metadata SXML Comparison Tool

  • Replay compare period reports

  • SQL Tuning Set comparisons

  • Sampling for Oracle Active Data Guard

  • Oracle Exadata simulation

  • Global Oracle RAC active session history report plus Automatic Database Diagnostic Monitor backward compatibility

  • Oracle multithreaded server and Oracle Streams support

  • Parallel query plus alt plan recording, export of SQL Tuning Set enhancements

  • Synchronization controls Unstructured-Data Management



Developers can achieve significant productivity gains with improvements to unstructured-data management capabilities:

 

  • Attribute extraction of requested attributes only

  • Client-side Digital Imaging and Communications in Medicine (DICOM) attribute extraction

  • DICOM enhancements

  • Watermarking and image processing enhancements

  • Visualization framework

  • Network data model enhancements

  • GeoRaster Java API

  • Raster reprojections and ground control point-based georeferencing

  • Routing and geocoding enhancements

  • Database file system

  • LZO support for SecureFiles

  • Binary XML enhancements

  • Oracle XML DB repository performance improvements and guidelines

  • XMLIndex enhancements

  • XMLType partitioning



Other Database Capabilities

Oracle Database 11g Release 2 supports other functionality as well, including improvements to Oracle Scheduler:

 

  • E-mail notification

  • File watcher

  • Multiple-destination jobs

  • Remote database jobs



Other enhancements improve database operations:

 

  • Flashback Data Archive support for data definition languages

  • Instance caging

  • Data Pump Legacy Mode utility

  • IPv6 support for JDBC thin clients, Java virtual machine, and Java Debugger

  • Oracle Enterprise Manager Support Workbench for Oracle Automatic Storage Management

 


 

David Baum ( david@dbaumcomm.com ) is a freelance business writer based in Santa Barbara, California 

 

  



Send us your comments