Oracle WebCenter Spaces Performance Analysis and Tuning in Practice - Part 1

by JayJay Zheng

A post-development performance tuning practice for addressing Oracle WebCenter Spaces performance issues

Write for OTN
Earn money and promote your technical skills by writing a technical article for Oracle Technology Network.
Learn more

April 2013

Read Part 1: Configurations: Oracle WebCenter Spaces and Web Cache Server
Read Part 2: JVM Monitoring and Tuning and Oracle WebCenter Spaces Application Tuning

Downloads
download-icon13-1Oracle WebCenter Portal
download-icon13-1Oracle JRockit

Introduction

I have been engaged with performance analysis and tuning of the Oracle WebCenter Spaces application for various customers. This article is based on my previously analyzed data but the principles described here can be applied to any generic Oracle WebCenter Spaces application.

This is a post-development performance tuning practice. The goal is to streamline the process of addressing Oracle WebCenter Spaces performance issues and align configurations and code modifications with best practices. To be specific, the WebCenter Spaces application examined here was taking an average of 3-5 seconds for page to page navigation with 10-20 concurrent users of the business requirement. The goal is to reduce page to page navigation to seconds.

The article covers the findings, modifications and recommendations and includes the best practices that can be applied to generic WebCenter Spaces applications. The following are listed as top performance areas and examined for potential performance improvement:

  • Oracle WebCenter Spaces OHS Configurations
  • Web Cache Server Configurations
  • JVM Monitoring and Tuning
  • Oracle WebCenter Spaces Application Tuning

This article is part one of a two-part series. Part one focuses on:

  • Oracle WebCenter Spaces OHS Configurations
  • Web Cache Server Configurations

The Oracle WebCenter Spaces application examined in this article is based on Oracle WebCenter Patch Set 4 with Oracle Weblogic Server 11.1.1.5. It is understood that there are potential performance improvements on Oracle WebCenter patch set 5. Several monitoring tools are used throughout this performance practice, such as Oracle Enterprise Manager Control, Oracle Weblogic Console, Oracle Web Cache Manager Console, Oracle JRockit Mission Control, and third party browser plugins including Firebug and, YSlow. A review of the relevant Oracle documentation is recommended for details of the various functionalities.

Limitations

Many factors can contribute performance limitations. Two are worthy of mention here:

  • Hardware Resources
    Performance objectives are limited by constraints, such as configurations of hardware: CPU type, disk size, disk speed, physical memory, etc. Hardware resources can be a bottleneck in a performance perspective if the CPU or memory usage of a server remains at a constant high level. It is always recommended to optimize hardware resources to the extent that budget allows.
  • Application Architecture
    Application architecture defines the interaction between applications, middleware components, databases and various business domains. It is a potential high-level application performance constraint if the application architecture does not adhere to best practices for performance, scalability and high availability.

It is recommended that capacity planning reflect both an assessment of system performance goals and a thorough understanding of the application. Information on the factors listed below can be gathered to understand the performance goals:

  • Anticipated number of concurrent users
  • Number and size of requests
  • Amount and consistency of data
  • Target CPU/memory utilization
  • Network latency

It is also important to understand user expectations. Application developers, database administrators, and system administrators must be careful to set appropriate performance expectations for users. When the system carries out a particularly complicated operation, response time may be slower than when it is performing a simple operation. Users should be made aware of which operations might take longer.

For example, as an Oracle WebCenter Spaces customer, you might want to ensure that 90% of the users experience response times no greater than three seconds, with a maximum response time for all users is 10 seconds. The application may include a variety of operations with different characteristics and acceptable response times. It is recommended to set measurable goals for each.

Application Architecture

The application architecture could be a high level performance bottleneck if not laid out properly. Figure 1 shows an architecture diagram for an Oracle WebCenter Spaces application with components of caching capabilities. The application is composed of a load balancer, clustered Oracle Web Cache servers, clustered Oracle WebCenter Spaces servers, a content server, and a database.

Figure 1
Figure 1: Oracle WebCenter Spaces application architecture

 

For performance best practices, it is highly recommended to have the distributed application on the clustered configurations. By setting up a clustered environment, performance can benefit vastly by distributing the loads over several managed servers so that the JVM garbage collection impact is reduced and the response stability is improved.

The caching mechanism can be applied on various components. As depicted in Figure 1, caching can be implemented in the following areas:

  • Client browser caching
  • Web Cache Server caching
  • Security caching
  • Oracle Coherence caching
  • Database caching

This article covers the first two caching mechanisms: client browser caching and web cache server caching. The other three caching methods are beyond the scope of this article.

Other caching mechanisms can also be implemented when applicable, such as Idoc Script caching if the site studio template is used in Oracle WebCenter Spaces, or portlet expiration caching when a portlet is used. Furthermore, an Oracle HTTP Server (reverse proxy) could be added to a UCM server for load distribution.

Oracle WebCenter Spaces OHS Configurations

In this exercise, the Oracle HTTP Server of Oracle WebCenter Spaces will be configured to enable browser caching. To improve performance, caching is an important area to review and usually an efficient way to gain improvements. Browser cache is a mechanism to store temporary data (such as HTML pages, images, CSS files, JavaScript files) locally on the client side in order to satisfy subsequent requests from local cache if conditions are met.

Besides caching, compression is another mechanism that can be used between web servers and web clients to make better use of available bandwidth and to provide faster transmission speeds between both. There are two main benefits that browser cache and compression can provide:

  • Lower Latency - since the requests are satisfied from the local cache and it takes less time for the application to display in the browser rather than retrieving from the web server.
  • Less network traffic - since fewer and smaller-size items are retrieved from the web server, consumption of network bandwidth is reduced, increasing transmission speed.

Enable server configuration for browser caching

Oracle WebCenter Spaces, like other web applications, has static resources that can be cached from the client browser side. To enable server configuration for browser caching, http request patterns must be analyzed first. The Firebug plugin for Firefox can be used for this purpose.

As shown in Figure 2, http requests are recorded during navigation to a Spaces page.

Figure 2
Figure 2: HTTP requests recorded during navigation

 

Every request has its request URL path and request header. By checking the entire request URL using the Firebug plugin, it is easy to identify the common request path. For example, the following paths have been identified in Oracle WebCenter Spaces (integrated with a WebCenter content server):

http://host/webcenter/adf/
http://host/webcenter/afr/
http://host/webcenter/javascript/
http://host/webcenter/webcenter-spaces-resources/
http://host/webcenter/content/
http://host/webcenter/faces/
http://host/cs/
http://host/get/

Please note that the URL pattern might be different in your environment. You can check all the requests from different pages within Oracle WebCenter Spaces to find out which data types are being requested. Pay attention to the data types and relative paths for caching. The images, JavaScript files, CSS files can be cached and compressed. But specific Oracle WebCenter Spaces pages should not be cached.

http://host/webcenter/faces/ is the path to request .jspx pages within Oracle WebCenter Spaces. It should not be cached at any time.

The following relative paths can be cached and compressed:

/webcenter/afr
/webcenter/adf
/webcenter/javascript
/webcenter/webcenter-spaces-resources
/content
/cs
/get

The http response header can be used to perform specific caching and assembly processes. Certain http header fields need to be added to the response so that the browser will cache the data that is included in the defined response path. The following fields can be used:

  • Last-modified: the last modified date for the requested object
  • Expires: the date/time after which the response is considered stale
  • Cache-Control: tells all caching mechanisms from server to client whether they may cache this object (Measured in seconds).

We will use max-age instead of Expire. The max-age directive specifies the maximum amount of time that a cached object is considered fresh. The advantage is that it is relative to the time of the request, rather than absolute. For this exercise, we've set max-age to 2592000 seconds (30 days).

The following header fields configuration will unset the last-modified and expires attributes and instead use Cache-control:

Header unset Last-Modified
Header unset Expires
Header set Cache-Control "max-age=2592000, public"

The response header configuration will be combined with http content compression configuration into one file and uploaded to the http server. Instruction for deployment of the changes is described in the next section.

Enable http content compression to reduce downloads

Oracle HTTP Server is based on Apache server and supports http compression. Apache uses the mod_deflate module for compression. For a complete reference please see http://httpd.apache.org/docs/2.0/mod/mod_deflate.html.

Here is a list of MIME types that can be compressed:

text/plain
text/xml
application/xhtml+xml
text/css
application/xml
image/svg+xml
application/rss+xml
application/atom+xml
application/x-javascript
text/html

Similarly, the http request pattern needs to be analyzed so we can compress certain data types within a relative request URL path. Since we have the request path from the previous section, we'll use it to to combine both caching and compression.

How to create the configuration file and deploy to the server

A *.conf file needs to be created. Caching and compression information discussed earlier needs to put into that file. The following is the content of the configuration file we used for browser caching and compression:

#WebCenter Spaces#

#compression#
<Location /webcenter>
  SetHandler weblogic-handler
  WeblogicHost mspcollab11
  WeblogicPort 8888
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>

#compression and caching#
<Location /cs>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /get>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /content>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /webcenter/afr>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /webcenter/adf>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /webcenter/content>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /webcenter/javascript>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>
<Location /webcenter/webcenter-spaces-resources>
  Header unset Last-Modified
  Header unset Expires
  Header set Cache-Control "max-age=2592000, public"
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/atom_xml
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE text/html
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
</Location>

LoadModule deflate_module        "${ORACLE_HOME}/ohs/modules/mod_deflate.so"

After the file is created, put the file into the following directory on OHS:

/$OHS_Instance_Home/config/OHS/ohs1/moduleconf

Note: Every file with 'conf' extension will be loaded during the Oracle HTTP Server start process.

Re-bounce the OHS to load the configuration changes. Go to /$OHS_Instance_Home/bin and do the following:

>> ./opmnctl stopall
>> ./opmnctl startall

Oracle Web Cache Server Configurations

Oracle Web Cache is a content-aware server accelerator, or a reverse proxy, for the Web tier. Oracle Web Cache is the primary caching mechanism provided with Oracle Fusion Middleware. Caching improves the performance, scalability, and availability of Web sites that run on Oracle Fusion Middleware by storing frequently accessed URLs in memory. It can also improve the performance, scalability, and availability of Web sites that run on any Web server or application server, such as Oracle HTTP Server and Oracle WebLogic Server.

Oracle Web Cache server also can be restrained by its hardware resources and memory availability. It performs best with two CPUs - or one very powerful CPU. Because Oracle Web Cache is an in-memory cache, it is rarely limited by CPU cycles. Additional CPUs do not increase performance significantly. However, the speed of the processors is critical. It is recommended to use the fastest CPU that budget allows. For memory configuration, it is recommended to set at least 512 MB. To get a close approximation of the maximum amount of memory required, please refer to the Oracle documentation for an estimation formula to calculate the max memory usage using total number of cachable objects and average size of cachable object: http://docs.oracle.com/cd/E12839_01/web.1111/e10143/getstarted.htm#CHDGAJHH

Another way to figure whether the CPU and memory is suitable for the current Cache server is to monitor the performance metrics using Oracle Enterprise Manager Console.

Before using Web Cache server, some configuration is necessary, including origin server connection, site definitions, site-to-server mapping, listen ports, and expiration policy definition and caching rules.

Caching Rules is the one configuration that we need to focus on. The configuration and the efficiency of the caching rules influences the caching server capacity as well as the performance of the Spaces application. Caching rules can be configured and reconfigured to increase its efficiency.

For initial caching rule configuration, it is important to understand the user behaviors and http request patterns. Through the list of request paths, we will focus on two things: the request patterns, and the frequency of each pattern. We will need to create caching rules to cover all the static resource request patterns and modify the sequence of caching rules in the appearance of Cache Server Manager. The sequence of the cache rules defined through Cache Server Manager will decide the caching rule precedence. Those at the top take precedence over those at the bottom.

After the request pattern and frequency have been identifed, the caching rules are created using Cache Server Manager. Caching rule url match criteria uses the following three options: regular expression, file extension and path prefix.

Another way to look at the http request pattern and frequency is to use monitor statistics from Cache Server Manager. Log in to the Manager Console, go to Monitor —> Popular Requests. The console displays a table with lots of valuable caching information, such as:

  • URI: the request path pattern
  • HTTP Method: whether it's a Get/Get with query string/Post method
  • Post Body: post body when it's a post method
  • Multi-Version: used for cookie configuration to increase the cache hit
  • Size of the cached/noncached object
  • Whether it's cached
  • Whether it's compressed

Rule No.1 is to make sure that the popular requests are cached as much as possbile. In order to determine the efficiency of the caches rules it is helpful to examine the "Caching, Personalization and Compression Rules", in which the cache and compression rules are defined. This page displays the caching 'Request Statistics' for each of the defined caching rules. The statistics information includes: number request matches, number of cache hits and number of cache misses. By adjusting the seqence in which the cache rules appear on this table, the efficiency of caching can be improved. Rule No.2 is to make the most popular cache matches and the cache misses/cache hits ratio appear first in the list.

Figure 3 shows a sample of caching rule definitions:

Figure 3
Figure 3: Caching rule definitions

 

To check the efficiency of the caching defined in the Web Cache server, log in to the Cache Manager Console, go to Monitoring and choose Popular Requests. Review if there are any cachable objects that are not cached. Also, turn on Firebug in the Firefox browser and check for the request URL to see if any of them that should be cached have been missed. Figure 4 shows a screenshot for the request URL table in the Firebug Firefox plugin after the caching.

Figure 4
Figure 4: Request URL table in Firebug

 

The sizes of the request objects are greatly reduced by compression, as much as 1% of the initial object size. The following table lists comparison of numbers of requests and page sizes before and after the caching and compression changes:

  BEFORE AFTER
Page Name # of Requests Page Size # of Requests Page Size
Page 1 47 791.9Kb 2 14Kb
Page 2 107 1.7Mb 2 21.2Kb
Page 3 109 1.8Mb 2 21.1Kb
Page 4 66 904.2kb 2 [20.1Kb
Page 5 118 1.8Mb 2 20.7Kb
Page 6 59 838.8Kb 2 21Kb
Page 7 70 976.5Kb 2 20.8Kb
Page 8 79 1Mb 2 20.4Kb
Page 9 88 1.2Mb 2 20.9Kb
Page 10 94 1.3Mb 2 21.2Kb
Page 11 125 1.3Mb 2 23.9Kb

To summarize, the Web Cache Server helps to cache all the popular requests in the Spaces application and significantly reduces the number and size of requests. It is difficult to provide a measurable page loading time benefits without completing an established testing cycle. However, by means of observations and preliminary testing data, a 0.4-0.6 second improvement has been seen using practices in the above two areas.

Continue to Part 2

About the Author

JayJay Zheng is a Solution Architect and certified Oracle WebCenter implementation specialist with AurionPro SENA. An Oracle Fusion Middleware evangelist, JayJay has over seven years of experience consulting in Oracle ADF, Oracle WebCenter Portal/Content, and SOA technologies. Twitter Blog LinkedIn