How to Optimize Your Web Commerce Infrastructure

By Tuning Oracle ATG Web Commerce Applications on SPARC SuperCluster

by Christopher J. Martin and Laura Verdibello

How to deploy a "best experience" consumer Website using Oracle ATG Web Commerce 10.1 software on an Oracle SPARC SuperCluster.


Published May 2013


Introduction
Architecture for Deploying Oracle ATG Web Commerce on SPARC SuperCluster
   More About Oracle ATG Web Commerce, SPARC SuperCluster, and the SPARC T4-4 Server
Workload Sizing Approach for Oracle ATG Web Commerce
   Simulating a Web Commerce Workload
   Determining the Optimal Workload
Examining the User Experience and Website Performance
   Product Search and Product Selection Experiences
   Product Purchasing Experiences
   Website Metrics During Optimum User Workload
Scaling Oracle ATG Web Commerce on SPARC SuperCluster
Best Practices for Deploying Oracle ATG Web Commerce on SPARC SuperCluster
   Configuring Oracle SPARC SuperCluster with Oracle VM Server for SPARC
   Setting Up the Oracle WebLogic Server Zone
   Setting Up the Oracle ATG Web Commerce Zone
   Modifying the General Purpose Domain
   Modifying the Oracle ATG Web Commerce Environment
   Modifying the Oracle WebLogic Server Environment
   Modifying the Database Server
Conclusion
See Also
About the Authors

Introduction

There are many ways to achieve a high-performing e-commerce merchandising presence on the internet. Having the right solution on the right platform is a critical step towards this end.

Want to comment on this article? Post the link on Facebook's OTN Garage page.  Have a similar article to share? Bring it up on Facebook or Twitter and let's discuss.

This following are the main sections of this article, which describe how to deploy a "best experience" consumer Website using Oracle ATG Web Commerce 10.1 software on an Oracle SPARC SuperCluster:

  • The architecture section describes the high-level architecture and how it is deployed.
  • The workload sizing section describes the workload sizing approach and the outcome of the sizing analysis and the response time analysis.
  • The user experience and Website performance section presents more details from the response time analysis by examining various user experiences, and it provides information about the overall performance of the Website.
  • The scaling section provides information about how the solution can be scaled to accommodate a larger workload.
  • The best practices section describes best practices for deploying SPARC SuperCluster, Oracle ATG Web Commerce, and Oracle Solaris to approximate the results achieved in this article.

Architecture for Deploying Oracle ATG Web Commerce on SPARC SuperCluster

The solution that was tested uses a half-rack SPARC SuperCluster configuration to deploy Oracle ATG Web Commerce 10.1 software. The software is deployed on a single SPARC T4-4 server (node) from Oracle, which is part of SPARC SuperCluster and contains four SPARC T4 processors. As shown in Figure 1, the configuration takes advantage of Oracle's built-in virtualization capabilities, including Oracle VM Server for SPARC and Oracle Solaris Zones.

For this solution, you can use either SPARC SuperCluster 1.0 or 1.1. For this article, SPARC SuperCluster 1.0 was used and it was preconfigured as enumerated in Configuration 23 for SPARC SuperCluster 1.1 of the Oracle Catalog. In this configuration, the SPARC T4 node is divided into two domains using Oracle VM Server for SPARC virtualization technology:

  • The database domain has a single SPARC T4 processor reserved for use by Oracle Database 11g Release 2.
  • The general purpose (GP) domain has the remaining three processors of the server reserved for the Web commerce infrastructure software.

Oracle Solaris Zones are used to further subdivide the GP domain so that specific system resources can be dedicated to Oracle ATG Web Commerce and Oracle WebLogic Server. Note that although the GP domain has two processors devoted to processing Oracle ATG Web Commerce transactions, it was further subdivided giving the Oracle ATG Web Commerce zone only a single processor for the purposes of measurement and sizing.

Figure 1

Figure 1. SPARC T4-4 architecture for Oracle ATG Web Commerce 10.1

More About Oracle ATG Web Commerce, SPARC SuperCluster, and the SPARC T4-4 Server

The challenge for e-commerce software is to engender customer loyalty and return business that increases revenue. Oracle ATG Web Commerce does this by offering a complete commerce software platform that enables the delivery of customer-specific buying experiences across all customer touch points, including the Web, contact center, mobile devices, social media, physical stores, and more.

SPARC SuperCluster is a highly available, prebuilt, and pretested engineered system that runs a complete Oracle hardware and software stack, eliminating the need for complex, multitier, multivendor hardware configurations. Because the entire environment is engineered and optimized to work together, it can be easily and rapidly deployed with e-commerce applications such as Oracle ATG Web Commerce.

The SPARC T4-4 server—one of the core building blocks of SPARC SuperCluster—is a versatile, high-performance, highly scalable four-socket server based on the SPARC T4 processor and optimized for a wide range of enterprise workloads, including consolidated Web application solutions. Each SPARC T4-4 server includes 32 CPU cores in four SPARC T4 processors and a 1-TB memory footprint. The compact five-rack unit (5RU) height provides increased computational density, making it a good fit for SPARC SuperCluster.

Workload Sizing Approach for Oracle ATG Web Commerce

For e-commerce Websites, it's important to understand Website behavior as customers and potential customers visit the Website to research products, conduct product searches, and add items to shopping carts. It's especially important to understand the different conditions under which users either abandon the Website without purchase or check out and complete a purchase. Many e-commerce Websites and their respective software applications are advanced, complex, and capable of accommodating hundreds of thousands of simultaneous users. As such, the best way to test and overcome potential performance problems is to use testing tools that simulate the end-user behavior.

Simulating a Web Commerce Workload

An analysis tool was used for the purposes of sizing and examining Oracle ATG Web Commerce on SPARC SuperCluster to discover Website behavior under load. The analysis tool consists of the following components:

  • Virtual User Generator: This is where real-world user behavior is encapsulated in the form of a virtual user. Each virtual user is given a set of Website transaction types to execute against the Oracle ATG Web Commerce–equipped Website on the SPARC SuperCluster–based configuration. These user behaviors are often called scripts and emulate behaviors such as the following:

    • Product research: Customers searching for products on the Website
    • Customer checkout: Customers editing their shopping cart and finalizing their purchase
    • Order placement: Customer's performing financial tasks in order to finish their purchase

    There are numerous other behaviors that were tracked during the sizing and response time analysis; however, only the behaviors above are illustrated in this article while measuring Website hits per second, page views per second, overall throughput, and customer response time metrics.

  • Controller: Here, we run the scripts generated above to control the various load generators and scenarios associated with them.
  • Analysis: This component provides detailed results and presents reports, charts, and graphics.

This environment was used to conduct a series of tests to characterize performance and user experience when large numbers of users are simultaneously using Oracle ATG Web Commerce on SPARC SuperCluster. The following sections describe the results of these tests.

Determining the Optimal Workload

Determining the optimal loading for the Website is critical to preserving the users' e-commerce experiences, which in turn increases the probability of garnering revenue during each user experience. Experimentation helped to determine the number of virtual users required to deliver the optimum Website loading.

When the workload was below the optimum number of virtual users, the overall infrastructure was underutilized and, thus, had the potential for realizing more revenue from internet e-commerce transactions. On the other hand, it was determined that even if the workload increased beyond the optimum number of virtual users and there were some response time anomalies, the user experience was largely preserved and the overall infrastructure remained stable throughout.

In order to determine infrastructure behavior and what the user experience was like with workloads that were less and greater than the optimum workloads, a stepwise testing methodology was used to collect not only system statistics but user response time metrics as well. The stepwise approach is illustrated in Figure 2. Starting at 50 percent of the optimal number of virtual users, the number of virtual e-commerce users was increased at 50-minute intervals and system-based statistics were collected using tools such as vmstat, netstat, and iostat as well as key elements of program-level memory cleanup (aka GC logging). The response time metrics were also recorded for each workload interval using the analysis tool.

Figure 2

Figure 2. Stepwise analysis approach used to determine and support the optimum number of virtual users

The more-refined objective of this analysis was to determine how many users the infrastructure can support while providing optimum response. It is important to have as many potential users on the Website storefront as possible to maximize revenue. It is also important to provide users with a "best experience" to prevent them from abandoning the storefront.

Figure 3 shows response time metrics as the number of virtual users was increased throughout the analysis. The analysis revealed that the optimal number of users is 1,600, since response time (RT) metrics were most favorable with that workload.

Figure 3

Figure 3. Response time metrics as Oracle ATG Web Commerce workload was incrementally increased

Examining the User Experience and Website Performance

It is useful to examine average response time metrics versus the number of users and the numerous transaction events that an e-commerce user might experience during a typical session. What follows are observed response time metrics plotted versus the number of sessions across the time intervals that coincide with the optimum number of users.

During the stepwise analysis, the numbers of users was increased stepwise to about 120 percent of the optimum number of e-commerce users during the six-hour analysis. The goal was to discover the consequences of exceeding the optimum number of users and investigate the user experience during that peak workload.

Product Search and Product Selection Experiences

The shopping experience—which comprises searching for products and selecting products—is important because users are most prone to abandon a Website during this time. They do this for a variety of reasons, including marginal response time performance, poor product search and selection experiences, and lack of ease of use.

In Figure 4, the product search and product selection response times are examined. In order to emulate a real-world environment as closely as possible, the load generator tool simulated multiple users refining their searches and customizing their selections (for example, for color and size). Average response times were collected for each user. Also, for the product search and product selection categories, in order to aggregate the response data and avoid taking "averages of averages," the median of the average response times was taken for each time series category to retain statistical integrity.

Figure 4

Figure 4. User shopping experience using Oracle ATG Web Commerce on SPARC SuperCluster during optimal workload

The product selection experience produced fairly consistent, subsecond average response times throughout the time series. These observations indicate that users will remain on the Website and abandon it only if their expectations for product selection are not met.

The average "Add to Shopping Cart" response times were higher than the average response times for product search and product selection. This indicates, again, that the Oracle ATG Web Commerce software and Oracle Database on SPARC SuperCluster are doing the necessary work to provide a good shopping experience.

Product Purchasing Experiences

Of utmost importance to an e-commerce business is the experience a customer has purchasing the product from the Website. The load generator tool collected average response time data for transactions related to purchasing a product from the Oracle ATG Web Commerce–empowered Website.

Figure 5 illustrates three transaction types: Add to Shopping Cart, Check Out, and Place Order.

Figure 5

Figure 5. User experience during product purchasing phase

Add to Shopping Cart transactions had the best response times, followed closely by the Place Order response times. The Check Out response times were higher, implying that the workload associated with a Check Out transaction requires more resources and varied the most. This could be driven by the number of items selected and the financial and database requirements associated with each item.

Website Metrics During Optimum User Workload

The load generator tool used for testing Oracle ATG Web Commerce processes user loads, but it also collects other metrics that are external to the user experience. This gives a complete view of not only what the user experience is, but also how well the Website responds to demand.

The observations were taken during the optimum user workload. During the time series, observations were taken for Website hits per second, Website pages downloaded per second, and overall data throughput in and out of the Website to the community of internet users (see Figure 6).

Throughout the interval, Pages Downloaded per second remained steady along with data throughput. The Website hits per second varied between about 2,000 hits per second to upwards of 2,190 hits per second during the interval.

Figure 6

Figure 6. Website metrics during optimum load

Scaling Oracle ATG Web Commerce on SPARC SuperCluster

This article focuses on using a single-node, single-processor configuration for Oracle ATG Web Commerce, but it makes sense to ask whether the application can scale on SPARC SuperCluster. Additional tests were conducted on SPARC SuperCluster to determine how Oracle ATG Web Commerce would scale.

In this subsequent testing, more virtual users were added to the user load for a total of 2.6 times the optimum number of users determined for the single processor tests or 4,200 users, and the compute platform was expanded to an Oracle Solaris Zone that contains three SPARC T4 processors.

Figure 7 shows graphically the dramatic increases in throughput as additional users and processor resources are added to the solution. The key business element here is the total number of transactions, which can relate directly to potential increases in revenue on the e-commerce Website with little to no additional investment, since the additional processor resources are readily available on SPARC SuperCluster and need only be configured using Oracle Solaris Zones.

Figure 7

Figure 7. Scaling Oracle ATG Web Commerce on Oracle SPARC SuperCluster

Also, in Table 1, system-wide metrics show how well Oracle ATG Web Commerce scales on SPARC SuperCluster. For example, by improving the compute capacity to three processors, the number of users increased by 162 percent, the number of hits per second improved by 266 percent, and page views improved by 270 percent. In addition, average throughput in megabytes per second improved by over 265 percent with an average user response time of 0.5 seconds (not shown in the table).

This was all accomplished with a lower average CPU utilization and lower peak CPU utilization because of the additional processors that were added to the zone configuration. While this clearly demonstrates how Oracle ATG Web Commerce can scale on SPARC SuperCluster, it also demonstrates that a larger workload with the resulting increased number of transactions can be supported when more processor, memory, and network resources are added, thus offering revenue opportunities as the configuration scales up.

Table 1. Analysis Metrics
Metric Oracle ATG Web Commerce Single-Processor Zone Oracle ATG Web Commerce Three-Processor Zone
Duration of analysis during optimum user load 50 minutes 50 minutes
Number of users 1,600 virtual users 4,200 virtual users
Average throughput 46 MB/sec 122 MB/sec
Total hits 6.3 million 16.8 million
Average hits per second 2,088 5,616
Total number of transactions 456,814 2,511,145
Average CPU utilization USR 73%, SYS 7.2%, Idle 20.00% USR 64.01%, SYS 6.8%, Idle 29.14%
Peak CPU utilization 93% 82%
Average network bytes read/second 1,189 1,548
Peak network bytes read/second 1,952 3,932
Average network bytes written/second 25,303 32,759
Maximum network bytes written/second 38,839 82,678
Page views/second 154.8 417.2

Best Practices for Deploying Oracle ATG Web Commerce on SPARC SuperCluster

The descriptions that follow include block diagrams as well as recommendations for changing parameters and command-line flows with SPARC SuperCluster.

Configuring Oracle SPARC SuperCluster with Oracle VM Server for SPARC

By default, all SPARC SuperCluster engineered systems come with Oracle VM Server for SPARC, which allows you to separate each SPARC T4-4 server into logical domains to utilize resources more fully and to tune each server for its particular application.

As mentioned earlier, SPARC SuperCluster 1.0 was preconfigured as enumerated in Configuration 23 for SPARC SuperCluster 1.1. This configuration allows each SPARC T4-4 server (node) to be separated into two domains, as shown in Figure 8: a database domain, which is reserved for Oracle Database 11g Release 2, and a general purpose (GP) domain, which is reserved for applications.

Figure 8

Figure 8. SPARC SuperCluster 1.1 as configured for Configuration 23 in the Oracle Catalog

For the purposes of deploying Oracle ATG Web Commerce, the GP domain is divided into zones using Oracle Solaris Zones. This allows you to further optimize the configuration for the applications that are required. Note, that when the Oracle SPARC SuperCluster 1.1 is deployed as Configuration 23, the domains cannot be changed. However, deploying Oracle Solaris Zones requires additional steps to partition the GP domain into two zones. The zones are configured such that the first zone is an Oracle WebLogic Server zone while the second zone is for the Oracle ATG Web Commerce instance. The zones share all three sockets except during the sizing analysis, Oracle ATG Web Commerce was restricted to only one of the three sockets. Since the database domain and GP domain are already set up and preconfigured, the next step is to set up the zones on the GP domain.

Setting Up the Oracle WebLogic Server Zone

The first task is to set up a dedicated zones disk to separate disk workloads from the root partition.

  1. Find a spare internal disk and create a zpool on that device, and then run the zfs list command:

    root@GPNode1:~# zpool create zones c0t5000C50043541C3Bd0
    root@GPNode1:~# zfs list zones
    NAME   USED  AVAIL  REFER  MOUNTPOINT
    zones  840KM  48.2G   43K  /zones
    
  2. Next, create the Oracle WebLogic Server zone, as shown in Listing 1. In the GP domain, keep in mind that the following will have to be modified to reflect the correct addresses and physical network interfaces for implementation environments.

    root@GPNode1:~# zonecfg -z wlserver
     wlserver: No such zone configured
     Use 'create' to begin configuring a new zone.
     zonecfg:wlserver> create -b
     zonecfg:wlserver> set zonepath=/zones/wlserver
     zonecfg:wlserver> set autoboot=true
     zonecfg:wlserver> set limitpriv=default,proc_priocntl,proc_lock_memory,dtrace_proc,dtrace_user
     zonecfg:wlserver> set ip-type=shared
     zonecfg:wlserver> add net
     zonecfg:wlserver:net> set address=<public IP address for zone>/<netmask>
     zonecfg:wlserver:net> set configure-allowed-address=true
     zonecfg:wlserver:net> set physical=<network device name>
     zonecfg:wlserver:net> set defrouter=<IP address for default router>
     zonecfg:wlserver:net> end
     zonecfg:wlserver> add net
     zonecfg:wlserver:net> set address=<IB IP address for zone>
     zonecfg:wlserver:net> set physical=<IB device name>
     zonecfg:wlserver:net> end
     zonecfg:wlserver> verify
     zonecfg:wlserver> commit
     zonecfg:wlserver> exit
    

    Listing 1

  3. Install and boot up the zone, as shown in Listing 2:

    root@GPNode1:~# zoneadm -z wlserver install
    A ZFS file system has been created for this zone.
    Progress being logged to /var/log/zones/zoneadm.20130312T131935Z.wlserver.install
           Image: Preparing at /zones/wlserver/root.
           Install Log: /system/volatile/install.4892/install_log
           AI Manifest: /tmp/manifest.xml.RLaqIj
           SC Profile: /usr/share/auto_install/sc_profiles/enable_sci.xml
           Zonename: wlserver
    	.
    	.
    	. 
    Installation: Succeeded
    Done: Installation completed in 1293.008 seconds.
    Next Steps: Boot the zone, then log into the zone console (zlogin -C) to complete the configuration process.
    Log saved in non-global zone as /zones/wlserver/root/var/log/zones/zoneadm.20130312T131935Z.wlserver.install
    root@GPNode1:~# zoneadm -z wlserver boot
    

    Listing 2

  4. To complete the configuration, run the following command and answer the subsequent prompts:

    root@GPNode1:~# zlogin -C appserver
    

Setting Up the Oracle ATG Web Commerce Zone

The next task at hand is to set up an Oracle Solaris Zone for the Oracle ATG Web Commerce instance. Keep in mind that you need to modify the following commands to correctly reflect the addresses and physical network interfaces for your implementation environment.

  1. First, create a dedicated file system for installation of the Oracle ATG Web Commerce software. Place /export/home/atguser in a UFS. After finding a disk that can be used, run the command shown in Listing 3.

    root@GPNode1:~# newfs /dev/rdsk/c0t5000C5001B198BEBd0s6
    newfs: construct a new file system /dev/rdsk/c0t5000C5001B198BEBd0s6: (y/n)? Y
    /dev/rdsk/c0t5000C5001B198BEBd0s6: 585105408 sectors in 95232 cylinders of 48 tracks, 128 sectors
    285696.0MB in 5952 cyl groups (16 c/g, 48.00MB/g, 5824 i/g)
    super-block backups (for fsck -F ufs -o b=#) at: 32, 98464, 196896, 295328, 393760, 492192, 590624, 
    689056, 787488, 885920,
    Initializing cylinder groups: .............
    super-block backups for last 10 cylinder groups at:
    584129312, 584227744, 584326176, 584424608, 584523040, 584621472, 584719904,584818336, 584916768, 585015200
    

    Listing 3

  2. Next, create the appserver zone, as shown in Listing 4:

    root@GPNode1:~# zonecfg -z appserver
    Use 'create' to begin configuring a new zone.
    zonecfg:appserver> create -b
    zonecfg:appserver> set zonepath=/zones/appserver
    zonecfg:appserver> set autoboot=true
    zonecfg:appserver> set limitpriv=default,proc_priocntl,proc_lock_memory,dtrace_proc,dtrace_user
    zonecfg:appserver> set ip-type=shared
    zonecfg:appserver> add net
    zonecfg:appserver:net> set address=<public IP address for zone>/<netmask>
    zonecfg:appserver:net> set configure-allowed-address=true
    zonecfg:appserver:net> set physical=<network device name>
    zonecfg:appserver:net> set defrouter=<IP address for default router>
    zonecfg:appserver:net> end
    zonecfg:appserver> add net
    zonecfg:appserver:net> set address=<IB IP address for zone>
    zonecfg:appserver:net> set physical=<IB device name>
    zonecfg:appserver:net> end
    zonecfg:appserver> add fs
    zonecfg:appserver:fs> set directory=/export/home/atguser
    zonecfg:appserver:fs> set special=/dev/dsk/c0t5000C5001B198BEBd0s6
    zonecfg:appserver:fs> set raw=/dev/rdsk/c0t5000C5001B198BEBd0s6
    zonecfg:appserver:fs> set type=ufs
    zonecfg:appserver:fs> end
    zonecfg:appserver> verify
    zonecfg:appserver> commit
    zonecfg:appserver> exit
    

    Listing 4

  3. Now, install and boot up the zone:

    root@GPNode1:~# zoneadm -z appserver install; zoneadm -z appserver boot
    
  4. To complete the configuration, run the following command and answer the subsequent prompts:

    root@GPNode1:~# zlogin -C appserver
    

Modifying the General Purpose Domain

At this time, complete a standard basic installation of all Oracle ATG Web Commerce software into the appserver zone. The following are recommendations for hosting Oracle ATG Web Commerce in the appserver zone on SPARC SuperCluster.

  1. Change parameters in /etc/system to optimize Oracle Solaris performance. To do this, edit /etc/system and add or modify the items shown in Table 2.

    Table 2
    Item Notes
    set lgrp_mem_pset_aware=1 Needed only if you are using processor sets; this allows lgroups to be pset-aware.
    set rlim_fd_cur=131072 Increase the soft limit for file descriptors allowed by a process (default is 256).
    set rlim_fd_max=131072 Increase the hard limit for file descriptors (default is 65536).
    set max_uheap_lpsize=0x10000000 Set large page size support to 256M pages maximum (default is 0x400000).
    set mmu_ism_pagesize=0x10000000 Change ISM to include 256M pages (default is 0x400000).
  2. Now modify TCP parameters to optimize for large amounts of Web server traffic:

    root@GPNode1:~# ipadm set-prop -p _time_wait_interval=6000 tcp
    root@GPNode1:~# ipadm set-prop -p _conn_req_max_q=64000 tcp
    root@GPNode1:~# ipadm set-prop -p _conn_req_max_q0=64000 tcp
    root@GPNode1:~# ipadm set-prop -p send_buf=524288 tcp
    root@GPNode1:~# ipadm set-prop -p recv_buf=524288 tcp
    root@GPNode1:~# ipadm set-prop -p smallest_anon_port=10000 tcp
    root@GPNode1:~# ipadm set-prop -p _naglim_def=1 tcp 
    
  3. View the TCP properties by running the ipadm show-prop tcp command, as shown in Figure 9:

    Figure 9

    Figure 9. System output showing the TCP properties

  4. In addition to making the changes above for improving TCP performance, make sure the values are not overridden. In the directory /opt/oracle.supercluster/ssctuner/tune/, edit the ndd file as follows:

    Change the following lines:

    /dev/tcp tcp_xmit_hiwat 1048576 
    /dev/tcp tcp_recv_hiwat 1048576 
    

    to:

    /dev/tcp tcp_xmit_hiwat 524288 
    /dev/tcp tcp_recv_hiwat 524288 
    
  5. Then, finish this step by refreshing and restarting ssctuner:

    root@GPNode1:~# svcadm refresh ssctuner 
    root@GPNode1:~# svcadm restart ssctuner
    

Modifying the Oracle ATG Web Commerce Environment

Although standard basic installation instructions are followed to install Oracle ATG Web Commerce, some modifications are necessary. First, the following aliases were used on the appserver and are used here for purposes of abbreviation:

alias cdweb='cd /export/atguser/weblogic/Oracle/Middleware/user_projects/domains/base_domain' 
export cdweb /export/atguser/weblogic/Oracle/Middleware/user_projects/domains/base_domain 

  1. In $cdweb, make changes to the generic startWebLogic.sh by inserting immediately before the startWebLogic.sh line the following command line, which places processes started in bin/startWebLogic.sh in the FX class.

    /usr/bin/priocntl -s -c FX $$ 
    
  2. Next, to minimize memory management processing (sometimes referred to as "garbage collection" or GC), save the GC logs to their individual locations. Note that these logs were for testing and debugging purposes only; in production, the LOG_NAME parameter must be removed.
  3. Continuing the additions to $cdweb/startWebLogic.sh, position memory management parameters and GC logging as appropriate and preload libumem with the appropriate settings, as shown in Listing 5.

    case "${SERVER_NAME}" in
       'AdminServer') 
          MEM_ARGS="-server -Xms256m -Xmx512m -XX:MaxPermSize=256m" 
          export MEM_ARGS 
       ;; 
       'atg_lock_production') 
          MEM_ARGS="-server -Xms2048m -Xmx2048m -Xrs" 
          export MEM_ARGS 
       ;; 
       'atg_publishing_lockserver') 
          MEM_ARGS="-server -Xms2048m -Xmx2048m -Xrs" 
          export MEM_ARGS 
       ;;
        'atg_production') 
          LOG_NAME="-Xloggc:/tmp/gc1.log" # for testing and debugging only 
          export LOG_NAME 
       ;;
    LD_PRELOAD_64=/usr/lib/sparcv9/libumem.so ${JAVA_HOME}/bin/java ${JAVA_VM} ${MEM_ARGS} \
    ${LOG_NAME} -Dweblogic.Name=${SERVER_NAME} \
    -Djava.security.policy=${WL_HOME}/server/lib/weblogic.policy \
    ${JAVA_OPTIONS} ${PROXY_SETTINGS} ${SERVER_CLASS}
    

    Listing 5

  4. Now that editing startWebLogic.sh files has been completed, in $cdweb/bin, edit setDomainEnv.sh. First, set Java tunables depending on memory size. Either setting can be used, as appropriate for a particular installation. In the script, note the comments regarding removal of the -verbose:gc and PrintGC flags so as not to adversely affect performance.

    Also, note the two different WLS_MEM_ARGS_64BIT settings. If you have a large memory system, for example, what was used in this analysis configuration with a full terabyte of memory, use -Xms16g. Otherwise, use -Xms8g. After changing these lines, edit the last two lines regarding memory sizing.

    WLS_MEM_ARGS_64BIT="-Xms8g -Xmx8g -XX:NewSize=3g \
    -XX:MaxNewSize=3g -XX:+CMSIncrementalMode -XX:CICompilerCount=8 \
    -XX:ParallelGCThreads=24 -XX:+AlwaysPreTouch -XX:+UseConcMarkSweepGC \
    -XX:+UseParNewGC -XX:SurvivorRatio=20 -XX:TargetSurvivorRatio=90 \
    -XX:MaxTenuringThreshold=8 -XX:+UseBiasedLocking \
    -XX:+UseLargePages -XX:LargePageSizeInBytes=256M \
    -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses \
    -XX:ReservedCodeCacheSize=1536m -XX:CMSFullGCsBeforeCompaction=1 \
    -XX:+CMSScavengeBeforeRemark -XX:CMSInitiatingOccupancyFraction=60 \
    -XX:+UseCMSInitiatingOccupancyOnly -verbose:gc -XX:+PrintGCDetails \
    -XX:+PrintGCTimeStamps -Dweblogic.SocketReaders=10 -XX:MaxPermSize=400m \
    -XX:PermSize=400m "
    

    Or:

    WLS_MEM_ARGS_64BIT="-Xms16g -Xmx16g -XX:NewSize=8g -XX:MaxNewSize=8g \
    -XX:CICompilerCount=8 -XX:ParallelGCThreads=8 -XX:+AlwaysPreTouch \
    -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=20 \
    -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseBiasedLocking \
    -XX:+UseLargePages -XX:LargePageSizeInBytes=256M \
    -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses \
    -XX:ReservedCodeCacheSize=1536m -verbose:gc \
    -XX:+PrintGCDetails -XX:+PrintGCTimeStamps \
    -Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=8 \
    -Dweblogic.threadpool.MinPoolSize=128 \
    -Dweblogic.threadpool.MaxPoolSize=128 -Dweblogic.GatheredWritesEnabled=true \
    -Dweblogic.ScatteredReadsEnabled=true -Doracle.net.SDP=true \
    -XX:MaxPermSize=800m  -XX:PermSize=800m"
    # Note: -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps are variables used 
    # for testing and debugging only.
    

    Then:

    MEM_PERM_SIZE_64BIT="-XX:PermSize=800m" (from 128m)
    MEM_MAX_PERM_SIZE_64BIT="-XX:MaxPermSize=800m" (from 256m)
    

Modifying the Oracle WebLogic Server Environment

What follows are the only changes made to Oracle WebLogic Server's Oracle HTTP Server environment. For illustration purposes, the following aliases are used in this article. When reviewing the follow-on modifications, refer to these aliases to determine the actual location of files and the fully qualified path names.

cdohs='cd /home/atguser/ohs/Oracle/Middleware/Oracle_WT1/instances/instance1/bin' 
export cdohs /home/atguser/ohs/Oracle/Middleware/Oracle_WT1/instances/instance1/bin 

  1. In $cdohs, change to the directory ../config/OHS/ohs1/ and edit the httpd.conf file. Make the changes illustrated in Listing 6. Note that two particular changes have to be accomplished twice in httpd.conf. First, AcceptMutex must be changed from fcntl to sysvsem. Second, LockFile appears on two different lines in the file, and both should be commented out so they have no effect.

    MaxKeepAliveRequests 0   # (From 100)
    StartServers 1           # (from 2) 
    ServerLimit 200          # (added) 
    MaxClients 12400         # (from 150) 
    MinSpareThreads 50       # (from 25)
    MaxSpareThreads 100      # (from 75) 
    ThreadsPerChild 62       # (from 25) 
    MaxRequestsPerChild 0    # (unchanged) 
    # AcceptMutex fcntl      # (commented out)
    AcceptMutex sysvsem      # (was fcntl )
    #                        # LockFile appears twice in the file, comment both out)
    # LockFile \
    "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/http_lock" </IfModule>
    # 
    Listen 10.196.95.85:7777 # (was 7777)
    

    Listing 6

  2. Next, in the file mod_wl_ohs.conf, comment out the following lines and set the IP addresses as appropriate for your implementation.

    # ServerName 10.129.193.195
    # AllowEncodedSlashes On
    # RewriteEngine On
    # RewriteOptions inherit
    # Set the IP addresses for the various atg_production* servers:
    WebLogicCluster
    192.168.68.86:7003
    # for multiple instances of the atg_production* servers:
    # 192.168.68.86:7003,192.168.68.86:7103,192.168.68.86:7203,192.168.68.86:7303,192.168.68.86:7403
    

    Listing 7

Modifying the Database Server

The final modification to be made is to change the Oracle Database configuration so that it can function properly with respect to Web server demand, since the workload requires that a large number of processes concurrently access the database. Make this change to the Oracle Database pfile, or make the change to the spfile using the Oracle Database alter command.

# Increase the number of concurrent database processes:

*.processes=5000

Conclusion

The objective of deploying an internet storefront or what is commonly called an e-commerce Website is to produce revenue through product offerings and subsequent purchases at the Website while at the same time "learning" about customers and their preferences. Ultimately, the goal is to make it easy for customers to research and purchase items on the site while encouraging customers to purchase related products and services. Oracle ATG Web Commerce provides these e-commerce features while delivering an effective and responsive user experience.

Delivering a positive user experience also requires fast response time. For this reason, it is important to provide a high-performing platform infrastructure so that customers do not abandon the Website experience for competing Websites.

The Oracle SPARC SuperCluster environment described in this article provides both a highly available and high-performance infrastructure for Oracle ATG Web Commerce. Oracle's documented test results for performance and user experience not only demonstrate good response times for large user loads but also take the guesswork out of sizing the hardware infrastructure. In addition, the best practices described in this article make it easy to replicate these results using Oracle technology.

See Also

For more information on Oracle's technology see the following resources.

Websites:

Data sheets:

White paper and Webcast:

Oracle services and support:

About the Authors

Chris Martin is a Principal Product Manger in Oracle Optimized Solutions. Prior to coming to Sun Microsystems and subsequently Oracle, Chris held positions at Lockheed-Martin in Orlando, Florida supporting information technology services as well as in Colorado Springs, Colorado where he provided architecture engineering for high-performance computing and storage. Just prior to coming to Sun Microsystems, Chris worked in San Francisco, CA for EMC, Inc. and Silicon Graphics, Inc. as a storage and server pre-sales engineer. Today Chris supports Oracle Optimized Solutions as they relate to ISV partners and their products when deployed on Oracle Red Stack Technologies.

Laura Verdibello is a Principal Software Engineer at Oracle, originally coming from Sun Microsystems. During her career at Sun and Oracle, she has worked with a wide range of Independent Software Vendors to integrate their applications on Sun platforms with the Oracle Solaris operating system. She has been involved with many ISV projects, working to improve the performance of applications and to demonstrate the advantages of application in the Oracle environment. Today Laura is in the Oracle Platform Integration group primarily sharing her knowledge with software organizations within Oracle.

The authors would like to give special thanks for the valuable contributions from ATG Web Commerce performance team on this project.

Revision 1.1, 05/21/2013

Follow us:
Blog | Facebook | Twitter | YouTube