Creating Prebundled Machine Images for the Cloud

by Kiran C. Nair

Learn how to create a custom virtual machine image prebundled with Oracle WebLogic Server 11g and Oracle Database XE, as well as the basics of creating init scripts and configuring services to run at user-defined runlevels

Published November 2009

Downloads
 Oracle WebLogic Server Installer for Linux
 Oracle Database 10g Express Edition for Linux

Platform virtualization is used by cloud service providers such as Amazon Web Services (AWS) to provide remote access to software running on shared hardware infrastructure. Platform virtualization uses the concept of machine images, which can be hosted virtually on remote platforms, allowing users unrestricted access to the underlying infrastructure.

In this tutorial, you will see how to create a custom virtual machine image prebundled with Oracle WebLogic Server 11g and Oracle Database 10g Express Edition (XE). You will also learn the basics of creating init scripts and configuring services to run at user-defined runlevels. This will enable the created images to fully utilize the on-demand scalability of cloud computing; prebundled applications or services will be able to start servicing requests as soon as the instance starts up.

The process followed in this tutorial is a generic one. The prebundled applications and utilities may be customized according to user preferences or demands. It should also be noted that the created images are not restricted to AWS but are fully compatible with any cloud that uses Xen as the hypervisor layer (for example, Eucalyptus Open Cloud).

(Note: Oracle certifies/supports and licenses many of its products on AWS and also supplies free, prebuilt virtual machine templates for several Oracle products on Oracle Enterprise Linux; see the Cloud Computing Center for a full list.)

 

Before You Start



This tutorial provides a step-by-step guide to the creation of custom images—including installing applications, configuring the images, writing startup scripts, and configuring services to run at specified runlevels—so that these images are ready to be deployed onto a cloud infrastructure. In this tutorial, we will:

  • Prebundle a Centos 5.2 base image with Oracle WebLogic Server and Oracle Database XE
  • Install additional packages such as vi, FTP, and necessary dependencies
  • Configure networking, password, and so on
  • Create init scripts
  • Configure services (FTP, networking, and so on) and applications (Oracle WebLogic Server and Oracle Database XE) to run at instance startup

Prerequisites

Mounting the Base Image Using Loopback

Resize the image file

The base image file may not have sufficient storage space for accommodating custom applications, so we may need to resize it. We can resize it to suit our own requirements. Here we will resize a 1GB image to 4GB:

[root@localhost Image]# ls -s
total 1026032
                               
1026032 centos.5-2.img
[root@localhost Image]# dd if=/dev/zero of=
centos.5-2.img
bs=1M conv=notrunc count=1 seek=4000 #This resizes the image to 4GB. [root@localhost Image]# losetup /dev/loop1
centos.5-2.img
#loop1 had to be used as xen was already running a different domU via loop0. [root@localhost Image]# e2fsck -f /dev/loop1 [root@localhost Image]# resize2fs /dev/loop1 [root@localhost Image]# e2fsck -f /dev/loop1 [root@localhost Image]# losetup -d /dev/loop1 [root@localhost Image]# ls -s total 4100032
4100032 centos.5-2.img

Mount the image file through loopback

The resized image file can now be mounted into a local mount-point so that it can be modified and configured according to our requirements. We will also mount the local processor into the ./mount-point/proc folder so that it functions as the temporary processor for the duration of our exercise.

#make a mountpoint
mkdir centos

#Mount Image 
mount -o loop ./centos.5-2.img ./centos

#Mount the Proc
mount -t proc none ./centos/proc/

Install additional packages

The mounted base image will contain the minimal set of packages needed to support a root file system (Centos 5.2 in our case). Additional packages can be added to provide extra functionalities. Here we will install three sets of additional packages: vsftpd Server (a popular FTP server for UNIX like systems), vi (the popular command line text editor), and dependencies required for Oracle Database XE.

Note: For this tutorial, the following RPMs should be downloaded:

vim-common-7.0.109-3.el5.3.i386.rpm (vi Editor)
vim-minimal-7.0.109-3.el5.3.i386.rpm (vi Editor)
logrotate-3.7.4-8.i386.rpm (dependency for vsftpd)
vsftpd-2.0.5-12.el5.i386.rpm (vsftpd)
libaio-0.3.96-7.i386.rpm (dependency for Oracle Database XE)
bc-1.06-21.i386.rpm (dependency for Oracle Database XE)

Now we will install these additional packages along with their dependencies into the mounted image:
                               
#Install Additional Packages

[root@localhost Image]# rpm --root=/home/kiran/Oracle/Image/centos -Uvh 
/home/kiran/Oracle/Image/RPMs/vim-common-7.0.109-3.el5.3.i386.rpm
Preparing...               ########################################### [100%]
   1:vim-common            ########################################### [100%]

[root@localhost Image]# rpm --root=/home/kiran/Oracle/Image/centos -Uvh 
/home/kiran/Oracle/Image/RPMs/vim-minimal-7.0.109-3.el5.3.i386.rpm
Preparing...               ########################################### [100%]
   1:vim-minimal           ########################################### [100%]

[root@localhost Image]# rpm --root=/home/kiran/Oracle/Image/centos -Uvh 
/home/kiran/Oracle/Image/RPMs/logrotate-3.7.4-8.i386.rpm
Preparing...               ########################################### [100%]
   1:logrotate             ########################################### [100%]

[root@localhost Image]# rpm --root=/home/kiran/Oracle/Image/centos -Uvh 
/home/kiran/Oracle/Image/RPMs/vsftpd-2.0.5-12.el5.i386.rpm
Preparing...               ########################################### [100%]
   1:vsftpd                ########################################### [100%]


#Install XE Dependencies

[root@localhost Image]# rpm --root=/home/kiran/Oracle/Image/centos -Uvh 
/home/kiran/Oracle/Image/RPMs/libaio-0.3.96-7.i386.rpm
warning: /home/kiran/Oracle/Image/RPMs/libaio-0.3.96-7.i386.rpm: 
Header V3 DSA signature: NOKEY, key ID 025e513b
Preparing...               ########################################### [100%]
   1:libaio                ########################################### [100%]

[root@localhost Image]# rpm --root=/home/kiran/Oracle/Image/centos -Uvh 
/home/kiran/Oracle/Image/RPMs/bc-1.06-21.i386.rpm
Preparing...               ########################################### [100%]
   1:bc                    ########################################### [100%]

                            

Working Inside chroot jail

Move Oracle Database XE installer and Oracle WebLogic Server (installers +pre-created domain .tgz) into the image mount-point

Installation of Oracle Database XE and Oracle WebLogic Server will be done from inside the image mount-point. Before doing so, these installers and necessary files need to be moved into the necessary folder inside the image mount-point. We will create a folder (./centos/root/install) inside the image mount-point.

[root@localhost Image]# mkdir ./centos/root/install
#Moving the Oracle XE RPM
[root@localhost Image]# mv /home/kiran/Oracle/Image/RPMs/
oracle-xe-univ-10.2.0.1-1.0.i386.rpm ./centos/root/install/
#Moving the Oracle Weblogic Binary Installer
[root@localhost Image]# mv /home/kiran/Oracle/Image/RPMs/wls1031_linux32.bin 
./centos/root/install/
[root@localhost Image]# mv /home/kiran/Oracle/Image/RPMs/CloudDomain.tgz 
./centos/root/install/

Change root to mount-point

The image mount-point (./centos) that has been created contains all the files and directory structure needed for a fully functioning base Linux system. We can now reroot into this directory using the chroot command. This directory is called "chroot jail."

[root@localhost Image]# cd ./centos
[root@localhost centos]# chroot .
bash-3.2#

Change root password for the image and configure remote root login

Here we will set a custom password for root user and configure ssh to allow root login. (Note: For high-security applications, this is discouraged. Use SSH Key-Pairsinstead.)

#Set root Password
bash-3.2# pwconv
bash-3.2# passwd
Changing password for user root.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
For allowing remote root login, change PermitRootLogin no to PermitRootLogin yes inside the file /etc/ssh/sshd_config.

Installing and Configuring Oracle Database XE

Install Oracle Database XE RPM

We will now install Oracle Database XE from the installer present in /root/install in the jail.

bash-3.2# rpm -Uvh /root/install/oracle-xe-univ-10.2.0.1-1.0.i386.rpm
Preparing...               ########################################### [100%]
   1:oracle-xe-univ        ########################################### [100%]
Executing Post-install steps...

You must run '/etc/init.d/oracle-xe configure' as the root user to
configure the database.
bash-3.2# rm -rf /root/install/oracle-xe-univ-10.2.0.1-1.0.i386.rpm
5.2 Configure Oracle Database Express Edition by running /etc/init.d/
oracle-xe configure
Run /etc/init.d/oracle-xe configure to open the command-line Oracle 
configuration wizard. 
Remember to enter "y" for the question "Do you want Oracle 
Database 10g Express Edition to be started 
on boot (y/n) [y]:" if you want the Oracle Database Express Edition 
service to be started at instance boot.
#Configure Oracle XE
bash-3.2# /etc/init.d/oracle-xe configure

Oracle Database 10g Express Edition Configuration
-------------------------------------------------
This will configure on-boot properties of Oracle Database 10g Express
Edition.  The following questions will determine whether the database should
be starting upon system boot, the ports it will use, and the passwords that
will be used for database accounts.  Press <enter> to accept the defaults.
Ctrl-C will abort.

Specify the HTTP port that will be used for Oracle Application Express [8080]:8080

Specify a port that will be used for the database listener [1521]:1521

Specify a password to be used for database accounts.  Note that the same
password will be used for SYS and SYSTEM.  Oracle recommends the use of
different passwords for each database account.  This can be done after
initial configuration:
Confirm the password:

Do you want Oracle Database 10g Express Edition to be started on boot (y/n) [y]:y

Starting Oracle Net Listener...Done
Configuring Database...Done
Starting Oracle Database 10g Express Edition Instance...Done
Installation Completed Successfully.
To access the Database Home Page go to http://127.0.0.1:8080/apex

Add required environment variables accessible at startup

Scriptlets that are run at instance startup are contained in rc.local. Necessary environment variables like ORACLE_BASE and ORACLE_HOME need to be present for Oracle Database XE to access these at runtime.

Edit /etc/rc.local and add the following lines:
ORACLE_HOME = /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/
ORACLE_BASE = /usr/lib/oracle/xe/app/oracle/
PATH=$PATH:/usr/lib/oracle/xe/app/oracle/product/10.2.0/server/bin:$HOME/bin
export PATH
export ORACLE_HOME
export ORACLE_BASE

Installing and Configuring Oracle WebLogic Server

Run Oracle WebLogic Server binary installer

The binary installer that was copied into /root/install folder will now be used to install Oracle WebLogic Server:

#Install WebLogic
bash-3.2# chmod +x /root/install/wls1031_linux32.bin
bash-3.2# /root/install/wls1031_linux32.bin

########### Follow the steps in the Command Line Installation Wizard ########

bash-3.2# rm -rf /root/install/wls1031_linux32.bin

Oracle WebLogic Server will now be present under /root/Oracle/Middleware/.

Unpack the pre-created domain into /user_projects/domains/

We will be using a domain created on a local machine instead of manually creating one. A domain can also be manually created by using the script /root/Oracle/Middleware/wlserver_10.3/common/bin/config.sh.

#Move CloudDomain.tgz into folder and extract
bash-3.2# mkdir -p /root/Oracle/Middleware/user_projects/domains/
bash-3.2# mv /root/install/CloudDomain.tgz /root/Oracle/Middleware
/user_projects/domains/
bash-3.2# cd /root/Oracle/Middleware/user_projects/domains/
bash-3.2# tar -xzvf CloudDomain.tgz
bash-3.2# rm -rf CloudDomain.tgz

This domain is now ready to be started using the script /root/Oracle/Middleware/user_projects/domains/CloudDomain/bin/startWebLogic.sh.

Create init Script for Starting Domain at Instance Startup

On-demand scalability provided by the cloud necessitates application startup during instance boot, with minimal user interaction. Once a new instance is booted up on the cloud, it should immediately be able to start servicing requests coming to it. For custom applications to fully utilize this capability, they need to be set as services, initialized and running at system boot-up.

Let us create an init script for starting our Oracle WebLogic Server Domain at instance startup. In the init script shown below, I have taken /root/Oracle/Middleware/user_projects/domains/CloudDomain/bin/startWebLogic.sh as the script to start the domain and /root/Oracle/Middleware/user_projects/domains/CloudDomain/bin/stopWebLogic.sh as the script to stop the domain. The script created is named as initWeblogic and is put into the /etc/init.d/ directory.

 #################Script Start#########################
#!/bin/sh
#
# chkconfig: 345 99 10
# description: Custom Script
#

SCRIPT_CMD="/root/Oracle/Middleware/user_projects/domains/CloudDomain
/bin/startWebLogic.sh"
SCRIPT_STOP="/root/Oracle/Middleware/user_projects/domains/CloudDomain
/bin/stopWebLogic.sh"
if [ -f /etc/init.d/functions ]; then
    . /etc/init.d/functions
elif [ -f /etc/rc.d/init.d/functions ]; then
    . /etc/rc.d/init.d/functions
else
    exit 0
fi

. /etc/sysconfig/network

[ ${NETWORKING} = "no" ] && exit 0

RETVAL=0

start() {
    echo -n "Starting WebLogic "
    $SCRIPT_CMD &
    RETVAL=$?
    [ $RETVAL -eq 0 ] && success || failure
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/myWebLogic
    echo
    return $RETVAL
}

stop() {
    echo -n $"Stopping WebLogic "
    $SCRIPT_STOP
    RETVAL=$?
    [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/myWebLogic
    echo
    return $RETVAL
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "Usage: $0 {start|stop|restart}"
        exit 1
        ;;
esac

exit $RETVAL
#################Script End#########################

This script is reusable and can be used to add any custom application into init.d directory.

Activating/Deactivating Services for Specific Runlevels

Services that start automatically at specific runlevels are specified in chkconfig. The following sequence must be followed:

  • Add a service to chkconfig --list (if it is not already present)
  • Specify runlevels at which the specific service should be on/off

The process is detailed below:

bash-3.2# chkconfig --add initWeblogic
bash-3.2# chkconfig --level 345 initWeblogic on
bash-3.2# chkconfig --level 345 oracle-xe on
bash-3.2# chkconfig --level 345 vsftpd on
bash-3.2# chkconfig --list
initWeblogic    0:off   1:off   2:off   3:on    4:on    5:on    6:off
mcstrans        0:off   1:off   2:on    3:on    4:on    5:on    6:off
netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off
netfs           0:off   1:off   2:off   3:off   4:off   5:off   6:off
netplugd        0:off   1:off   2:off   3:off   4:off   5:off   6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off
oracle-xe       0:off   1:off   2:on    3:on    4:on    5:on    6:off
rdisc           0:off   1:off   2:off   3:off   4:off   5:off   6:off
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off
vsftpd          0:off   1:off   2:off   3:on    4:on    5:on    6:off

This will enable the services initWeblogic, oracle-xe, and vsftpd to start at boot-up and be ready for servicing requests as soon as a new instance of this particular image is created on the cloud.

Stopping Services and Unmounting the Image

Stop the services that were started during image creation, and exit from chroot jail

Services started inside the chroot jail and accessing resources from inside the rerouted folder will make the umount command used to unmount these files back to the image throw a "device busy: error. So let?s stop all the services that were started inside the chroot jail:

bash-3.2# service oracle-xe stop
Shutting down Oracle Database 10g Express Edition Instance.
Stopping Oracle Net Listener.
bash-3.2# service vsftpd stop
Shutting down vsftpd:                              [  OK  ]
bash-3.2#exit
exit
[root@localhost centos]#

Unmount the modified file system back to the image

This is the last step for giving you a single image file (centos.5-2.img) ready to be bundled and uploaded into AWS:

[root@localhost centos]# cd ..
[root@localhost Image]# umount ./centos/proc/
[root@localhost Image]# umount ./centos

Bundle Image and Upload It into AWS

Bundle image using ec2-bundle-image

[root@localhost Image]#ec2-bundle-image -i ./centos.5-2.img -k 
<your ec2-key> -c <your ec2-cert> --user <your AWS-user-id> 

Refer to AWS docs for further information.

Upload bundle using ec2-upload-bundle

[root@localhost Image]#ec2-upload-bundle -b OracleBucket -m 
/tmp/centos.5-2.img.img.manifest.xml -a <your ec2-key> -s 
<your secret-key> 
Refer to AWS docs for further information.

Register image

[root@localhost Image]# ec2-register OracleBucket/centos.5-2.img.img.manifest.xml -k 
<your ec2-key> -c <your ec2-cert> --user <your AWS-user-id> 

Once the image has been registered in AWS, it becomes an Amazon Machine Image (AMI). We can start an instance of this AMI on EC2. Once this instance boots up, it will automatically start all the services that we have configured to run during startup. Thus, once this instance is running, we will be able to directly access Oracle Database XE and Oracle WebLogic Server hosted on the newly created domain.

Screenshots

Oracle Database XE is hosted at http://<cloud-domain>:8080/apex.

Transport configuration

Oracle WebLogic Server 11g is hosted at http://<cloud-domain>:7001/console.

Transport configuration

Conclusion

In this tutorial, we saw how to create a custom virtual machine image prebundled with Oracle WebLogic Server 11g and Oracle Database XE. We also learned the basics of creating init scripts and configuring services to run at user-defined runlevels. The approach taken here, being generic, can be used to create custom prebundled images or to modify existing AMIs to provide additional functionalities or to better utilize the on-demand scalability of PaaS clouds.


Kiran C. Nair specializes in JEE, client-server architecture, and performance lifecycle analysis at SETLabs, the research wing of Infosys Technologies Ltd.