by Muriel L. Rambeloarison and Angelo Rajadurai
Published January 2013More About the Oracle Solaris Remote Lab
This article shows how we used Oracle Solaris 11 to develop a cloud infrastructure for the Oracle Solaris Remote Lab, which independent software vendors can use to validate and certify their applications on Oracle Solaris 11.
Access to the Oracle Solaris Remote Lab is a benefit of Oracle PartnerNetwork (OPN) members at the Gold level and above who are also members of the Oracle Solaris Knowledge Zone. However, the main goal of this article is to show how the cloud infrastructure of the lab is organized so that you can set up your own clouds.
Note: It is not the goal of this article to provide a standard for how a developer cloud should be implemented. The intent is to show how cloud and virtualization technologies featured in Oracle Solaris 11 have been used to create a secure and performing remote virtual lab. The implementation details for any other cloud platforms remain at the discretion of the developers of those cloud platforms.
The Oracle Solaris Remote Lab is based on many cloud technologies that are at the core of Oracle Solaris 11. It is a virtual lab that grants users remote access to Virtual Machines (VMs) through a secure Web browser. Within minutes, instead of hours or even days, users can have access to testing environments. The Oracle Solaris Zones technology and the ZFS file system, which are both central to Oracle Solaris 11, are used to create VMs with a single click.
The ready-to-use VMs can be either SPARC-based or x86-based with Oracle Database and/or Oracle Fusion Middleware preinstalled. Oracle Secure Global Desktop technology provides access to the VMs through terminal sessions or full-screen graphical Oracle Solaris desktops. Oracle Secure Global Desktop also provides the ability to transfer files from user's local computers to the secure storage assigned to them within the Oracle Solaris Remote Lab.
By providing users the opportunity to skip the typical steps of first acquiring servers and then provisioning them with the required software, the Oracle Solaris Remote Lab provides significant cost and time savings.
The Oracle Solaris Remote Lab provides users with access to remotely accessible environments to test and tune their applications on the latest major release of Oracle Solaris. The lab has been designed to take advantage of the many cloud technologies packaged with Oracle Solaris 11.
Figure 1 shows the structure of the lab, which is logically split into a front end and a back end separated by a firewall.
Figure 1. Overview of the Oracle Solaris Remote Lab Front End and Back End
(*Oracle SGD stands for Oracle Secure Global Desktop)
The remainder of this article focuses on the following building blocks, which were used to create the lab, and demonstrates how Oracle technologies, primarily features of Oracle Solaris 11, have been used to deliver the cloud architecture shown in Figure 1:
The user interface of a developer cloud is the main portal for interaction between users and the machines on their assigned cloud. Figure 2 illustrates the user dashboard of the Oracle Solaris Remote Lab, which is the primary user interface users are directed to when they log in.
Figure 2. Dashboard of the Oracle Solaris Remote Lab
Through this dashboard, users generate requests to check out VMs, request visual access to these VMs, and set up file transfers, as detailed in the following sections.
Via their dashboard, users can create up to five VMs, which can be either x86 or SPARC machines or a combination of the two. The dashboard also allows users to reboot within seconds and securely delete VMs.
Each VM has 4 GB of RAM and 10 GB of disk space. Various VM images provisioned with preinstalled database or middleware software are available. All of these images include the Oracle Solaris Studio compiler and tools suite. At the time of writing, the following VMs are available for creation:
VMs are implemented as Oracle Solaris Zones, which are virtualized operating system environments created within a single instance of the Oracle Solaris 11 operating system (see the "Oracle Solaris Zones" section for more details).
When users request the creation of a new VM from the dashboard, this request is inserted into the MySQL database in the front end. Provisioning is done in the back end, where a multitude of servers initiate and perform the creation of the VM, which is ready for use within five minutes.
The MySQL database is the main communication channel in the Oracle Solaris Remote Lab. It acts as a queuing system where all the requests for the back end provisioning services are recorded. Java agents from servers in the back end regularly poll the database for any outstanding job requests, which they then perform.
From the Oracle Solaris Remote Lab dashboard, users can request terminal sessions or full-screen Oracle Solaris 11 desktops to visually interact with their VMs. This graphical desktop access is implemented with Oracle Secure Global Desktop, which uses a remote display protocol (Adaptive Internet Protocol or AIP) to provide users with secure, remote access to desktop applications running on Oracle Solaris 11 (and other operating systems) from a variety of computers and mobile devices. AIP, combined with various other performance features provided by Oracle Secure Global Desktop, allows for excellent performance even over high-latency WAN links.
With Oracle Secure Global Desktop, users can access their Oracle Solaris Remote Lab VMs remotely from a variety of client devices. The only requirement is a Java-enabled Web browser. As a Java-based technology, Oracle Secure Global Desktop ensures that the target operating system being accessed is isolated from the device and the Web browser being used to access the operating system.
Whenever users register with the Oracle Solaris Remote Lab, an Oracle Secure Global Desktop server zone is created and assigned to them. This server zone, which sits on the VLAN where the user's VMs are located, is connected to an Oracle Secure Global Desktop Gateway located in the front end of the lab. The role of this gateway is to direct each user's network traffic to the correct VLAN and, thus, to the correct VMs. The "Network Security" section provides more details on how the VLAN and the overall network virtualization scheme of the Oracle Solaris Remote Lab are organized.
In the Oracle Solaris Remote Lab, permanent data storage between VMs is implemented using an NFS server. A shared directory,
/data, is present on each VM in a user's VLAN. Each user has a dedicated NFS server zone present on their VLAN, and
/data is mounted there. Sharing data between the VMs is then as simple as copying files to and from
File transfers between a user's local computer and the Oracle Solaris Remote Lab VMs are handled by file upload and file download mechanisms, which also make use of the user's dedicated NFS server zone. The upload mechanism allows users to upload files from their local machines to their shared
/data directory, making the files accessible to all of the user's VMs. Similarly, users can download any files from their VMs to their local machines by placing the files in the
/data directory and initiating a file download. Because these mechanisms take advantage of the user's NFS server zone, they allow users the opportunity to upload and download files regardless of whether they have any existing VMs.
Virtualization technologies are a central part of the Oracle Solaris Remote Lab as a developer cloud. By allowing for resource sharing, virtualization increases to a more efficient and adaptive level the utilization of the servers and storage running the services in the back end of the lab.
To implement compute, data, and network virtualization, the lab relies on three Oracle Solaris 11 virtualization features: Oracle Solaris Zones, the ZFS file system, and the Oracle Solaris 11 network virtualization technology.
Figure 3. Building Blocks of Virtualization in the Oracle Solaris Remote Lab
The following sections define these technologies and explain how they are used in the Oracle Solaris Remote Lab.
Oracle Solaris Zones provide a virtualized operating system environment that is created within a single instance of the Oracle Solaris 11 operating system. When applications and processes are installed on Oracle Solaris Zones, they are isolated from one another on the same operating system, which provides an isolated application execution environment.
Each zone is a complete resource-controlled environment in which resources such as memory, networking, storage, and CPU usage can be specifically and independently allocated. An Oracle Solaris Zone also provides an abstract layer that separates applications from the physical attributes (for example, physical devices) of the machine on which it is deployed.
Oracle Solaris Zones are widely used in the Oracle Solaris Remote Lab: user VMs, NFS servers, and Oracle Secure Global Desktop servers are all implemented as zones. These zones are created through a process called zone cloning, during which a new zone is provisioned by copying an existing working zone. The new zone includes all the changes and configurations that were made to customize the source zone.
In the Oracle Solaris Remote Lab, each of the systems providing user VMs has zones preinstalled with application software that represents the VM images that the user can select when creating a new VM. When a new VM is requested, a clone of the selected VM image's zone is produced. Zone cloning requires very little overhead and significantly less time than fresh creation of a new zone. As such, this zone cloning mechanism provides users with access to ready-to-use VMs within minutes, and those VMs are already configured with application software, such as an Oracle Database instance or Oracle Fusion Middleware.
The ZFS file system is the default file system included in Oracle Solaris 11 and in the Oracle Solaris Remote Lab. With its self-healing capabilities, its transparent encryption functionality, and its snapshot feature, this file system helps simplify the deployment and management of petabyte-scale storage. The essence of the ZFS file system's architecture lies in the concept of a virtual storage pool, which decouples the file system from physical storage in the same way that virtual memory abstracts the address space from physical memory, allowing for much more efficient use of storage devices.
In the Oracle Solaris Remote Lab, whenever a VM zone is created via cloning, a ZFS file system is also cloned underneath it to act as local storage for the VM. ZFS cloning consists of creating a writable snapshot of a ZFS file system, which is a nearly instantaneous operation in Oracle Solaris 11.
A ZFS file system clone initially shares all its disk space with the original snapshot, thus consuming no extra disk space. Each ZFS clone uses the copy-on-write (COW) optimization strategy, whereby only the changes made to the original ZFS file system are saved. This strategy reduces the need for overhead memory and significantly increases the speed and performance of the Oracle Solaris Remote Lab when it loads.
The encryption functionality of ZFS is built in, inheritable to descendant file systems, and completely transparent to the applications and files present on the file system. When it is enabled, all the data and metadata of the file system is encrypted when stored permanently in the ZFS pool. In the Oracle Solaris Remote Lab, whenever a ZFS file system is cloned, a key server in the back end randomly generates a new key with which the file system is encrypted. In accordance to a new feature in Oracle Solaris 11, this key is accessible via HTTPS and is not saved in a file.
The built-in network virtualization technology of Oracle Solaris 11 (previously known as Project Crossbow) provides network virtualization and bandwidth resource control that virtualizes the core networking stack and Network Interface Controllers (NICs) around any service protocol (for example, HTTP, HTTPS, FTP, and so on) or VM.
The architecture of the network virtualization is built around virtual NICs (VNICs), which result from virtualizing a physical NIC into a multitude of virtual ones. At the time it is created, each VNIC is assigned its own MAC address and an optional VLAN ID. It can then be assigned to the VMs sharing the physical NIC. Network virtualization also allows VNICs to be assigned specific resources, such as bandwidth and CPU usage limits, which leads to more effective sharing of networking resources.
As previously mentioned, in the Oracle Solaris Remote Lab, all the VMs for a given user sit on a non-routable VLAN, which is assigned when the user first registers with the lab. Because each VM is implemented as an Oracle Solaris Zone, it can be assigned its own VNIC and also its own IP stack and address.
When a user creates a new VM, this VM is given an IP address and it is associated with the user's VLAN; the VMs of the user are, thus, located within a virtual network that is accessible only by the user.
A developer cloud should offer several measures of security to ensure that each user's data, applications, and network remain private. Given that virtualization technologies revolve around resource sharing, it is even more important to provide security at different levels in a developer cloud. In parallel, the integrity of the machines implementing the developer cloud should be ensured at all times in order to detect, prevent, and avoid any attacks from intruders.
In the Oracle Solaris Remote Lab, a significant level of security is achieved through isolation, not only in terms of physical architecture, servers in the front end and the back end are isolated from one another, but also through the features of the virtualization technologies that are used. The following sections highlight some of the security and isolation layers as observed through three main angles: data, computing resources, and network.
A major part of the data security of the Oracle Solaris Remote Lab is provided by the ZFS file system. As mentioned in the "Oracle Solaris 11 ZFS File System" section, the ZFS file system of each user is encrypted with a unique, randomly generated encryption key. This key is saved in a key store, which is accessible only through a secure connection.
Each VM created by an Oracle Solaris Remote Lab user has a local ZFS file system associated with it, and each of these local ZFS file systems has its own unique encryption key. Each user also has a unique NFS server. All of a user's VMs and, thus, the local file systems associated with them, as well as the user's NFS server are isolated on a specific VLAN. These measures ensure that the security of user's data is maintained when they are using VMs.
The ZFS technology allows changes to be made to a ZFS file system encryption key without bearing the overhead of re-encrypting the data set present on the file system. In the Oracle Solaris Remote Lab, this feature is used when users delete VMs.
When a VM is set for deletion, the ZFS encryption key for its local file system is changed to a random set of bytes that is not saved. The VM and its ZFS file system are then deleted with this new key. This method provides a fast secure delete, which ensures any data that was stored in the VM cannot be accessed even after the VM has been discarded.
As previously mentioned, the Oracle Solaris Remote Lab is logically split between a front end and a back end separated by a firewall. This firewall is only selectively open, so only a very restricted selection of ports is open to ensure the required communication between both ends. All communications between the front end and back end happen through the MySQL database in the front end, which keeps a record of any outstanding job requests forwarded by the servers in both ends of the lab. This mechanism allows the servers to be physically isolated from one another; indeed, no server in the back end ever directly calls or accesses any server in the front end and vice versa, thus reducing the risk of intruders' attacks spreading easily.
Another layer of security for the computing resources is provided by the structure of the Oracle Solaris Zones. By default, each system has a global zone, which is used to administer all the other non-global zones. In the Oracle Solaris Remote Lab, the VM zones, the NFS server zones, and the Oracle Secure Global Desktop server zones, among others, are all non-global. Access to the global zone has been restricted to only the administrators of the Oracle Solaris Remote Lab. Additionally, while it is possible to access a non-global zone by logging in through the global zone, the inverse is not possible. Thus, this configuration ensures that if intruders were to break into any non-global zone, they would still remain unprivileged users and be unable to spread their attack to the other building blocks of the lab.
It is also important to mention that every system in the Oracle Solaris Remote Lab is located in a Demilitarized Zone (DMZ), which is a sub-network between the internet and the Oracle WAN. This DMZ offers protection to the Oracle WAN, because it provides the network with a layer of security from the outside world. As such, even if intruders were successful in attacking any of the hosts running in the Oracle Solaris Remote Lab, they would not be able to access the Oracle WAN.
All the VMs for a given user sit on a private non-routable VLAN. As a security measure, each user is assigned to only one VLAN, and each VLAN is assigned to only one user. As previously described, an Oracle Secure Global Desktop server zone is present on each VLAN.
The Oracle Secure Global Desktop Gateway in the front end of the lab acts as a bridge between the VLANs and the internet to direct the network traffic for a given user to the correct VLAN when the user logs in. Along with an enterprise-class network switch located in the front end, this gateway also manages the network traffic in and out of the Oracle Solaris Remote Lab.
The Oracle Secure Global Desktop Gateway comes with Oracle Secure Global Desktop, and it is a proxy server designed to sit in a DMZ and connect to an array of Oracle Secure Global Desktop servers. The gateway authenticates all connections to avoid any unwanted access to the Oracle Secure Global Desktop servers. These servers communicate on standard ports, so additional ports need not be opened between the corporate firewall.
This Oracle Secure Global Desktop connection is secure and SSL-encrypted. Moreover, users do not know the details of the VLAN that has been assigned to them, thereby decreasing the risk of possible intruder attacks. Users are automatically forwarded to their VLANs by smart agents in the Oracle Secure Global Desktop Gateway.
As another security feature, the Oracle Secure Global Desktop server zones in the VLANs are implemented as immutable non-global zones. Immutable zones are read-only zones, and modifications to system binaries or system configurations in such zones are blocked after the zones have been created. With this configuration, if intruders are able to break into a user's VLAN, they will not be able to modify the Oracle Secure Global Desktop server zone in an attempt to open a connection to other machines and gain access to another user's VLAN.
To ensure the proper operation of a developer cloud, several management systems should be implemented to keep track of the logistics for users and their VMs. The following sections highlight how account management, software management, and auditing features are implemented in the Oracle Solaris Remote Lab.
When users visit the Oracle Solaris Remote Lab, they log in using the SSO capability provided by Oracle Access Manager 11g. Oracle PartnerNetwork members use their Oracle PartnerNetwork credentials. All secure access to the lab is handled by SSO; moreover, no passwords for accessing the lab are saved in any database. Therefore, in addition to making the login process simple and streamlined, SSO also provides a significant layer of security to the lab.
Additions and updates of software packages are managed with the Image Packaging System, the network-based software packaging and delivery feature of Oracle Solaris 11. Package repositories for both x86 and SPARC software have been preconfigured on the Oracle Solaris Remote Lab. Users are able to easily update, install, and uninstall packages on their VMs using simple commands in a terminal window.
The main role of the auditing feature in the Oracle Solaris Remote Lab is to prevent and detect any possible intruders by keeping track of who accesses the VMs and any of the servers in the back end. As a security measure, the cloud infrastructure of the lab is built so that auditing and monitoring not only never stops, but also cannot be turned off. It is also not possible for any user to access any auditing and monitoring log.
Auditing is run on a global zone; therefore, even if intruders are able to break into a non-global zone (such as a VM zone, an Oracle Secure Global Desktop server zone, or an NFS server zone), they will not be able to stop the auditing and monitoring.
This article provided details on the implementation of the Oracle Solaris Remote Lab, an Oracle PartnerNetwork resource that helps partners qualify their applications on Oracle Solaris 11. In minutes and with only a few clicks, partners can provision and access dedicated VMs in a secure network and securely upload, store, and delete data to greatly reduce install and qualification times.
The Oracle Solaris Remote Lab was entirely implemented using published Oracle Solaris 11 features, for example, Oracle Solaris Zones as the low-overhead but secure virtualization mechanism, ZFS as the encrypted data store, and network virtualization technology to provide VLANs used for network isolation. Oracle Secure Global Desktop is used for secure data and desktop access from the internet.
A goal of this article was to provide partners and non-partners with ideas on how to use the many cloud features of Oracle Solaris 11 to build a cloud platform. In particular, it focused on the features used in the Oracle Solaris Remote Lab as building blocks for a developer cloud: the user interface, virtualization capabilities, security measures, and management systems. However, partners and non-partners can use these same features to set up their own cloud platforms.
Muriel L. Rambeloarison is a software engineer at Oracle, where she is a member of the Oracle ISV Engineering team. She started at Oracle in 2012 after completing a bachelor's degree and a master's degree in electrical engineering and computer science at MIT.
Angelo Rajadurai is the architect of the Oracle Solaris Remote Lab. He has worked at Sun and Oracle for the last 20 years. He is a technology evangelist for Oracle Solaris technologies in the Oracle ISV Engineering team, where he works with Oracle's partners to adopt Oracle Solaris 11.
|Revision 1.0, 01/07/2013|