What You See Is What You Get Element

Securing a Cloud-Based Data Center

With Oracle Solaris 11

by Orgad Kimchi, Ron Larson, and Richard Friedman

How to protect and secure your cloud infrastructure using Oracle Solaris technologies such as Oracle Solaris Zones, ZFS, and network virtualization.

Published August 2014

right arrow Part 1 - What's Involved in Building a Cloud-Based Data Center
right arrow Part 2 - Securing a Cloud-Based Data Center

Want to comment on this article? Post the link on Facebook's OTN Garage page.  Have a similar article to share? Bring it up on Facebook or Twitter and let's discuss.
Table of Contents
Security in the Cloud
Oracle Solaris 11 Security Features
How Oracle Solaris Remote Lab Achieves Its Security Goals with Oracle Solaris 11
See Also
About the Authors

No doubt, with all the media reports about stolen databases and private information, a major concern when committing to a public or private cloud must be preventing unauthorized access of data and applications. In this article, we discuss the security features of Oracle Solaris 11 that provide a bullet-proof cloud environment. As an example, we show how the Oracle Solaris Remote Lab implementation utilizes these features to provide a high level of security for its users.

Security in the Cloud

When we build a cloud, the following aspects related to the security of the data and applications in the cloud become a concern:

  • Sensitive data must be protected from unauthorized access while residing on storage devices, during transmission between servers and clients, and when it is used by applications.
  • When a project is completed, all copies of sensitive data must be securely deleted and the original data must be kept permanently secure.
  • Communications between users and the cloud must be protected to prevent exposure of sensitive information from "man in a middle attacks."
  • Limiting the operating system's exposure protects against malicious attacks and penetration by unauthorized users or automated "bots" and "rootkits" designed to gain privileged access.
  • Strong authentication and authorization procedures further protect the operating system from tampering.
  • Denial of service (DoS) attacks, whether they are started intentionally by hackers or accidentally by other cloud users, must be quickly detected and deflected, and the service must be restored.

In addition to the security features in the operating system, deep auditing provides a trail of actions that can identify violations, issues, and attempts to penetrate the security of the operating system. Combined, these threats and risks reinforce the need for enterprise-grade security solutions that are specifically designed to protect cloud environments.

With Oracle Solaris 11, the security of any cloud is ensured. This article explains how.

Oracle Solaris 11 Security Features

Security features are inherent throughout the Oracle Solaris 11 operating system. Some of these features and the threats they address are introduced here. (See the Oracle Solaris security documentation for a more detailed description.)

Protection of Sensitive Data

In every cloud environment, data in storage is always at risk from unintended errors or actual data theft. Usually an enterprise's local IT group is responsible for data and system security. But in a cloud environment, the cloud provider must guarantee complete security.

The Oracle Solaris 11 ZFS file system provides several mechanisms, such as encryption of data and file system metadata, to secure sensitive data in storage. Encryption and a centralized key management system extend protection out to any service using the ZFS file system, including NFS and images of Oracle Solaris Zones virtualization environments.

Secure Deletion of Sensitive Data

Clearly, public cloud implementations that reuse physical resources to improve resource utilization must ensure that any data belonging to a previous cloud tenant cannot be retrieved by a new cloud tenant.

ZFS file system encryption provides a simple and secure way to delete copies of sensitive data. While the copy-on-write feature in ZFS defeats the usual overwrite strategy for deleting data files, an encrypted file system can be made unreadable simply by changing the file system's encryption key.

Secure Communications

The cloud infrastructure must protect data in motion between the end user and the public cloud or private cloud or between the private cloud and the public cloud in a hybrid cloud environment.

One method of protecting data in motion is to encrypt ingress and egress across the internet using the Secure Sockets Layer (SSL) protocol. Oracle Secure Global Desktop provides encryption of all network traffic using SSL between the client connection and the cloud's main portal.

Encryption by itself is highly compute-intensive. Architectures without hardware-assisted cryptographic capabilities experience a significant performance penalty that impacts the entire system, with the potential for bogging down the speed of other business applications running on the system. Current SPARC processors from Oracle include an integrated hardware cryptographic accelerator that eliminates performance degradation while also reducing cost.

Oracle Secure Global Desktop provides a gateway capability to enable isolation of Oracle Secure Global Desktop servers from the network. Oracle Secure Global Desktop gateway is deployed in a demilitarized zone (DMZ) in front of an array of Oracle Secure Global Desktop servers. This configuration enables the array to be located on an organization's internal network. All connections to the gateway are authenticated in the DMZ before any connections are made to the servers.

The Oracle Secure Global Desktop gateway also manages load balancing of Secure HyperText Transfer Protocol (HTTPS) connections.

Ensuring Network Isolation Through Virtualization

Network virtualization provides improved security by logically segregating the network traffic of users. Oracle Solaris 11 network virtualization allows any physical network topology to be built rapidly by utilizing software components rather than hardware. This includes virtual network interface cards (VNICs), virtual switches, and more-sophisticated network components, such as load balancers, routers, firewalls, and virtual local area network (VLAN) network-segregation technology.

Using network virtualization eliminates the need to invest in additional network equipment, significantly reducing infrastructure costs.

With all the network building blocks based on software, not on hardware, deployment is rapid, allowing cloud system administrators to implement software-defined network topologies for their data centers.

In every cloud environment, it is critical to segregate each tenant's network traffic from the traffic of all other tenants sharing the same infrastructure. Oracle Solaris 11 offers network segregation technologies such as VLANs and firewalls to support the needed segregation.

VLANs provide the ability to maintain network traffic isolation between tenants of the cloud. This enables subdivision of a local area network (LAN) at the data link layer of the protocol stack (layer 2). VLANs also make network administration easier by organizing networks into smaller groups.

Oracle Solaris 11 provides a built-in firewall enabling the isolation of network traffic between the cloud tenants' networks and between applications.

Packet filtering provides basic protection against network-based attacks. The IP Filter feature of Oracle Solaris 11 can filter network activity by IP address, port, protocol, network interface, and traffic direction, as well as by an individual source IP address, a destination IP address, by a range of IP addresses, or by address pools.

IP Filter executes a sequence of steps as a packet is processed. Figure 1 illustrates the packet processing steps and how filtering is integrated with the TCP/IP protocol stack.

Packet processing steps and the IP filtering

Figure 1. Packet processing steps and the IP filtering.

Preventing Spoofing Attacks

Network virtualization technology also provides data link protection through Oracle Solaris Zones technology, which can allow privileged root access within a zone while disallowing the ability to send spoofed outgoing packets with a different source IP address or MAC address, or packets that aren't IPv4, IPv6, or ARP.

Cloud administrators can set the following spoofing protection modes:

  • ip-nospoof: Requires that any outgoing IP, ARP, or NDP packet have an address field that matches either a DHCP-configured IP address or one of the addresses listed in the allowed-ips link property.
  • mac-nospoof: Prevents the root user from changing the zone's MAC address. An outbound packet's source MAC address must match the data link's configured MAC address.
  • dhcp-nospoof: Prevents client ID/DUID spoofing for DHCP.
  • restricted: Allows only IPv4, IPv6, and ARP protocols. Using this protection type prevents the link from generating potentially harmful layer 2 control frames.

The following example demonstrates how enforcing link protection can be used to prevent zone IP address changes by the zone's root user.

  1. Create the VNIC:

    root@global_zone:~# dladm create-vnic vnic0 -l net0
  2. Enable the link protection modes on the VNIC:

    root@global_zone:~# dladm set-linkprop -p \
    protection=mac-nospoof,restricted,ip-nospoof vnic0
  3. Specify IP address for the allowed-ips property on the vnic0 link:

    root@global_zone:~# dladm set-linkprop -p allowed-ips= vnic0
  4. Verify the link protection property values:

    root@global_zone:~# dladm show-linkprop -p \
    protection,allowed-ips vnic0
    LINK     PROPERTY            PERM VALUE        DEFAULT      POSSIBLE
    vnic0    protection          rw   mac-nospoof, --           mac-nospoof,
                                      restricted,               restricted,
                                      ip-nospoof                ip-nospoof,
    vnic0    allowed-ips         rw     --           --
  5. In Step 3, was set as the allowed IP address. Let's now log in to user-zone as root and attempt to change the zone's IP address. As you will see, we will be prevented from doing so:

    1. First, log in:

      root@global_zone:~# zlogin user-zone
    2. Now, let's remove the IP interface:

      root@user-zone:~# ipadm delete-ip vnic0
    3. Next, create a new IP interface:

      root@user-zone:~# ipadm create-ip vnic0
    4. Now try to assign a new IP address (

      root@user-zone:~# ipadm create-addr -a local= vnic0/v4
      ipadm: cannot create address: Permission denied

      As we can see, the root user can't change the zone's IP address.

  6. Let's try to change the IP address to the allowed IP address (

    root@user-zone:~# ipadm create-addr -a local= vnic0/v4
  7. Verify the IP address change:

    root@user-zone:~# ipadm show-addr vnic0
    ADDROBJ           TYPE     STATE        ADDR
    Vnic0/v4          inherited ok

    We can see that we were able to change the IP address.

Operating System Exposure to Attack

Placing public or hybrid cloud infrastructure outside a company firewall exposes the cloud infrastructure to attacks coming from the internet or from other cloud tenants. Oracle Solaris 11 uses a "secure by default" approach to mitigate these attacks.

"Secure by default" is an operating system posture that limits the system services at startup. For example, there is no default GUI.

Many daemons and administrative commands are assigned only the privileges that enable them to succeed. Many daemons are run from special administrative accounts that do not have root (UID=0) privileges, so they cannot be hijacked to perform other tasks. These special administrative accounts cannot log in.

Regardless of the configuration, the root user within the non-global zone cannot acquire all the privileges available to the global zone. By design, there are commands that users cannot run in their zones. This reinforces the security posture of Oracle Solaris Zones.

The list of privileges can be queried from within a zone by using the ppriv(1) command:

root@user-zone:~# ppriv -l zone 

Rootkits are serious threats that insert hostile code through custom kernel modules. Oracle Solaris Zones technology prevents loading or unloading kernel modules by denying zones the sys_config privilege. This limits the attack surface and prevents this form of attack.

In the following example, we can see that even the root user is unable to load a custom kernel module inside a zone:

root@user-zone:~# ppriv -De modload -p /tmp/systrace

modload[21174]: missing privilege "ALL" (euid = 0, syscall = 152)
    needed at modctl+0x52
Insufficient privileges to load a module

Oracle Solaris Zones technology also protects the system from malicious or accidental tampering with system binaries and configuration during runtime. This capability is known as Immutable Zones and is documented further in "Configuring and Administering Immutable Zones."

The mandatory write access control (MWAC) kernel policy enforces file system write privileges through a zonecfg file-mac-profile property. The global zone is not subject to the MWAC policy, so the global zone can write to a non-global zone's file system for installation, image updates, and maintenance.

The file-mac-profile can be set to one of the following values, which provide flexible OS hardening policies. The benefit of using these polices is gaining the flexibility to change the hardening policy based on the operating system's exposure to security risks.

For example, you can use the flexible-configuration policy for the internal network and use the more-secure fixed-configuration or strict policy in a less-secure environment such as a DMZ.

  • none (the default). Provides a standard, read-write, non-global zone, with no additional protection beyond the existing Oracle Solaris Zones boundaries. Setting the file-mac-profile value to none is equivalent to not setting its value.
  • strict. Enforces a read-only file system; no exceptions.

    • Oracle Solaris Image Packaging System packages cannot be installed.
    • Persistently enabled Oracle Solaris Service Management Facility services are fixed.
    • Service Management Facility manifests cannot be added from the default locations.
    • Logging and auditing configuration files are fixed. Data can only be logged remotely.
  • fixed-configuration. Permits updates to /var/* directories, with the exception of directories that contain system configuration components.

    • Image Packaging System packages, including new packages, cannot be installed.
    • Persistently enabled Service Management Facility services are fixed.
    • Service Management Facility manifests cannot be added from the default locations.
    • Logging and auditing configuration files can be local. syslog and audit configuration are fixed.
  • flexible-configuration. Permits modification of files in /etc/* directories, changes to root's home directory, and updates to /var/* directories.

    • Image Packaging System packages, including new packages, cannot be installed.
    • Persistently enabled Service Management Facility services are fixed.
    • Service Management Facility manifests cannot be added from the default locations.
    • Logging and auditing configuration files can be local. syslog and audit configuration can be changed.

Authentication and Authorization

Self-service with rapid elasticity is the very essence of cloud provisioning. This requirement forces the cloud infrastructure to be very dynamic and flexible. Still, user identities must be authenticated, and activities must be authorized according to the granting of specific privileges.

It's the operating system that must enforce the authority to provision, use, or release cloud services and authenticate users. Oracle Solaris provides a number of access profiles for granting specific authorizations, privileges, and rights.

RBAC: An Alternative to the Superuser Model

Access profiles distribute the responsibilities of the traditional superuser (root) across multiple roles in a scheme referred to as role-based access control (RBAC). RBAC makes it possible to replace the all-powerful superuser (to whom all privileges are granted) with a set of roles that have distinct privileges. For example, a role can be created to handle user account creation, while another could handle system file modification. When a role to handle a function or set of functions has been established, those functions can be removed from root superuser's privileges. Fine-tuning privileges into separate roles that can be assigned to individual accounts or isolated from each other removes any single point of vulnerability.

The root user has the ability to read and write to any file, run all programs, and send kill signals to any process. Specifically, an invader who becomes superuser can easily then modify a site's firewall, alter the audit trail, read confidential records, and shut down the entire network. Programs that run with root permissions, so-called setuid programs, are similarly all-powerful. And a hijacked setuid program could do great harm to the system.

As an alternative to superuser, RBAC uses the security principle of least privilege. Least privilege means that a user has just the privileges that are minimally necessary to perform a task. RBAC collects superuser capabilities into rights profiles. These rights profiles are assigned to special user accounts that are called roles. Regular users have just enough privileges to run their applications, check the status of their jobs, print files, create new files, and other typical activities. Capabilities beyond these are explicitly included in distinct rights profiles. Users who need to take an action that requires privileges beyond those basic privileges would be assigned a role that authorizes the appropriate rights and permissions for that action.

Rights profiles can provide broad capabilities. For example, the System Administrator rights profile enables a role that can perform tasks that are not related to security, such as printer management and cron jobs. Rights profiles can also be narrowly defined. For example, the Cron Management rights profile manages at and cron jobs. When you create roles, the roles can be assigned broad or narrow capabilities or both.

List all authorizations:

root@global_zone:~# getent auth_attr | more
solaris.:::All Solaris Authorizations::help=AllSolAuthsHeader.html
solaris.account.:::Account Management::help=AccountHeader.html
solaris.zone.login:::Zone Login::help=ZoneLogin.html
solaris.zone.manage:::Zone Deployment::help=ZoneManage.html

List all rights profiles:

root@global_zone:~# getent prof_attr | more
All:::Execute any command as the user or role:help=RtAll.html
Audit Configuration:::Configure Solaris
Zone Management:::Zones Virtual Application Environment 
Zone Security:::Zones Virtual Application Environment   
solaris.auth.delegate;help=RtZoneSecurity.html ...

Denial of Service Attacks

While denial of service (DoS) attacks can take many forms, Oracle Solaris 11 has features designed specifically to guard against them. Often both accidental and intentional DoS attacks take the form of a task starving legitimate users of resources such as memory, processor cycles, or network bandwidth. Some forms of DoS attacks and the features that deal with them are discussed here.

Oracle Solaris resource management features enable restricted access to a specific resource and prevent an application from indiscriminately consuming resources.

Preventing Memory Starvation

A malicious or poorly written process can attempt to deny other processes the ability to properly allocate memory. One way repeatedly allocates memory until all the virtual memory (RAM and swap space) is consumed. Further allocation attempts by any process fail. In fact, few applications can continue to operate normally under these conditions. The maximum amount of virtual memory a zone is allowed to allocate can be set when the zone is created by using the zonecfg command.

Preventing CPU Attacks

The most common DoS attacks are caused by an arbitrary code that runs on the target computer consuming as many CPU cycles as possible. Legitimate workloads also running on the system are starved of the system's CPU(s), might become unable to function correctly, and might fail. This can be prevented by setting a limit to the CPU shares for a zone.

In the following example, zonecfg creates the environment for a new zone, setting a limit on the number of fair-share scheduler (FSS) CPU shares for the zone. The example also shows how to select a given resource for modification.

root@global_zone:~# zonecfg -z user-zone
user-zone: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:user-zone> create
zonecfg:user-zone> set zonepath=/zones/user-zone
zonecfg:user-zone> set autoboot=true
zonecfg:user-zone> set cpu-shares=5
zonecfg:user-zone> add capped-memory
zonecfg:user-zone:capped-memory> set physical=5g
zonecfg:user-zone:capped-memory> set swap=6g
zonecfg:user-zone:capped-memory> end
zonecfg:user-zone> exit

Preventing Process Table Attacks

One well-known DoS attack to over-consume system resources is a fork bomb. This method does not necessarily consume a great deal of memory or CPU resources, but rather it seeks to saturate the process slots in the Oracle Solaris kernel's process table. In Oracle Solaris, a running process starts with just one thread of execution, also called a light-weight process (LWP). Many programs generate new threads, becoming multithreaded processes. By default, Oracle Solaris systems with a 64-bit kernel can run over 85,000 LWPs simultaneously. A booted zone that is not yet running any applications has approximately 100 to 150 LWPs. To prevent a zone from using too many LWPs, a limit can be set using the zonecfg max-lwps property, which limits the maximum number of LWPs simultaneously available to this zone.

The following command sets a limit of 300 LWPs for a zone:

root@global_zone:~# zonecfg -z user-zone 'set max-lwps=300'

Reboot the zone in order to enforce this limitation:

root@global_zone:~# zoneadm -z user-zone reboot 

The number of LWPs used by a zone can be monitored using the prstat -LZ command from the global zone.

Note: Oracle Solaris can limit other types of resources such as shared memory; see zonecfg(1M) for more examples.

The resource controls of a running zone can be retrieved using the prctl command. For example, the following command shows how to retrieve the user-zone resource controls:

root@global_zone:~# prctl -i zone user-zone

For more examples of system resource monitoring, see "Performance Analysis in a Multitenant Cloud Environment."

Using Network Controls

Oracle Solaris 11 comes with two network resource management technologies: bandwidth control and flows.

Bandwidth control can be configured on a VNIC of the global zone to limit the network traffic sent and received from a specific zone through that zone's VNIC.

Cloud administrators can limit the throughput of the VNIC using the maxbw property of the VNIC and the dladm command.

For example, setting the VNIC vnic0 (which we created in a previous example) maximum throughput to 500 Mb/sec effectively limits its use to a portion of the bandwidth of the physical data link (network port net0) connection.

root@global_zone:~# dladm set-linkprop -p maxbw=500M vnic0

Runtime data link statistics can be monitored using the dlstat command to detect peaks in network traffic.

Through the use of flows, Oracle Solaris 11 provides a sophisticated quality of service (QoS) with finer-grained network bandwidth control.

A flow is a customized way of categorizing packets to further control how resources are used to process these packets. Network packets can be categorized according to an attribute. Packets that share an attribute constitute a flow and are labeled with a specific flow name. The flow can then be assigned specific resources.

The attributes that serve as the basis for creating flows are derived from the information in a packet's header. Packet traffic can be organized into flows according to one of the following attributes:

  • IP address
  • Transport protocol (UDP, TCP, or SCTP)
  • Application port number, for example, port 443 for HTTPS

Both bandwidth control and flows can be applied on the same data link.

In the following example, HTTPS (TCP port 443) traffic is limited to 100 Mb/sec on the vnic1 network interface. This is useful in order to mitigate DoS attacks against a web server. The impact of a DoS attack on the rest of the infrastructure is limited by setting the zone's HTTPS maximum network bandwidth to 100 Mb/sec.

In the following example, SSL traffic is limited to 100 Mb/sec on the network interface:

  1. Create the VNIC:

    root@global_zone:~# dladm create-vnic vnic1 -l net0
  2. Create the flow on top of this VNIC:

    root@global_zone:~# flowadm add-flow -l vnic1 -a \
    transport=TCP,local_port=443 https-flow

    In the command above, -l specifies the link name and -a specifies the flow's attributes: transport (TCP) and port (443). https-flow is the name of the flow.

  3. Implement resource controls on the flow:

    root@global_zone:~# flowadm set-flowprop -p maxbw=100M https-flow

    maxbw is the maximum amount of the link's bandwidth that packets identified with this flow can use.

    The limitations implemented in the example above are shown in Figure 2:

     Example of implementing a flow and resource controls

    Figure 2. Example of implementing a flow and resource controls.

Flow statistics for a VNIC can be monitored using the flowstat command from the non-global zone or from the global zone.

For more network monitoring examples, see "Advanced Network Monitoring Using Oracle Solaris 11 Tools."

With these limits set, the impact of any DoS attack on this web server is minimized on the rest of the infrastructure.

Operating System Auditing

Preserving an audit trail of system activities is critical. Auditing helps detect potential security breaches by revealing suspicious or abnormal patterns of system usage. Auditing also provides a way to trace suspicious actions back to a particular user. If users know that their activities are being audited, they are less likely to attempt malicious activities.

Protecting a computer system, especially a system on a network, requires mechanisms that control activities before system processes or user processes begin. Security requires tools that monitor activities while in progress and reports on them after the activities have ended.

Oracle Solaris 11 auditing features can be used to provide a detailed record of all system activities and administrative actions.

The system generates audit records for specific events, such as the following:

  • System startup and system shutdown
  • Login and logout
  • Process creation or process destruction, or thread creation or thread destruction
  • The opening, closing, creating, destroying, or renaming of objects
  • Use of privilege capabilities or RBAC

The audit service monitors the entire system, including activities in zones. A system that has installed non-global zones can run a single audit service to audit all zones identically. Or, it can run one audit service per zone, including the global zone.

The advantages of per-zone auditing are a customized audit trail for each zone, and the ability to disable auditing on a zone-by-zone basis. But there is some administrative overhead. Each zone administrator must manage the auditing. And, each zone must run its own audit daemon, with its own flags and audit policy.

The audit service can be instructed to copy some or all of the audit records in the audit queue to a remote syslog server. By using syslog to store audit records remotely, you protect log data from alteration or deletion by an attacker. The syslog records provide greater convenience and flexibility. For example, you can collect the syslog data from a variety of sources and correlate the different types of data, in addition to having a centralized audit repository.

The following example selects audit classes to be sent to the syslog service.

root@global_zone:~# auditconfig -setplugin audit_syslog active \ 

This configures the syslog service and adds an audit.notice entry to the syslog.conf file. The entry includes the name or IP address of the remote syslog server:

The entry for the remote system can be seen by running the following command:

remote1.root@global_zone:~# cat /etc/syslog.conf
audit.notice       @remote1

The audit.notice entry in the syslog.conf file on the remote1 system points to the log file:

root@remote1 # cat /etc/syslog.conf
audit.notice       /var/adm/auditlog

On the remote system, create the log file:

root@remote1:~# touch /var/adm/auditlog

Refresh the configuration information for the syslog service:

root@global_zone:~# svcadm refresh system/system-log

Refresh the audit service. The audit service reads the changes upon refresh:

root@global_zone:~# audit -s

Note: Instead of a remote syslog, you can send the audit files to a remote audit repository. For more information, see the audit_remote(5) man page.

How Oracle Solaris Remote Lab Achieves Its Security Goals with Oracle Solaris 11

As a real-world example of how native Oracle Solaris 11 features can be used to provide a high degree of security for user data and applications, let's take a look at Oracle Solaris Remote Lab. Introduced in the Part 1 of this series on cloud building, Oracle Solaris Remote Lab is a virtual lab that grants users remote access through a secure web browser to virtual machines running Oracle Solaris 11. Implemented with Oracle Solaris 11, Oracle Solaris Remote Lab is a showcase for Oracle Solaris 11 virtualization and security features.

Ready to use "virtual machines" in the Lab are implemented as Oracle Solaris Zones and can be any combination of SPARC-based or x86-based zones. These virtual machines are created with Oracle Database, Oracle Fusion Middleware, and/or Oracle Solaris Studio preinstalled.

When users first access the Lab, they are assigned a unique VLAN, a private NFS server, and a private Oracle Secure Global Desktop server (see Figure 3). The NFS server and Oracle Secure Global Desktop server for a user are linked to the user's unique VLAN. The virtual machines a user creates are implemented as Oracle Solaris 11 Zones and are linked to that user's VLAN. Thus, the VLAN facilitates all network traffic associated with the user and serves to segregate that user's network traffic from other users' network traffic.

Oracle Solaris Remote Lab implemented with Oracle Secure Global Desktop servers, NFS servers, and SPARC- and x86-based Oracle Solaris Zones.

Figure 3. Oracle Solaris Remote Lab implemented with Oracle Secure Global Desktop servers, NFS servers, and SPARC- and x86-based Oracle Solaris Zones.

The private NFS servers assigned to each user are actually implemented as individual Oracle Solaris 11 Zones on a server that has significant mass storage. By making each user's NFS server a zone, the storage assigned to each user is segregated from the storage assigned to other users.

Similarly, the private Oracle Secure Global Desktop servers handling traffic between the user's client system and the user's virtual environment are also implemented as individual Oracle Solaris 11 Zones on a server that has a significant amount of memory. Again, the use of zones serves to segregate the user's traffic from the traffic of other users. This private server provides the user with access to their virtual machines through command-line terminal sessions or full-screen Oracle Solaris desktops, as well as providing remote file transfer capabilities between the user's local system and assigned secure storage in the Lab.

The actual implementation of the Lab is a little more complex than implied by the user view diagram shown in Figure 4. It incorporates three front-end systems in a secure data center that handle direct communications with the user over the internet communicating with back-end systems in another data center that provide the virtual machine servers, Oracle Secure Global Desktop server, NFS server, ZFS centralized key management, and an audit server. Firewalls protect both data centers and communications between them is encrypted.

Oracle Solaris Remote Lab implemented with Oracle Secure Global Desktop servers, NFS servers, and SPARC- and x86-based Oracle Solaris Zones.More details for user view of the Oracle Solaris Remote Lab implementation

Figure 4. More details for user view of the Oracle Solaris Remote Lab implementation.

Protection of Sensitive Data

Oracle Solaris Remote Lab makes extensive use of features in the ZFS file system, including a centralized key management system to protect all the user data, applications, and virtual machines in the Lab.

Physical disk storage is accessed only through ZFS file systems, and never as raw devices. Each zone in the Lab has its own file system and does not have access to other file systems.

A zone's root file system is encrypted, as are the local ZFS file systems created for users in their zones. The encryption keys for all file systems are stored in a centralized key management system and are not visible to the zone users—even to those who have zone root credentials—as shown in Figure 5.

Encryption keys for all file systems are stored in a centralized key management system

Figure 5. Encryption keys for all file systems are stored in a centralized key management system.

The following example demonstrates how to set up an encrypted ZFS data set with a centralized key management system and then create a zone with its root file system residing on the encrypted ZFS data set. Note that in this case, the encryption keys are stored in the centralized key management system outside of the zone's file system and aren't managed by or visible to the zone users (even root).

  1. Create the ZFS data set with a centralized key management system:

    ZFS can get the wrapping key or passphrase from any web service that supports a simple GET request on a Uniform Resource Identifier (URI).

    In the following command, -o encryption=on enables encryption, -o keysource=raw,https://keys.example.com/mykey specifies the URI of the centralized key management system, and rpool/zones/user-zones is the name of the ZFS data set.

    root@global_zone:~# zfs create -o encryption=on -o \
    keysource=raw,https://keys.example.com/mykey rpool/zones/user-zones
  2. Configure the zone and add the ZFS file system (which we created in the previous step):

    root@global_zone:~# zonecfg -z user-zone 'create ; set \   
  3. Install the zone:

    root@global_zone:~# zoneadm -z user-zone install
  4. Once the zone is installed and booted, retrieve the ZFS key location using the zfs get command, as shown in the following example:

    root@user-zone:~# zfs get encryption,keysource rpool/ROOT/solaris
    NAME                PROPERTY   VALUE   SOURCE
    rpool/ROOT/solaris  encryption on      inherited from $globalzone
    rpool/ROOT/solaris  keysource passphrase,https https://keys.example.com/mykey  inherited from $globalzone

    Notice that we inherited the keys from the global zone.

For more examples of Oracle Solaris Zones with encrypted ZFS data sets, see "Immutable Zones on Encrypted ZFS."

Secure Deletion of Data

Once a tenant's project in the Lab is completed, all the data in the virtual machines and the private NFS server must be deleted so it cannot be recovered by any means, even by trying to retrieve the data directly from the hard disk blocks by bypassing the file system layer.

The Lab's use of ZFS encryption with a secure key provides a very simple means of making the data unreadable at the end of a project. When a zone is set for deletion, the ZFS encryption key for its local file system is changed to a random set of bytes that is not saved. Note that even the cloud administrator does not know the new key since it is based on random bytes; this way, the data is also protected from the cloud provider's operations team. The zone and its ZFS file system are then deleted with this new key. This method provides a fast and secure deletion, which ensures any data that was stored in the zone cannot be accessed even after the zone has been discarded.

The following example ensures the deletion of ZFS data set rpool/zones/user-zone.

  1. Change the ZFS encryption key to a random set of bytes:

    root@global_zone:~# zfs key -c -o raw,file:///dev/random \ 
  2. Unmount the ZFS data set, and then unload the wrapping key for an encrypted data set:

    root@global_zone:~# zfs key -u rpool/zones/user-zone
  3. Destroy the ZFS data set:

    root@global_zone:~# zfs destroy rpool/zones/user-zone

    Note that all the zones in the Lab—whether they are used as user zones, NFS file systems, or Oracle Secure Global Desktop servers—use ZFS encryption, and when they are no longer needed, they are securely deleted using this method.

Secure Communications

In the Lab, communications between users and their virtual machines makes use of the gateway features of Oracle Secure Global Desktop deploying an Oracle Secure Global Desktop gateway in a DMZ in front of an Oracle Secure Global Desktop array.

The connection between the gateway and the array is secure and SSL-encrypted. In the Lab, cloud tenants are each assigned a unique VLAN to accommodate communications between their virtual machines (zones), their private Oracle Secure Global Desktop server (a member of the array described above), and their private NFS file system. Users do not have access to the details of their VLANs, which decreases the risk of possible intruder attacks. A user's traffic is forwarded to that user's VLAN segment by smart agents in the Oracle Secure Global Desktop gateway, as shown in Figure 6.

Secure communications are ensured by the Oracle Secure Global Desktop gateway

Figure 6. Secure communications are ensured by the Oracle Secure Global Desktop gateway

Ensuring Network Isolation

Oracle Solaris Remote Lab utilizes the VLAN capabilities of Oracle Solaris 11 by assigning a unique VLAN tag for each cloud tenant to isolate each tenant's network traffic from other tenants' traffic while sharing the same physical infrastructure. As noted earlier, each tenant is assigned to a unique VLAN, and the users do not have access to any of the details of their VLAN.

For example, Figure 3 shows three users, each belonging to a different cloud tenant, each cloud tenant having a private Oracle Secure Global Desktop server and a private NFS server, and VMs with network traffic segregated through the use of VLANs.

The Lab also makes use of the Oracle Solaris 11 firewall to isolate the network traffic between the tenants' VLANs and between different application components, such as the Lab's back end and front end.

Operating System Exposure to Attack

Applications can be deployed in Oracle Solaris Zones as an extension to the concept of a hardened OS. Oracle Solaris Zones add an additional layer of isolation between the application's network service and the operating system of the underlying global zone. A successful attack on the service would compromise only the zone, not the underlying operating system. This isolation prevents attackers from using methods such as "virtual machine escape" to expand their control beyond the resources allocated to them as cloud tenants.

Using Immutable Zones

In the Lab, each zone for an Oracle Secure Global Desktop server is implemented as an immutable non-global zone that is read-only, so modifications to system binaries or system configurations in that zone are blocked. If intruders were able to break into a user's VLAN in an attempt to open a connection to other machines and possibly gain access to another user's VLAN, they would not be able to modify the Oracle Secure Global Desktop server zone.

The zone configuration, which is performed in the global zone using the zonecfg command, specifies the file-mac-profile property.

For example, the following command shows how to change a zone's mandatory write access control (MWAC) kernel policy and then reboot the zone to enforce this policy.

root@global_zone:~# zonecfg -z osgd-zone \
'set file-mac-profile=fixed-configuration'

root@global_zone:~# zoneadm -z osgd-zone reboot

If we log in to the Oracle Secure Global Desktop zone as root and attempt to install a new software package or to add a new user, we will be prevented from doing so, as shown in the following example:

root@global_zone:~# zlogin osgd-zone
root@osgd-zone:~# pkg install wireshark
pkg install: Could not complete the operation on /var/pkg/lock: read-only filesystem.

root@osgd-zone:~# useradd alice

UX: useradd: ERROR: Cannot update system - login cannot be created.

Note: You can enable the file-mac-profile property during zone creation.

Isolating Processes

One of the key features of Oracle Solaris Zones technology is the ability to isolate processes in one zone from those running in other zones (even the global zone). This isolation prevents processes that are running in one zone from monitoring or affecting processes that are running in other zones. Even a process running with superuser capabilities in a zone cannot view or affect activity in other zones.

Consider the following MySQL process running in the global zone:

root@global_zone:~# ps -ef | grep mysqld
   mysql 14423 14246   0   Feb 06 ? 13:00 /usr/mysql/5.1/bin/mysqld  …

As we can see below, if we try to observe this process from a non-global zone, the process isn't visible:

root@user-zone:~# ps -fp 14423
     UID   PID  PPID   C    STIME TTY         TIME CMD

Similarly, processes running in one non-global zone are not visible to other non-global zones.

Authentication and Authorization

Several examples relating to authentication and authorization in Oracle Solaris 11 were given in the previous "Authentication and Authorization" section. The Lab makes use of all these Oracle Solaris 11 features by default.

Denial of Service Attacks

Several examples of DoS attacks and the Oracle Solaris 11 features that deal with them were given in the previous "Denial of Service Attacks"section. When a cloud tenant's virtual machines are created, several limits are put in place capping their use of memory, processors, and process tables.

Auditing Capability in Oracle Solaris Remote Lab

The Lab makes extensive use of auditing to provide a method of tracking any failures in the Lab back to their root cause and tracking possible security violations back to their source.


Oracle Solaris Remote Lab is an example of a secure cloud platform based on Oracle Solaris 11 features. The Lab's implementation addresses security requirements at every layer in the cloud: networking, storage, and operating system virtualization It is an example of a holistic security approach, since all the building blocks are built in the Oracle Solaris 11 operating system.

Security measures are "a must" to ensure that cloud tenants do not pose a risk to one another in terms of data loss, data misuse, or privacy violation. Multitenancy protections must be offered by cloud service providers for all layers of their offerings, for example, infrastructure as a service (IaaS) and software as a service (SaaS).

See Also

About the Authors

Orgad Kimchi is a principal software engineer on the ISV Engineering team at Oracle (formerly Sun Microsystems). For six years he has specialized in virtualization, big data, and cloud computing technologies.

Ron Larson is a veteran of forty-five years in the computing industry with extensive experience in computer and software systems design, prototyping, and implementation. He is currently the project manager for the Oracle Solaris Remote Lab project.

Richard Friedman is a freelance technical writer with over thirty years of experience working in high-performance computing, software application development, and programming languages.

Revision 1.0, 08/06/2014

Follow us:
Blog | Facebook | Twitter | YouTube