Exposing Coherence Cache Data Using Coherence 3.7 and REpresentational State Transfer (REST)

Overview

    Purpose

    Coherence allows applications written in Java, C++, and .NET to access caches and other clustered services using a native client library. Starting with the 3.7 release, Coherence extends support to applications written in any language with a Representational State Transfer (REST) client library, including Pearl, PHP, Ruby, and any other language that supports REST.

    This tutorial covers how to extend cache objects, modify configuration, and start REST-aware servers to expose Coherence Cache data using REpresentational State Transfer or REST as it is more commonly known.

    Time to Complete

    Approximately 90 minutes 

    Scenario

    In this tutorial, you perform the following:

    • Modify several classes to support JAXB or JSON.
    • Create a REST configuration in support of exposing REST-aware classes.
    • Configure Coherence servers to support REST.
    • Test Coherence REST using cURL.

    Introduction

    Coherence 3.7 provides a number of important features, one of which is support for REpresentational State Transfer or REST. Using Coherence REST, languages other than Java, C++ and .NET can access Coherence cache data and reap all the benefits that Coherence provides. In this tutorial, we will examine how to modify a cache object to support REST, how to modify a Coherence configuration to support REST, and how to start both stand-alone and WebLogic Server-based Coherence REST servers. 

    Hardware and Software Requirements

    The following is a list of hardware and software requirements:

    • Coherence 3.7.1, which can be downloaded here
    • Coherence REST OBE sample code, which can be downloaded here

    Prerequisites

    Before starting this tutorial, you should:

    • Have a basic working knowledge of Linux commands and command line syntax
    • Have access to and have already installed Coherence 3.7.1
    • Have downloaded and unpacked the REST OBE sample code. This code is designed to run on Linux and similar operating systems but can be adapted to Windows easily.
    • Have downloaded and installed an Oracle Java JDK version 1.6.0_26 or later. The Oracle JDK kits and associated products can be found here. The installation of Java is outside the scope of this OBE. For more information, see the JDK download and installation pages.

    Note that this tutorial does not intend to teach the overall concepts of Java annotations, JSON, JAXB, or REST but rather the mechanics only. However, this tutorial does attempt to provide guidance, pointers, and reference links throughout.

    Before running this tutorial, you should download and install Oracle Coherence 3.7.1 and the associated example source code. The following steps review the installation process of Coherence 3.7.1, as well as the expected installation locations.  Note that if you have previously installed Coherence 3.7 you may skip this step.

    Using the link above, accept the license agreement and download Coherence 3.7.1. Note that for this tutorial, it is assumed that files are downloaded to/tmp/.  and installed into /opt.

    Open a command prompt and change directory to the location where Coherence will be installed.

    Unzip the contents of Coherence zip using a command that is similar to:

    unzip /tmp/coherence-java-3.7.1...zip 

    Note that the exact name of the file may vary.

    For this tutorial, all files will be unzipped to /opt/coherence. 


    Unzip the contents of the OBE example code using a command that is similar to:
    unzip /tmp/REST.OBE.zip


Modify Classes to Support REST Using JAXB and JSON Annotations

    Representational State Transfer (REST) is an architecture style for large-scale applications that is based on clients and servers sending specifically formatted requests and responses. Such requests come in the form of a URL and an action, such as POST, PUT, GET, DELETE, and so on, with appropriate context. The requests then return a response using a specific representation. In this example, we will examine two representations: JSON and JAXB. Note that in this tutorial, we will introduce annotations for JAXB and JSON but continue using JAXB. Also note that the JAXB representation that comes with Coherence is capable of reading and writing JSON, as well as XML.

    • JavaScript Object Notation (JSON) is a lightweight data interchange format. It is easy for both people and machines to read and write, and is based on a subset of the JavaScript Programming Language. Many browsers and javascript parsers support reading and writing JSON. To learn more about JSON, see www.json.org.
    • Java API for XML Binding (JAXB) is both a data interchange format based on XML and a set of APIs for accessing that data. It allows for reading and writing Java objects into and out of an XML format. Other languages that support JAXB as a data interchange format can also use JAXB. For more information about JAXB, see http://jaxb.java.net

    Note that as JAXB annotations support the generation of both XML and JSON.  As such is JAXB is used in the throughout this example.

      Copy and Modify a Class to Support JAXB Annotations


      In this subsection, we will annotate the Person.java and Address.java classes to expose their properties using JAXB.

      JAXB annotated classes have three minimal requirements:
      • The class must implement some form of serializable, examples implement java.io.Serializable.
      • Each class must specify an XML root element name, which becomes the top or outermost element in the XML representation.
      • Each class must specify a mechanism for exposing its fields. 

      JAXB provides two annotations for these purposes:XmlRootElement and XmlAccessorType. XMLRootElement is somewhat self-explanatory, and provides the name of the outermost XML element. The XmlAccessType attribute specifies how data is extracted. In our examples, we used a type of FIELD, which exposes any public fields and any specifically annotated property methods. PROPERTY is the inverse of field, and exposes any public property fields and specifically annotated fields.

        Confirm you are in the root directory of the OBE.  For this OBE is is assumed to be /opt/REST.OBE,
        Copy the original sources from src/ into the work/ directory.
        cp src/* work/

        Open the Person.java in your favorite text editor.

        Note that completed versions of the classes can be found in the completed/jaxb subdirectory.
        Add the required import and annotation.
        Imports (all shown):

        import javax.xml.bind.annotation.XmlAccessType;
        import javax.xml.bind.annotation.XmlAccessorType;
        import javax.xml.bind.annotation.XmlRootElement;

        Annotations:
        @XmlRootElement(name="person")
        @XmlAccessorType(XmlAccessType.FIELD)


        The completed class should resemble:



        Open the work/Address.java class and annotate it to expose PROPERTYs rather than fields.

        The completed class should resemble:

        Save the changes.
        Close and exit the editor.

      Compile the Updated Source Code

        In a command prompt window, ensure that you are in the /opt/REST.OBE directory.
        If you installed the sources to another directory, ensure that you are in the correct directory.

        For convenience, a command script has been provided that compiles the sources.

        Open build.sh and examine its contents. If required, change the location of installation of Coherence.

        Note that the script assumes that Coherence 3.7.1 was installed in /opt/coherence and that the coherence-rest.jar file is used (via its manifest) to resolve the imports for the Jackson libraries.

        Save any changes and exit the editor.
        Compile the classes using a command that is similar to:

        ./build.sh

        Correct any compilation errors if required.

      Copy and Modify a Class to Support JSON Annotations


      Note that this section and the following compile subsection are optional and informational only.
      JAXB can read and write JSON. JSON Annotations should be considered when some special JSON representation feature is required in addition to what is supported by JAXB. It should be noted that many of the provided aggregator functions cannot consume strict JSON      .

      In this subsection, we will annotate the Person and Address classes to expose their properties using both JSON annotations. Coherence uses the Jackson code libraries to serialized and deserialized between JSON and Java. For more information, visit http://jackson.codehaus.org/. 

      Each class must provide several pieces of information in order to be serialized and de-serialized into JSON/Java. This information is provided via the JsonTypeInfo annotation, which takes several properties:

      • use = The kind of type metadata that is to be used for serializing and de-serializing type information for instances of annotated type
      • include = The mechanism that is used for including type metadata
      • property = The property name that is used to identify the type 

      The JsonTypeInfo annotation can be found in the org.codehaus.jackson.annotate package.

        In a command prompt copy the original sources from src/ into the work/ directory.
        cp src/* work/

        Open the Person.java in your favorite text editor. Note that the Person class is a simple JavaBean style serializable object.

        gedit Person.java

        Modify the class to include the required JSON imports.

        The required imports include:

        import org.codehaus.jackson.annotate.*;
        import org.codehaus.jackson.annotate.JsonTypeInfo.*        ;

        Annotate the class to expose its fields as properties and the classes type information using the @type property.
        @JsonTypeInfo(use=JsonTypeInfo.Id.CLASS, 
                      include=JsonTypeInfo.As.PROPERTY,
                      property="@type")

        Save your changes.

      Modify the Address Class to Support JSON and Define Explicit Field Names

        Open Address.java in your favorite text editor and add the required import and annotation statements.

        gedit Address.java

        Imports:
        import org.codehaus.jackson.annotate.*;

        Annotation:
        @JsonTypeInfo(use=JsonTypeInfo.Id.CLASS,include=JsonTypeInfo.As.PROPERTY,property="@type")

        Modify the street, city, and country fields to define explicit names using the JsonProperty annotation, which is a field-level annotation.
        Annotation:
        @JsonProperty("name to be applied to field")

      Compile the Updated Source Code

        Open a command prompt and change directory to /opt/REST.OBE.

        For convenience, a command script has been provided that compiles the sources.
        Open build.sh and examine its contents.
        Note that the script assumes Coherence 3.7.1 was installed in /opt/coherence. 
        If required, change the location of installation of Coherence. 
        Also note that the script uses the coherence-rest.jar file to resolve the imports for the Jackson libraries.

        Save any changes and exit the editor.
        Compile the classes using a command that is similar to:

        ./build.sh

        Correct any compilation errors as required.

Modify Configuration to Support Exposing REST-Aware Objects

    Modifying classes to support REST is only a portion of the work required to expose cache objects via REST. Coherence requires two configuration changes to expose newly REST-aware objects.

    • REST configuration: To map HTTP requests to Coherence objects, Coherence must know which cache the objects are in, the class name of the value object, and a key class that can be used to convert complex multi-part keys in their appropriate representation. The REST configuration includes the resource elements, which define each of these components.

      Note that the default file name for a REST configuration file is coherence-rest-config.xml. This default value can be overridden using the system variable tangosol.coherence.rest.config.

    • Proxy configuration: A proxy server is required to intercept and forward HTTP requests to the correct cache for service. Proxy-scheme elements are used to define a proxy, which can accept and respond to HTTP operations.

    Complete coherence-rest-config.xml

      In a command prompt window, create the config directory and copy the baseline configuration files into it.

      cp src/config/* config/


      In a text editor, open the config/rest-configuration.xml file that has been partially completed for you. Note that you will be adding resource elements that resemble:
      <resource>
          <cache-name>somecache</cache-name>
          <key-class>fully qualified path to key class</key-class>
          <value-class>fully qualified path to value class</value-class>
      </resource>

      Such elements are inserted in a resources element for each value object to be supported.

      REST-aware objects are exposed using <resource> elements inside the resources element within a rest configuration file.

      Find the comment (<!--...-->) and replace it with an element that resembles:

      <cache-name>dist-http-example</cache-name>
          <key-class>java.lang.String</key-class>
          <value-class>example.Person</value-class>
      </resource>

      The updated configuration file should resemble that shown below. Note that the new element specifies a cache name of dist-http-example, which exists in the cache configuration.

      Save the updated file. 


    Complete rest-cache-configuration.xml

      Ensure that you are in the root of the REST OBE source and open the config/rest-cache-configuration.xml file that has been partially completed for you.

      REST-aware objects are served using a proxy server. Proxy servers are defined using a proxy-scheme element in a cache configuration file.
       
      Find the proxy scheme comment and beneath it specify a proxy-scheme element. The proxy-scheme element should be defined with a service name of ExtendHttpProxyService, a thread count of 5 and should also be marked autostart false.
      The new element should resemble:
       <proxy-scheme>
         <service-name>ExtendHttpProxyService</service-name>
         <thread-count>5</thread-count>
         <autostart system-property="proxy.enabled">false</autostart>
      </proxy-scheme>

      Note that the auto-start element includes a system-property attribute. This property allows you to enable or disable proxy settings using a comment line element that is similar to:

      -Dproxy.enabled=true

      The updated cache configuration should resemble that shown below. Note the new proxy-scheme element and the system property.

      The proxy service must define an acceptor that describes how it accepts incoming requests. Multiple acceptor types exist. However, REST support requires that the proxy accept HTTP connections. Add an acceptor-config that defines an http-acceptor as shown below. Note that the acceptor should be defined between the thread-count element and the autostart element. Note that the address and port elements have system properties defined for them such that they may also be overridden. Port 8080 is the typical default port, but is overridden normally at the command line.

      The new element should resemble:
      <acceptor-config>
          <http-acceptor>
              <local-address>
                  <address system-property="proxy.adddress">localhost</address>
                  <port system-property="proxy.port">8080</port>
              </local-address>
          </http-acceptor>
      </acceptor-config      >

      The completed cache configuration should resemble: 

      Save the updated file. 


Start Servers to Support REST

    Coherence REST Servers are dependent on a number of libraries, most of which are provided with Coherence itself. The start-server.sh and start-proxy.sh scripts require specific libraries to run. For convenience the prepare-lib.sh script has been provided, which creates and populates a directory with all the required libraries.

    These libraries include:

    Library
    		 Description
    		 Comprised of:
    Jersey 1.8 The reference implementation of JAX-RS
    (JSR 311, the Java API for RESTful web services)
    • jerrsy-core1.8.jar
    • jersey-json-1.8.jar
    • jersey-server-1.8.jar
    • jersey-grizzly2-1.8.jar,
      which must be downloaded from http://grizzly.java.net
    Jackson 1.1.1, JSON Serialization support
    • jackson-all-1.8.1.jar

    If you need to download the Jersey Grizzly jar, which provides an HTTP server that is capable of serving REST, you can find it in the Container section of the downloads, under the Grizzly HTTP web server as shown below:
     

    Ensure that you are in the root of the REST OBE source and open the bin/prepare-lib.sh script in your chosen test editor.

    Confirm that the COHERENCE_HOME and OBE_HOME variables are correct. If Coherence or the OBE sources were installed in another directory, correct these variables.

    Save any changes and exit the editor.

    Execute the script using a command that is similar to:

    ./bin/prepare-lib.sh

    The script will execute, copying files to the the /lib directory and echoing its actions as it runs, and should produce output that is similar to:

    Open the provided server-start script bin/start-server.sh in your favorite editor.

    Note the following and correct any values that do not match your environment:

    1. CLASSPATH

    • The lib directory holds all the required REST libraries. Note that the coherence-rest jar references other library jar files via its manifest.
    • The classes directory holds the compiled versions of the classes from the earlier build steps.
    • The config directory holds the updated rest and cache configuration.

    2. Cache and REST configuration
    • tangosol.coherence.cacheconfig=rest-cache-configuration.xml
    • tangosol.coherence.rest.config=rest-configuration.xml

    Open the provided server-proxy script bin/start-proxy.sh in your favorite editor.
    Confirm that the classpath and file names match your environment.

    Note that the proxy contains two additional properties, which define the port and enable the starting of the backing proxy service. This OBE uses port 9000, if you need to change the port to something else, note that the various helper scripts in the bin directory will also need to be adjusted to reference the correct port.

    Start the server using a comment that is similar to:

    ./bin/start-server.sh > server.log 2>&1 &

    This starts the server in the background, piping all output (both normal and error) to server.log.

    To track the start of the server, consider using a command that is similar to:

    tail -f server.log

    When the server shows started, continue to the next step.

    Start the proxy server using a comment that is similar to:

    .bin/start-proxy.sh > proxy.log 2>&1 &

    This starts the server in the background, piping all output (both normal and error) to server.log

    To track the start of the proxy server, consider using a command that is similar to:

    tail -f proxy.log

    When the proxy server shows started, continue to the next step.

Testing REST Using cURL

    cURL is a client application that can be used to get or send documents/files to a server, using any of the supported protocols (HTTP, HTTPS, FTP, and others). The command is designed to work without user interaction or any kind of interactivity, and can be used to script server interaction. 

    The cURL command is native to OSX and Linux, and can be downloaded for use with Windows. Windows users can find a version of cURL here.

    cURL uses a variety of commands to interact with a Coherence*Extend proxy server, which include:

    • PUT - Update the existing resource defined by the URL. If the resource does not exist, create it.
    • POST - Create a new resource that is subordinate to the result URL.
    • GET - Return the resource associated with the URL. Note that partial GETs are possible. In addition, GET can be used to execute Coherence functions and aggregators such as count, sum, and others.
    • DELETE - Remove the resource associated with the URL.


    Determine the URL of the Entity Being Accessed.

    Using the hostname, port, and cache name specified in the configuration and startup scripts, determine the root URL for all cURL requests:

    • localhost - Everything is running locally.
    • 9000 - The start-proxy.sh script specifies a port of 9000.
    • dist-http-example - This gives the name of the cache being used from config/rest-cache-configuration.xml.

    The complete partial URL should resemble http://localhost:9000/dist-http-example. The remainder of the URL depends on the operation. 

    Add an Entry to the Cache.

    Adding elements to the cache is done via a PUT, which adds or updates an existing element that is subservient to the URL provided.
    The general form of the cURL command using the PUT operation is:

     curl -i -H "Content-type:application/type" -X PUT -d data http://hostname[:port]/cache-name/keyspec

    Important parts of the command are:
         type - The type of input, either JSON or XML
         data - A data string that represents the contents of the object
         keyspec - The resource identifier for the object being added 

    For example, to add an example.Person object with key 3, Name Rebecca, and age 32, enter a cURL command that is similar to:

     curl -i -H Content-type:application/json -X PUT -d '{"@type":"example.Person",""name":"Rebecca","age":"32"}'
              http://localhost:9000/dist-http-example/3

    Note that this command must be entered on a single line. The following data was used as examples in this OBE:

    Key
    		 Name
    		 Age
    1
    		 Rusty
    		 52
    2
    		 Al
    		 49
    3
    		 Rebecca
    		 32
    4
    		 Bill
    		 39

    For your convenience, the bin/put.sh script can be used to add objects. The script takes three arguments: the key, name, and age, and adds the object, as well as echoes the cURL command used. For example:

    ./bin/put.sh 3 Rebecca 32

    This produces output that is similar to:

    Return Entries from a Cache.

    Getting elements from the cache is done via a GET, which returns one or more elements from the cache.

    The general form of the cURL command using the GET operation is:

     curl -i -H "Content-type:application/type" -X GET http://hostname[:port]/cache-name/keyspec{.type}

    Important parts of the command are:
        type - The type of input, either JSON or XML
        keyspec - The resource identifier for the object being returned

    For your convenience, the bin/get.sh script can be used to return single objects. This script takes two arguments: the identifier of the object to be returned and a type (xml or json), for example:

     ./bin/get.sh 3 json
    This produces output that is similar to:


    Return Multiple Entries from a Cache.
    REST and Coherence support returning multiple elements at the same time via a form of key specification that is similar to {key[,key]}. For example, {1,2} would return the element associated with these keys.

    To return the first and third element, enter a key specification that is similar to:

     curl -i -X GET http://localhost:9000/dist-http-example/\{1,3\}

    Experiment with the GET command to return other combinations of keys used in previous PUT operations.

    Return Partial Entries from a Cache.
    REST and Coherence support returning parts of objects, as well as selections of objects, by extending the URL using a list of field names. Field names are appended to the URL using a ; followed by some number of comma separated fields. For example:

     curl -i -X GET http://hostname[:port]/cache-name/{1,3};A,B,C

    Return the fields A, B, and C for keys 1 and 3.

    To return the Name and Age fields for keys 2 and 4, enter a command that is similar to:

     curl -i -X GET http://localhost:9000/dist-http-example/{2,4};name,age

    The person object has four fields: Name, Age, Sex, and IsValidPerson. Explore the GET command using these fields.

    Return Data using Aggregators.
    Coherence supports executing aggregators via a combination of GET operations and a key specification, which specifies an aggregator function rather than a key specification. Depending on the data, any number of aggregators can be executed at the command line.

    The general format for using aggregators is:
     curl -i -X GET http://hostname[:port]/cache/[optional key spec][/aggregators([arguments])

    For example, the count aggregator can be used to return a count of all objects in the cache. To use count, execute the following (note that slashes are used to "escape" the "(" and ")" characters, are required by BASH, and are not part of the command):

     curl -i -X GET http://localhost:9000/dist-http-example/count\(\)

    There are a number of out-of-the-box aggregators, including the following:
    Function
    		 Description
    max, long-max
    		 Return the maximum value of an integer or long field.
    min, long-min
    		 Return the minimum value of an integer or long field.
    sum,long-sum, double-sum
    		 Return the integer, long, or double sum of a field.
    count
    		 Return the total number of elements in a cache.

    Experiment with the various methods. Note that age is a long and must be used with the long functions.

    To return the sum of all ages in the cache, execute:
     curl -i -X GET http://localhost:9000/dist-http-example/long-sum\(age\)

    Remove Entries from a Cache.
    Objects are deleted from the cache using a notation that is similar to GET but by specifying the DELETE operation. Note that single and multiple deletes are supported.

    The general format for performing deletes is:
     curl -i -X DELETE http://hostname[:port]/cache/multi-or-single-keyspec

    For example, to delete elements 1 and 3, enter a command that is similar to:
     curl -i -X DELETE http://localhost:9000/dist-http-example/{1,3}

    To delete element 5, enter a command that is similar to:

     curl -i -X DELETE http://localhost:9000/dist-http-example/5

    Explore the DELETE command. Note that you can always use the provided bin/out.sh script to re-add an element.

Summary


    In this tutorial, you should have learned how to:

    • Modify classes to support JAXB or JSON
    • Create a REST configuration in support of exposing REST-aware classes
    • Configure Coherence servers to support REST
    • Start rest servers and proxies
    • Test Coherence REST using cURL

    Resources

    • The Oracle Coherence 3.7.1 documentation, which is found here
    • Various Oracle Coherence Development and Administration classes and advance training that can be found here
    • To learn more about Oracle Coherence, additional OBEs in the Oracle Learning Library

    Credits

    • Lead Curriculum Developer: Al Saganich
    • Other Contributors: Jason Howes, Tim Middleton, Noah Arlis, and others

To help navigate this Oracle by Example, note the following:

Hiding Header Buttons:
Click the Title to hide the buttons in the header. To show the buttons again, simply click the Title again.
Topic List Button:
A list of all the topics. Click one of the topics to navigate to that section.
Expand/Collapse All Topics:
To show/hide all the detail for all the sections. By default, all topics are collapsed
Show/Hide All Images:
To show/hide all the screenshots. By default, all images are displayed.
Print:
To print the content. The content currently displayed or hidden will be printed.

To navigate to a particular section in this tutorial, select the topic from the list.