March 2010

 

Oracle Enterprise Manager System Monitoring Plug-in
for Sun Storage 7000 Unified Storage Systems




 

Introduction

This document provides a description of the Sun Storage 7000 Plug-in for Oracle Enterprise Manager Grid Control and high-level instructions for use and various common questions and operations with the plug-in.

Description

The Sun Storage 7000 Plug-in for Oracle Enterprise Manager Grid Control provides first-class monitoring of the Sun Storage 7000 Appliance Family with the ability to:

  • Monitor Sun Storage 7000 Appliances (including all available appliance models, 7110, 7210, 7310, 7410)

  • Gather storage system information, configuration information and performance information of accessible storage components

  • Raise alerts and violations based on thresholds and monitoring information collected by the tool

  • Provide out-of-the-box reports that compliment Sun Storage 7000 Analytics. For details of reports, see Table 1 - Reports.

  • Support monitoring by remote agents.

Once an appliance is configured to be monitored by Oracle Enterprise Manager Grid Control, Sun Storage 7000 Appliances contain analytics worksheets and datasets that bridge the administrator's view to the deeper level of detail provided by the Real-time Analytics available within the Appliance.

Required Versions

Deployment of the Sun Storage 7000 Appliance Enterprise Manager Grid Control Plug-in requires the following versions and pre-requisites, these should already be installed properly according to the respective documentation available in each product:

  • Oracle Enterprise Manager Grid Control 10g Release 2 and above

  • Sun Storage 7000 Software version 2010.Q1 and above

 

There are additional sub-requirements for proper operation of the plug-in, such as required Java version, but as long as the above are met, the sub-requirements will be properly met.

Pre-Requisites for Monitoring Sun Storage 7000 Appliances

Prior to monitoring a Sun Storage 7000 Appliance with the appropriate software revision, the storage administrator must run a workflow on the appliance with several parameters that properly configure the system for monitoring.

During this process, several steps occur:

  • A user with a restricted role is created for use by the monitoring agent: oracle_agent

  • One or more analytics worksheets are created to ensure smooth communication between the Oracle Enterprise Manager Grid Control administrator and the Storage Administrator

For each system, enter the Sun Storage 7000 Appliance Browser User Interface (BUI) as "root"or another user with the proper role to create worksheets, create users and create roles on the appliance. Once in the BUI:

  • Go to the Maintenance / Workflows tab

  • Run the "Configure for Oracle Enterprise Manager Monitoring"workflow, entering and recording the following information (information with an * is required by the Oracle Enterprise Manager Grid Control administrator):

    • Password for user oracle_agent*

Communicate the information above denoted with an "*"along with:

  • The IP Address for the network interface that you want Oracle Enterprise Manager to use to access management information (this interface must allow administration)

  • The full DNS name that is used to locate the system and corresponds to the IP Address

  • The port that is used by the BUI (typically 215 unless your company had a requirement to change management ports)

In the event Oracle Enterprise Manager Grid Control will monitor resources available on a clustered Sun Storage 7000 Appliance, only one cluster head must have the configuration step completed, the user creation and worksheet will automatically be created on the peer. In the event that the head containing the monitored resources fails, Oracle Enterprise Manager Grid Control monitoring will "automatically"start monitoring resources on the head that takes over the exported resources.

For reports to work that include Database to Appliance Mappings, the Database and the Host that the Database resides on must be monitored by Oracle Enterprise Manager Grid Control. Please refer to your documentation that came with Oracle Enterprise Manager Grid Control or to the available online documentation for instructions on monitoring host storage and Database instances.

Deploying the Plug-in

From the sun.com/download web site, Accept the Terms and Conditions, download compressed file and unzip into an appropriate directory within your local system. You should see license files, a README, and the sun_storage_7000.jar file that contains the management plug-in.

Log into Grid Control as a Super Administrator (typically SYSMAN) and complete the following steps::

  • Click the "Setup"link (in the navigation footer)

  • Find the "Management Plug-ins"link in the navigation bar and click it

  • From the Management Plug-ins Page, locate and click the "Import"button

  • From the Import Management Plug-ins page, locate the Browse button and find the sun_storage_7000.jar file unpacked from the downloaded .zip file and select it

  • Choose to "List Archive", you will see the sun_storage_7000 listed, select it and finish the Import (if multiple versions of the plug-in are listed, import the latest version unless there is a compelling reason in this documentation or your environment to use a previous version of the plug-in)

  • When returned to the Management Plug-Ins page, you should see the sun_storage_7000 agent listed

  • Deploy the plug-in to one or more Oracle Enterprise Manager Grid Control Agents

    • Choose "Deploy"icon next to the Plug-in

    • Choose "Add Agents"

    • Locate the Agent System in the Search Window and Choose Select

    • Choose "Next"and finish the deployment

Once the above steps are complete, it takes some amount of time before all artifacts relating to the Sun Storage 7000 Appliance are available. For example, SYSTEM Reports tend to take 5-10 minutes to appear under the Reports tab. The Sun Storage 7000 Appliance should be available immediately to set as a target from the agent view.

Adding Instances for Monitoring

Once the Management Plug-in is deployed and the target Sun Storage 7000 Appliance (or appliances) is properly configured, a Sun Storage 7000 Appliance can be monitored by configuring an instance within an Oracle Enterprise Manager Grid Control agent:

  • Choose "Setup"from the header or footer

  • In the menu bar, locate the "Agents"link

  • Choose an agent that you deployed

  • From the "Add"drop-down Combo Box, Choose "Sun Storage 7000 Appliance"and press "Go"

  • Enter the appropriate information as requested (Instance Variables)

    • Host Name or IP Address: <IP Address or DNS Accessible host name of target system's administration interface supplied by the Storage Administrator>

    • User ID: oracle_agent (default, only change if instructed by the Storage Administrator)

    • User Password: <password supplied by Storage Administrator>

  • The system name should appear in the Monitored Targets List

Clustered systems require some consideration by the Oracle Enterprise Manager Grid Control administrator. Unless Grid Control is being used as a general purpose Configuration Management Database (CMDB) or general purpose asset and resources tracking, only the head or heads containing active exported resources in use by other monitored software resources should be monitored by Oracle Enterprise Manager Grid Control.

In the event that a storage head fails and an alternate head takes over the resources, the new head owning the logical resources will become monitored (if the takeover head was properly configured). Within the metric reports, the status of the cluster is given. The cluster status is highly relevant as shared resources in an active / active environment will become even more pressured and the actual event of a cluster failover can cause disruption in quality of service objectives.

Reports and Information Available

There are two primary avenues for gathering information about monitored instances of a Sun Storage 7000 Appliance. To see the raw information collected from a Sun Storage 7000, go to the system's home page, locate the link at the bottom of the page for "All Metrics"and view the categories and information collected.

As a rule, there is more "point in time"information displayed within the raw information than is available in the reports. The use of All Metrics does not show the changing of values over time, nor collected information that is related and intended to be viewed in a single view, as Reports do. It is possible to configure alerts and threshold localized to Oracle Enterprise Manager Grid Control within the All Metrics screens.

In addition to the All Metrics page containing raw data from the last collection interval, the following reports are available:

 

Table 1: Reports

Report

Description

Purpose

Monitored System Home Page (10.2.0.5 or beyond only)

Contains a series of graphs that give a very high level view of the system throughput and the clients accessing the system. It is important to note the semantics of a system being "Up"or "Down". A system is "Up"if the management interface is available and simple properties can be queried that do not require login. A system is "Down"if the management interface is not available. If a system is setup with more than one interface that allows access to the data path, a system could conceivably be "Down"when the data path interface is still accessible. Regardless of this situation, the management interface being flagged as Down is sufficient to warrant attention to the system.

Give at-a-glance status of the system.

Database to Appliance Mapping Report

Mapping of Databases using NFS to access data residing on a Sun Storage 7000 Appliance. Use Detail Report Links to view deeper information on mappings and component activity. Detail Reports Provide:
  • Detailed database file mapping to exports residing on appliances
  • Information about export that files reside on (including capacity consumption and cache activity)
  • Information about the appliance that a share resides on
  • Information about the network interface and it's activity that a database file is accessed through.

This report is available from the Reports Tab or from a pre-selected Target Report's Tab.

Provides a detailed view of how a Sun Storage 7000 Appliance is being accessed for information via Oracle Databases. Provides easy navigation to drill into the configuration and activity of Sun Storage 7000 Appliance Components.

Storage Capacity Report

Sun Storage 7000 Appliance Capacity Usage Report gives detailed information about how storage is used on an individual Sun Storage 7000 Appliance.

This report is available from the Reports Tab or from a pre-selected Target Report's Tab.

 

Network Interface Information Report

Provide details of network configuration and generalities about network activity over the past 24 hours.

This report is available from the Reports Tab or from a pre-selected Target Report's Tab.

Provides a detailed view of how ports on a Sun Storage 7000 Appliance are configured and the aggregate activity of the interfaces.

Federated Database to Appliance Mapping Report

Mapping of Databases using NFS to access data residing on all Sun Storage 7000 Appliances managed by Grid Control. Use Detail Report Links to view deeper information on mappings and component activity. Detail Reports include:
  • Detailed database file mapping to exports residing on appliances
  • Information about export that files reside on (including capacity consumption and cache activity)
  • Information about the appliance that a share resides on
  • Information about the network interface and it's activity that a database file is accessed through.

This report is available in the Reports Tab only. Locate it underneath the "Storage"→ "Network Attached Storage"group.

Provides a detailed view of how a Sun Storage 7000 Appliance is being accessed for information via Oracle Databases. Provides easy navigation to drill into the configuration and activity of Sun Storage 7000 Appliance Components.

Federated Capacity and Configuration Report

Sun Storage 7000 Appliance Configuration Report, gives high level information about all monitored Sun Storage 7000 Appliances or capacity and configuration decisions.

This report is available in the Reports Tab only. Locate it underneath the "Storage"→ "Network Attached Storage"group.

Provides a side by side comparison of all Sun Storage 7000 Appliances to make decisions on where to allocate capacity and what software is installed on the appliances.

Reports accessed via a target instance differ when they are accessed via the Reports tab within Oracle Enterprise Manager Grid Control. All instance reports are available under the Reports tab, Storage category and Network Attached Storage sub-category. When accessed with this route, Reports have configurable time spans for graph display where these are applicable.

 

All system reports can be cloned and customized by administrators.

Several metrics can have custom thresholds set within Oracle Enterprise Manager Grid Control. The alerts received are contained within the product and are not set as Alerts and Thresholds on the Sun Storage 7000 Appliance itself. The table below shows the metrics that have thresholds set with their default values:

Table 2: Metrics

Metric

WARNING

CRITICAL

Purpose

Status

N/A

EQ SUN7000.CONNECT.EXCEPTION

Provides information to the home page on availability of the system's management path.

Physical Resource Status / Cluster State

EQ AKCS_OWNER

EQ AKCS_STRIPPED

State of the cluster. OWNER would imply this head is the owner of the cluster and that resources are likely in contention. STRIPPED means that the head's resources are owned by it's peer and needs to have the takeover re-done.

Logical Status / Share Percentage Utilized

80.00%

90.00%

Any share issues these warnings. A Grid Control administrator can make specific shares trigger on other values.

Logical Capacity Totals / Pool Total Used Capacity

80.00%

90.00%

Issued when the total capacity available on this systems crosses the threshold.

CPU Utilization

80.00%

90.00%

Issued after 3 separate polling intervals see the CPU Utilization above the percentage. Overuse of CPU can result in lower performance.

Network Devices / Up

N/A

N/A

Because a device being down does not necessarily indicate a problem (since the device may not be configured or operating by plan), this threshold is enabled but left off.

Active Problem / Problem Type

M* (matches Minor or Major problems)

Crit* (matches Critical problems)

Problems on a system that require attention are categories as Minor, Major and Critical faults. These can be used to ensure systems receive attention from the database administrator as well as the storage administrator. The Storage Administrator can log into the Sun Storage 7000 Appliance Console and see the problem and more detailed information.

For the most part, alerts and thresholds for the appliance should be managed by the Storage Administrator, they have many more statistics available to them within the Sun Storage 7000 Appliance User Interface. Further, the Storage Administrator will have real-time indications of performance issues as opposed to the slower polling interval required by Oracle Enterprise Manager Grid Control.

Metrics, Reports, and information presented on charts can take up to 60 minutes to populate as the polling intervals for relatively static information is long. Be patient for the complete reports to populate. Further, do to differing polling cycles between statistics, some information may populate prior to other information.

Un-Deploying the Plug-in

Follow instructions provided within the Oracle Enterprise Manager documentation to un-deploy a plug-in. At a high level, the Oracle Enterprise Manager Grid Control administrator must:

  • Ensure all Sun Storage 7000 instances being monitored are removed from the agent monitoring configuration

  • Un-deploy the Plug-in from all agents

  • Remove the Plug-in

 

Once completed, communicate with the Sun Storage 7000 Storage Administrator(s) and have them run the "Unconfigure Oracle Enterprise Manager Monitoring"workflow.

Communicating with a Sun Storage 7000 Administrator

A Sun Storage 7000 Administrator has an advantage of real-time appliance analytics that, like Oracle Enterprise Manager Grid Control, can be viewed in various time-spans dating back to the start of the time when the system was configured to be monitored by Oracle Enterprise Manager Grid Control. Some data sets are available from the first power on of the appliance itself.

 

The most common use of the Oracle Enterprise Manager Grid Control plug-in (besides simple capacity monitoring and high-level information collection) will be the analysis of application performance degradation. Using the information within Oracle Enterprise Manager Grid Control, print or deliver the information to the Sun Storage 7000 Administrator that you believe is the leading indicator of the Sun Storage 7000 Appliance being a root cause or related to a performance degradation in an application.

 

Once that information is presented to the Storage Administrator, they can start the analysis process by entering the "Oracle Enterprise Manager"worksheet. This worksheet displays a "down to the second"view of the data that you presented to them, as opposed to the much coarser view of the data available to Oracle Enterprise Manager Grid Control (often a 10 or 20 minute collection cycle).

 

From the worksheet, or other data sets available to the Storage Administrator, they can root cause down to the hardware component or software component level if they believe there is an issue in the hardware. In the event of a pure resource contention issue, they have deep levels of client access information available to them with how and when individual clients are accessing a system along with the resources they are accessing (network interfaces, logical resources such as shares and more).

 

Once initial communication is complete, the Storage Administrator should complete the analysis of the problem and return with a way to resolve a disruption in Quality of Service or with data from the system indicating that the problem is likely not associated with the Sun Storage 7000 Appliance.

 

There is a direct mapping of information displayed within Oracle Enterprise Manager Grid Control to datasets and information collected in the target Sun Storage 7000 Appliances. Storage Administrators should ensure that the following datasets on the Sun Storage 7000 Appliance remain enabled or notify the Oracle Enterprise Manager Grid Control administrator that they will start receiving errors when they are disabled:

Table 3: Metric Information

Metric Name

Column (* indicates a key field, ! Indicates Conditions Enabled)

Sun Storage 7000 Dataset

Polling Interval (Minutes)

Response

Status

N/A (request for hostname)

5

Logical Status

Share Identifier*

System Attribute from Workflow

20


Available (GB)


Total Allocated (GB)


Data (GB)


Share Percentage Used (%)!


Quota (GB)


Reserved (GB)


Unused Reserved (GB)


Appliance Capacity Totals

Storage Pool*

System Attribute from Workflow

20


Pool Owner


Pool Status


Data Profile


Available (GB)


Total Used (GB)


Pool Used Percentage (%)!


Total Capacity (GB)


Simple Metrics

Primary Cache Size (MB)

arc.size

20


Secondary Cache Size (MB)

arc.l2_size


CPU Utilization %!

cpu.utilization


I/O Operations per Second

io.ops


I/O Bytes per Second

io.bytes


Network Interface KB per Second

nic.kilobytes


NFS V2 Operations per Second

nfs2.ops


NFS V3 Operations per Second

nfs3.ops


NFS V4 Operations per Second

nfs4.ops


CIFS Operations per Second

smb.ops


ISCSI Operations per Second

iscsi.ops


ARC Activity

Activity*

arc.accesses[hit/miss]

20


Accesses per Second

arc.accesses[hit/miss]


L2ARC Actvity

Activity*

arc.l2_accesses[hit/miss]

20


Accesses per Second

arc.l2_accesses[hit/miss]


ARC Activity by Share

Share*

arc.accesses[share]

20


Accesses Per Second

arc.accesses[share]


L2ARC Activity by Share

Share*

arc.l2_accesses[share]

20


Accesses Per Second

arc.l2_accesses[share]


NFS V3 Operations By Share

NFS V3 Share*

nfs3.ops[share]

20


Operations Per Second

nfs3.ops[share]


NFS V4 Operations By Share

NFS V4 Share*

nfs4.ops[share]

20


Operations Per Second

nfs4.ops[share]


Network I/O Direction

Direction*

nic.kilobytes[direction]

20


Kilobytes Per Second

nic.kilobytes[direction]


Network I/O by Device

Device*

nic.kilobytes[device]

20


Kilobytes Per Second

nic.kilobytes[device]


Network Interface Performance

Interface*

net.kilobytes[interface]

20


Kilobytes Per Second

net.kilobytes[interface]


Physical Component Status

Cluster State!

System Attributes from Workflow

20


Cluster State Description


Cluster Peer State


Cluster Peer State Description


Appliance Information

Reported Node Name

System Attributes from Workflow

60

Product Model

Appliance Serial Number

Chassis Serial Number

Appliance Software Version

Install Time

Last Update Time

Power On Time

Up Time


Clustered


Cluster Peer


Appliance Logical Information

Share Identifier*

System Attributes from Workflow

60


Share Type


Mount Point


Data Profile


Data Compression


Cache Usage


NFS


CIFS


IP Network Interfaces

Interface*

System Attributes from Workflow

30


IP Address*


CIDR


IP Version


Management Enabled


IP Network Interface to Datalink Relationship

Interface*

System Attributes from Workflow

30


Datalink*


Jumbo Frames


IP Datalink to Device Relationship

Datalink*

System Attributes from Workflow

30


Link*


Class


IP Network Devices

Network Device*

System Attributes from Workflow

30


Device Speed


Device Up!


Active Problem List

Problem ID*

System Attributes from Workflow

20

Description

Problem Type!

Message ID

 

From the above list, information that is collected via "System Attributes from Workflow"indicate items that cannot be "enabled / disabled"by an administrator, they are simply collected via scripts that are called on each Sun Storage 7000 Appliance.

Debugging Monitoring Deployments

There are a variety of reasons that a monitored instance in the Oracle Enterprise Manager Grid Control may not be returning information properly. These issues are typically associated with mis-configuration, incorrect permissions or changing configuration within one of the products.

Table 4: Symptoms and Resolutions

Symptom

Resolution

Appliance does not appear to be "Up"after being added as an instance to be monitored

If the appliance previously reported "Up"but is now indicating it is "Down", this is an indication that the Management Path is no longer available on the target Sun Storage 7000 Appliance. The data path may still be available on the appliance but this critical problem must be addressed.f this the first time an appliance was added and it has never reported as being "Up", a variety of mis-communication problems between the Storage Administrator and the administrator adding the instance to be monitored may have taken place, among them, check:
  • That the designated DNS name or IP Address is correct in the asset's Monitoring Configuration
  • That the port number is correct (Sun Storage 7000 Appliances default to port 215)
  • That the named asset is accessible from the agent system
Note that the Up / Down indicator does not require authentication to the target Sun Storage 7000 Appliance and, as such, the User ID and Password are not relevant here.

Checking the "value"of the Response Metric should show SUN7000.CONNECT.EXCEPTION. This indicates that the Agent properly invoked the metric collection program but the metric collection program could not connect to the target Sun Storage 7000 Appliance given the parameters in the Monitoring Configuration.


Appliance is "up"but all metrics are not being collected

Check the metric collection errors. The most common problem is misconfiguration of the password used for the oracle_agent user within Oracle Enterprise Manager Grid Control.

For most metric collection errors, a summary message and a Java "Stack Trace"is generated and viewable by clicking on the "Metric Collection Errors"link underneath the home page reports, and clicking through the text "Exception in thread "main...". The initial summary message should contain enough information about a problem.


Appliance is "up"but some metrics are not being collected.

Determine what metrics are not being collected. Most likely, it is a group of metrics like:
  • ARC Activity by Share
  • L2ARC Activity
  • L2ARC Activity by Share
  • NFS V3 Operations by Share
  • NFS V4 Operations by Share
There are usually three reasons for selective metrics not being collected:
  • A component does not exist on a system "This is common with the L2ARC metrics. In this case, you will not receive a metrics collection error but instead, the "Last Upload"time in the "All Metrics"page will remain empty (or include a ""-). This is not a problem and is common with the Sun Storage 7110.
  • The "Configure for Oracle Enterprise Manager Monitoring"workflow was not run on the target Sun Storage 7000 Appliance "this problem embodies in metrics that are not collected by default on a target appliance, such as those listed above. In this case, Metric Collection Errors will occur within the Oracle Enterprise Manager Grid Control until the target Sun Storage 7000 Appliance is correctly configured.
  • A Storage Administrator disabled or removed datasets on the target system. Request that they re-enable the datasets by communicating the dataset name that is in error by using the table shown in the Section "Communicating with a Sun Storage 7000 Administrator".


The list of shares in the Appliance Logical Configuration and the Appliance Logical Status appears to be out of sync.

The layout of file systems and their values are collected at different intervals than the capacity information for them. In the worst case, a metric could be as much as 60 minutes out of sync with a storage appliance. If you have a need for real-time information, please consult the Sun Storage 7000 Appliance Browser User Interface.


A Report or Metric Detail is not showing a change made on the system.

Metrics and information about system values is collected on varying intervals. In the worst case, a metric could be as much as 60 minutes out of sync with a storage appliance. If you have a need for real-time information, please consult the Sun Storage 7000 Appliance Browser User Interface.


Unexpected Workflow Errors in running the Configuration Workflow on the Sun Storage 7000 Appliance such as "aksh: uncaught error exception (code EAKSH_GENERIC_DATASET_INVALID)"

Be sure you are running the configuration script as a user with "root"access or equivalent with respect to creating and manipulating datasets and users.


Database mapping reports do not show database artifacts that are mapped to NFS v3 or v4 shares on a Sun Storage 7000 Appliance.

Most likely, either the host, the database or both are not being monitored by Grid Control. Install the proper agents on the host to ensure storage is being monitored that is attached to the system.

Please refer to the Oracle Enterprise Manager Grid Control documentation for information on monitoring host and database assets.


Database mapping reports do not show database artifacts that are mapped to iSCSI or Fibre Channel shares on a Sun Storage 7000 Appliance.

iSCSI and Fibre Channel mappings are not supported in this release.


No graphs shown on Homepage of a Target Appliance.

You must be using Oracle Enterprise Manager Grid Control Version 10.2.0.5 or above.

If the version of Grid Control does support Homepage Reports, Check the "Version"of the Plug-in that is installed. Most likely, Version 1.00 is installed. Version 1.00 is compatible with back-level Oracle Enterprise Manager 10g Grid Control. To attain compatibility, the Homepage Report and Charts are removed. Upgrade the Plug-in to Version 1.01.


 

There are also a set of common configuration problems when the "Metric Collection Error". In most cases, Metric Collection Errors contain two parts, a short string followed by a longer Java Stack Trace. Do not concern yourself with the Java Stack Trace, this will be used by Sun Services if there is a more complex problem. The first string should contain enough information to isolate the problem. Common Metric Collection problems are as follows:

Table 5: Metric Collection Errors

Metric Collection Errors Symptom

Problem / Resolution

"login denied"

Problem:The User ID / Password combination on the target Sun Storage 7000 Appliance is incorrect

Causes:

  • The Monitoring Configuration in Oracle Enterprise Manager Grid Control is using the incorrect User ID / Password based on the Storage Administrator's instructions. The User ID should be oracle_agent and the Password should have been supplied by the storage administrator
  • The target Sun Storage 7000 Appliance does not have the proper User Ids and Role.
Solution:

Fix either the monitoring configuration to use the proper User ID / Password as instructed by the Storage Administrator, or (re)configure the target Sun Storage 7000 Appliance using the proper oracle_agent password.

"workflow parameters are invalid"

Problem:The parameters passed to the Sun Storage 7000 Appliance have become invalid.

Causes:

  • A dataset on the Sun Storage 7000 Appliance is not enabled
  • An internal error occurred and the agent is calling an incompatible workflow
Solution:
  • Run the "Unconfiguration"workflow on the target Sun Storage 7000 Appliance and remove the worksheet
  • Run the "Configuration"workflow on the target Sun Storage 7000 Appliance and Recreate the Worksheet
  • Have the Storage Administrator ensure that all of the datasets in the created worksheet are being collected properly

"cannot find specified workflow"

Problem:A workflow on the Sun Storage 7000 Appliance that collects and returns metrics to the agent cannot be found.

Causes:

  • The oracle_agent user (or user that you configured in the Monitoring Configuration) does not have workflow.*.* read authorization
  • A Sun Storage 7000 Appliance is being used that does not have the workflow on it, either because the version of the software on the Sun Storage 7000 Appliance is incorrect or the workflow was removed by a Storage Administrator
Solution:
  • Run the "Unconfiguration"workflow on the target Sun Storage 7000 Appliance and remove the oracle_agent user
  • Run the "Configuration"workflow on the target Sun Storage 7000 Appliance and Recreate the oracle_agent user
  • Have the Storage Administrator ensure the Configure / Unconfigure workflows are on the system
  • Contact Sun Service if the above fail to start the agent collecting the dataset again

"workflow failed"

Problem:The agent is accessing a Sun Storage 7000 Appliance, but the workflow (piece of logic that collects the metric information) is failing in some way.

Causes:

  • A problem in the metric collection logic is causing it to fail on the Sun Storage 7000 Appliance
Solution: Run the Configuration workflow on the Sun Storage 7000 Appliance. Choose to recreate the worksheet (recreating the user should be unnecessary but can be done).

If you are still receiving this error, contact Sun Services.


 

For detailed debug information, use the audit log on the target Sun Storage 7000 Appliance to determine actions that occurred with the workflows. Creation and deletion of artifacts (or the failure to create or delete an artifact) are traced in the audit log.

 

Debugging why an agent is not properly collecting statistics from a Sun Storage 7000 Appliance can be more difficult and should only be done by service personnel.

 

Most problems (other than version mismatches) will occur during collection of metrics or collection response information. Look for the following logs on the agent host. The agent host may differ from the management station location. In the following file locations, %AGENT_LOCATION% indicates the home directory of the agent, typically similar to "/export/home/oracle/OracleHomes/agent10g", an * indicates there may be several files with additional extensions.

  • %AGENT_HOME%/sysman/emd/sun_storage_7000_pipe.log* - Contains errors that occurred below the Oracle Enterprise Manager Grid Control Framework during attempts to communicate with the Sun Storage 7000 Appliance. These files will typically contain fine-grained details on connection problems with a Sun Storage 7000 Appliance or failures that occur while retrieving information.

  • %AGENT_HOME%/sysman/log/emagent.trc* - These logs also contain connection exceptions as well as any information on using the data returned from the system to populate database tables

  • %AGENT_HOME%/sysman/log/* - Remaining logs will contain finer grained details on various elements in the Grid Control Framework, 90% of the issues can be diagnosed with the above.

 

When interacting with Sun Services or Developers within your own help desk, please include the sun_storage_7000_pipe.log* files and emagent.trc* files for their use.

Documentation

  1. Sun Storage Unified Storage System Administration Guide, http://wikis.sun.com/display/FishWorks/Documentation

  2. Management of Heterogeneous Data Centers Using Oracle Enterprise Manager Plug-ins, An Oracle White Paper, http://www.oracle.com/technology/products/oem/pdf/dc_mgmt.pdf