Troubleshooting Large Projects in the Oracle Solaris Studio IDE

by Alexander Simon

This article describes some tips and tricks to help you improve the performance of the Oracle Solaris Studio IDE.


Published June 2012

Troubleshooting IDE Performance
Check the IDE's Memory Consumption
   How to Increase Memory for the IDE
   How to Use 32-Bit Java to Reduce Memory Usage
Checking for File System Issues
   How to Resolve File System Issues
Checking the Project Size
   How to Reduce the Project Size
   How to Exclude Problem Files in Project Sources
Checking the User Watch Limit on Linux
   How to Increase the User Watch Limit
Remote Development Solutions
Memory Use by ZFS on Oracle Solaris 11 Desktop Systems
   How to Change the ZFS Cache Limit on Oracle Solaris 11 Desktop Systems
Reporting IDE Issues
See Also
About the Author

OTN is all about helping you become familiar enough with Oracle technologies to make an informed decision. Articles, software downloads, documentation, and more. Join up and get the technical resources you need to do your job.

The Oracle Solaris Studio IDE can be used to develop large projects. When you are working on large projects in the IDE, at times, you might notice the IDE becoming less responsive, slow at parsing the project, or freezing.

This article shows you some tactics for determining what might be causing the slowdown, and it provides some suggestions for actions you can take to improve the performance of the IDE.

Troubleshooting IDE Performance

Before you take actions to improve the performance, you should troubleshoot to look at some likely causes of the performance issue. Table 1 shows common problems and provides links to steps you can take to determine the causes and solve the problems.

Table 1. Troubleshooting Information
Possible Issue How to Check How to Solve
The IDE is slow because of a lack of memory. Check the IDE's Memory Consumption How to Increase Memory for the IDE

How to Use 32-Bit Java to Reduce Memory Usage
Project parsing is slow due to file system issues. Checking for File System Issues How to Resolve File System Issues
Parsing is slow because the project is too large. Checking the Project Size How to Reduce the Project Size
On Linux, the IDE freezes because the system has run out of user watches. Checking the User Watch Limit on Linux How to Increase the User Watch Limit
Performance is very slow when using the IDE remotely. You are using VNC or X-Window forwarding. Remote Development Solutions
On Oracle Solaris 11, the IDE is slow due to ZFS memory use. Memory Use by ZFS on Oracle Solaris 11 Desktop Systems How to Change the ZFS Cache Limit on Oracle Solaris 11 Desktop Systems

Checking the IDE's Memory Consumption

A common cause of slowness in the IDE is that the Java Virtual Machine (JVM) that runs the IDE is using enough memory to come close to its heap utilization limit. This procedure shows how to determine whether the IDE is consuming too much memory.

  1. In the IDE, check the Memory toolbar indicator. The indicator is a small graph that shows the current memory used and the allocated heap memory.
  2. If you do not see the indicator, display it by choosing View -> Toolbars -> Memory from the IDE's menu bar or by right-clicking the toolbar and choosing Memory.
  3. To determine whether the IDE has a lack of memory, look at the Memory indicator while the IDE parses the project when you open the project.

Figure 1 shows how the memory indicator might look when there is a lack of memory. The first number in the graph is the memory in use at the moment, and the second number is allocated memory. This is a dynamic graph and the value should change over time as memory is used and freed. In this picture, the value stays at about the same level, indicating that all the memory is allocated.

Picture of memory indicator in the IDE toolbar

Figure 1. Memory Indicator in the IDE Toolbar

Due to the JVM's continual garbage collection to reclaim unused memory, IDE performance is greatly reduced when all the memory is allocated. Even if your computer has four cores and the IDE parses project files in four threads, a lack of available memory might decrease parsing speed so that it takes three times longer than normal to parse your project.

How to Increase Memory for the IDE

Use the following procedure to first determine how much memory the IDE has been allocated, and then change the settings to provide more memory .

  1. View the IDE log by choosing View -> IDE Log.

    The log is displayed in the Output window in the IDE and can help you to determine how much heap memory the IDE has been provided.

  2. Right-click in the Output window and choose Find to open a Find dialog box, so you can search for relevant items in the log.
  3. Search for the strings -Xmx and Heap memory usage in the log.

    You should find lines similar to the following:

    -Xmx1024m
    ...
    Heap memory usage: initial 64.0MB maximum 912.0MB
    

    In this example, the limit of the memory was set to 1024 megabytes with the -Xmx1024m option, and the IDE reports that the maximum available heap memory is 912 megabytes.

  4. Exit the IDE.
  5. Start the IDE using the following command and option to override the default heap memory limit and set the memory limit to 2000 megabytes.

    % solstudio -J-Xmx2000m
    
  6. Try working with your project again to see whether the parsing performance and IDE responsiveness have improved.

    If there is not enough improvement, try the procedure in the How to Use 32-Bit Java to Reduce Memory Usage section.

    If the improvement is satisfactory, you might want to always override the default heap memory limit by changing a setting in the IDE configuration file solstudio.conf. This file is located in the studio-install-dir/lib/solstudio/etc directory. If you add the option to this file, all users of this installation of Oracle Solaris Studio IDE are affected.

    To change the setting, find the line in solstudio.conf that begins with default_options= and change the -J-Xmx1024m option to -J-Xmx2000m, as shown in bold below.

    # command line switches
    default_options="--branding solstudio -J-XX:PermSize=64m 
    -J-XX:MaxPermSize=400m -J-Xms64m -J-Xmx2000m 
    -J-Dnetbeans.openfile.197063=true"
    

    If you want to always start the IDE with more memory but you do not want to edit the solstudio.conf file, you can create an alias in your shell configuration file (for example, .cshrc or .bashrc). For instance, you might add the following line to your .bashrc file:

    alias solstudio='solstudio -J-Xmx2000m'
    

How to Use 32-Bit Java to Reduce Memory Usage

If you use 64-bit Java version 1.6, you can reduce memory consumption of the IDE by using 32-bit Java 1.6 instead. Using 32-bit Java saves memory on each pointer and can cut memory consumption nearly in half without any noticeable performance regression.

  1. Determine which version of Java is used by the IDE by choosing Help -> About in the IDE.

    If the IDE is using 64-bit Java, the About window includes information similar to the following:

    Product Version: Oracle Solaris Studio 12.3
    Java: 1.6.0_27; Java HotSpot(TM) 64-Bit Server VM 20.2-b06
    
  2. Exit the IDE and restart using the -J-d32 option.

    % solstudio -J-d32
    

If you want to use 32-bit Java for each IDE session, edit the studio-install-dir/lib/solstudio/etc/solstudio.conf file and insert the -J-d32 option in the default_options= line. For example, the option is shown below in bold inside the default options settings.

# command line switches
default_options="--branding solstudio -J-XX:PermSize=64m 
-J-XX:MaxPermSize=400m -J-Xms64m -J-Xmx1024m -J-d32
-J-Dnetbeans.openfile.197063=true"

This change affects all users of this installation of Oracle Solaris Studio. If you want to affect only your individual session, use the command line switch or create an alias that starts the IDE with the command line switch.

Checking for File System Issues

Conditions in the file system can be another cause of poor IDE performance. If you have increased the heap memory for the JVM, but the IDE is still slow to parse the project, you might have a slow file system or broken symbolic links in the file system.

In a terminal window, go to the project root directory and use the following command to determine how long it takes to list the files:

% time ls -laRL | wc -l

real    2m1.262s
user    0m1.152s
sys     0m15.347s
    236341

In this example, more than two minutes of real time elapsed to list over 236,000 files.

If the real time taken to list the files is excessive for the number of files, the IDE will also spend more time than expected to access the project files. The result is slow performance of the IDE.

How to Resolve File System Issues

The problem could be caused by any of the following conditions:

  • The file system has symbolic links to slow network resources. The simple solution is to consider copying linked resources to the local hard disk and changing the links to use the local versions.
  • Your home directory is located on a server with slow NFS service, which slows the IDE's access to your Oracle Solaris Studio user directory, $HOME/.solstudio. If you always work on the same system, a solution is to start the IDE with the -userdir option to store the user directory locally to avoid the slow NFS service. The user directory stores your preferences and other information such as the cache, which is accessed frequently. You can increase performance if you keep this directory on a local disk. A sample command for creating the local user directory is solstudio --userdir /var/tmp/yourname/oss12.3.
  • The file system has broken symbolic links or links to offline resources. As a result, the IDE can hang while trying to access the resources. The simple solution is to fix or remove the broken links, if possible. Another option is to exclude the problem links from the project as described in How to Exclude Problem Files in Project Sources.
  • Your project has too many files. See Checking Project Size.

Checking the Project Size

When you open a project, a progress bar is displayed at the bottom of the IDE window to show the parsing progress. If you click the progress bar, the IDE shows the number of files in the parsing queue.

Figure 2 shows the progress bar of a project from existing sources after the user clicked the progress bar. You can see a count of parsed files as a portion of the total files.

Picture of the progress bar with parsing queue data

Figure 2. Progress Bar with Parsing Queue Data

The memory consumption, parsing time, and IDE responsiveness are all affected by project size.

The amount of memory that the IDE requires for a project depends on the number of files and the amount of code in the files. When running 32-bit Java, the IDE would typically need 1,000 to 1,500 megabytes of memory for a project with 10,000 files. The parsing time for a project depends on the amount of code, the number of CPUs, and the CPU frequency. The parsing speed is typically 20,000 to 80,000 lines per second on a computer with four 3-GHz CPUs.

How to Reduce the Project Size

Sometimes, you can reduce project size by removing unneeded components or including only needed compilation units. A project's size depends on how the project was created.

You can create a project from existing sources using two different project creation templates:

  • C/C++/Fortran Project with Existing Sources
  • C/C++/Fortran Project from Binary File

If your project has too many files, the best solution is to create a new project from an existing binary. When you create a project in this way, only the files actually used in building the binary are included in the project, so it is a quick way to remove unneeded files from the project. The files are not physically removed, but they are not included in the project.

To create a project from an existing binary in the IDE, select File -> New Project -> C/C++/Fortran -> C/C++/Fortran Project from Existing Binary. In the dialog boxes for creating the project, you can specify whether to include just the source files used to build the binary or to include all files in the source root directory. If you do not really need all the extra files in the source root, you should not include them.

A project created as a "Project with Existing Sources" includes all files from the source root. A source root often contains files that are not C or C++ header files, but the Project with Existing Sources template must reflect the real file system tree of the source root, so these files are included in the project. The extra files can reduce IDE performance as it accesses and parses all the files. You can improve the performance of the IDE if you specify which files are unneeded by specifying them in an exclude pattern.

A project created as a "Project from Binary File" includes only compilation units from which the binary was built. Figure 3 shows a dialog box in the New Project wizard for creating a project for a particular binary built from the ACE+TAO open source code:

Picture of creating project from existing binary

Figure 3. Creating a Project from an Existing Binary

Notice the Include into Project field is set to Source files used in build so that not all files are included; only the necessary files are included.

Figure 4 shows a comparison of the parsing progress bars for Project with Existing Sources and Project from Existing Binary:

Picture that compares parsing progress bars

Figure 4. Comparison of Parsing Progress Bars

You can see the project created from the binary has fewer files. The project created from existing sources includes the complete ACE+TAO source code, while the project created from existing binary includes the code for one component of ACE+TAO.

You might find that in many cases, creating a project from a binary is sufficient for what you are working on and enables the IDE to perform at a higher rate.

How to Exclude Problem Files in Project Sources

One way to deal with broken links that you cannot fix is to exclude them from the project.

For example, suppose you have a project folder called my_project with subfolder src and links to versions of a library huge_library.so. The link to one version of the library is broken. The project file layout is as follows:

+ - my_project                     
  | - src/                             
  | - huge_library.so -> huge_library.so.14
  | - huge_library.so.13 ->    broken link
  | - huge_library.so.14 -> /usr/lib/huge_lib.14/huge_lib.so

The source files in the src folder include the header files for the library. If you are only using the library in your project, not developing the library, you do not need to include the library in the project. The IDE will automatically resolve headers from the library. If you include the library in the project, the IDE has to process any extra files from the library, such as tests and examples. In most cases, this is probably not what you want. You want to work with files in the src folder and sometimes navigate to header files in the library.

If you create the project using Create Project with Existing Sources and specify the folder my_project as the source root, you will have a performance problem because of the broken link to huge_library.so.13 and because huge_library.so.14 is included in the project twice. If you create the project using Create Project with Existing Sources and specify the folder src as the source root, you avoid the performance problem caused by the broken link and the huge_library.so.14.

The most efficient option for omitting the libraries from the project is to delete the broken link to huge_library.so.13 and exclude huge_library.so.14 when you create the project from existing sources. To exclude the library, select the Custom configuration mode from the Select Mode screen of the New Project wizard, as shown in Figure 5.

Picture of New Project wizard's Select Mode dialog box specifying Custom configuration

Figure 5. Specifying the Custom Configuration Mode

After you progress through the wizard to the Source Files screen, you can add huge_library.so.14 to the Exclude Pattern field, as shown in Figure 6.

Picture of New Project wizard's Source Files dialog box Exclude Pattern field

Figure 6. Exclude Pattern Field

If you prefer not to recreate the project, you can also omit the library after creating the project. To do this, right-click the project in the Projects window, select Properties, click the General node, and then add the library name to the Ignored Folders Pattern field, which is highlighted in Figure 7.

Picture of Project Properties dialog box and Ignored Folders Pattern field

Figure 7. Ignored Folders Pattern Field

The Ignored Folders Pattern format is ^(dirname|dirname|dirname)$. Figure 7 shows ^(nbproject|huge_library.so.14)$, which causes the IDE's project folder and the huge_library.so.14 folder to be omitted from the project.

Checking the User Watch Limit on Linux

On Linux platforms that support inotify file system monitoring by user applications, sometimes the IDE editor freezes when the IDE window obtains focus. If you have a large project that contains more than 5,000 subfolders, the slowness can be caused by exceeding the limit of inotify user watches. If this is the problem, the IDE log likely contains warnings similar to the following:

Cannot add filesystem watch for [file path]: error 28

You can see the current setting for the maximum number of watches by using the following command:

% cat /proc/sys/fs/inotify/max_user_watches
8192

How to Increase the User Watch Limit

To fix the problem, you can temporarily increase the limit for user watches by using the following command:

% sudo sysctl -w fs.inotify.max_user_watches=50000

To make this change permanent, add the following line to the /etc/sysctl.conf file:

fs.inotify.max_user_watches=50000

Remote Development Solutions

If you have been using the IDE remotely, running on a server, and displaying to your local system over the network by using X-Window forwarding or VNC, you might wish you could run the IDE locally to avoid delays caused by the network traffic. Beginning with Sun Studio 12 update 1, you could use the IDE locally on Oracle Solaris and Linux machines and set up a remote system as the build host. This avoided the need to go across the network to interact with the IDE, and it enabled you to use the power of a large server to build large projects from your local machine. However, if your local machine was running Microsoft Windows, Mac OS X, or even Ubuntu, you could not take advantage of the remote development feature.

Starting with Oracle Solaris Studio 12.3, remote development has a desktop distribution alternative for users on those popular but previously unsupported client operating systems. The desktop distribution feature enables you to create a zip file that contains the software and data you need to run the IDE on other platforms. You can unzip the file on your local Microsoft Windows, Mac OS X, or Ubuntu machine and run the IDE there while accessing the compilers and tools on a remote system.

Most of your IDE settings will be exported from your remote IDE to the desktop distribution zip, and all of the open projects in your remote IDE will be available as full remote projects in the installed desktop distribution on your local system.

For more information, click the C/C++ Remote Development link in the IDE's start page to see "Generating a Desktop Distribution" in the IDE help. For more detailed information, including information about the platforms that are supported for desktop distribution, see the article "How to Develop Code from a Remote Desktop with Oracle Solaris Studio."

Memory Use by ZFS on Oracle Solaris 11 Desktop Systems

If you run the IDE on an Oracle Solaris 11 desktop machine and notice slow IDE performance, one issue could be that ZFS is using more memory than necessary. In Oracle Solaris 11, ZFS by default can use all available RAM minus 1 GB on systems with 4 GB or more, or 75% of available RAM on systems with less than 4 GB. Although this is good practice for Oracle Solaris 11 on a server, it can negatively affect GUI applications on an Oracle Solaris 11 desktop if the applications must frequently reclaim memory from the ZFS cache.

If you have system administrator privileges on your desktop machine, you can investigate ZFS memory usage and, if necessary, adjust a parameter to tune the ZFS memory cache, as described in the next section.

Note: For servers that are not used to run desktop applications, the ZFS cache limit should not need to be changed.

How to Change the ZFS Cache Limit on Oracle Solaris 11 Desktop Systems

If you have system administrator privileges, you can check the memory usage of ZFS by using the mdb utility. Enter the commands shown in bold in Listing 1:

# mdb -k
Loading modules: [ unix genunix specfs dtrace ....
... ufs ipc ]
> ::memstat
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     125414               489   24%
ZFS File Data              135535               529   26%
Anon                       189613               740   36%
Exec and libs                4629                18    1%
Page cache                  31336               122    6%
Free (cachelist)            14351                56    3%
Free (freelist)             21248                83    4%

Total                      522126              2039
Physical                   522125              2039

> (press Ctrl-D to exit)

Listing 1. Checking the Memory Usage of ZFS

You can see in the example in Listing 1 that ZFS File Data is 529 MB, which is 26% of the total memory. The IDE can probably function well in this situation, so if the IDE is running slowly on the system, the problem is not the memory usage by ZFS. The memory usage would need to be closer to the limit of 75% to be a problem.

Another way to check ZFS memory usage is with the kstat command. Use kstat commands, as follows, to see the number of bytes of memory in use by ZFS and the maximum bytes for the ZFS cache:

$ kstat -m zfs -s size
module: zfs                             instance: 0     
name:   arcstats                        class:    misc
        size                            534926600

$ kstat -m zfs -s c_max
module: zfs                             instance: 0     
name:   arcstats                        class:    misc
        c_max                           1603971072

If ZFS is using a large percentage of the system memory, you can tune the zfs_arc_max parameter to lower the maximum memory for the ZFS cache, which can make more memory available for user applications such as the IDE.

See the Oracle Solaris Tunable Parameters Reference Guide for information about tuning the Oracle Solaris kernel by setting the zfs_arc_max parameter.

You could, for example, try setting the zfs_arc_max value to half the current value of c_max to see if that helps the IDE performance.

Reporting IDE Issues

If you cannot solve performance issues with the IDE, you can ask for assistance at the Oracle Solaris Studio Tools forum. The forum also provides a link to information about reporting bugs at http://bugreport.sun.com/bugreport/index.jsp.

Oracle Solaris Studio is scheduled to transition to the Oracle bug database in 2012. If you try to submit a bug at the Sun bug report site after the transition and have any difficulty submitting a bug in the new system, you can submit a bug (for the IDE only) using the NetBeans issue tracking system. Select the product cnd (C native developer) in the bug report and specify in the description note that you are using the Oracle Solaris Studio IDE 12.3.

Before you file a bug at the Sun bug report site or in the NetBeans issue tracking system, do the following to gather information that can help in tracking down the bug:

See Also

Oracle Solaris Studio Topics Web page

About the Author

Alexander Simon has been working in IT for over 25 years. Before joining Sun in 2006, he worked for Togethersoft and Borland, participating in the development of enterprise-level systems and modern IDEs. Alexander is currently a Principal Software Engineer for Oracle in St. Petersburg, Russia developing the Oracle Solaris Studio IDE. His responsibilities include developing IDE modules for the tool collection, project creation, and the editor.

Revision 1.0, 06/19/2012

See sysadmin-related content for all Oracle technologies by following OTN Systems on Facebook and Twitter.