Berkeley DB Java Edition 3.0.12 Change Log

In JE 3.0.11 we introduced a new Plain Old Java Object (POJO)-based persistent object model which we called the Persistence API. Since then, we've decided to rename this API to the Direct Persistence Layer, to avoid confusion with the EJB3 Java Persistence API.

Version 3.0.12 contains bug fixes to the Direct Persistence Layer beta released in version 3.0.11. The changes between 3.0.11 and 3.0.12 are described directly below. A full list of all changes since JE 2.1.34 is also available below.

Changes since 3.0.11

Direct Persistence Layer Beta:

1. Fixed a bug where "UnsupportedOperationException: Class evolution is not supported" was thrown when performing class enhancement. This problem would occur when creating a store without enhancing classes and then using enhancement later, or when creating a store with class enhancement and then not using enhancement later. [#14651]
2. 
   
3. Fixed a bug where "ArrayIndexOutOfBoundsException: 2147483646" was thrown when updating a record with a secondary key. The problem occurred when a single instance was used for both the primary key and secondary key fields. [#14665]
4. 
   
5. Fixed a bug where "IllegalArgumentException: Class is not persistent" was thrown for a class that was correctly made persistent using a proxy class. The problem occurred when one proxy class referenced another. [#14665]
6. 
   
7. Before, if a custom key class (composite key class) was defined but the @KeyField annotation was not included, an UnsupportedOperationException would occur when trying to put() a record, and the exception message was not meaningful. Now, an IllegalArgumentException will occur when getting the index, with a meaningful exception message. [#14673]
8. 
   
9. An optional verbose option was added to the ClassEnhancer to print the name of each class file that is enhanced. See ClassEnhancer for more information. [#14680]

Full list of changes since JE 2.1.34

All changes from JE 3.0.11, 3.0.12 are listed here.

For additional details, please see the documentation and Release Notes included in your download package or on our website.

Log File On-Disk Format Changes:

1. JE 3.0.11 has moved to on-disk file format 4.
   
   The change is forward compatible in that JE files created with release 2.1.30 and earlier can be read when opened with JE 3.0.11. The change is not backward compatible in that files created with JE 3.0.11 cannot be read by earlier releases. Note that if an earlier environment is opened read/write, a new log file is written by JE 3.0.11 and the environment can no longer be read by earlier releases.
2. 
   
3. An upgrade problem was inadvertently introduced between JE 2.0.90, which was released in Nov 1, 2005 and the following JE 2.1.34 release in January 2006. The problem does not apply to you if you have an environment created with JE 2.0.90 or earlier, and have since successfully opened it with JE 2.1.30. Likewise, a 2.0.90 and earlier environment that has been closed cleanly with Environment.close() will not encounter this problem, nor will most environments that have more than 30 Mb of data. The problem manifests as the following stack trace. A workaround is described below.

   com.sleepycat.je.DatabaseException: JE 3.0.11: searchResult=class com.sleepycat.
   je.tree.IN nodeId=479 nEntries=0
          at com.sleepycat.je.tree.Tree.getParentBINForChildLN(Tree.java:1031)
   ...
   Caused by: java.lang.ClassCastException: com.sleepycat.je.tree.IN
          at com.sleepycat.je.tree.Tree.getParentBINForChildLN(Tree.java:1021)
   

New Features:

1. A new Persistence API adds a built-in Plain Old Java Object (POJO)-based persistent object model, which provides support for complex object models without compromises in performance. For an introduction to the Persistence API, see Getting Started with the Persistence API. This is a beta release of the Persistence API. Please see the Release Notes for more information. [#13894]
2. 
   
3. A new "deferred write" database mode provides better performance for temporary databases or databases which rapidly modify data by reducing the amount of I/O incurred. In a normal durable database, every new version of a data record is written to disk; in deferred write mode, JE avoids disk I/O until cache is exceeded or Database.sync() is called. See the Getting Started Guide, Chapter 3 (Databases) for more information. [#13936]
4. 
   
5. com.sleepycat.je.util.DbBackup is a new helper class which simplifies backups by managing JE background activity in an open environment. See the Getting Started Guide, Chapter 7 (Backing up and Restoring Berkeley DB Java Edition Applications) and the Javadoc for DbBackup for more information. [#14022]
6. 
   
7. The com.sleepycat.collections package is now fully compatible with the Java Collections framework. In previous releases, Collections.size() was not supported, and collection iterators had to be explicitly closed. These incompatibilities have been addressed to provide full interoperability with other Java libraries that use the Java Collections Framework interfaces. [#14154] [#12986]
8. 
   
9. The com.sleepycat.bind.tuple package now has bindings that correctly sort negative floating point numbers. See the following new APIs:

     SortedFloatBinding, SortedDoubleBinding,
     TupleInput.readSortedFloat, TupleInput.readSortedDouble,
     TupleOutput.writeSortedFloat, TupleOutput.writeSortedDouble.
   

   [#14127]

General Environment Changes:

1. Shared latches are now used on all internal nodes of the b-tree, which increases concurrency for many operations. Shared latches are only enabled when je.env.sharedLatches is set to true; the default value is false. The default may change in a future release. [#14168]
2. 
   
3. Fixed a bug where an internal latch was not taken during the deletion of log cleaning metadata or database metadata. [#14214]
4. 
   
5. Fixed a bug where internal node latches were not released in a corner case where the database has duplicate records and there are concurrent readers and writers. [#14230]
6. 
   
7. Calling com.sleepycat.je.Transaction.toString() after the transaction was closed resulted in a NullPointerException. This is now fixed. [#14349]
8. 
   
9. In a corner case requiring a certain pattern of inserts coupled with btree splits and checkpoints, a data record could be lost after an environment recovery following a non-clean shutdown. This has been fixed. [#14424]
10. 
    
11. The following NullPointerException could be seen when je.cleaner.threads > 0. This has been fixed.

    java.lang.NullPointerException
            at com.sleepycat.je.cleaner.UtilizationProfile.getCheapestFileToClean(UtilizationProfile.java:201)
            at com.sleepycat.je.cleaner.FileSelector.selectFileForCleaning(FileSelector.java:205)
            at com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:201)
            at com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:143)
    

    [#14431]
12. 
    
13. Cursor.getSearchBothRange() was incorrectly using the custom duplicate key comparator instead of the main btree key comparator, if the database was configured with a custom duplicate comparator. [#14464]
14. 
    
15. A user reported an OutOfMemoryError in a deployment using fairly large (512K) records. We finally tracked the problem down to the behavior of the standard java.util.zip.Adler32 class, which JE uses to checksum log records. Our research found JVM implementation notes which said that a JVM may temporarily disable garbage collection when executing the native code for Adler32.

    The workaround for the user was to use -Dje.disable.java.adler32=true at runtime. This forces JE to its own, slower Adler32 implementation. It's an existing flag that was used to work around a Java 1.4 problem with the Adler32 implementation.

    While this is not an implementation change, the situation and workaround may be of interest to our users. See the sleepycat blog entry on this topic for more details. [#14149]

API Changes:

1. Iterators returned by StoredCollection.iterator() no longer need closing. The original StoredIterator is now available using the new StoredCollection.storedIterator() method. See the Release Notes for more information on collection API changes. [#12986]
2. 
   
3. Database.entry now supports equals() and hashCode(). [#13778]
4. 
   
5. Key comparators can now be stored as serialized objects using the new DatabaseConfig.setBtreeComparator(Comparator) and DatabaseConfig.setDuplicateComparator(Comparator) methods. This allows a single comparator class to handle more than one key format. [#14422]
6. 
   
7. SecondaryDatabase.delete has been corrected to delete all keys (and their associated primary records) in the presence of duplicates. Before, only the first of a duplicate set (and its associated primary) was deleted. Please note this behavior change. [#14506]
8. 
   

Utility Changes:

1. DbVerify will now verify all databases in the environment if the "-s" flag is omitted. In previous versions, "-s" was a mandatory parameter. A new option, "-c" has been added which will check log cleaner metadata. [#14239]

Configuration, Documentation, and Build Changes:

1. Ant 1.6.0 or later is required to build JE from the provided sources.
2. 
   
3. The directions in the installation page on how to compile a non-debug jar were updated.
4. 
   
5. The "Writing Transactional Applications" guide has been corrected to make it clear that nested transactions are not yet supported in JE. [#14079]
6. 
   
7. A variety of problems noted by our faithful readers have been fixed.
   • The examples now specify a character encoding when constructing a String object.
   • The examples use the base interface type when possible, rather than a specific collection class.
   • ExampleDatabaseLoad.loadFile() now closes its FileInputStream.
   • Two examples in the Concurrency chapter of the "Writing Transactional Applications" manual were updated to correctly return after a commit.

Direct Persistence Layer Beta:

1. Fixed a bug where "UnsupportedOperationException: Class evolution is not supported" was thrown when performing class enhancement. This problem would occur when creating a store without enhancing classes and then using enhancement later, or when creating a store with class enhancement and then not using enhancement later. [#14651]
2. 
   
3. Fixed a bug where "ArrayIndexOutOfBoundsException: 2147483646" was thrown when updating a record with a secondary key. The problem occurred when a single instance was used for both the primary key and secondary key fields. [#14665]
4. 
   
5. Fixed a bug where "IllegalArgumentException: Class is not persistent" was thrown for a class that was correctly made persistent using a proxy class. The problem occurred when one proxy class referenced another. [#14665]
6. 
   
7. Before, if a custom key class (composite key class) was defined but the @KeyField annotation was not included, an UnsupportedOperationException would occur when trying to put() a record, and the exception message was not meaningful. Now, an IllegalArgumentException will occur when getting the index, with a meaningful exception message. [#14673]
8. 
   
9. An optional verbose option was added to the ClassEnhancer to print the name of each class file that is enhanced. See ClassEnhancer for more information. [#14680]

Workaround for upgrade issue between JE 2.0.90 and JE 2.1.30

If you are upgrading from a release of JE that is 2.0.90 or earlier, then you should ensure that you call Environment.close() prior to upgrading. If you do not close the environment cleanly, and you have very recently created a new database, you may see the following exception:

com.sleepycat.je.DatabaseException: JE 3.0.11: searchResult=class
com.sleepycat. je.tree.IN nodeId=479 nEntries=0
      at com.sleepycat.je.tree.Tree.getParentBINForChildLN(Tree.java:1031)
...
Caused by: java.lang.ClassCastException: com.sleepycat.je.tree.IN
       at
       com.sleepycat.je.tree.Tree.getParentBINForChildLN(Tree.java:1021)

The data in the environment was not corrupted, and the environment can be successfully opened with the following steps which remove any new files create by JE 3.0.11, and then call Environment.close() prior to upgrading.

To remove any data created by JE 3.0.11:

• First take a directory listing of your Environment and examine all the *.jdb files written at the time of the attempted upgrade. For example:

  -rwx------   1 cwl      other      19629 May  5 11:26 00000000.jdb
  -rw-r--r--   1 cwl      other       3014 May  8 11:36 00000001.jdb <-- new file
  

• Examine all the new files to verify that they use file version 4 by doing:

  java com.sleepycat.je.util.DbPrintLog -h <envhome> -s <lastfilenumber>
  

  where <lastfilenumber> is a hex number of the last file (e.g. '-s 0x1' in the case above). You should see output like this:

  <DbPrintLog>
  <entry lsn="0x1/0x0" type="FileHeader/0" prev="0x0" size="24" cksum="800588407"><FileHeader num="0x1" lastEntryInPrevFileOffset="0x4c87"
  logVersion="0x4" time=" 2006-05-08 12:50:20.63"/></entry>
  
             ...
  

  The first file header record shows a logVersion of 0x4. These files were written during the attempted upgrade and can be deleted.

To close the environment cleanly after any JE 3.0.11 data has been removed:

• Once the version 4 log files have been deleted, perform a checkpoint using the old (pre-2.0.90) version of JE. You can do this by calling Environment.close() from your application code or by invoking the unadvertised DbRunAction utility:

  java -cp "je-1.7.1.jar:je-3.0.11.jar" com.sleepycat.je.util.DbRunAction -h <envdir>
  

  Since DbRunAction is not present in releases prior to 2.1.30, you should include the latest release in the classpath, but after the old release. That will run a checkpoint on the old environment using the old version of JE. Several lines of output will be produced. Verify that the last log file was written to by taking a directory listing.
• 
  
• Finally, perform another checkpoint using the newer version of JE. After this, the environment will be fully upgraded.

  java -cp "je-3.0.11.jar" com.sleepycat.je.util.DbRunAction -h <envdir>