Berkeley DB Java Edition 1.7.0 Change Log

Changes since Release 1.5.3.

For additional details, please see the documentation and Release Notes included in your download package or on our website.

Log File On-Disk Format Changes:

  1. JE 1.7.0 has moved to on-disk file format 2.
    The change is forward compatible in that JE files created with release 1.5.3 and earlier can be read when opened with JE 1.7.0.  The change is not backwards compatible in that files created with JE 1.7.0 cannot be read by earlier releases. Note that if a 1.5.3 environment is opened read/write, a new log file is written by JE 1.7.0 and the environment can no longer be read by earlier releases.

New Features:

  1. Added a new transaction commit mode. Previously, JE had the ability to commit transactions synchronously or asynchronously (nosync mode).  A synchronous commit is one where JE guarantees full transaction durability by writing and flushing JE log buffers to disk at commit time. "Write" means to move bytes from the JVM's buffers to the operating system's buffers, using the java.nio.channel.FileChannel.write() call and "flush" means to force the operating system's buffers to durable storage using the java.nio.channel.FileChannel.force() call.

    Alternatively, the user could commit transactions in "noSync" mode which does not write or flush log buffers and therefore has better performance, but carries a greater risk of losing the transaction if the JVM crashes.

    JE 1.7.0 has a new "commitWriteNoSync()" mode which does write but does not flush log buffers at commit time. The transaction will be durable if the JVM crashes, but will not be durable if the OS crashes. CommitWriteNoSync is faster than a synchronous commit and slower than a asynchronous commit. Synchronous, writeNoSync, and noSync modes can be specified through EnvironmentMutableConfig, TransactionConfig, or through Transaction commit(), commitWriteNoSync(), commitNoSync(). [#11131]

General Environment Changes:

  1. A deadlock will no longer occur when inserting data after a truncate using the same database and transaction handles. [#10386]

  3. Improve the log file cleaner. In some cases the amount of disk space used will be reduced by 50%, but the results will vary depending on your application.  Please be aware that when an environment created with older releases is first opened, the initial cleaning process will take more overhead than usual.  The cleaner may be active for some time as old log files are processed.  If you wish to speed up this initial cleaning process, you may perform a dump-load on all your databases; see the DbDump and DbLoad utilities for details. [#10395]

  5. Fix a bug in the recovery of deleted internal nodes that contained duplicate data items. [#10395]

  7. Improve JE memory management for transactions that hold many locks. JE now includes transactions and lock objects in its calculations of memory consumption.  A transaction that modified many data records and therefore held many locks could cause a java.lang.OutOfMemoryError even though unused memory was available in the JE cache.  This has been corrected. [#10887]

  9. The evictor is now run during the last phase of recovery to avoid out-of-memory errors.  This makes it less likely for an application to run out of memory during recovery (while opening an environment) after an abnormal exit. Running out of memory during recovery will only occur if a smaller heap than normally used by the application is specified. [#11076]

  11. Improve space utilization by the internal tree.  More user objects will fit in the cache and recovery will use less memory. [#10952]

  13. A secondary database can no longer be associated with two different primary databases. [#10755]

  15. Transactions may only be aborted after a deadlock or timeout operation thrown on their behalf.  They may not perform further operations. [#10955]

  17. Fix a bug where Cursor.getSearchBothRange() could return the key prior to the specified key, when the specified key was not present in the database. [#11119]

  19. Fix a bug that caused a ClassCastException in a transaction T1 when T1 was waiting to update a record that was locked by another transaction T2, and a second data item (duplicate) was added. [#11195]

  21. Fix a bug that could cause the database log to become corrupted when I/O errors occur under certain conditions. [#11236]

  23. Fix a bug that could cause exceptions during log cleaning. The stack trace contained the Cleaner.processLN method. [#11248]

  25. Change the Environment.verify() method so that it does not consider open transactions an error.  The verify method is  called when no application transactions are active, although transactions may be active in JE daemon threads. [#11256]

  27. Fix a bug that caused timeouts when accessing the internal database tree. [#11293]

  29. Cursor.getFirst() and Cursor.getLast() will no longer return NOTFOUND when the first (or last) record is deleted and other records are present in the database.  This would occur when the first (or last) internal node in the database was empty but compression of the node was only partially complete. [#11297]  

  31. A “latch not held exception” will no longer occur during recovery. [#11307]

  33. Fix a race condition where the recovery thread tries to write a new file header with a read-only file handle. [#11328]

  35. JE daemon threads would die after throwing an Exception. They now retry. [#11450]

  37. Fix a problem assigning node ids with multiple environments in a single JVM. [#11484]

API Changes:

  1. The environment cache size properties (je.maxMemory and je.maxMemoryPercent) can now be changed by the application after opening the environment. [#11337]

  3. Database.truncate() is deprecated and a new method Environment.truncateDatabase() is added.

    Previous versions of JE had a bug where the results of a Database.truncate() call could be seen by other methods run from the same Database handle before the transaction committed, even from other transactions. Environment.truncateDatabase() was introduced to fix the bug without incurring a performance penalty on general operations. [#10339]

  4. By default JE provides Repeatable Read isolation, which permits phantom records to appear between calls. Phantoms are records that are not seen at one point in transaction A but are seen at a later point in transaction A, when they are inserted by transaction B and transaction B is committed. Now, phantoms are not permitted if the Serializable isolation level is configured. This isolation level is configured by calling TransactionConfig.setSerializationIsolation or EnvironmentConfig.setTxnSerializationIsolation. See the Transactions and Concurrency section of the Getting Started Guide for more information. [#10477] [#11381]

  6. JE applications run on Mac OS 1.4.2_* must explicitly set the environment cache size at startup. JE is unable to determine a default cache size because the value returned by Runtime.maxMemory() is unreliable. [#11463]

  8. Make two minor improvements to TransactionStats: 1- In the nested class Active change the public fields to getter methods; 2- For convenience add a toString method that lists all data fields. [#11255]

Utility Changes:

  1. Change DbDump/DbLoad format. DbLoad now accepts either {true|false} or {0|1} for boolean parameters specified at the start of the file, for compatibility with Berkeley DB dump/load format. DbDump dumps boolean parameters as {0|1}.

    Remove output of database=<name> parameter since we do not allow multiple db dumps. [#10861]

Configuration, Documentation, and Build Changes:

  1. Made several Javadoc corrections and clarifications. [#10963]
    • Environment.beginTransaction() is free threaded.
    • Non-transactional Cursors can only be used by a single thread.
    • BtreeStats.getLeafNodeCount() provides an accurate count of data records when stats are called in a quiescent system.

  3. Change package name of the collections example programs to accommodate tar pathname length limits. Change "shipment" to "ship" and "helloworld" to "hello". [#11039]