Berkeley DB Java Edition 4.0.71 Change Log

Changes in 4.0.71

Log File On-Disk Format Changes:

The change is forward compatible in that JE files created with release 3.3 and earlier can be read when opened with JE 4.0.71. The change is not backward compatible in that files created with JE 4.0 cannot be read by earlier releases. Note that if an existing environment is opened read/write, a new log file is written by JE 4.0 and the environment can no longer be read by earlier releases.

Changes in API and exception handling create a binary incompatibility between JE 4.0 and JE 3.X. Users are advised to recompile their applications when moving to JE 4.0.

New Features:

1. This release features JE High Availability, which permits the use of JE with replication. See the High Availability Guide for an introduction to the product, and the com.sleepycat.je.rep javadoc for API specifications.


2. JE now provides two JMX MBean implementations for monitoring a running JE application. A JConsole plugin is also provided for accessing the new JEMonitor and RepJEMonitor classes. Environment statistics can be collected and graphed, and other state information can be perused. See the how-to for more information.


3. It is critical that invalid files are not added to a backup set, since then both the live environment and the backup will be invalid. Two new classes now support verification of log files prior to making a backup.

   The com.sleepycat.je.util.LogVerificationInputStream class enables log verification programmatically as files are being copied. The com.sleepycat.je.util.DbVerifyLog class is a simple command line utility for inclusion in backup scripts. [#16476]



4. Cursor and Database record retrieval has been optimized to significantly reduce I/O when DatabaseEntry.setPartial is used to suppress return of the data item and the READ_UNCOMMITTED isolation mode is used. See "Using Partial DatabaseEntry Parameters"in the Cursor javadoc for more information. [#16859]


5. The EnvironmentConfig.setNodeName(String nodeName) and EnvironmentConfig.getNodeName() methods have been added to match those same methods in ReplicationConfig. Setting the EnvironmentConfig node name will cause java.util.logging messages and thread names to include the user-specified name. This was previously only true for replicated environments, and has been extended to work for non-replicated environments, as a debugging aid when using multiple environments in the same JVM. The feature was originally requested on the OTN forum. [#17629] (changed in 4.0.70)

1. The close() method may now be called repeatedly on Database, Cursor, Environment, JoinCursor, and EntityStore instances, and abort() may be called repeatedly on Transaction instances. Whereas they used to throw an exception if close() or abort() was called multiple times, they now return silently on subsequent calls. [#16214]


2. XAEnvironment.getXATransaction() has been removed from the Javadoc. This is an internal entrypoint and should not be used by general users. getXATransaction()'s behavior has been changed so that it will no longer return a Transaction instance for a transaction which is prepared, but not yet committed or rolled back. The only user operations which can be performed on prepared, but not yet committed/rolled-back transactions are XAEnvironment.commit() or rollback(). [#16375]


3. Added the DatabaseConfig.setUseExistingConfig and DatabaseConfig.getUseExistingConfig methods. This allows a user application to open a Database readonly and determine its existing DatabaseConfig (e.g. whether the database has sorted duplicates or not). [#16504]


4. Several major improvements have been made to the JE exception hierarchy. In most cases, no source code changes to applications are required. However, users are strongly encouraged to read the new exception documentation and follow the recommended approach for exception handling. In one case, a compatibility flag must be set if DeadlockException or LockNotGrantedException is explicitly caught, and the use of these exceptions is not changed as described below. Also, if an instance of DatabaseException is being created directly by the application, this must be changed as described below.
   
   
   • DatabaseException is now a runtime exception rather than a checked exception. In other words, it now extends RuntimeException rather than Exception. Since all other JE exceptions extend DatabaseException, all JE exceptions are now runtime exceptions. This change allows catching and handling JE exceptions at the level of the call chain most appropriate for a given situation, without having to declare DatabaseException in the throws clause of every method or wrap it in a runtime exception. In some cases JE exceptions should be handled by the immediate caller of a JE method, but in other cases should be handled at a higher level. Using runtime exceptions makes it easier for applications to make the appropriate choice. No application changes are required, although many users may wish to remove DatabaseException from the throws clause of methods where it is no longer needed, or remove the use of runtime exception wrappers. [#16704]
   
   
   • The JE exception hierarchy has been expanded and all exceptions are now classified as either operation failures (base class OperationFailureException) or environment failures (base class EnvironmentFailureException). For both types of failures, the general approach for handling the exception has been defined in the javadoc for these base classes. To support handling all exceptions in a general way, the Transaction.isValid and Environment.isValid methods have been added. [#16704]
   
   
   • RunRecoveryException has been deprecated and replaced by EnvironmentFailureException; however, application code that catches RunRecoveryException need not be changed immediately. EnvironmentFailureException extends RunRecoveryException, so existing code that catches RunRecoveryException will now catch EnvironmentFailureException. However, please be aware that EnvironmentFailureException is now thrown for more error cases than RunRecoveryException was in JE 3.3 and earlier. Applications are strongly encouraged to use the new approach for handling EnvironmentFailureException, which is described in the javadoc for this class. [#16704]
   
   
   • The DatabaseException class is now abstract. This ensures that a specific exception, having well defined rules for handling it, is always thrown. Applications that are throwing DatabaseException directly must be changed to throw a different exception, and not a JE exception. Although JE exceptions have public constructors, these are excluded from the javadoc and marked as Internal Use Only. JE exceptions should only be created and thrown by JE. [#16704]
   
   
   • Several improvements have been made to the exception hierarchy for lock conflicts (lock timeouts, transaction timeouts, deadlocks, etc). New exceptions and the new base class LockConflictException are available. Now, LockConflictException should be caught to handle lock conflicts in a general manner, instead of catching DeadlockException. New exceptions are now thrown as follows:
     • LockTimeoutException is now thrown when a lock timeout occurs, rather than DeadlockException.
     • TransactionTimeoutException is now thrown when a transaction timeout occurs, rather than DeadlockException.
     • LockNotAvailableException is now thrown when a lock conflict occurs for a no-wait transaction, rather than LockNotGrantedException.
     These three new exceptions are subclasses of LockConflictException. DeadlockException is also now a subclass of LockConflictException, but is not currently thrown by JE because true deadlock detection is not used in JE. Currently, lock timeouts are used instead. When true deadlock detection is added to JE in the future, DeadlockException will be thrown. LockNotGrantedException has been deprecated and replaced by LockNotAvailableException. Unless EnvironmentConfig.LOCK_OLD_LOCK_EXCEPTIONS is set to true (which enables the old exception behavior) the following changes must be made to JE applications that upgrade from JE 3.3 or earlier.
     1. All occurrences of DeadlockException must be replaced with LockConflictException or one of its non-deprecated subclasses (LockTimeoutException, TransactionTimeoutException, or LockNotAvailableException). It is strongly recommended to replace it with LockConflictException, since catching this exception will catch true deadlocks in the future and other types of lock conflicts. All lock conflicts all typically handled in the same way, which is normally to abort and retry the transaction.
     2. All occurrences of LockNotGrantedException must be replaced with LockNotAvailableException. LockNotGrantedException has been deprecated because it misleadingly extends DeadlockException.
     [#16264]
   
   
   • To go along with the lock conflict exception changes above, example code is now provided that shows the canonical approach for implementing a transaction retry loop. Adopting this approach is especially important for compatibility with the use of replication. Please see the Performing Transaction Retries section in the javadoc for com.sleepycat.je.LockConflictException (for the base API) and com.sleepycat.persist.EntityIndex (for the DPL). [#14573]
   
   
   • In addition, many other new exceptions have been defined so they may be specifically caught and handled. For example, this allows detecting a secondary constraint violation by catching SecondaryConstraintException, or one of its more specific subclasses, rather than by catching DatabaseException. Please see the javadoc for the following new exceptions. Note that some new exceptions were added for replication support.
     
     • DatabaseExistsException
     • DatabasePreemptedException
     • DeleteConstraintException
     • DuplicateDataException
     • EnvironmentNotFoundException
     • ForeignConstraintException
     • LockConflictException
     • LockNotAvailableException
     • LockPreemptedException
     • LockTimeoutException
     • LogOverwriteException
     • LogWriteException
     • SecondaryConstraintException
     • SecondaryIntegrityException
     • SecondaryReferenceException
     • SequenceExistsException
     • SequenceIntegrityException
     • SequenceNotFoundException
     • SequenceOverflowException
     • ThreadInterruptedException
     • TransactionTimeoutException
     • UniqueConstraintException
     • VersionMismatchException
     • XAFailureException
     [#16704]
   
   
   • The JE javadoc has been updated to specify all exceptions that are thrown by each method, including the standard Java runtime exceptions (IllegalStateException, UnsupportedOperationException, and IllegalArgumentException). [#16704]
   
   
   See the DatabaseException javadoc for an overview of all JE exceptions.


5. The Environment.getLockStats() method and LockStats class are deprecated. All of the lock stats may be obtained using Environment.getStats(). The same get methods which exist on LockStats are now present on EnvironmentStats. [#16792]


6. Added the Database.compareKeys(DatabaseEntry, DatabaseEntry) and Database.compareDuplicates(DatabaseEntry, DatabaseEntry) methods which may be used for comparing two DatabaseEntry instances using either the btree comparator or the duplicate comparator. [#16816]


7. The com.sleepycat.je.ForeignKeyDeleteAction, com.sleepycat.je.LockMode, and com.sleepycat.je.OperationStatus classes have been changed from classes to Java enums. This is a compatible change and does not require a recompilation unless you use LockMode.DIRTY_READ. LockMode.DIRTY_READ was previously deprecated and has now been removed from the code base. Use of the LockMode.DIRTY_READ field in class files compiled with earlier versions of JE will cause java.lang.NoSuchFieldError to be thrown at runtime.

   In addition, the following previously deprecated methods, classes, and constants have been removed:

   com.sleepycat.collections.StoredCollections.dirtyReadCollection,
   com.sleepycat.collections.StoredCollections.dirtyReadList,
   com.sleepycat.collections.StoredCollections.dirtyReadMap,
   com.sleepycat.collections.StoredCollections.dirtyReadSet,
   com.sleepycat.collections.StoredCollections.dirtyReadSortedMap,
   com.sleepycat.collections.StoredCollections.dirtyReadSortedSet,
   com.sleepycat.collections.StoredContainer.isDirtyReadAllowed,
   com.sleepycat.collections.StoredContainer.isDirtyRead,
   com.sleepycat.je.CursorConfig.DIRTY_READ,
   com.sleepycat.je.CursorConfig.setDirtyRead,
   com.sleepycat.je.CursorConfig.getDirtyRead,
   com.sleepycat.je.TransactionConfig.setDirtyRead,
   com.sleepycat.je.TransactionConfig.getDirtyRead,

   [#16984]



8. The SecondaryCursor.dupSecondary() method has been deprecated and replaced by SecondaryCursor.dup(). The SecondaryDatabase.openSecondaryCursor() method has been deprecated and replaced by SecondaryDatabase.openCursor(). The SecondaryDatabase.getSecondaryConfig() method has been deprecated and replaced by SecondaryDatabase.getConfig(). The SecondaryDatabase.openSecondaryCursor() method has been deprecated and replaced by SecondaryDatabase.openCursor(). The cloneConfig() methods for DatabaseConfig, EvolveConfig, and StoreConfig have been deprecated and replaced by clone(). The clone() methods for CheckpointConfig, CursorConfig, EnvironmentConfig, JoinConfig, PreloadConfig, SecondaryConfig, SequenceConfig, StatsConfig, TransactionConfig, and VerifyConfig have been added. [#16985]


9. All public void setFoo() methods in com.sleepycat.je.*Config have been changed to return this instead of void. This allows multiple set calls to be placed on one line, similar to StringBuffer calls. For example, it is now possible to do the following:

   EnvironmentConfig envConf = new EnvironmentConfig();
   envConf.setAllowCreate(true).setTransactional(true);

   This will require users to recompile their code. [#17021]


10. In JE 4.0.70, an additional change was made so that all setter methods in the DPL StoreConfig EvolveConfig now return this rather than having a void return type. This change requires that applications using the DPL be recompiled. [#17021] (changed in 4.0.70)


11. Previous versions of JE would allow conflicting settings in TransactionConfig and EnvironmentConfig for TxnNoSync, TxnWriteNoSync, and TxnSync. TransactionConfig no longer allows setting more than one of TxnNoSync, TxnWriteNoSync, or TxnSync to true. An IllegalArgumentException will be thrown if the settings of these configuration parameters are conflicting. [#17705]


12. The 4.0 release provides support for replicated environments. JE defines a new class: Durability, as a single consistent way to specify durability for both standalone and replicated environments. Accordingly, the methods: setTxnNoSync, getTxnNoSync, setTxnWriteNoSync, getTxnWriteNoSync in EnvironmentMutableConfig and methods setNoSync, getNoSync, setWriteNoSync, getWriteNoSync in TransactionConfig are deprecated, the methods setDurability, getDurability should be used in their place. A new Transaction method, commit(Durability) has also been provided for use in both standalone and replicated environments.


13. Removed any code which uses NIO for IO to the file system. Deprecated the je.log.useNIO and je.log.directNIO parameters. If EnvironmentConfig.LOG_USE_NIO or EnvironmentConfig.LOG_DIRECT_NIO is used by the application, a deprecation warning will result. If these parameters are set, they will have no effect on JE, and no warning (other than the deprecation warning) will be given to the user.


14. JE 3.X and earlier provided various proprietary JE properties named "java.util.logging.XXX" which were meant to be set either in the je.properties file or in the EnvironmentConfig class. These properties configured JE's use of the java.util.logging framework, but did not obey the conventions of standard java.util.logging naming and configuration management. These proprietary properties have been removed. Instead, JE's use of java.util.logging can now be configured in the standard fashion through the configuration properties described by the java.util.logging package.

    To display all logging output to the console, set com.sleepycat.je.util.ConsoleHandler.level = ALL as a java.util.logging property. By default, JE records logging output of level INFO or higher is recorded in <envdir>/je.info. To change the level of output displayed in the file, set com.sleepycat.je.util.FileHandler.level to the desired level. The logging output is particularly helpful in a replicated application. An example of the output can be found here.



15. The following Environment configuration properties now allow specifying a time unit. In the past, values could only be expressed as microseconds. For these properties, the default unit has always been microseconds and that has not changed. For more information about specifying units with configuration properties, see the Time Duration Properties section of the EnvironmentConfig class. These properties are:
    • EnvironmentConfig.LOCK_TIMEOUT
    • EnvironmentConfig.TXN_TIMEOUT
    • EnvironmentConfig.LOG_FSYNC_TIMEOUT
    • EnvironmentConfig.ENV_BACKGROUND_SLEEP_INTERVAL
    • EnvironmentConfig.COMPRESSOR_WAKEUP_INTERVAL
    • EnvironmentConfig.COMPRESSOR_LOCK_TIMEOUT
    • EnvironmentConfig.CHECKPOINTER_WAKEUP_INTERVAL
    • EnvironmentConfig.CLEANER_LOCK_TIMEOUT

    In addition, the following new methods have been added to allow specifying a unit along with a time duration. The equivalent older methods, without a unit argument, have been deprecated. The new methods are:

    • EnvironmentConfig.setLockTimeout(long,TimeUnit)
    • EnvironmentConfig.getLockTimeout(TimeUnit)
    • EnvironmentConfig.setTxnTimeout(long,TimeUnit)
    • EnvironmentConfig.getTxnTimeout(TimeUnit)
    • Transaction.setLockTimeout(long,TimeUnit)
    • Transaction.getLockTimeout(TimeUnit)
    • Transaction.setTxnTimeout(long,TimeUnit)
    • Transaction.getTxnTimeout(TimeUnit)


16. All methods that return a new cursor in a replicated environment now allow passing a null Transaction. This includes Database.openCursor, EntityIndex.entities, EntityIndex.keys, and StoredContainer.storedIterator. Before, a null Transaction was prohibited in a replicated environment unless read-uncommitted isolation was configured. Note that a null transaction has always been allowed (and still is) in a non-replicated environment. [#16513] (changed in 4.0.70)


17. The Transaction.commit and CurrentTransaction.commitTransaction family of methods no longer throw LockPreemptedException. Note that LockPreemptedException is only applicable in replicated environments. [#16513] (changed in 4.0.70)


18. An IllegalArgumentException is now thrown if LockMode.RMW is passed to the Database.get and Database.getSearchBoth methods, and a null Transaction is also passed. Without a transaction it is not possible to use these methods to perform a read-modify-write operation that initially acquires a write lock, so the combination of parameters does not make sense. [#16513] (changed in 4.0.70)


19. IllegalStateException is now thrown if Transaction.getCommitToken() is called before the transaction has finished. Previously, it would return a null commit token. [#17377] (changed in 4.0.70)

1. Fixed a bug which could cause a deadlock if a Database handle is closed while another thread is still using it. Removed code which automatically closes cursors that are still open at Database.close() time. [#15413]


2. If a call to com.sleepycat.je.Transaction.commit() throws an exception because the transaction has open cursors, a follow-on call to Transaction.abort() could fail with the following stack trace:

       Transaction 3 has been closed. java.lang.IllegalStateException: Transaction 3 has been closed.
           at com.sleepycat.je.txn.Txn.checkState(Txn.java:1580)
           at com.sleepycat.je.txn.Txn.abortInternal(Txn.java:819)
           at com.sleepycat.je.txn.Txn.abort(Txn.java:805)
           at com.sleepycat.je.Transaction.abort(Transaction.java:90)
           at com.sleepycat.je.txn.TxnEndTest.testTxnClose(TxnEndTest.java:
   

   This was a bug and has been fixed so the call to abort() will complete successfully. [#16214]


3. Improved synchronization on Database instances obviating the need to use separate Database handles in high-concurrency environments. Previously, this practice was recommended, but is no longer necessary. [#16346]


4. Improved internal concurrency by using shared (vs exclusive) latches on internal nodes of the B+Tree. Added a new stat to Database and Environment stats to record the number of "relatches" (latch upgrades) that have occurred on both the particular Database and overall for the Environment. [#16346]


5. When opening different environments in the same process, if recovery for one of the environments took a long time, the other environment openings would block. Changed the behavior such that opening additional environments is not blocked by another environment's recovery. [#16350]


6. Improved JE write performance when doing synchronous transaction commits on certain Linux filesystems, including ext3.

   In JE, when the durability specified for a transaction dictates force-writing to disk (for example, a commitSync() call), it calls fsync(). On modern disk drives an fsync() will take on the order of several milliseconds. This is a relatively long time compared to the time required to do a write() call, which only moves data to the operating system's file cache.

   JE's group commit mechanism seeks to reduce the number of fsyncs required by issuing one fsync for batches of multiple transaction commits. For example, if a thread T1 requires an fsync(), and JE determines that no fsync() is in progress, it will execute that fsync() immediately. If, while T1 is executing the fsync(), some other thread(s) require an fsync(s), JE will block those threads until T1 finishes. When T1's fsync() completes, a new fsync() executes on behalf of the blocked thread(s).

   The past JE group commit implementation assumed that the underlying platform (OS + file system combination) allow IO operations like seek(), read() and write() to execute concurrently with an fsync() call (on the same file, but using different file descriptors). On Solaris and Windows this is true. Hence, on these platforms, a thread which is performing an fsync() does not block another thread performing a concurrent write(). But, on several Linux file systems, ext3 in particular, an exclusive mutex on the inode is grabbed during any IO operation. So a write() call on a file (inode) will be blocked by an fsync() operation on the same file (inode). This negates any performance improvement which might be achieved by group commit.

   The JE group commit code has been improved to batch write() and fsync() calls, rather than just fsync() calls. Just before a write() call is executed, JE checks if an fsync() is in progress and if so, the write call is queued in a (new) Write Queue. Once the fsync() completes, all pending writes in the Write Queue are executed.

   This change in behavior is enabled by default and may be disabled by setting the je.log.useWriteQueue configuration parameter to false. The size of the Write Queue (i.e. the amount of data it can queue until any currently-executing IO operations complete) can be controlled with the je.log.writeQueueSize parameter. The default for je.log.writeQueueSize is 1MB with a minimum value of 4KB and a maximum value of 32MB. The Write Queue does not use cache space controlled by the je.maxMemory parameter. [#16440]



7. Trace messages are now written to the log at recovery and shutdown time. This applies mostly as a field support aid. [#16452]


8. Fixed a memory leak in applications which set EnvironmentConfig.setLocking(false) and used multiple environments and shared caches. JE could continue to maintain a reference to internal data structures after an Environment is fully closed. This report was originally mentioned in this forum post. [#16453]


9. Fixed a circular static initialization dependency between LockType and LockUpgrade. This was discovered by a user whose use of reflection caused a change in the class loading order. [#16496]


10. Fixed a bug that causes incorrect secondary key values to be created when a partial DatabaseEntry is passed to a Cursor or Database put() method, if secondaries are configured. Before, the partial entry was passed to the SecondaryKeyCreator. Now, the resulting merged data is passed to the key creator. Thanks to archie172 for reporting this on OTN. [#16929]


11. A bug in which the DbPrintLog utility would sometimes display negative numbers for the size of the key/data bytes when using the -S option has been fixed. A -SC option has also been added which prints the summary information in CSV format. [#17196]


12. Fixed a bug where com.sleepycat.je.util.DbPrintLog could throw a NullPointerException. [#17456]


13. Fixed a bug that would allow the Total Memory Usage stat to become negative on certain architectures. [#17462]


14. Removed a concurrency hot spot in TxnManager.thread2Txn, which has a positive influence on highly concurrent applications with many transactions . [#17542]


15. Two new environment configuration parameters are available to control the use of "proactive migration" in the log cleaner:
    • EnvironmentConfig.CLEANER_FOREGROUND_PROACTIVE_MIGRATION
    • EnvironmentConfig.CLEANER_BACKGROUND_PROACTIVE_MIGRATION
    See the javadoc for these parameters for more information. By default, proactive migration is now disabled. In earlier releases, proactive migration was unconditionally enabled for both foreground and background activities. This sometimes caused operation delays and long checkpoints, and these problems should now be reduced. But because of this change in behavior, in rare cases an application may now need to configure more cleaner threads, or enable proactive migration, in order to prevent a cleaner backlog. [#17634]


16. In rare cases requiring a combination of delete and repeated insert operations and environment recoveries, Database.count() could be incorrect, usually by a small number of records. Note that Database.count() does have the general caveat that is it not guaranteed to be accurate when there concurrent updates. This miscount has been fixed. [#17770] (4.0.61)


17. In JE 4.0.60, read operations such as Database.get(), when executed on a replicated environment, and used with a null transaction parameter, did not properly obey the default environment replica read consistency policy. This has been fixed. [#16513] (changed in 4.0.70)


18. Fixed a bug that sometimes caused a NullPointerException in a replicated environment, when a database was renamed on the Master. If a secondary database was renamed, the app running on the Replica could see a NullPointerException rather than the intended DatabasePreemptedException. [#17015] (changed in 4.0.70)


19. Fixed a bug where a LogFileNotFoundException could be seen in an environment which had repeated inserts and deletes of records with the same key and repeated environment recoveries. [#17879] (changed in 4.0.70)


20. A bug could cause replicated environments which contain databases with duplicates to see the following exception with this distinctive stacktrace:

    (JE 4.0.60)... LOG_FILE_NOT_FOUND: Log file missing, log is likely
     invalid. Environment is invalid and must be closed.
    	at com.sleepycat.je.tree.ChildReference.
    fetchTarget(ChildReference.java:147)
    	at com.sleepycat.je.tree.DIN.getDupCountLN(DIN.java:155)
    	at com.sleepycat.je.dbi.CursorImpl.lockDupCountLN(CursorImpl.java:2596)
    	at com.sleepycat.je.tree.Tree.insertDuplicate(Tree.java:2705)
    	at com.sleepycat.je.tree.Tree.insert(Tree.java:2652)
    	at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1076)
    	at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1769)
    	at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1726)
    	at com.sleepycat.je.Cursor.putNotify(Cursor.java:1659)
    	at com.sleepycat.je.Cursor.putLN(Cursor.java:1609)
    	at com.sleepycat.je.DbInternal.putLN(DbInternal.java:125)
    	at com.sleepycat.je.rep.impl.node.Replay.applyLN(Replay.java:738)
    	at com.sleepycat.je.rep.impl.node.Replay.replayEntry(Replay.java:483)
    	at com.sleepycat.je.rep.impl.node.Replica.doRunReplicaLoopInternalWork(Replica.java:414)
    	at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:341)
    	at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:283)
    	at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:886)
    

    The problem would only occur in a fairly tight timing window, and has been fixed. [#17879] (changed in 4.0.70)


21. A replicated environment which had been running in a replication group and restarted as a replica could see this exception at startup time:

    
     (JE 4.0.60) ... GroupCBVLSN: 231,655 is outside VLSN range: first=231,700 last=583,909 sync=583,909 txnEnd=583,909
    UNEXPECTED_STATE_FATAL: Unexpected internal state, unable to continue. Environment is invalid and must be closed.
    Problem seen replaying entry ...
    	at com.sleepycat.je.EnvironmentFailureException.unexpectedState(EnvironmentFailureException.java:392)
    	at com.sleepycat.je.rep.impl.node.GlobalCBVLSN.recalculate(GlobalCBVLSN.java:179)
    	at com.sleepycat.je.rep.impl.node.RepNode.recalculateGlobalCBVLSN(RepNode.java:650)
    	at com.sleepycat.je.rep.impl.node.Replay.replayEntry(Replay.java:444)
    	at com.sleepycat.je.rep.impl.node.Replica.doRunReplicaLoopInternalWork(Replica.java:414)
    	at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:341)
    	at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:283)
    	at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:886)
    
    

    This has been fixed. [#17885] (changed in 4.0.70)

1. Report a meaningful IllegalArgumentException when @Persistent is incorrectly declared on an enum class. Before, the confusing message Persistent class has non-persistent superclass: java.lang.Enum was reported. Thanks to Lowell for reporting this problem on OTN. [#15623]


2. Report a meaningful IllegalArgumentException when @Persistent is incorrectly declared on an interface. Before, a NullPointerException was reported. Thanks to Tony Clifton for reporting this problem on OTN. [#15841]


3. Report a meaningful IllegalArgumentException when @Persistent or @Entity is incorrectly used on an inner class (a non-static nested class). Before, the confusing message No default constructor was reported. [#16279]


4. Entity subclasses that define secondary keys must now be registered prior to storing an instance of the class. This can be done in two ways:
   • The EntityModel.registerClass method may be called to register the subclass before opening the entity store.
   • The EntityStore.getSubclassIndex method may be called to implicitly register the subclass after opening the entity store.

   Failure to register the entity subclass will result in an IllegalArgumentException the first time an attempt is made to store an instance of the subclass. An exception will not occur if instances of the subclass have previously been stored, which allows existing applications to run unmodified in most cases.

   This behavioral change was made to increase reliability. In several cases, registering an entity subclass has been necessary as a workaround, as described here, for example. The requirement to register the subclass will ensure that such errors do not occur in deployed applications. [#16399]



5. When a secondary index database is opened, it is now auto-populated only if it did not previously exist, rather than if it is empty. This prevents repeated auto-population of the secondary from occurring when the secondary keys for all entities in the primary index happen to be null. This problem was reported by jhalex on OTN. Note that the workaround mentioned in the OTN thread -- to create a dummy record that contains a non-null value for each secondary key -- is no longer necessary.

   To go along with this change, the EntityStore.truncateClass method now removes all secondary databases rather than truncating them, so that they will be auto-populated if they are accessed again. The primary database is truncated, as before. [#16399]



6. The com.sleepycat.collections.TransactionRunner.handleException method has been added to allow overriding the default transaction retry policy. See the javadoc for this method for more information. [#16574]


7. Fixed a bug that causes an assertion to fire or a NullPointerException (when assertions are disabled) from the EntityStore constructor. The problem occurs only when the previously created EntityStore contains an entity with a secondary key definition in which the key name has been overridden and is different than the field name. The key name can be overridden with the name property of the SecondaryKey annotation. Below are examples of the assertion and exception.

   java.lang.NullPointerException
       at com.sleepycat.persist.impl.ComplexFormat.checkSecKeyMetadata(ComplexFormat.java:1143)
       at com.sleepycat.persist.impl.ComplexFormat.evolveFieldList(ComplexFormat.java:1428)
       at com.sleepycat.persist.impl.ComplexFormat.evolveAllFields(ComplexFormat.java:1245)
       at com.sleepycat.persist.impl.ComplexFormat.evolve(ComplexFormat.java:1019)
       at com.sleepycat.persist.impl.Evolver.evolveFormatInternal(Evolver.java:440)
       at com.sleepycat.persist.impl.Evolver.evolveFormat(Evolver.java:248)
       at com.sleepycat.persist.impl.PersistCatalog.(PersistCatalog.java:357)
       at com.sleepycat.persist.impl.Store.(Store.java:180)
       at com.sleepycat.persist.EntityStore.(EntityStore.java:165)
   

   java.lang.AssertionError
       at com.sleepycat.persist.impl.ComplexFormat.evolveFieldList(ComplexFormat.java:1327)
       at com.sleepycat.persist.impl.ComplexFormat.evolveAllFields(ComplexFormat.java:1245)
       at com.sleepycat.persist.impl.ComplexFormat.evolve(ComplexFormat.java:1019)
       at com.sleepycat.persist.impl.Evolver.evolveFormatInternal(Evolver.java:440)
       at com.sleepycat.persist.impl.Evolver.evolveFormat(Evolver.java:248)
       at com.sleepycat.persist.impl.PersistCatalog.(PersistCatalog.java:357)
       at com.sleepycat.persist.impl.Store.(Store.java:180)
       at com.sleepycat.persist.EntityStore.(EntityStore.java:165)
   

   Thanks to Lukasz Antoniak for reporting this bug accurately and clearly on OTN. [#16819]


8. Key cursors have been optimized to significantly reduce I/O when the READ_UNCOMMITTED isolation mode is used. See EntityIndex.keys for more information. [#16859]


9. Report a meaningful IllegalArgumentException when NULLIFY is used with a @SecondaryKey and the field is a primitive type. Before, the confusing message Key field object may not be null was reported. Thanks to Erick for reporting this problem on OTN. [#17011]


10. Enum fields may now be used as DPL keys, including primary keys, secondary keys, and fields of composite key classes. Comparators are supported for composite key classes containing enum fields.

    A bug was also fixed that prevented enum values from being inserted before existing values in an enum declaration. Now, new values may be inserted before existing values in the declaration, as well as appended to the end of the declaration. However, note that renaming and deletion of enum values is not supported. [#17140]



11. Fixed a bug that prevented the use of custom key comparisons (composite key classes that implement Comparable) for secondary keys defined as ONE_TO_MANY or MANY_TO_MANY. An example stack trace is below.

    java.lang.IllegalArgumentException: Key class is not a simple type or a composite key class (composite keys must include @KeyField annotations): java.util.Set
    at com.sleepycat.persist.impl.PersistKeyBinding.(PersistKeyBinding.java:38)
    at com.sleepycat.persist.impl.Store.getKeyBinding(Store.java:1294)
    at com.sleepycat.persist.impl.Store.setBtreeComparator(Store.java:1312)
    at com.sleepycat.persist.impl.Store.getSecondaryConfig(Store.java:1115)
    at com.sleepycat.persist.impl.Store.openSecondaryIndex(Store.java:666)
    at com.sleepycat.persist.impl.Store.openSecondaryIndexes(Store.java:636)
    at com.sleepycat.persist.impl.Store.getPrimaryIndex(Store.java:384)
    at com.sleepycat.persist.EntityStore.getPrimaryIndex(EntityStore.java:259)

    [#17207]


12. Improve performance of StoredCollection.removeAll. This method no longer iterates over all records in the stored collection. Thanks to ambber on OTN for suggesting and testing this improvement. [#17727]


13. A hot upgrade of an application (without taking down the replication group) is now allowed when persistent DPL classes have been changed. Such changes take place when the DPL schema has been updated and class evolution is required. Before, to deploy such a change, the entire replication group had to be bought down. For more information see "Upgrading a Replication Group" in the javadoc package summary for the com.sleepycat.persist.evolve package. [#16655] (changed in 4.0.70)

1. Changed DbPrintLog -S so that it can display values in the trillions. Fixed a bug that didn't allow -S or -q to come before any other flags on the command line. [#16370]


2. Fixed a bug which caused an InvocationTargetException to be displayed (masking the real exception) when invoking a utility with the je.jar file in this way:

     java -jar je.jar <UtilityName>
   

   [#16649]

1. JE support in Android has been improved by the addition of a lib/je-android.M.N.P.jar file. Rather than requiring the Android application developer to modify and recompile the JE sources so that references to javax.transaction.xa.* classes are removed, the jar file can be simply included in the project's libs directory. The JE and Android directions reflect these changes. (changed in 4.0.70)


2. The Monitoring JE with JConsole and JMX directions had a small typo. The sentence "(e.g. using -DJEMonitor-true on the command line)" has been corrected to say "-DJEMonitor=true". (changed in 4.0.70)