The change is forward compatible in that JE files created with release 3.2 and earlier can be read when opened with JE 3.3. The change is not backward compatible in that files created with JE 3.3 cannot be read by earlier releases. Note that if an existing environment is opened read/write, a new log file is written by JE 3.3 and the environment can no longer be read by earlier releases.
A number of these on-disk format changes were made to reduce disk-space usage. [#15399]
Also note the binary and compile time incompatibilities between JE 3.3 and JE 3.2 described in the API Changes sections.
The changes between 3.2 and 3.3.98 are described below. The changes are listed separately for each patch release, with the most recent patch release at the top.
Database.count()to return incorrect values when an environment is opened read-only. [#18180]
Database.preload()which prevents a latch on a database root node from being released. This occurs when the preload is incomplete because the cache fills or the preload takes more than the configured maximum time. The unreleased latch causes other operations on the database to block indefinitely. [#18396]
EVICT_LN. The new modes can help to improve cache memory usage and reduce Java GC costs for some applications. See the Javadoc for more information. [#18410]
ArrayIndexOutOfBoundsExceptionduring a checkpoint or a call to
Environment.sync(). The stack trace is similar to the following:
java.lang.ArrayIndexOutOfBoundsException: -1 IN.java:862 Checkpointer.java:1323 ...[#18474]
Exception in thread "Checkpointer" java.lang.AssertionError at com.sleepycat.je.cleaner.FileSelector.updateFilesAtCheckpointEnd(FileSelector.java:393) at com.sleepycat.je.cleaner.Cleaner.updateFilesAtCheckpointEnd(Cleaner.java:631) at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:534)[#18121]
EnvironmentStats.getCleanerBacklogstat is incorrect, which could lead the application to unnecessarily increase the number of cleaner threads.
EnvironmentConfig.CLEANER_MAX_BATCH_FILESis set to a non-zero value, log cleaning is disabled when the number of deleted files in the backlog reaches this limit.
fsync()calls on the same file may cause file corruption. 3.3.92 detects if JE is on Windows 7 and uses a workaround. [#17865]
java.util.concurrent.locks.ReentrantReadWriteLocktaken during construction of a
ThreadLocal. Changed our code to make the initialization lazy and conditional on the actual case where it is being used (stats collection). [#17944]
java.lang.NullPointerException at com.sleepycat.je.tree.LN.log(LN.java:534) at com.sleepycat.je.tree.LN.log(LN.java:373) at com.sleepycat.je.tree.BIN.logDirtyLN(BIN.java:1016) at com.sleepycat.je.tree.BIN.evictInternal(BIN.java:985) at com.sleepycat.je.tree.BIN.evictLNs(BIN.java:940) at com.sleepycat.je.evictor.Evictor.evictIN(Evictor.java:761) at com.sleepycat.je.evictor.Evictor.evictBatch(Evictor.java:344) at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:244) at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:269) at com.sleepycat.je.dbi.CursorImpl.close(CursorImpl.java:711) at com.sleepycat.je.Cursor.close(Cursor.java:326) at com.sleepycat.je.Database.putInternal(Database.java:999) at com.sleepycat.je.Database.put(Database.java:885)Thanks to tkram01 for reporting this on OTN. [#17551]
EnvironmentConfig.COMPRESSOR_PURGE_ROOThas been deprecated. This parameter no longer has any effect. Compression of the root node has no benefit and this feature has been removed. This feature was also bug-prone, as reported by penemue on OTN. [#17546]
DbDumpif neither the
-Rswitch is specified.
2009-04-27 17:27:09:132:MEST SEVERE Cleaner deleteSafeToDeleteFilesLog file 0xab could not be deleted. This operation will be retried at the next checkpoint com.sleepycat.je.log.LogException: (JE 3.3.x) Couldn't delete /envHome/000000ab.jdb at com.sleepycat.je.log.FileManager.deleteFile(FileManager.java:683) at com.sleepycat.je.cleaner.Cleaner.deleteSafeToDeleteFiles(Cleaner.java:530) at com.sleepycat.je.cleaner.Cleaner.updateFilesAtCheckpointEnd(Cleaner.java:642) at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:534) at com.sleepycat.je.recovery.Checkpointer.onWakeup(Checkpointer.java:257) at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:141) at java.lang.Thread.run(Thread.java:619[#17079] (3.3.81)
... Caused by: java.lang.NullPointerException: lsn1=21814149475543 lsn2=-1 at com.sleepycat.je.utilint.DbLsn.compareTo(DbLsn.java:76) at com.sleepycat.je.cleaner.RecoveryUtilizationTracker.isDbUncounted(RecoveryUtilizationTracker.java:151) at com.sleepycat.je.cleaner.RecoveryUtilizationTracker.countObsoleteIfUncounted(RecoveryUtilizationTracker.java:107) at com.sleepycat.je.recovery.RecoveryManager.redoUtilizationInfo(RecoveryManager.java:2322) at com.sleepycat.je.recovery.RecoveryManager.redoLNs(RecoveryManager.java:1247) ...[#16774] (3.3.77)
BufferOverflowExceptionwhile writing transactional records. This could occur if multiple threads were writing to an environment while using the same
java -jar je.x.y.z.jar DbPrintLog -h DIR -Sand examine the line labeled MapLN on the left. If the amount of the log taken by MapLNs is 10% or greater, or if you see this number increasing steadily over time, then your application is probably experiencing this problem.
By installing JE 3.3.75 or later, the excess disk space will automatically be reclaimed over time, as ordinary checkpoints and log cleaning occur. If you wish to recreate your database rather than wait for this to occur gradually, you can use DbDump and DbLoad to do so.
We'd like to express our appreciation and sincere thanks to Jules and the other folks at Xoopit who reported this problem and helped us to diagnose it. We would not have found or fixed this problem as quickly as we did without their help.[#16610]
This bug can occur only for databases with a Btree that is 4 or more levels deep. For the default maximum entries per Btree node (128), this means the database must have grown to at least 10 million records to be a candidate for occurrance of this problem. This is only an approximation and may be larger or smaller if DatabaseConfig.setNodeMaxEntries has been called.[#16523]
Environmentis fully closed, preventing the Java GC from reclaiming the memory for the closed environment. This can occur when locking is disabled. This problem was originally mentioned in this forum post. [#16453]
Caused by: java.lang.NullPointerException at com.sleepycat.je.recovery.RecoveryManager.redoUtilizationInfo(RecoveryManager.java:2276) at com.sleepycat.je.recovery.RecoveryManager.redoLNs(RecoveryManager.java:1247)[#16515]
Caused by: java.lang.NullPointerException at com.sleepycat.persist.impl.ComplexFormat.newInstance(ComplexFormat.java:478) at com.sleepycat.persist.impl.ComplexFormat$EvolveReader.newInstance(ComplexFormat.java:2003) at com.sleepycat.persist.impl.PersistEntityBinding.readEntity(PersistEntityBinding.java:88) at com.sleepycat.persist.impl.PersistEntityBinding.entryToObject(PersistEntityBinding.java:60) at com.sleepycat.persist.PrimaryIndex.get(PrimaryIndex.java:518This exception could also be caused by two class changes that evolve the entity twice, such that the second change reverts the first change to the original version of the class. In other words, this could occur when there are three versions of a class A, A-1, A-2 and A-3, and A-3 is identical to A-1.
Thanks to jhalex for reporting the problem in this OTN forum thread and helping to diagnose it. As mentioned in the forum thread, a workaround for this problem is to call EntityModel.registerClass for all entity subclasses.
transientkeywork for Java serialization. To override the normal rules for DPL field persistence, the @NotPersistent and @NotTransient annotations may be specified. Normally these keywords are not needed because the transient keyword is sufficient to mark a field as not persistent. However, some applications may wish for a field to be transient with respect to the DPL but not transient with respect to Java serialization, or vice-versa. The new annotations are provided for such applications. The feature was requested in this forum thread. [#16297]
getCacheMode. See the
com.sleepycat.je.CacheModeclass for more information. [#16239]
StoredMapclass now implements the standard
java.util.concurrent.ConcurrentMapinterface. In the 3.3.62 release, the
ConcurrentMapmethods were implemented by the
StoredMapclass, but it was not declared to implement the
In addition, two internal changes have been made to durable deferred-write database behavior. These changes are compatible with the existing use of deferred-write database, but may influence performance characteristics. These changes were necessary to ensure that the log cleaner operates properly for deferred-write databases.
return operation and byte counts for the number of random and sequential disk IOs. All values are approximate and may differ from the actual number of operations/byte-counts depending on the type of disks and file system, disk geometry, and file system cache size. [#16086]
EnvironmentStats.getNRandomReads() EnvironmentStats.getNRandomWrites() EnvironmentStats.getNRandomReadBytes() EnvironmentStats.getNRandomWriteBytes() EnvironmentStats.getNSequentialReads() EnvironmentStats.getNSequentialWrites() EnvironmentStats.getNSequentialReadBytes() EnvironmentStats.getNSequentialWriteBytes()
EnvironmentStats.getNFileOpens()can indicate a need to tune the size of the file handle cache through increasing the
je.log.fileCacheSizeproperty. Also, general improvements have been made to improve the file handle cache, and to remove a bottleneck in concurrency in that area. [#16166]
If the application used a custom comparator defined as
DatabaseConfig.setBtreeComparator(Class<? extends Comparator<byte>>)
DatabaseConfig.setDuplicateComparator(Class<? extends Comparator<byte>>)
public class MyCompare implements Comparator, Serializablethat should now be declared as
public class MyCompare implements Comparator<byte>, SerializablePlease note that while these are fine:
Comparator<byte> compareInstance = new MyCompare(); dbConfig.setBtreeComparator(compareInstance);or
dbConfig.setBtreeComparator(MyCompare.class);the following previously legal line will now provoke a compile error:
dbConfig.setBtreeComparator(compareInstance.getClass());The compile error is puzzling and regrettably breaks compile time compatibility with JE 3.2 and earlier, but comes about because in Java, due to its type erasure based generics scheme, all the instances of a generic class have the same runtime class. Instead, the application has to apply the following cast:
dbConfig.setBtreeComparator((Class<? extends Comparator<byte>>) compareInstance.getClass());