These notes describe the difference between Derby release 10.3.1.4 and the preceding release 10.2.2.0.
Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs.
Derby functionality includes:
- Embedded engine with JDBC drivers
- Network Server
- Network client JDBC drivers
- Command line tools: ij (SQL scripting), dblook (schema dump) and sysinfo (system info)
Derby 10.3.1.4 runs on the following platforms:
- JDK 1.4 and Java 5 - Here Derby supports the JDBC 3.0 interface.
- Java 6 - Here Derby supports the JDBC 4.0 interface.
- CDC/Foundation 1.1 - Here Derby supports the JSR 169 interface.
This is a minor release. The following table lists new features that were added and notable improvements that were made:
Feature | Description | Issue Id(s) |
DBA Powers | Control who can shutdown, encrypt and upgrade databases. | Master JIRA: DERBY-2264 |
Secure Server | Make the Network Server secure by default. | Master JIRA: DERBY-2196 |
Language Based Ordering | Add built in language based ordering and like processing to Derby. | Master JIRA: DERBY-1478 |
Alter Table | You can now DROP or RENAME a column. Together with a number of enhancements in 10.2, this means that most dynamic schema modifications are now possible | DERBY-396 DERBY-1489 DERBY-1490 DERBY-1926 DERBY-1909 DERBY-2042 |
SSL/TLS | Implement SSL/TLS communication between client and server | DERBY-2108 DERBY-2356 DERBY-2272 DERBY-2273 |
Blob/Clob API | Support all JDBC API methods for Blob/Clob, both for embedded driver and client driver | DERBY-1341 , DERBY-1285 , DERBY-1286 , DERBY-2443 , DERBY-2444 , DERBY-2730 |
Client Side Tracing | Provide a way to enable client tracing without changing the application | DERBY-1275 |
Import/Export of Blob/Clob | Add support for import/export of tables with clob, blob and other binary type columns | DERBY-378 |
JDBC methods for autogenerated keys | Implement JDBC methods for autogenerated keys for Embedded | DERBY-2631 |
CREATE TABLE AS | Enable Create of a new empty table based upon a sub query | DERBY-64 |
XATransaction timeout | Support for XAResource.setTransactionTimeout | Master JIRA: DERBY-2432 |
SYSCS_UTIL.SYSCS_SET_USER_ACCESS SYSCS_UTIL.SYSCS_GET_USER_ACCESS | Add a system procedure to set a user's connection level authorization. | DERBY-2735 |
SYSCS_UTIL.SYSCS_EMPTY_STATEMENT_CACHE | Create a procedure to empty the statement cache exposing the existing functionality. | DERBY-2772 |
support for diagnostic vti tables that take parameters | Implement support for diagnostic vti tables that take parameterss. | DERBY-2152 |
FOR EACH/MODE DB2SQL in CREATE TRIGGER optional | Make FOR EACH clause and MODE DB2SQL phrase in CREATE TRIGGER statement optional. | DERBY-1953 |
ANSI TRIM | Add ANSI TRIM implementation | DERBY-1623 |
Performance | Reduce CPU usage in embedded Derby. Main areas being worked on are: Lock manager and latching, reduced use of synchronized data structures, optimize use of bit sets. | DERBY-1704 DERBY-2107 DERBY-2149 DERBY-2150 DERBY-2191 DERBY-2226 |
Performance | Improve Derby's treatment of IN-lists to allow the optimizer to consider using indexes when appropriate. | DERBY-47 |
Testing | Switch testing to be pure JUnit based. | DERBY-1952 & many others |
Memory Usage | Avoid having to materialize entire LOBs in network client. The client will use locators when requesting operations to be performed on LOBs stored on the server side. | Master JIRA: DERBY-208 |
Platforms | Minimum JDK support will change to JDK 1.4.2 for J2SE & CDC/Foundation 1.1 for J2ME. (Removes support for JDK 1.3 and J2ME/CDC/Foundation 1.0) | DERBY-1983 DERBY-1985 DERBY-2121 |
The following issues are addressed by Derby release 10.3.1.4. These issues are not addressed in the preceding 10.2.2.0 release. This list includes bugs and improvements, and sub-tasks if the super-task is not fixed in 10.3.1.4, but not issues with only test or web content changes.
Issue Id Description DERBY-2973 With collation TERRITORY_BASED, insert into table after changing type of column causes assert failure and loss of connection DERBY-2966 loss of connection with TERRITORY_BASED collation. DERBY-2963 AccessControlException connection from remote client. DERBY-2960 Group by substr() on collated database causes ERROR XJ001 DERBY-2955 ERROR 42ZA2 creating table with check constraint with literal comparison DERBY-2941 With 10.2, Closing a resultset after retrieving a large > 32665 bytes value with Network Server does not release locks DERBY-2925 Prevent export from overwriting existing files DERBY-2882 Remove references to JDK 1.2 and 1.3 in the documentation DERBY-2868 BUILDING.txt doesn't mention junit.jar in list of jars installed in tools/java DERBY-2858 Export exceptions swallow useful information DERBY-2849 Add a documentation for derby.jdbc.xaTransactionTimeout system/database property DERBY-2837 Update docs on STRONG_PASSWORD_SUBSTITUTE_SECURITY/ENCRYPTED_USER_AND_PASSWORD_SECURITY and JCE support DERBY-2809 Expressions with a parameter can be assigned the incorrect type DERBY-2806 calling getByteLength on org.apache.derby.impl.jdbc.StoreStreamClob makes BinaryStream, fetched before this call, unusable DERBY-2794 Document ansi trim functionality DERBY-2793 Ensure LIKE predicate follows correct rules for determing collation DERBY-2789 DatabaseMetaData .locatorsUpdateCopy() should return true DERBY-2784 With JDBC 4 autoloading DriverManager.getProtocol("jdbc:derby:") throws java.sql.SQLException No suitable driver DERBY-2753 org.apache.derby.impl.drda.DDMWriter might swallow unexpected exceptions DERBY-2748 TimeSlice and Socket-Timeout bounds checking wrong DERBY-2737 Change documentation on permissions needed to include read/write for system property derby.storage.jvmInstanceId DERBY-2694 org.apache.derby.impl.drda.DDMWriter uses wrong algorithm to avoid spliting varchar in the middle of a multibyte char. DERBY-2692 Client driver doesn't chain exceptions received from the server DERBY-2689 Deadlock with GenericPreparedStatement DERBY-2686 The skip method for some InputStreams and Readers return invalid values DERBY-2673 If derby.system.home does not exist Derby should only attempt to create that specific folder, not any missing parents (ie. use File.mkdir(), not File.mkdirs()) DERBY-2671 Errors/messages early in starting the network server through NetworkServerControl.start() are not reported to the PrintWriter passed into start(). DERBY-2654 Document newly-supported (in embedded mode) JDBC methods for autogenerated keys. DERBY-2652 Clob.setCharacterStream differs between embedded and client driver DERBY-2619 A Derby source release must include the documentation source files DERBY-2613 upgrade test problem when attempting to test 10.0.2.1 - DERBY-2610 Queries in metadata.properties allow tablepattern for JDBC methods that do not allow patterns DERBY-2607 DatabaseMetaData is not consistent about throwing SqlException when tablename=null DERBY-2606 Derby should print the parameters to failed statements to the derby.log when it logs the error DERBY-2604 Implement Clob support for locators DERBY-2603 Minor erratum in page of VARCHAR in Derby Reference manual DERBY-2597 Language result sets should not reuse current isolation level across executions DERBY-2594 Revoking a privilege from an SQL Object should invalidate statements dependent on that object DERBY-2593 Add documentation for the CREATE TABLE as subquery clause DERBY-2591 DataDictionaryImpl.getSystemSQLName() may generate duplicates DERBY-2584 Creating a database with JPOX SchemaTool sometimes gives ArrayIndexOutOfBoundsException when getIndexInfo() is called DERBY-2581 Callers of SanityManager.THROWASSERT should chain the exceptions when possible DERBY-2580 SanityManager.THROWASSERT(String,Throwable) ignores message argument DERBY-2579 AssertFailure class should use JDK's built-in chaining of exceptions DERBY-2570 Create a utility which generates Release Notes DERBY-2566 OutOfMemory/Sanity-assert failed when updating database DERBY-2558 client trhows ArrayIndexOutOfBounds exception instead of parameter out of range DERBY-2556 Code paths for db restore do not use doPrivileged-calls, causing SecurityException DERBY-2551 Global Xid value garbled in syscs_diag.transaction_table. DERBY-2549 ArrayIndexOutOfBoundsException in SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE DERBY-2538 Update documentation to describe the expected behavior when a JDBC 4 app creates a JDBC 3 datasource. DERBY-2526 Wrong results with queries that use the JOIN ... ON syntax to join with views or other non-base table expressions. DERBY-2521 Building derby outputs (from ant) various information messages that are marked as warning severity. DERBY-2520 Document new restrictions of database shutdown, encryption and hard upgrade powers DERBY-2511 reference manual's description of JDBC4 features has misleading sections DERBY-2501 Batch scripts in bin\ report extraneous errors when DERBY_HOME is invalid DERBY-2500 Assertion failure preparing query with AND and OR in where clause DERBY-2493 Use unsynchronized collections in BackingStoreHashtable DERBY-2489 Document the policy-reloading system procedure. DERBY-2488 When loaded by jvm1.6 - EmbeddedConnectionPoolDataSource is not returning a JDBC 4 compliant PooledConnection object DERBY-2480 DriverManager.getConnection leaks memory when connecting to a non-existent database DERBY-2472 Use Throwable.initCause() to improve error reporting DERBY-2462 org.apache.derby.impl.store.access.BackingStoreHashTableFromScan does not honor ResultSet holdability DERBY-2459 Ordering on a CASE-expression casues a NullPointerException when using a UNION DERBY-2456 File stream is left open when an exception occurs while setting up a character stream for data export. DERBY-2450 Clob.Position returning wrong value when operating on Reader DERBY-2444 Implement not implemented methods Blob.getBinaryStream(long pos, long length) and Clob. getCharacterStream(long pos, long length) in the Network Client DERBY-2443 Implement ResultSet updateClob/updateBlob methods on the NetworkClient DERBY-2431 Documentation for DatabaseMetaData should reflect that getColumnPrivileges and getTablePrivileges are implemented DERBY-2425 ResultSetMetaData.getColumnDisplaySize() returns a negative value for BLOB columns for client DERBY-2406 XAResource.end does not set the XA transaction state correctly when the XAException is thrown DERBY-2400 Javadoc - clean up Cloudscape references in javadoc DERBY-2391 "Derby and standards" section of Developer's Guide needs reorganization DERBY-2390 DOCS - Merge Working with Derby and Getting Started Guide DERBY-2387 DOCs - Reorder the topics in the Ref Manual into Alphabetical order - functions, procedures DERBY-2386 timestampdiff function fails when using SQL_TSI_FRAC_SECOND for datepart parameter, except for very small intervals DERBY-2383 ReuseFactory should use the constants in java.lang.Boolean DERBY-2381 ParameterMappingTest fails due to ArrayIndexOutOfBoundsException executing a procedure DERBY-2377 Document language based ordering which will be implemented by code related sub-tasks of DERBY-1478. DERBY-2376 Patch available to make .classpath entries portable - relative to ECLIPSE_HOME DERBY-2372 Document the secure-by-default network server DERBY-2371 Setting a default value for a VARCHAR column fails when column contains data DERBY-2370 EXISTS may return the wrong value for sub-queries involving set operations DERBY-2369 NetworkServerControl.shutdown() takes at least 1.5 seconds, could be faster. DERBY-2365 Brushing up pages for MAX and MIN in Derby Reference Manual DERBY-2364 improve documentation to explain logged/unlogged operations DERBY-2361 Documentation should give examples for using the different security mechanisms DERBY-2355 Wrong URL in Eclipse-Plugin Lab Example Jays.java DERBY-2350 Use of XML values in the action statement of a trigger throw exceptions. DERBY-2331 Disallow code in installed jars from resolving classes in the org.apache.derby.* namespace except for public apis. DERBY-2330 Disallow user-defined SQL routines to resolve to entry points (methods in classes) in the org.apache.derby.* namespace DERBY-2323 Update Graphic in Dev Guide - Embedded deployment application overview DERBY-2301 Documentation of different executeBatch error handling between embedded and DerbyNetClient needed DERBY-2296 getProperties method deprecated on ClientDataSource DERBY-2281 Update the Tuning Guide figure about using the statement cache DERBY-2279 JDBC3 driver is loaded instead of JDBC4 when running with jdk1.7 DERBY-2275 XSLT changes for PDF output DERBY-2272 SSL Documentation DERBY-2263 Update the copyright dita files to mark Derby logo images as non-substantive images (for accessibility) DERBY-2262 DatabaseMetaData.getTypeInfo returns incorrect MAXIMUM_SCALE value for DECIMAL and NUMERIC types DERBY-2260 DatabaseMetaData.getTypeInfo() returns incorrect precision for VARCHAR FOR BIT DATA DERBY-2259 DatabaseMetaData.getTypeInfo() SEARCHABLE column returns incorrect information for types that cannot be searched. DERBY-2258 DatabaseMetaData.getTypeInfo() does not list supported Derby SQL types correctly. DERBY-2256 Wrong Results: Use of decimal values in an IN-list with INTEGER left operand can lead to extra rows. DERBY-2244 DatabaseMetaData.supportsExpressionsInOrderBy() returns false DERBY-2243 DatabaseMetaData.supportsANSI92EntryLevelSQL() returns false for embedded, true for client driver DERBY-2237 Cleanup copyrights in the DITA source and generated docs DERBY-2230 AssertFailure: ByteCode Conditional then/else stack mismatch DERBY-2228 Support Derby on J2ME/CDC/Foundation 1.1 DERBY-2226 Move column bitset computation to IndexToBaseRowNode DERBY-2223 Let BasePage.fetchFieldFromSlot use the special single-col FetchDescriptor ctor DERBY-2222 'show indexes in SCHEMANAME' does not work with the client driver DERBY-2220 Uncommitted transactions executed throught XAResource will held locks after the application terminates (or crashes during the transaction). DERBY-2218 Null Pointer Exception when an untyped NULL subquery ("values null") appears outside of the FROM list in a SELECT query. DERBY-2216 Allow demo SimpleApp to work in J2ME environment DERBY-2214 Fix Getting Started file to reflect classpath change DERBY-2208 setNetworkServerCP scripts need not add derby.jar into the CLASSPATH DERBY-2202 DROP PROCEDURE depends on SET SCHEMA DERBY-2195 Nested triggers not working properly after maximum trigger count exception is thrown DERBY-2193 [import] ERROR 38000: StringIndexOutOfBoundsException was thrown while evaluating an expression. DERBY-2191 Cleanup of FormatableBitSet DERBY-2183 Trigger recompilation problem when trigger action has its table not qualified with a schema DERBY-2166 Implement proper handling of SocketTimeoutException in DRDAConnThread DERBY-2152 Support diagnostic vti tables that take parameters, such as SpaceTable DERBY-2150 Reduce use of synchronized collections in GenericLanguageConnectionContext DERBY-2149 Replace Vectors and Hashtables with ArrayLists and HashMaps in RAMTransaction DERBY-2147 LIKE predicate does not accept a pure column reference as righthand operand (gives ERROR 42824) DERBY-2141 BlobClob4BlobTest.testPositionBlob() fails with NullPointerException DERBY-2138 Remove DataDictionaryContext and associated code DERBY-2137 CALL (PROCEDURE) statement documentation in reference manual has incomplete syntax for arguments DERBY-2124 Incorrect method name in error message for Connection.setTransactionIsolation method DERBY-2123 Remove workaround for old JIT bug from StoredPage DERBY-2122 Optimize ContainerLock.isCompatible() DERBY-2121 Remove JDK 1.3 build dependency in network server DERBY-2118 Change some boundary checks in ArrayInputStream to ASSERTs to improve performance DERBY-2114 Let Clock embed a HashMap rather than inherit from Hashtable DERBY-2112 Nullpointer on executeBatchRequestX when preparedStatement has no parameters DERBY-2107 Move page latching out of the lock manager DERBY-2103 After a Lexical Error due to syntax error , even a simple create table does not work on the same connection. DERBY-2099 Make BasePage.getPageId() final DERBY-2096 Change the Parser.parserStatement() to return the more specific StatementNode instead of QueryTreeNode. DERBY-2093 Error in initSlotTable() can cause NPE or ASSERT rather than reporting page number in corrupt page message. DERBY-2088 Update the documentation templates. Add comment about changing the reference ID DERBY-2083 Shutting down a database loaded from a jar leaves an open file reference to the jar file containing the database. DERBY-2067 Assert failure in EmbedConnection.restoreContextStack() when running lang/closed.java DERBY-2060 SET CURRENT ISOLATION in ref.man refers java.sql.Connection.setTransactionLevel instead of java.sql.Connection.setTransactionIsolation DERBY-2057 SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE documentation or implementation error on its arguments. DERBY-2053 Dev Guide: Syntax errors in SQL tips -> Tricks of the VALUES clause -> Multiple rows DERBY-2050 Manipulating CachedItems could be more efficient DERBY-2046 Make class org.apache.derby.iapi.store.raw.PageKey final DERBY-2042 Provide documentation for new RENAME COLUMN statement DERBY-2040 Setting derby.database.classpath to contain installed jars causes the database to be unbootable when a Securitymanager is installed. DERBY-2019 IJ's describe command does not handle quotes very well DERBY-2018 NullPointerException in CREATE VIEW ... VALUES NULL; DERBY-2001 Add DITA templates for the 3 topic types into the trunk DERBY-1995 Add base schema scripts for order entry DERBY-1993 Check in the demo used by the Java in the Database session at Apachecon 2006 DERBY-1983 Change build system so that base level is JDK 1.4 DERBY-1965 NetworkServerControlImpl never closes the socket or streams it opens in setUpSocket. DERBY-1964 Update the documentation of SYSCS_UTIL.SYSCS_COMPRESS_TABLE for the changes that went in as part of DERBY-737 DERBY-1959 10.2 'Derby Developer's Guide' error/ambiguity. DERBY-1956 Remove stale code from the statement classes in the client driver DERBY-1955 Unquoted path in .bat files may cause errors (Win) DERBY-1953 Make FOR EACH clause and MODE DB2SQL in CREATE TRIGGER statement optional DERBY-1949 locate function documentation should clarify behavior when first parameter is empty string DERBY-1947 OutOfMemoryError after repeated calls to boot and shutdown a database DERBY-1942 There exists difference between behavior of setNull(Types.TIME) and setTiime(null). DERBY-1935 Reference Manual - Derby Limitations DERBY-1934 Reference Manual updates - J2EE Compliance: Java Transaction API and javax.sql Extensions DERBY-1930 Move JDBC implementation notes into the published javadoc DERBY-1929 SYSTABLEPERMS and SYSCOLPERMS documentation needs to be updated DERBY-1928 Update table "Support for SQL-92 Features: Basic schema manipulation" for GRANT/REVOKE DERBY-1926 Provide documentation for ALTER TABLE DROP COLUMN DERBY-1922 readme.html under frameworks does not mention about Derby client and some minor typos in example.html DERBY-1920 DOCS - Improve topic titles for vague and duplicate topics DERBY-1919 Top level index.html page should link to release notes and other items in top-level of a release. DERBY-1918 INCREMENT of IDENTITY column described as allowing a value of zero in reference manual DERBY-1917 Clob.position fails with Embedded driver and large Clobs DERBY-1890 improve XSDFI error message DERBY-1879 Save meta data related information for an EmbedResultSet at the plan level instead of the ResultSet level improves performance. DERBY-1868 Merge argument descriptors into SQLState strings so that SQLState documentation can be generated by a program DERBY-1861 Column ordering ASSERT when combining column references and expressions in same ORDER BY DERBY-1852 Wrong results: duplicate rows returned for nested UNIONs when they should be eliminated. DERBY-1847 SELECT statement asserts with XJ001 when attempted to select a newly added column in SQL authorization mode DERBY-1828 Access rule violations should use a SQL state starting with '42' according to the SQL standard. DERBY-1822 LOCK TABLE example and description in reference manual should get replaced by a 'real' example DERBY-1816 Client's ResultSet.getTime() on a SQL TIMESTAMP column loses the sub-second resolution and always has a milli-second value of zero. DERBY-1815 In admin guide examples to start network server on windows with .bat scripts, $DERYBY_INSTALL Is used , I think it should be %DERBY_INSTALL% DERBY-1805 Links to element ids inside a topic are broken in PDFs and HTML Books DERBY-1797 Building toursdb would go a little faster if it used autocommit off mode. DERBY-1795 Graphics not copied over for PDF and HTML-single manuals DERBY-1765 Update ALTER TABLE documentation to reflect DERBY-119 (ALTER COLUMN [NOT]NULL) DERBY-1753 Doc for SYSCS_INPLACE_COMPRESS_TABLE has incorrect procedure name in the java examples. DERBY-1752 Fix javadoc to account for changes required by new licence header policy. DERBY-1709 Deprecate scripts in frameworks directory DERBY-1673 Compling with jikes not longer works due to recent changes that added -target -source command line flags to each compile DERBY-1665 Incorrect JavaDoc for Qualifier interface DERBY-1662 Document derbyrun.jar DERBY-1644 NPE when inserting values to a table that has a column declared as generated by default as identity DERBY-1624 use of direct column name rather than alias make aggregation fail (Hibernate depends on that) DERBY-1623 Add ANSI TRIM implementation DERBY-1620 SQL CASE statement returns ERROR 42X89 when including NULL as a return value DERBY-1610 Resolve difference of type compatibility between Embedded and NetworkServer/NetworkDriver DERBY-1590 Consolidate the *conrefs.dita files in the documentation source tree to a single file. DERBY-1589 CREATE TABLE throws NullPointerException in Derby SQL Standard Authorization after DROPs and REVOKES DERBY-1588 Link "Getting Started...." and "Apache Derby Server ...." in demo.html needs to be linked to actual documents instead of manuals page DERBY-1570 The derby configuration, logging and diagnostic properties such as derby.language.logStatementText are hard to find in the documentation DERBY-1535 Trial 2 for DERBY-550, improve use of Engine from NetworkServer and reduce memory usage DERBY-1526 build should be able to locate the Java runtime libraries from properties not sourced from ${user.home}, but inside the current subversion checkout. DERBY-1520 Document new SYSCS_DIAG tables DERBY-1519 'setAsciiStream' uses different encodings for embedded and client DERBY-1501 PreparedStatement#setNull(int parameterIndex, int sqlType) throws SQL Exception if given sqlType is LONGVARBINARY in embedded mode DERBY-1494 PreparedStatement.setNull(int, int) checks type compatibility on embedded, but not on the client DERBY-1490 Provide ALTER TABLE RENAME COLUMN functionality DERBY-1489 Provide ALTER TABLE DROP COLUMN functionality DERBY-1484 Client and embedded behave differently when the table name is null in DatabaseMetaData methods DERBY-1440 jdk 1.6 client driver omits SQLStates and chained exceptions in error messages DERBY-1434 Client can send incorrect database name to server after having made multiple connections to different databases. DERBY-1381 Document ij.exceptionTrace property DERBY-1377 Update copyright headers to comply with new ASF policy DERBY-1355 ClientDriver ResultSetMetaData.isAutoIncrement(column) always returns false DERBY-1285 Finish JDBC3 Blob implementation DERBY-1275 Provide a way to enable client tracing without changing the application DERBY-1132 Truncation Error with Concat DERBY-1054 Starting Derby with the NetServlet inside of tomcat does not allow binding to non localhost interface. DERBY-931 Until DERBY-911 gets fixed, document the difference in behavior between Nework Client Driver and Embedded Driver for setReadOnly DERBY-908 YEAR,SECOND,MONTH, MINUTE, HOUR and DAY functions have incorrect information on durations. DERBY-815 Prevent unneeded object creation and excessive decoding in parseSQLDTA_work() DERBY-806 One each deleted or updated from a heap row a new RowPosition object is created. DERBY-801 Allow parallel access to data files. DERBY-790 SQLException used by the networked interface to Derby is not serializable DERBY-729 Scalar time and date functions return 0 instead NULL when argument is NULL DERBY-681 Eliminate the parser's rewriting of the abstract syntax tree for queries with GROUP BY and/or HAVING clauses DERBY-680 In ij, executing a prepared statement with numeric/decimal parameter fails with NullPointerException in J2ME/CDC/FP DERBY-630 create trigger fails with null pointer exception DERBY-606 SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails on (very) large tables DERBY-595 Using derby.language.logStatementText=true can mask certain exceptions and lead to incorrect behavior in some cases DERBY-552 Fetching resources using getResourceAsStream from a jar stored in a database that is archived in a jar file fails DERBY-538 Investigate using the standard java.net.URLClassLoader for database class loading. DERBY-537 SQLJ.INSTALL_JAR and SQLJ.UPDATE_JAR fail when running with a SecurityManager enabled. DERBY-415 sysinfo with -cp client option should not print error saying DB2 jar file and driver class are missing DERBY-396 Support for ALTER STATEMENT to DROP , MODIFY, RENAME a COLUMN DERBY-391 Tools and Utilities guide does not document ij.datasource, ij.user, nor ij.password DERBY-378 support for import/export of tables with clob/blob and the other binary data types will be good addition to derby, DERBY-264 This enhancement to allow expressions in ORDER BY clause will require documentation changes. DERBY-234 Documentation of DateTime types is incomplete DERBY-208 Add support to retrieve lobs for Network Server by locator rather than matierializing the LOB DERBY-183 Parameter names required in CREATE FUNCTION DERBY-147 ERROR 42X79 not consistant ? - same column name specified twice DERBY-64 Create a table with a query DERBY-47 Some possible improvements to IN optimization
Compared with the previous release (10.2.2.0), Derby release 10.3.1.4 introduces the following new features and incompatibilities. These merit your special attention.
Note for DERBY-2925: Prevent export from overwriting existing files
Note for DERBY-2757: Security enhancements to the Network Server may slow down Derby's runtime performance, and they may trigger SecurityExceptions when Derby executes user-written functions and procedures.
Note for DERBY-2729: Blob and Clob objects are released when the transaction ends and when the connection is closed.
Note for DERBY-2610: The table name can no longer be a pattern in calls to DatabaseMetaData methods getBestRowIdentifier, getColumnPrivileges, getIndexInfo, getVersionColumns, getPrimaryKeys, getImportedKeys, getExportedKeys and getCrossReference.
Note for DERBY-2526: Queries which use the JOIN ... ON syntax to join with views or other non-base table expressions may now return different results.
Note for DERBY-2443: Added unimplemented methods introduced in the java.sql.ResultSet interface.
Note for DERBY-2430: The application will see an error in the event of calling setObject(int targetType, Blob source) and setObject(int targetType, Clob source) with a targetType other than Blob and Clob. This will be in conformance with the embedded behaviour.
Note for DERBY-2386: The return type of the timestampdiff function has been changed from INT to BIGINT.
Note for DERBY-2370: EXISTS predicates with subqueries that use set operators (UNION, INTERSECT, EXCEPT) may now behave differently.
Note for DERBY-2296: ClientDataSource now supports the createDatabase and shutdownDatabase properties.
Note for DERBY-2264: Henceforth, when authentication is enabled (
derby.connection.requireAuthentication
has the valuetrue
) and SQL Authentication is enabled (derby.database.sqlAuthentication
has the valuetrue
) some database level operations are restricted to the database owner.Note for DERBY-2256: Use of decimal values in an IN predicate whose left operand is an INTEGER may now return different results.
Note for DERBY-2196: The user should customize the security policy which the Network Server now installs by default.
Note for DERBY-2108: SSL/TLS implemented for client/server communication.
Note for DERBY-2020: Writing of the transaction log to disk has been changed to open log files in "rwd" mode instead of "rws" if the JVM supports it.
Note for DERBY-1942: The use of the TIME data type is more restricted.
Note for DERBY-1852: Queries with nested set operators (UNION, INTERSECT, EXCEPT) in a FROM list may now return different results.
Note for DERBY-1828: Most authorization failures have new error codes.
Note for DERBY-1816: ResultSet.getTime() on an SQL TIMESTAMP value now has millisecond resolution with the Derby client driver.
Note for DERBY-1610: In a number of usage cases of setNull(int, int) and set*(int, null, int) methods for PreparedStatement and CallableStatement, Derby's Client implementation has been changed, to correctly behave in the same way as the Embedded implementation.
Note for DERBY-1519: Streams obtained by calling get-/setAsciiStream in the client driver use encoding "ISO-8859-1" instead of "US-ASCII".
Note for DERBY-1484: The table name can no longer be null in calls to DatabaseMetaData methods getBestRowIdentifier, getColumnPrivileges, getIndexInfo, getVersionColumns, getPrimaryKeys, getImportedKeys and getExportedKeys.
Note for DERBY-1341: Added unimplemented methods introduced in the JDBC 3.0 and 4.0 specification. Blob and Clob now support all the methods of JDBC 3.0 and 4.0.
Note for DERBY-729: Scalar time and date functions should return NULL when the argument is NULL
Note for DERBY-415: Sysinfo now presents separate command switches to check the classpath for the presence of the Derby client and the DB2 JCC driver.
Note for DERBY-208: Network Client: Locator-based implementation of Blob/Clob operations.
Note for DERBY-2925
Summary of Change
Prevent export from overwriting existing files
Symptoms Seen by Applications Affected by Change
Due to security concerns, and to avoid accidental file damage, Export processing will not overwrite existing files. As a result, applications may observe some behavioral changes with respect to Export operations.
Incompatibilities with Previous Release
- SYSCS_UTIL.SYSCS_EXPORT_TABLE: Exports of all the data from a table to an existing file is no longer possible and the user/application must remove the existing file, otherwise ERROR XIE0S will be returned.
- SYSCS_UTIL.SYSCS_EXPORT_QUERY: Exports of all the data returned from the SELECT statement to an existing file is no longer possible and the user/application must remove the existing file, otherwise ERROR XIE0S will be returned.
- SYSCS_UTIL.SYSCS_EXPORT_QUERY_LOBS_TO_EXTFILE: Export of the result of a SELECT statement to a main data output file, and the LOB data into a large object auxiliary file to an existing file is no longer possible and the user/application must remove the existing data file and/or large object auxiliary file, otherwise ERROR XIE0S will be returned if the data file exists or ERROR XIE0T will be returned if the large object auxiliary file exists.
Rationale for Change
Due to security concerns and to avoid accidental file damage, Export processing will no longer overwrite an existing file.
Application Changes Required
The application needs to specify a different filename which does not exist or delete the existing file before performing the Export operation.
Note for DERBY-2757
Summary of Change
Security enhancements to the Network Server may slow down Derby's runtime performance, and they may trigger SecurityExceptions when Derby executes user-written functions and procedures.
Symptoms Seen by Applications Affected by Change
Startup performance for networked applications may degrade after upgrading to 10.3. In addition, after upgrade, user-written functions and procedures may raise SecurityExceptions.
Incompatibilities with Previous Release
In previous releases, the Network Server booted without installing a default security manager. Now, the Network Server installs a security manager if the user forgets to. Application startup may slow down as the security manager performs initial access checks on Derby tables. Once all user tables have been touched, the application should reach steady state and the drag induced by the security manager should be negligible.
In addition, SecurityExceptions may occur if user-written functions and procedures perform sensitive operations such as reading/writing files and getting/setting system properties.
Rationale for Change
In a client/server configuration, it is particularly important to protect against other users' mistakes and hacking. Derby is enhancing security for this configuration. Now if you neglect to configure a Java security manager, the Network Server attempts to install its own security manager with a default policy.
Application Changes Required
SecurityExceptions can be avoided by installing your own security manager with your own policy file, which grants the appropriate privileges to your user-written code. Instructions on how to do this can be found in the Derby Developer's Guide in the section titled "Running Derby under a security manager" and in the Derby Server and Administration Guide in the section titled "Running the Network Server under the security manager".
If for some reason this is not practical, or if the startup performance drag is intolerable, then you can instruct the server to not install a security manager. You do this by booting the server with the "-noSecurityManager" command line option as explained in above-mentioned section of the Derby Server and Administration Guide.
Note for DERBY-2729
Summary of Change
Blob and Clob objects are released when the transaction ends and when the connection is closed.
Symptoms Seen by Applications Affected by Change
Applications now get a SQLException with SQL STATE XJ215 when accessing Blobs and Clobs after the transaction ends or the connection is closed.
Incompatibilities with Previous Release
In the previous release, Blobs and Clobs were sometimes usable even after the transaction ended or the connection was closed. Now Blobs and Clobs are not usable after these events.
Rationale for Change
Now Blobs and Clobs store their data in temporary files. These temporary files are deleted when the transaction ends. The Blob.free() and Clob.free() methods also delete these temporary files. This erases the transient state of these large objects and makes them unusable.
Application Changes Required
Applications should be revised to not use Blobs and Clobs after the transaction ends or the connection is closed.
Note for DERBY-2610
Summary of Change
The table name can no longer be a pattern in calls to DatabaseMetaData methods getBestRowIdentifier, getColumnPrivileges, getIndexInfo, getVersionColumns, getPrimaryKeys, getImportedKeys, getExportedKeys and getCrossReference.
Symptoms Seen by Applications Affected by Change
In the previous release, the methods returned information on all tables that matched the pattern in the schema. A table name value of null was treated as a wildcard. Table names now have to match what is stored in the database.
Incompatibilities with Previous Release
Calls to the specified methods now fail if the table name parameter is a pattern (no information is returned).
Rationale for Change
The previous behavior violated the JDBC standard. The new behavior is correct.
Application Changes Required
Users must recode applications to specify the table names. When information on multiple tables is required, the application first has to get the table names, e.g. by using the DatabaseMetaData method getTables(), and then use the returned table names as input to the method in question.
Note for DERBY-2526
Summary of Change
Queries which use the JOIN ... ON syntax to join with views or other non-base table expressions may now return different results.
Symptoms Seen by Applications Affected by Change
Applications which use the explicit JOIN ... ON syntax to perform joins between three or more FROM expressions, at least one of which is a view, subquery, or other non-base table expression, may have been seeing incorrect results prior to this release.
As an example, take the following:
create table t1 (c0 int);
create table x (c1 int, c2 int);
create table t2 (c3 int, c4 int);
insert into t1 values 1;
insert into x values (0, 1);
insert into t2 values (0, 2);With these tables, the following query should return one row, but was returning zero rows in previous releases:
select t1.* from t1 JOIN (select * from x) vw(c1,c2) ON (t1.c0 = vw.c2) JOIN t2 ON (vw.c1 = t2.c3)
This problem has been fixed in Derby 10.3.
Incompatibilities with Previous Release
The fix for the bug shown above does not introduce any functional incompatibilities. However, applications may now see different results if they happen to use queries that rely on the JOIN ... ON syntax to join with views, subqueries, or other non-base table expressions.
Rationale for Change
Due to an error in column reference mappings, it was possible for earlier versions of Derby to confuse ON predicate column references with each other, thereby leading to incorrect computation of transitive closure. This in turn could lead to the addition of invalid predicates to the query, which caused wrong results in certain cases.
By fixing this bug we ensure that the affected queries will always return the correct results.
Application Changes Required
No application changes should be needed.
Note for DERBY-2443
Summary of Change
Added unimplemented methods introduced in the java.sql.ResultSet interface.
Symptoms Seen by Applications Affected by Change
Existing application won't be effected by it as these methods are new implementations. Applications won't be getting Not Implemented exception anymore.
Rationale for Change
To add implementations for unimplemented Blob and Clob related methods in the ResultSet interface.
Application Changes Required
Applications can now make use of the following new methods
void updateBlob(int columnIndex, Blob x) throws SQLException void updateBlob(String columnName, Blob x) throws SQLException void updateClob(int columnIndex, Clob x) throws SQLException void updateClob(String columnName, Clob x) throws SQLException void updateBlob(int columnIndex, InputStream x, long length) throws SQLException void updateBlob(String columnName, InputStream x, long length) throws SQLException void updateClob(int columnIndex, Reader x, long length) throws SQLException void updateClob(String columnName, Reader x, long length) throws SQLExceptionDetailed description of these methods can be found in api docs of Java SE 6: ResultSet.
Note for DERBY-2430
Summary of Change
Derby's Client implementation will return an error matching the behavior with Embedded in the event when setObject(int targetType, Blob source) and setObject(int targetType, Clob source) is called with a targetType other than Blob and Clob.
Symptoms Seen by Applications Affected by Change
The application will see an error in the event of calling setObject(int targetType, Blob source) and setObject(int targetType, Clob source) with a targetType other than Blob and Clob. This will be in conformance with the embedded behaviour.
Rationale for Change
Derby's Client implementation and Embedded implementation should behave the same way from the point of view of an application whenever possible.
Application Changes Required
Applications relying on the Client behaving in the old way will have to be adjusted.
Note for DERBY-2386
Summary of Change
The return type of the timestampdiff function has been changed from INT to BIGINT.
Symptoms Seen by Applications Affected by Change
In the previous release(s), using the builtin timestampdiff function using the SQL_TSI_FRAC_SECOND for the datepart parameter would return an integer overflow error except with very small intervals, less then a second, because the result would exceed the range of an INT. For intervals larger than a second, SQL_TSI_SECOND had to be used. Now, the result type has been changed to BIGINT and so one can use intervals > 1 sec with SQL_TSI_FRAC_SECOND. However as a result of the change SQL functions that take the result of the timestampdiff function as a parameter of datatype INT will no longer be resolved.
Incompatibilities with Previous Release
As a result of the change, the result type for the timestampdiff function is now BIGINT. Applications which use the return value as a parameter to an SQL function will find that the SQL function can no longer be resolved.
Rationale for Change
The previous behavior required the application developer to know ahead of time the range for the timestampdiff. Returning BIGINT enables the function to cope with a larger range.
Application Changes Required
SQL functions taking the result of a timestampdiff function as a parameter will have to be modified to take a BIGINT parameter instead of INT.
Note for DERBY-2370
Summary of Change
EXISTS predicates with subqueries that use set operators (UNION, INTERSECT, EXCEPT) may now behave differently.
Symptoms Seen by Applications Affected by Change
Applications which specify set operations inside of an EXISTS predicate may have been seeing incorrect results for such queries. For example, the following query should return a single row with column "OK":
select * from ( values 'OK' ) as T where exists (values 1 except values 2)Prior to Derby 10.3, though, that query would incorrectly return zero rows.
Incompatibilities with Previous Release
Prior to Derby 10.3 a user could specify "SELECT *" within an EXISTS predicate's set operation and Derby took the "*" to be a single column. As a result, some queries would compile and execute without error even when they should have failed. As an example, the following query would succeed even if table T2 had more than one column:
select * from ( values 'OK') as T where exists (select i from T1 union select * from T2)As of version 10.3 Derby no longer allows this. Changes to fix the incorrect query results make it so that the above query will now throw an error (42X58) if T2 has more than one column, because then the left and right result sets of the union would not have the same number of result columns.
Rationale for Change
Due to the way in which Derby internally handles result columns within an EXISTS predicate, it was possible for queries having EXISTS predicates with set operators to return incorrect results. By internally rewriting the EXISTS subquery when it is a set operator, we can ensure that Derby now evaluates such predicates correctly. For more on the details of the rewrite, see DERBY-2370.
Application Changes Required
As a result of the new internal rewrite, any "*" that appears within the EXISTS set operation will now be properly expanded. This means that applications which specify set operations inside of an EXISTS predicate must now ensure that the result sets to which a given set operation applies have the same number of result columns. See the "incompatibilities" section above for details. No other application changes are required.
Note for DERBY-2296
Summary of Change
ClientDataSource now supports the createDatabase and shutdownDatabase properties.
Symptoms Seen by Applications Affected by Change
The functionality is new for ClientDataSources. Previously, applications could not call these methods. Now these methods work:
- setCreateDatabase (String)
- getCreateDatabase()
- setShutdownDatabase (String)
- getShutdownDatabase()
These methods should behave similarly to the ones in Embedded, i.e. only valid value for setCreateDatabase is "create", and for setShutdownDatabase is "shutdown". In essence, at connection time, if set to a valid value, the property is passed on to the server side with the ConnectionAttributes.
Note, that the result for setting contradicting properties for createDatabase and ShutdownDatabase, whether through setCreateDatabase("create") and setShutdownDatabase("shutdown") or via setConnectionAttributes("create=true") or ("shutdown=true") is undefined.Incompatibilities with Previous Release
No incompatibilities were introduced.
Rationale for Change
With version 10.2.1.6 an incompatibility with 10.1.1.0 was introduced - since revision 393003, the undocumented, non-standard but previously public method, getProperties for Client DataSources was removed for security reasons. This left only the setConnectionAttributes method available for creating or shutting down a database, and caused hardship for applications that had previously taken advantage of the getProperties method. For instance, in Geronimo the getProperties method was previously used to inquire and set whether a create=true was set for a database.
Support for the four methods already existing for EmbeddedDataSource was requested for ClientDataSource to compensate for the disappearance of getProperties as a public method.Application Changes Required
Application code will need to be adjusted if they want to take advantage of these new methods for ClientDataSources.
Note for DERBY-2264
Summary of Change
Henceforth, when authentication is enabled (
derby.connection.requireAuthentication
has the valuetrue
) and SQL Authentication is enabled (derby.database.sqlAuthentication
has the valuetrue
) some database level operations are restricted to the database owner.Symptoms Seen by Applications Affected by Change
When connecting to an existing database toan SQLException with SQLState "08004" is thrown. For the English locale the exception string will be one of:
- shut down the database
- encrypt a previously unencrypted database
- re-encrypt an encrypted database with a new bootPassword or a new encryption key
- perform a full (as opposed to soft) upgrade of the database from a previous version
- User user cannot shut down database database. Only the database owner can perform this operation.
- User user cannot (re)encrypt database database. Only the database owner can perform this operation.
- User user cannot hard upgrade database database. Only the database owner can perform this operation.
Incompatibilities with Previous Release
In release 10.2, any authenticated user would be able to:
In 10.3, if both authentication and sqlAuthentication are enabled, an application which tries to perform any such operation by connecting as any user other than the database owner, will see an SQLException with SQLState "08004" and the operation will not be performed.
- Shut down the database.
- Encrypt a previously unencrypted database.
- Re-encrypt an encrypted database with a new bootPassword or a new encryption key.
- Perform a full (as opposed to soft) upgrade of the database from a previous version.
The check on full upgrade pertains to upgrades from 10.2.* (and subsequent) releases of Derby only. When upgrading from 10.0 or 10.1 any valid user can still do the full upgrade. This user then becomes the database owner. The reason for this difference is that the database owner concept was first introduced in 10.2. Note that once you upgrade, the database owner can not be changed.
Rationale for Change
These changes were introduced to enhance Derby security by limiting operations which impact all users of a database - with potentially far-reaching effects - to the data base owner.
Application Changes Required
Impacted applications must now be changed to perform these operations as the database owner, that is, supplying the authorization identifier (i.e. user name) of the database owner when connecting to perform these operations.
Note: The database owner is identical to the authorization identifier (i.e. user name) used when the database was created or user name used when upgrading from 10.0 or 10.1 as the case may be. If the database was created or upgraded without supplying a user name (authentication not enabled) the database owner defaults to "APP".
The following query can be used to show the database owner:
select authorizationid from sys.sysschemas where schemaname = 'SYS';
Note for DERBY-2256
Summary of Change
Use of decimal values in an IN predicate whose left operand is an INTEGER may now return different results.
Symptoms Seen by Applications Affected by Change
Applications which use an IN predicate to see if an integer value is contained within a list of decimal values may have been seeing incorrect results prior to this release. In some cases rows may have been missing from the result; in other cases, additional (incorrect) rows may have been returned.
As an example, take the following:
create table t1 (i int);
insert into t1 values 1, 2, 3, 4, 5;
-- Following query was returning zero rows when it should return 1 row.
select * from t1 where i in (4.23, 4);
-- Following query should return zero rows but was returning 1 row. select * from t1 where i in (2.8, 4.23);
This problem has been fixed in Derby 10.3.
Incompatibilities with Previous Release
The fix for the bug shown above does not introduce any functional incompatibilities. However, applications may now see different results if they happen to use queries which rely on an IN predicate to check for integer values within a list of decimal values.
Rationale for Change
Derby's behavior in previous releases was incorrect and could lead to wrong results.
Application Changes Required
No application changes should be needed.
Note for DERBY-2196
Summary of Change
The user should customize the security policy which the Network Server now installs by default.
Symptoms Seen by Applications Affected by Change
When booted from the command line, the Network Server now installs a security manager with a default policy. This policy does not expose the application to any additional risks. However, the policy is overbroad and the user should customize it in order to reduce security threats.
Rationale for Change
Derby is providing more security support.
Application Changes Required
Instead of relying on the default policy installed by the Network Server, the user should further limit the scope of privileged operations. In particular, the user should fine-tune the blanket read/write privilege granted on the entire server file system. The user should narrow this privilege to just the directories needed for backup/restore, import/export, and jar file loading. For instructions on how to refine the policy file, please consult the Derby Server and Administration Guide section titled "Customizing the Network Server's security policy".
This is also an opportunity for the user to enable user authentication if the Network Server currently runs without authentication. Running in a client/server configuration without authentication exposes the application and the server machine to many threats. It is strongly discouraged. For instructions on enabling user authentication, please consult the Derby Developer's Guide section titled "Working with user authentication".
Note for DERBY-2108
Summary of Change
SSL/TLS implemented for client/server communication.
Symptoms Seen by Applications Affected by Change
Several error messages have been changed to reflect failure scenarios which may involve SSL:
SQLExceptions:
The message
ERROR 58009: A communications error has been detected
has been extended to give the underlying cause. E.g. if the underlying cause is an SSL problem, you get
ERROR 58009: A communications error has been detected. Unrecognized SSL message, plaintext connection?
The message
ERROR 58009: A network protocol error was encountered and the connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected. Reason: 0x3.
has been changed to
ERROR 58009: A network protocol error was encountered and the connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected. Reason: 0x3. Plaintext connection attempt to an SSL enabled server?
Other error messages:
If the server socket can't be established when a server is started, the message
Could not listen on port NNNN on host XXXX
has been extended to give the underlying cause, e.g:
Could not listen on port NNNN on host XXXX: java.net.BindException: Address already in use
If a plaintext server is connected by an SSL enabled client, you will see messages like this in the server log:
Execution failed because of a Distributed Protocol Error: DRDA_Proto_SYNTAXRM; CODPNT arg = 0; Error Em Value = 3. Plaintext connection attempt from an SSL enabled client?
If a plaintext administration command is used towards an SSL server, the message
Invalid reply header from network server: Invalid string xxxx.
has been changed to
Invalid reply header from network server: Invalid string xxxx. Plaintext connection attempt to an SSL enabled server?
Incompatibilities with Previous Release
Applications which rely on the content of error message texts may fail.
Rationale for Change
The messages had to be extended due to more failure scenarios when connecting a client to a Derby server.
Application Changes Required
Users must recode applications to recognize the changed messages.
Note for DERBY-2020
Summary of Change
Writing of the transaction log to disk has been changed to open log files in "rwd" mode instead of "rws" if the JVM supports it.
Symptoms Seen by Applications Affected by Change
With JVMs/OSs that support the functionality, such as JDK version 1.4.2 and up on Solaris, a performance improvement may be noticed. On other platforms, no change may be noticeable.
Rationale for Change
For writing the transaction log to disk Derby uses a RandomAccessFile. Before this change, if it was supported by the JVM, the log files were opened in "rws" mode making the file system take care of syncing writes to disk. "rws" mode ensured that both the data and the file meta-data was updated for every write to the file. On some operating systems (e.g. Solaris) this lead to two write operations to the disk for every write issued by Derby. This was limiting the throughput of update intensive applications. By changing the file mode to "rwd" the number of updates to the disk is reduced. Some JVMs have a bug in the support for "rws" and "rwd" mode. Derby will check for this bug, and if it is detected, Derby will revert back to using "rw" mode and print an appropriate message indicating this in derby.log.
Application Changes Required
No changes are needed or possible to benefit from this change. If your JVM supports it, it will be used. If Derby detects that your JVM has a bug in the support for "rwd", a message will be printed to derby.log:
------------ BEGIN ERROR MESSAGE -------------
LogToFile.checkJvmSyncError: Your JVM seems to have a problem with implicit syncing of log files. Will use explicit syncing instead.
------------ END ERROR MESSAGE -------------
Note for DERBY-1942
Summary of Change
The use of the TIME data type is more restricted.
Symptoms Seen by Applications Affected by Change
Applications might encounter an error that indicates incompatible data types, when values are passed using the TIME data type.
Incompatibilities with Previous Release
In previous releases, applications could call the setTime method for variables that use the TIMESTAMP data type, if the passing value was null. Starting with this release, calling the setTime method fails for variables that use the TIMESTAMP data type, regardless of the value that is passed.
Moreover, applications were allowed to pass values that used the TIME data type to a parameter of the DATETIMEDIFF function in the previous release. Starting with this release, instead of using the TIME data type, the TIMESTAMP data type must be used to pass a parameter of the DATETIMEDIFF function.
Rationale for Change
The TIME data type is classified as a data type for time information. Unlike the TIMESTAMP and DATE data types, the TIME data type is special case. The TIME data type does not represent a specific instant or one time frame but represents all instants, every day. For example, if TIME is 20:42, this represents 20:42 regardless of the date. 20:42 today, 20:42 tomorrow, and so forth. Starting with this release, the TIME data type is regarded completely different from the TIMESTAMP data type.
Application Changes Required
Applications using the TIME data type must use the TIMESTAMP data type instead, if date information is needed besides time information.
Note for DERBY-1852
Summary of Change
Queries with nested set operators (UNION, INTERSECT, EXCEPT) in a FROM list may now return different results.
Symptoms Seen by Applications Affected by Change
The Derby documentation indicates that if the 'ALL' keyword is not specified when using a set operator, the default behavior is to remove duplicate rows from the result. There is, however, a specific type of query in which previous versions of Derby would incorrectly include duplicate rows, even if "ALL" was not specified. The queries in question have all of the following characteristics:
- A chain of two or more set operators appears in the FROM list of a SELECT query; and
- There exists at least one nested set operator in the chain which:
a) is *not* the top-level set operator in the chain, and
b) which does *not* include "ALL"; and- The result of the set operator from #2 includes duplicate rows; and
- None of the set operators which sit above the set operator from #2 in the query tree removes duplicates.
A query which satisfies all of these conditions would return incorrect results (duplicate rows) in previous releases. Consider the following simple example:
create view vw(i) as values 1, 2 union values 2 union all values 3;
-- Both of the following queries are equally affected by this issue.
select * from vw;
select * from (values 1, 2 union values 2 union all values 3) x
In this example the chain of unions is as follows:
union_1 all / \ union_2 values 3 / \ values 1, 2 values 2Note that union_2 is nested within union_1, and that union_2 has duplicate values ("2"). Also note that union_1 has an "ALL" specified and thus does not remove duplicates. So we've satisfied the above criteria and we would expect that, since union_2 does not have the "ALL" keyword, the duplicate "2" rows should be removed before evaluation of union_1. But in previous releases of Derby this removal of duplicates did not happen. Thus the query would incorrectly return four rows, with the value "2" being duplicated.
Incompatibilities with Previous Release
The fix for the bug shown above does not introduce any functional incompatibilities. However, applications may see different results if they happen to use queries which satisfy the conditions outlined above. Such applications may also see slightly increased execution times for the relevant queries due to the fact that Derby internally sorts the nested results to remove duplicates.
Rationale for Change
The previous behavior gave incorrect results.
Application Changes Required
No application changes should be needed.
Note for DERBY-1828
Summary of Change
Most authorization failures have new error codes.
Symptoms Seen by Applications Affected by Change
In the previous release, authorization failures had error codes 2850x and 04501. In this release, most of these errors have new error codes. The code changes are: 04501, 2850H, 2850I and 2850J are now 08004. 28506-2850G are now 42500-4250A, 28501 is now 4250B, 28503-28505 are now 4250C-4250E. Only the error codes have been changed; error messages are not affected.
Incompatibilities with Previous Release
Applications that are dependant on authorization error codes may fail.
Rationale for Change
The old error codes violated the SQL standard. The new error codes are correct.
Application Changes Required
Applications that are dependant on authorization error codes must be recoded to expect the new codes.
Note for DERBY-1816
Summary of Change
ResultSet.getTime() on an SQL TIMESTAMP value now has millisecond resolution with the Derby client driver.
Symptoms Seen by Applications Affected by Change
JDBC applications which a) use the Derby client driver, and b) use ResultSet.getTime() to retrieve a java.sql.Time object from an SQL TIMESTAMP value would, in previous releases, always get a java.sql.Time object whose milliseconds field was zero. This would be the case even though a Derby SQL TIMESTAMP has nanosecond resolution.
With this release the Derby client driver now matches the embedded driver in that a call to ResultSet.getTime() on an SQL TIMESTAMP will return a java.sql.Time object with the correct millisecond resolution.
Incompatibilities with Previous Release
This change does not introduce any functional incompatibilities. Applications using the Derby client driver will now see correct millisecond values when they call ResultSet.getTime() to retrieve an SQL TIMESTAMP.
Rationale for Change
While it is true that a Derby SQL TIME value has by definition resolution of only a second, the java.sql.Time class is not a direct mapping to the SQL Type. Rather, it's a JDBC type, and the JDBC java.sql.Time class has a precision of milliseconds. So when retrieving a java.sql.Time value from an SQL TIMESTAMP, Derby should retain the millisecond precision.
Note that the Derby embedded driver correctly retained millisecond precision in previous releases. With the latest release, the client driver now matches that behavior.
Application Changes Required
No application changes should be needed.
Note for DERBY-1610
Summary of Change
In a number of usage cases of setNull(int, int) and set*(int, null, int) methods for PreparedStatement and CallableStatement, Derby's Client implementation has been changed, to correctly behave in the same way as the Embedded implementation.
Symptoms Seen by Applications Affected by Change
In a number of method calls involving null values for PreparedStatement and CallableStatement, Derby's Client implementation behaved differently from the Embedded implementation. Now, in most cases, the Client returns an error where Embedded returns an error and succeeds where Embedded succeeds. For instance:
Similar differences existed with the set<type>(int, null) calls.
- setNull(LONGVARCHAR) on parameter of type CHAR FOR BIT DATA (Types.BINARY). In previous releases, Embedded returned an error (SQLState 22005), but the Client did not. Now, both Embedded and Client will return an error.
- setNull(LONGVARBINARY) on parameter of type CHAR FOR BIT DATA (Types.BINARY). In previous releases, this call succeeded with Embedded, but the Client returned an error (SQLState 22005). Now, such a call will succeed with both Embedded and Client.
- setNull(LONGVARBINARY) on parameter of type VARCHAR FOR BIT DATA (Types.VARBINARY). In previous releases, the call succeeded with Embedded, Client returned an error (SQLState 22005). Now, such a call will succeed with both Embedded and Client.
- setNull(BINARY) on parameter of type LONG VARCHAR FOR BIT DATA (Types.LONGVARBINARY). In previous releases, such a call succeeded with Embedded, but Client returned an error (SQLState 22005). Now, such a call will succeed with both Embedded and Client.
- setNull(TIME) on parameter of TIMESTAMP. In previous releases, Embedded returned an error (SQLState 22005), but the Client did not. Now, both Embedded and client will return an error.
Note, that where an error is returned, the actual SQLState returned may still be different between the two implementations.Incompatibilities with Previous Release
Now, in the cases indicated above, the Client and Embedded implementation show the behavior previously shown only with Embedded.
Rationale for Change
Derby's Client implementation and Embedded implementation should behave the same way from the point of view of an application whenever possible.
Application Changes Required
Applications relying on the Client behaving in the old way will have to be adjusted.
Note for DERBY-1519
Summary of Change
Streams obtained by calling get-/setAsciiStream in the client driver use encoding "ISO-8859-1" instead of "US-ASCII".
Symptoms Seen by Applications Affected by Change
Strange symbols might appear in text, or text seems garbled. Where a question mark was printed previously, another character (printable or non-printable) may occur. The described symptoms only apply for the client driver.
Incompatibilities with Previous Release
Ascii streams in the client driver can now return 8-bit values in the range 0 - 255, whereas they previously returned 7-bit values in the range 0-127. As a consequence, all 8-bit values are also accepted as input.
Rationale for Change
The JDBC specification defines Ascii as values in the range 0 - 255. "US-ASCII" only contains 7-bit values, and cannot represent the full range. "ISO-8859-1" can represent the full range.
Application Changes Required
Applications accessing Ascii streams from Derby through the client driver must be extended to handle 8-bit values.
Note for DERBY-1484
Summary of Change
The table name can no longer be null in calls to DatabaseMetaData methods getBestRowIdentifier, getColumnPrivileges, getIndexInfo, getVersionColumns, getPrimaryKeys, getImportedKeys and getExportedKeys.
Symptoms Seen by Applications Affected by Change
In the previous release, a table name value of null was treated as a wildcard. Derby now raises an SQLException if the table name is null.
Incompatibilities with Previous Release
Calls to the specified methods now fail if the table name parameter is null (SQLException is thrown).
Rationale for Change
The previous behavior violated the JDBC standard. The new behavior is correct.
Application Changes Required
Users must recode applications to specify the table names. When information on multiple tables is required, the application first has to get the table names, e.g. by using the DatabaseMetaData method getTables(), and then use the returned table names as input to the method in question.
Note for DERBY-1341
Summary of Change
Added unimplemented methods introduced in the JDBC 3.0 and 4.0 specification. Blob and Clob now support all the methods of JDBC 3.0 and 4.0.
Symptoms Seen by Applications Affected by Change
Existing application won't be effected by it as these methods are new implementations. Applications won't be getting Not Implemented exception anymore.
Rationale for Change
To support all the methods in Blob and Clob introduced in the latest JDBC Specification.
Application Changes Required
Applications can now make use of following newly added methods
java.sql.Blob
InputStream getBinaryStream(long pos, long length) throws SQLExceptionjava.sql.Clob
OutputStream setBinaryStream(long pos) throws SQLException
int setBytes(long pos, byte[] bytes) throws SQLException
int setBytes(long pos, byte[] bytes, int offset, int len) throws SQLException
void truncate(long len) throws SQLException
Reader getCharacterStream(long pos, long length) throws SQLExceptionDetailed description of these methods can be found in api docs of Java SE 6
OutputStream setAsciiStream(long pos) throws SQLException
Writer setCharacterStream(long pos) throws SQLException
int setString(long pos, String str) throws SQLException
int setString(long pos, String str, int offset, int len) throws SQLException
void truncate(long len)throws SQLException
Note for DERBY-729
Summary of Change
Scalar time and date functions should return NULL when the argument is NULL
Symptoms Seen by Applications Affected by Change
In previous releases, the scalar time and date functions returned 0 when the argument is NULL. This has now been corrected.
Incompatibilities with Previous Release
With this release, the scalar time and date functions will return NULL when the argument is NULL.
Rationale for Change
The previous behavior was incorrect.
Application Changes Required
Applications that rely on the old behavior will have to be changed to expect/check for NULL as return value instead of 0.
Note for DERBY-415
Summary of Change
Sysinfo now presents separate command switches to check the classpath for the presence of the Derby client and the DB2 JCC driver.
Symptoms Seen by Applications Affected by Change
Previously, if you ran "java org.apache.derby.tools.sysinfo -cp client SimpleApp.class" you got a message indicating that the DB2 JCC driver was not found in your classpath, even though you had the Derby Client library in your classpath. Now "-cp client" only checks for the presence of the Derby client. To check for the presence of the DB2 JCC driver jar, you can pass the new "-cp db2driver" argument.
Incompatibilities with Previous Release
To check for the presence of the DB2 JCC driver, you must use the new "-cp db2driver" argument.
Rationale for Change
The behavior has changed in order to eliminate a confusing diagnostic message.
Application Changes Required
Users who want to check for the presence of the DB2 JCC driver will need to use the new command switch.
Note for DERBY-208
Summary of Change
Network Client: Locator-based implementation of Blob/Clob operations.
Symptoms Seen by Applications Affected by Change
Memory requirements for handling large Blob/Clob values in the Network Client are significantly reduced. As a result of this, applications may observe some behavioral changes with respect to Blob/Clob operations.
Incompatibilities with Previous Release
- Life span for Blob/Clob objects: Blob/Clob objects are no longer valid after the transaction in which they were created has been terminated (committed or rolled back).
- Stream access to Blob/Clob values: Streams obtained to read from Blob/Clob objects (E.g., Blob.getBinaryStream()), would in previous releases not reflect any changes made to the object after the stream was created. In Derby 10.3, it is unpredictable what a read operation on a stream will see of changes made between the creation of the stream and the time of the read operation. This means that applications that interleaves reads and writes of a Blob/Clob object may observe a changed behavior.
Rationale for Change
In earlier versions of the Derby Network Client, Blob and Clob operation caused the entire objects to be materialized in memory. For large values, this required large amounts of memory on the client side. To reduce the resource usage, a locator-based scheme has been implemented. The JDBC driver will hold references to server-side Blob/Clob values, and use these references to request the server to perform operations on the values. Hence, the Network Client will no longer need large amounts of memory too materialize large values in memory.
The incompatibilities are introduced because it was difficult to maintain the previous behavior when the Blob/Clob objects are no longer materialized in memory.
Application Changes Required
Applications that access Blob/Clob objects after the transaction they were created in has committed need to be changed. Make sure that the commit of the transaction is delayed until the objects are no longer needed. This means that if the same object is to be used in several statements, these statements will have to run in the same transaction (i.e., auto-commit may not be used in such cases). Also note that if auto-commit is used, a transaction will be committed when the result set is closed. Hence, when using auto-commit one must delay the closing of the result set until the Blob/Clob objects are no longer needed.
Open Bugs
For open bugs please refer to the JIRA bug tracking system: http://issues.apache.org/jira/browse/DERBY
However, of special interest in particular for existing applications are the following bugs open at the time of the 10.3.1.4 release:
Issue Id Description DERBY-2436 SYSCS_IMPORT_TABLE can be used to read derby files DERBY-2437 SYSCS_EXPORT_TABLE can be used to overwrite derby files DERBY-2892 Closing a resultset after retrieving a large > 32665 bytes value with Network Server does not release locks DERBY-2905 Shutting down embedded Derby does not remove all code, the AutoloadDriver is left registered in the DriverManager. Build Environment
Derby release 10.3.1.4 was built using the following environment:
- Branch - Source code came from the 10.3 branch.
- Machine - SunOS 5.11 snv_48.
- Ant - Apache Ant version 1.6.5 compiled on June 2 2005.
- JDK 1.4 - Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_12-b03).
- Java 6 - Java(TM) 2 Runtime Environment, Standard Edition (build 1.6.0-b105).
- OSGi - The osgi.jar was used to build org.apache.derby.osgi.EmbeddedActivator.
- Compiler - The 1.4.2_12-b03 javac was used to compile all classes except for the JDBC4 drivers. The JDBC4 driver classes were compiled using the 1.6.0-b105 javac.
- JSR 169 - J2ME support was built using java.sun.com/j2me (j2me_cdc_fp-1_0_2).