Changes:
Updated Snowflake driver to 3.16.1.
Updated Databricks driver to 2.6.36.
Updated BigQuery driver to 1.6.2.
Using INSERT with multiple values in
insertTable() for DataBricks for faster inserts.
Bugfixes:
Bugfixes:
Fixed dbFetch() for DBI drivers, no longer ignoring
n argument.
Fix bulk import for Postgres on MacOs.
Changes:
On Snowflake always using
QUOTED_IDENTIFIERS_IGNORE_CASE=TRUE to avoid name
mismatches when using quotes.
Updated Redshift drivers.
Added unit tests for all supported platforms.
Bugfixes:
Changes:
DatabaseConnectorConnection class is now defined at
runtime, preventing the “Found more than one class ‘Microsoft SQL
Server’” error when using the odbc package while
DatabaseConnector is installed.Bugfixes:
Fixing test for whether packages exist, which was causing errors on some Linux versions.
Fixing error about missing origin when fetching datetimes on older R versions.
Changes:
dbFetch() function now respects n = -1
and n = Inf arguments. Will throw warning if other value is
used.Bugfixes:
Fixing error about missing origin when fetching dates on older R versions.
Fixing RStudio connection panel information for DuckDB.
Changes:
Changing heuristic for detecting when almost running out of Java heap.
Setting default fetchRingBufferSize for RedShift to
100MB (instead of 1GB) to preven Java out of heap errors, and overall
better performance.
Using integers instead of strings to pass dates from Java to R for improved speed.
Using doubles instead of strings to pass datetimes from Java to R for improved speed.
Bugfixes:
Bugfixes:
rJava and rlang interaction
causing
no field, method or inner class called 'use_cli_format'errors
to be thrown when Java throws an error.Changes:
Can now use downloadJdbcDrivers() to download
BigQuery drivers (directly from Google).
Updating Spark JDBC driver to latest version.
Bugfixes:
Not asking to delete old files when not in interactive mode and
calling downloadJdbcDrivers().
Fixing error in RStudio table preview on Postgres when database name contains uppercase characters.
Several bugfixes for Spark.
Changes:
Bugfixes:
Fixed capacity < 0 error message when using a
large Java heap space.
Fixed ‘optional feature not supported’ error when connecting to DataBricks using JDBC.
Fixed insertTable() on Snowflake when data includes
POSIXct type.
Fixed ‘out of Java heap space’ when fetching data with (large) strings. This is achieved by checking the available Java heap space at every 10,000 rows, stopping the batch when less than half is still available. Additionally, all strings from the previous batch are de-referenced before starting a new batch.
Fixing DATETIME shifts on Snowflake.
Fixing unit tests for upcoming Andromeda version (using arrow instead of SQLite).
Changes:
Complete overhaul of DBI interface. DBI functions now apply
translation to all incoming SQL, thus making the DatabaseConnector DBI
interface an interface to a virtual database platform speaking OHDSISql
as defined in SqlRender. This should aid write once -
execute everywhere code development. It also allows using
dbplyr with DatabaseConnector.
Adding requiresTempEmulation() and
assertTempEmulationSchemaSet() functions.
The dropEmulatedTempTables() function can be called
on all platforms (it doesn’t do anything on platforms not requiring temp
table emulation).
Ensuring dbms(), insertTable(),
renderTranslateExecuteSql(),
renderTranslateQuerySql(), and
renderTranslateQuerySqlToAndromeda() functions work with
connection pool objects.
Adding logging at ‘TRACE’ level for performance profiling using
the ParallelLogger package. Use
options(LOG_DATABASECONNECTOR_SQL = TRUE) to enable. The
query times can be automatically extracted using the new
extractQueryTimes() function.
Adding the computeDataHash() function.
The inserTable() function now converts logical
fields to integer fields (and throws a warning if it does so).
The dbListTables() and getTableNames()
functions return results in lowercase instead of uppercase for
consistency with DBI functions.
Throwing informative error if user calls
createConnectionDetails() with credential arguments that
cannot be evaluated in a multi-threading setting.
Supporting external tables on RedShift.
Adding support for ODBC connections to Spark.
Querying to Andromeda when using a DBI driver (instead of a JDBC driver) now also uses batching to avoid running out of memory.
Adding appendToTable argument to
querySqlToAndromeda(),
renderTranslateQuerySqlToAndromeda(), and
lowLevelQuerySqlToAndromeda().
Bugfixes:
insertTable() and all column
names require quotes.Changes:
Supporting upcoming version of Andromeda using
arrow backend.
Adding new dbms() function, which is the preferred
way to establish the dbms of a connection.
Adding support for Snowflake.
Updating RedShift driver to V2.1.0.9. Fixes error when uploading data.
Bugfixes:
Fix erroneous dates when inserting dates in a tibble into a SQLite database.
Ensuring errors when inserting data turn into R errors.
Throwing informative error when using insertTable()
on BigQuery without specifying the
tempEmulationSchema.
Bugfixes:
connectionString is empty string
(instead of NULL).Changes:
Bugfixes:
Fixing ‘DBMS not supported’ error when connecting to Hive.
Fixing error when bulk uploading to Postgress with NULL values.
Fixing warning when automatically converting
Integer64 to numeric in R 4.2.0.
Changes:
Updating Spark JDBC driver in response to Log4J vulnerability.
Using new Andromeda functions for better abstraction.
More informative error messages when user forgets to provide a connection property (like a password).
Also deprecating oracleTempSchema in DBI
functions.
Adding existsTable() function.
Splitting vignette into two, because many users will only need to now how to connect to their database.
Improved error messaging related to the driver folder.
Bugfixes:
getTableNames() when the database or schema name
contains escaped characters.Changes:
dropEmulatedTempTables() function.Bugfixes:
Changes:
32-bit integers are also converted to numeric by default because (a) the output of some SQL functions (e.g. DATEDIFF) is INT on some platforms, BIG_INT on others, and (b) dplyr often throws errors when working with both integer and numeric vectors.
Reusing byte buffer when passing 64-bit integers from Java to R for efficiency.
Adding support for connecting to Spark.
Adding renderTranslateQueryApplyBatched()
function.
Throw informative error when provided dbms argument
does not match any of the expected values.
Adding getAvailableJavaHeapSpace() function for
debugging purposes.
Bugfixes:
Fixing field type of numeric fields on Oracle when fetching data.
Fixing issues when fetching dates from SQLite (needed casting to numeric in some scenarios before conversion to date in R).
Fixing insertTable() when using a
databaseSchema argument on SQLite.
Fixing insertTable() for inserting into temp table
on RedShift when table name does not start with ‘#’.
Bugfixes:
Changes:
Using SqlRender’s new
tempEmulationSchema argument. Avoids deprecation
warning.
Throwing an informative error when the user points the
pathToDriver argument to a file instead of a
folder.
Changes:
JDBC drivers are no longer pre-packaged in the
DatabaseConnectorJars package. Instead, drivers need to be
downloaded manually to a user-specified location in the local file
system. The downloadJdbcDrivers() function has been added
to make downing drivers easier. The path to the drivers can be set in
the ‘DATABASECONNECTOR_JAR_FOLDER’ environmental variable.
Support for 64-bit integers using bit64’s integer64
type, although this is disabled by default because of the danger of
silent errors when using integer64 in R. Each query
function has a integer64AsNumeric argument, and overall
behavior can be set
usingoptions(databaseConnectorInteger64AsNumeric = FALSE).
INT fields are now translated to R integers and back.
ConnectionDetails delays evaluation of sensitive arguments until needed for improved security.
Removing deprecated schema argument from
createConnectionDetails and connect.
Deprecating useMppBulkLoad argument of
insertTable() function in favor of bulkLoad
argument.
Deprecating oracleTempSchema argument in various
functions in favor of tempEmulationSchema argument, which
can be set via the “sqlRenderTempEmulationSchema” option.
Adding bulk upload mode for PostgreSQL.
Batch mode in executeSql divides SQL into batches
(1000 statements per batch) to avoid running our of Java heap
memory.
Removed message that ‘JDBC driver supports batch updates’ when running executeSql in batch mode.
Added optional databaseSchema argument to
insertTable for consistency with general OHDSI
framework.
Bugfixes:
Fixed ‘Warning: no non-missing arguments to max; returning -Inf’ when a column only has NA values, or no values at all.
Fixed error when trying to insert data from a tibble.
Error report now includes all SQL in a batch when failing in batch mode.
Changes:
Dropping support for orphaned ff package.
Adding support for Andromeda package to replace ff.
Error report file name now defaults to errorReportSql.txt to avoid confusion with other error reports.
Enforcing delay if about to exceed query quota on BigQuery.
Changes:
Deprecating schema argument in connect and createConnectionDetails functions.
Checking table and column names for SQL reserved words when inserting a table.
Switching to Java 1.8 as minimum requirement.
Preparing for DatabaseConnectorJars v1.2.0, which will have JDBC v4.2 drivers (instead of JDBC v4).
Bugfixes:
Setting append = TRUE when not creating table in insertTable for RSQLite.
On RedShift, converting schema name to lowercase before getting table names to fix empty results.
Fixed insertTable on BigQuery.
Fixing autocommit not supported error on BigQuery.
Changes:
Bugfixes:
Changes:
Bugfixes:
Preventing scientific notation when bulk uploading to PDW to avoid error.
Fixing null error when calling getSchemaNames for BigQuery.
Changes:
Bugfixes:
Changes:
Bugfixes:
Not adding ‘#’ prefix when performing insert into RedShift.
Disabling autocommit when sending updates to RedShift to prevent errors with new JDBC driver.
Preventing ‘FeatureNotSupportedError’ from terminating query on platforms that do no support autocommit.
Fixing some errors related to insertTable when determining correct field type for columns.
Changes:
Adding support for SQLite through RSQLite (mainly for demonstration and testing)
Adding convenience functions renderTranslateExecuteSql, renderTranslateQuerySql, and renderTranslateQuerySql.ffdf
Dropping Starschema BigQuery driver (in favor of newer Simba driver)
Added support for inserting BIGINTs (large integers stored as numeric in R)
Applying CTAS hack to improve insertion performance for RedShift (was already used for PDW)
Bugfixes:
Changes:
Bugfixes:
Changes:
Changes:
Changes:
Bugfixes:
Changes: initial submission to CRAN