arkdb 0.0.16

arkdb 0.0.15

Patch for test suite for Solaris. arrow package installs on Solaris, but functions do not actually run correctly since the C++ libraries have not been set up properly on Solaris.

Added ability to name output files directly.
Add warning when users specify compression for parquet files.
Added callback functionality to the ark function. Allowing users to perform transformations or recodes before chunked data.frames are saved to disk.
Added ability to filter databases by allowing users to specify a “WHERE” clause.
Added parquet as an streamable_table format, allowing users to ark to parquet instead of a text format.

Make cached connection opt-out instead of applying only to read_only. This allows cache to work on read-write connections by default. This also avoids the condition of a connection being garbage-collected when functions call local_db internally.

Better handling of read_only vs read_write connections. Only caches read_only connections.
Includes optional support for MonetDBLite

Support vroom as an opt-in streamable table
Export process_chunks
Add mechanism to attempt a bulk importer, when available (#27)
Bugfix for case when text contains # characters in base parser (#28)
Lighten core dependencies. Fully recursive dependencies include only 4 non-base packages now, as progress is now optional.
Use “magic numbers” instead of extensions to guess compression type. (NOTE: requires that file is local and not a URL)
Now that duckdb is on CRAN and MonetDBLite isn’t, drop built-in support for MonetDBLite in favor of duckdb alone.

ark()’s default keep-open method would cut off header names for Postgres connections (due to variation in the behavior of SQL queries with LIMIT 0.) The issue is now resolved by accessing the header in a more robust, general way.

unark() will strip out non-compliant characters in table names by default.
unark() gains the optional argument tablenames, allowing the user to specify the corresponding table names manually, rather than enforcing they correspond with the incoming file names. #18
unark() gains the argument encoding, allowing users to directly set the encoding of incoming files. Previously this could only be set by setting options(encoding), which will still work as well. See FAO.R example in examples for an illustration.
unark() will now attempt to guess which streaming parser to use (e.g csv or tsv) based on the file extension pattern, rather than defaulting to a tsv parser. (ark() still defaults to exporting in the more portable tsv format).

Ensure the suggested dependency MonetDBLite is available before running unit test using it.

Overwrite existing tables of same name (with warning and interactive proceed) in both DB and text-files to avoid appending.