In R 4.3.0 as.character(lubridate::now()) adds microseconds, which breaks the SQL upload. This is now replaced by cstime::now_c().
Changed “success” to “succeeded” in update_config_tasks_stats
DBPartitionedTableExtended_v9$info() bug fixed with argument collapse=TRUE.
Inclusion of confirm_indexes in DBPartitionedTableExtended_v9.
DBPartitionedTableExtended_v9$nrow() now has a new argument collapse=FALSE that provides partion-specific results
DBPartitionedTableExtended_v9$info() now includes sizes in MB
Bug fix in DBPartitionedTableExtended_v9$nrow()
Extension of DBPartitionedTableExtended_v9 so that it is easier to use multiple partitioning variables.
Inclusion of partitions_randomized in DBPartitionedTableExtended_v9 so that when running in parallel, the database tables don’t get locked.
Inclusion of remove_table in DBPartitionedTableExtended_v9
Fixed an error in RAM calculation in parallel for get_config_tasks_stats
DBTableExtended_v9 now automatically includes a column for all tables, called auto_last_updated_datetime, which is automatically calculated each time that row is changed.
Creation of DBPartitionedTableExtended_v9, which allows for one dataset to be partitioned amongst multiple SQL tables automatically.
SurveillanceSystem_v9 constructor now takes in an argument called implementation_version, which can be used to identify what version of analytics code is currently being run.
update_config_last_updated has now been replaced by update_config_tables_last_updated (which contains when the tables were last updated) and config_tasks_stats (which contains all the runtimes of the tasks).
SurveillanceSystem_v9 now uses an internal R6 class DBTableExtended_v9 (which extends csdb::DBTable_v9) instead of using csdb::DBTable_v9 directly. DBTableExtended_v9 calls update_config_last_updated after altering a database table.
sc8 is deprecated in favor of sc9.
Allows for multiple databases to be used for different access levels.
copy_into_new_table_where now also copies indexes.
V8 schemas now have a nice print function.
V8 redirects now have a nice print function.
copy_into_new_table_where uses tablock.
upsert_at_end_of_each_plan and insert_at_end_of_each_plan can now take named lists as the return value from the action_fn.
Custom progressr handler.
When using sc8::add_task_from_config_v8 the schema list is checked to make sure they are actually schemas. This will solve the issue where people incorrectly add non-existent schemas to the task.
insert_data, upsert_data, drop_all_rows_and_then_insert_data are now the recommended ways of inserting data
schemas now use load_folder_fn, which should dynamically check if a user has permission to write to a folder, solving permissions errors
Both granularity_time AND granularity_geo are now included in db censors
Requires R >= 4.1.0
sc8::config$plan_attempt_index now exists. When running plans in parallel, if a plan fails it is retried five times. This lets a user track which attempt they are on. This is mostly useful so that emails and smses are only sent when sc8::config$plan_attempt_index==1
(Disabled) TABLOCK is disabled right now due to issues where data would not be uploaded.
(Disabled) Data is sorted before sending it to bcp to speed up in/upserts.
Release of schema redirects that allow for restricted and anonymous datasets to be seamlessly used by people with different access rights
Consistent naming of task_from_config_v8 and add_schema_v8
db_insert_data, db_upsert_data, db_drop_all_rows_and_then_upsert_data are now the recommended ways of inserting data
update_config_datetime and get_config_datetime now automatically record database table updates as well
Updating default db schemas to be more explicit with the useage of isotime.
qsenc_save and qsenc_read to save/read to/from encrypted files.
task_from_config_v3 sets a new direction for creation of tasks and management of tasks
describe_tasks and describe_schemas help with automatic documentation
task_inline_v1 allows for easy inline task creation
Corresponding RStudio addin for inline tasks that copy from one db table to another
copy_into_new_table_where allows for the creation of a new table from an old table
First RStudio addin
write_data_infile now checks for Infinite/NaN values and sets them to NA
Task now includes action_before_fn and action_after_fn
validator_field_contents_sykdomspulsen now allows baregion as a valid granularity_geo
tm_get_plans_argsets_as_dt provides an overview of the plans and argsets within a task
keep_rows_where now also retains the PK constraints