Programming for clinical trial data analysis tends to be very standardized. With data standards such as CDISC, expectations tend to be very clearly defined. Within these programming activities, there’s ample room for the use of metadata. Metadata can be used for several different purposes, such as applying dataset attributes, establishing sort sequences, working with controlled terminology, and more. Despite CDISC standards, organizations tend to have their own means of storing metadata, be it in excel spreadsheets, databases, and more.
The purpose of metacore is to establish a common foundation for the use of metadata within an R session. This is done by creating an R object that can hold the necessary data in a standardized, immutable structure (using R6) that makes it easy to extract out necessary information when needed. Users can read in their metadata from their various sources. To make this easy, we’ve provided some helper functions - and even have readers that can read directly from Define.xml 2.0. By establishing a common and consistent object in memory, further packages that support these work flows can have a common foundation upon which tools can be built that leverage metadata in the future. This reduces the need to hold different data structures containing metadata and instead allows programs to pull this information from a centralized source.
You can install the current development version of metacore from github with:
A metacore object is made-up of 6 different tables, which are connected with a series of identifying columns. The goal of these tables is to normalize the information as much as possible, while keeping together like information. Each table has a basic theme to make them easier to remember. They are as follows:
ds_spec: Contains dataset level information
ds_vars: Bridges the dataset and variable level information
var_spec: Contains variable level information
value_spec: Contains value level information
derivations: Contains all derivations
codelist: Contains information about code/decodes, permitted values and external libraries
supp: Contains information specific to supplemental variables
Here is a schema of how all this fits together:
This table covers the basic information about each dataset. There is only a single row per dataset, with the following information:
dataset: The abbreviated name of the dataset (e.g. AE)
Structure: Value structure of the dataset as a sting
Label: Dataset label
This table contains the information that bridges between purely dataset level and purely variable level. There is one row per dataset per variable:
dataset: The abbreviated name of the dataset. This will match to the name in ds_spec
variable: Variable name
key_seq: Sequence key, which are the variables used to order a dataset. This is a column of integers, where 1 is the first sorting variable and 2 is the second etc. If the variable is not used in sorting it will be left NA
order: Order sets the order of the columns to appear in the dataset. This is also a numeric value
keep: Logical value about if the variable needs to be kept
core: ADaM core, which should be one of the following values: “Expected”, “Required”, “Permissible”, “Conditionally Required”, “Conditionally Expected”, or NA. For more information about core see CDISC
supp_flag: Logical to determine if the variable is in the supplemental datasets
This table contains the information the purely variable level information. The goal is there is a single row per variable, which is common across all datasets. This helps ensure variables follow the CDISC standard. But, this isn’t always possible, so if information for a given variable differs across datasets, the variable will be recorded as dataset.variable in the variable column.
variable: Variable name, which should match the name in ds_spec. Unless the variable needs to be duplicated, then the name will be a combination of the the dataset name and variable name from ds_spec (dataset.variable)
type: Variable class
length: Variable length (while not relevant to R datasets, this is important for when creating XPT files)
label: Variable label
common: Common across ADaM datasets
format: Variable format
This table contains the information the information at the value level. There will be at least one row per dataset/variable combination. There is more than one row per dataset/variable combination if the combination has values which have differing metadata. For instance LBORRES that are different data types depending on the value. The information contained are as follows:
dataset: The abbreviated name of the dataset. This will match to the name in ds_spec
variable: Variable name. This will match to the name in ds_vars
type: String of the value type
origin: Origin of the value
sig_dig: Significant digits of the value
code_id: ID for the code list to match the id in the codelist table
where: Value of the variable
derivation_id: ID for the derivation to match with the derivation table
This table has all the derivation information, with one row per derivation ID and the following information:
derivation_id: The ID, which should match to value_spec
derivation: Text describing the derivation
This table contains the code lists, permitted value lists, and external libraries nested within a tibble. There is only a single row per list/library, with the following information:
code_id: the ID used to identify the code list. This should be the same as the code_id in val_spec
name: Name of the code list
code: A list of tibbles (for code / decode combinations) and vectors (for permitted values and libraries), which contain all the codes
type: An indicator of if the information in the code column is a code/decode table, permitted value, or external library
To see a metacore object in about please see our vignettes
This table contains the information needed to create supplemental tables. If you want to add a variable which will go into a supplemental qualifier then you can create as normal (i.e. label information going to the var_spec table and derivation and origin going into the value_spec table), but you need to flag it as supplemental in the ds_vars table and add a row to the supp table. There is only a single row per dataset/variable, with the following information:
dataset: The abbreviated name of the dataset. This will match to the name in ds_spec
variable: Variable name. This will match to the name in ds_spec
idvar: ID variable used for the supplemental variable. Can be left missing if not needed
qeval: Evaluator for the supplemental variable
To see a metacore object in about please see our vignettes
This is an alpha release of this package, so if you have ideas on future improvements please add them to the issue log.