What is GGIR?

GGIR is an R-package to process multi-day raw accelerometer data for physical activity and sleep research. GGIR will write all output files into two sub-directories of ./meta and ./results. GGIR is increasingly being used by a number of academic institutes across the world.

What is mMARCH.AC?

mMARCH.AC is an R-package to data processing after running GGIR for accelerometer data. In detail, all necessary R/Rmd/shell files were generated for data processing after running GGIR for accelerometer data. Then in module 1, all csv files in the GGIR output directory were read, transformed and then merged. In module 2, the GGIR output files were checked and summarized in one excel sheet. In module 3, the merged data was cleaned according to the number of valid hours on each night and the number of valid days for each subject. In module 4, the cleaned activity data was imputed by the average ENMO over all the valid days for each subject. Finally, a comprehensive report of data processing was created using Rmarkdown, and the report includes few explortatory plots and multiple commonly used features extracted from minute level actigraphy data in module 5-7. This vignette provides a general introduction to mMARCH.AC.

Software Architecture

The R package mMARCH.AC has been released with an open-source GPL-3 license on CRAN, and mMARCH.AC can run on Windows and Linux. Parallel computing in Linux is recommended due to the memory requirements associated with reading in multiple of the large data files. The package contains one primary function for users which, when run, generates all necessary R/R Markdown/shell executable files for data processing after running GGIR for accelerometer data; load, read, transform and merge long activity data; examine and summarize GGIR outputs; clean the merged activity data according to the number of valid hours per night and the number of valid days per subject; activity data imputation by taking the average across the valid days for each subject; build a comprehensive report of data processing and exploratory plots; extract multiple commonly used features and study feature structure by the covariance decomposition. Figure 1 presents a flowchart for each step in this process which is described in greater detail below. The procedure, R functions, inputs, and outputs are all described in this package vignette. In addition, more documentation and example data could be found in mMARCH.AC repository on GitHub (URL: https://github.com/WeiGuoNIMH/mMARCH.AC).

Software Dependencies

All mMARCH.AC code is written in R and reports generated in R Markdown. The R packages ActFrag and ActCR are used for the calculation of certain physical activity and circadian rhythmicity features. The R package r.jive is used to perform the feature interaction analysis and to study the joint and individual variation structure by JIVE.

Install R and RStudio

Download and install R

Download and install RStudio (optional, but recommended)

Download GGIR with its dependencies, you can do this with one command from the console command line:

install.packages("mMARCH.AC", dependencies = TRUE)

Prepare folder structure

1. folder of .bin files for GGIR or a file listing all .bin files
   • R program will check the missing in the GGIR output by comparing with all raw .bin files
2. foder of the GGIR output with two sub-folders
   • meta (./basic, ./csv, etc)
     
   • results (modulesummary.csv)

Create a template shell script of mMARCH.AC

library(mMARCH.AC) 
create.shell()

The function will create a template shell script of mMARCH.AC in the current directory, names as STUDYNAME_module0.maincall.R.

cat STUDYNAME_module0.maincall.R

 
argv = commandArgs(TRUE);  
print(argv) 
print(paste("length=",length(argv),sep=""))  
mode<-as.numeric(argv[1])  
print(c("mode =", mode))
# (Note) Please remove the above lines if you are running this within R console 
#        instead of submitting jobs to a cluster.
 
#########################################################################   
# (user-define 1) you need to redefine this according different study!!!!
######################################################################### 
# example 1 
filename2id.1<-function(x)  unlist(strsplit(x,"\\."))[1] 
 
#  example 2 (use csv file =c("filename","ggirID")) 
filename2id.2<-function(x) {
  d<-read.csv("./mMARCH.AC/inst/extdata/example/filename2id.csv",head=1,stringsAsFactors=F)
  y1<-which(d[,"filename"]==x)
  if (length(y1)==0) stop(paste("Missing ",x," in filename2id.csv file",sep=""))
  if (length(y1)>=1) y2<-d[y1[1],"newID"] 
  return(as.character(y2))
} 


#########################################################################  
#  main call
######################################################################### 
  
mMARCH.AC.shell<-function(mode,filename2id=NULL){ 

library(mMARCH.AC) 
packageVersion("mMARCH.AC")
#  ?mMARCH.AC.maincall  # run help to see all argumengts 

#########################################################################  
# (user-define 2) Fill in parameters of your ggir output
########################################################################## 
  
currentdir = 
studyname =
bindir = 
outputdir =  
setwd(currentdir) 

rmDup=FALSE   # keep all subjects in mMARCH.AC
PA.threshold=c(50,100,400)
part5FN="WW_L50M100V400_T5A5" 
epochIn = 5
epochOut =  60
use.cluster = FALSE
log.multiplier = 9250
QCdays.alpha = 7
QChours.alpha = 16 
QCnights.feature.alpha = c(0,0)
DoubleHour= "average" 
QC.sleepdur.avg=NULL
QC.nblocks.sleep.avg=NULL
useIDs.FN=NULL 
Rversion="R" 
desiredtz="US/Eastern" 
RemoveDaySleeper=FALSE  
NfileEachBundle=20 
holidayFN=NULL
trace=FALSE
#########################################################################  
#   remove duplicate sample IDs for plotting and feature extraction 
######################################################################### 
if (mode==3 & rmDup){
# step 1: read ./summary/*remove_temp.csv file (output of mode=2)
keep.last<-TRUE #keep the latest visit for each sample
sumdir<-paste(currentdir,"/summary",sep="")  
setwd(sumdir)  
inFN<-paste(studyname,"_samples_remove_temp.csv",sep="")
useIDs.FN<-paste(sumdir,"/",studyname,"_samples_remove.csv",sep="") 

#########################################################################  
# (user-define 3 as rmDup=TRUE)  create useIDs.FN file
######################################################################### 
# step 2: create the ./summary/*remove.csv file manually or by R commands
d<-read.csv(inFN,head=1,stringsAsFactors=F)
d<-d[order(d[,"Date"]),]
d<-d[order(d[,"newID"]),]
d[which(is.na(d[,"newID"])),]
S<-duplicated(d[,"newID"],fromLast=keep.last) #keep the last copy for nccr
d[S,"duplicate"]<-"remove"
write.csv(d,file=useIDs.FN,row.names=F)

} 

#########################################################################  
#   call afterggir
######################################################################### 

setwd(currentdir)  
mMARCH.AC.maincall(mode=mode,
          useIDs.FN=useIDs.FN,
          currentdir=currentdir,
          studyname=studyname,
          bindir=bindir,
          outputdir=outputdir,
          epochIn=epochIn,
          epochOut=epochOut, 
          log.multiplier=log.multiplier,
          use.cluster=use.cluster,
          QCdays.alpha=QCdays.alpha,
          QChours.alpha=QChours.alpha,
          QCnights.feature.alpha=QCnights.feature.alpha, 
          DoubleHour= DoubleHour,
          QC.sleepdur.avg=QC.sleepdur.avg,
          QC.nblocks.sleep.avg=QC.nblocks.sleep.avg,
          Rversion=Rversion,
          filename2id=filename2id,
          PA.threshold=PA.threshold,
          desiredtz=desiredtz,
          RemoveDaySleeper=RemoveDaySleeper,
          part5FN=part5FN,
          NfileEachBundle=NfileEachBundle,
          holidayFN=holidayFN,
          trace=trace) 

} 
#########################################################################
         mMARCH.AC.shell(mode)   
######################################################################### 

#   Note:   mMARCH.AC.shell(mode)
#        mode = 0 : creat sw/Rmd file
#        mode = 1 : data transform using cluster or not
#        mode = 2 : summary
#        mode = 3 : clean 
#        mode = 4 : impu
 

Edit shell script

Three places were marked as “user-define” and need to be edited by user in the STUDYNAME_module0.maincall.R file. Please rename the file by replacing your real studyname after the edition.

Run R script

mMARCH.AC.shell(mode,filename2id)   

Run script in a cluster

#!/bin/bash
#
#$ -cwd
#$ -j y
#$ -S /bin/bash
  source ~/.bash_profile
  module load R ; 
     R --no-save --no-restore --args  < studyname_module0.maincall.R  0
     R --no-save --no-restore --args  < studyname_module0.maincall.R  1
     R --no-save --no-restore --args  < studyname_module0.maincall.R  2
     R --no-save --no-restore --args  < studyname_module0.maincall.R  3
     R --no-save --no-restore --args  < studyname_module0.maincall.R  4 
 
     R -e "rmarkdown::render('module5_studyname_Data_process_report.Rmd'   )"  
     R -e "rmarkdown::render('module6_studyname_NonWear.report.Rmd'   )" 
     R -e "rmarkdown::render('module7a_studyname_calculate_newfeatures.Rmd'   )" 
     R -e "rmarkdown::render('module7b_studyname_merge_GGIRfeatures.Rmd'   )" 
     R -e "rmarkdown::render('module7c_studyname_runJIVE.Rmd'   )" 
     R -e "rmarkdown::render('module7d_studyname_calculate_WD_WE_avg_features.Rmd'   )" 

Module 1

The functions of Module 1 read, merge, and transform the lower resolution summary of raw acceleration data for all subjects. Raw data are summarized using Euclidian Norm Minus One (ENMO) (Migueles et al., 2019), a rotationally invariant measure of volume of acceleration, in epochs specified by the user in their call to GGIR. Merged data, originally in long format, are saved in wide format, an intuitive data structure for accelerometer data (Leroux 2019). Depending on the time zone on which the devices were initialized, a day may have between 23 and 25 hours due to daylights savings time. On daylight savings crossovers, ENMO is averaged for duplicate timestamps between 1:00 AM and 2:00 AM, allowing for straightforward comparisons to standard 24-hour days. In addition to recording ENMO at each epoch, the angle of the z-axis (ANGLEZ) relative to the horizontal plane (degrees), used for estimating sleep periods, is merged and saved into an excel file. Additional data recorded by certain devices are merged for all participants and saved under the ./data directory including the light mean, light peak, temperature mean, clipping score, and the Euclidian norm metric (EN) if available.

Each row represents the corresponding ENMO and ANGLEZ values at a timestamp per 5 seconds epoch, which is specified by GGIR parameter (windowsizes) when running GGIR. After running Module 1, the ENMO and ANGLEZ data are transformed into wide matrix in which each row represents 24 hours data for a day. For example, the ENMO data is formated as follows,

Finally, the data was merged for all days and all subjects.

Module 2

The functions of Module 2 create descriptive variables of all accelerometer files output by GGIR. In Module 2, an excel file is output under the ./summary directory, which includes ten pages as follows, (1) List of files in the GGIR output (2) Summary of numbers of output files (3) List of duplicate IDs (4) ID errors (5) Number of valid days (6) Table of number of valid/missing days (7) Missing pattern (8) Frequency of the missing pattern (9) Description of all accelerometer files (10) device information. Multiple plots are generated in a pdf file including the number of valid hours, days, missing pattern, etc. First, the completeness of GGIR run is inspected by detecting the missing files in each GGIR step when user specified the path of GGIR output. Additionally, the descriptive variables are created to form a comprehensive data quality report based on summary results generated by GGIR for the purpose of examining data quality in the Module 2 of mMARCH.AC.

Module 3 and Module 4

The functions of Module 3 introduce ‘flag’ variables for data cleaning of the merged ENMO and ANGLEZ data. By default, days with more than 16 hours are marked as valid days and subjects with more than 7 valid days are marked as valid samples. Users can set these two parameters in the call to the main function (QCdays.alpha = 7, QChours.alpha = 16). Further, the data can be aggregated (averaged) in lower resolution epochs (e.g. minute-level or hour-level) as desired by users. In Module 4, ENMO data during estimated non-wear periods are imputed by taking the subject-level mean over all the valid days for each subject at that time, limiting potential bias in imputation due to diurnal patterns of activity. All output is saved under the ./data directory. In the output file, the following description variables are included: (1) number of valid hours; (2) missing pattern for each subject; (3) non-wear time in minutes; (4) an indicator variable to indicate if the visit should be removed for having multiple visits for some subjects which might lead to invalidity of independent and identically distribution in statistics and (5) the number of missing values after imputation. When the number of missing values after imputation is not zero, it means the activity data is missing on same timestamps among all days and therefore could not be imputed, therefore, such samples would be removed when systematic missingness was observed.

Module 5

The functions of Module 5 generate a comprehensive report in .html format. This report includes data quality checks and an exploratory data analysis using valid days of data. First, the numbers and missingness of GGIR output files are summarized for all GGIR parts. Second, duplicate samples are checked and marked, where the duplication might be caused by having multiple visits for some samples but only one visit will be kept in the data analysis such as functional principal component analysis (FPCA). Third, as shown in Figure 1, data quality is presented visually using the number of valid days, non-wear time, and missing data pattern. As an exploratory analysis, the data correlation, and the output of FPCA analysis are plotted in the report.

Module 6 (Optional)

Estimated non-wear periods are loaded from the \(M\$metalong\$nonwearscore\) variable of the R data that was stored in the folder of /meta/basic of the GGIR output, which generate the matrix to clarify when data was imputed for each long epoch time window and the reason for imputation. This function will generate a non-wear matrix at minute level, coded as 0/1 for wear/non-wear time. This function will generate a non-wear matrix at minute level, and it could be skipped if user chose to use the imputation data in the JIVE application as default.

Module 7

In Module 7, 88 features were extracted from minute level activity data of three domains of sleep, physical activity, and circadian rhythmicity, which were based on outputs from GGIR v2.4.0 and calculated by R ActFrag and ActCR packages. The standard deviation across days on each subject was also created for each feature. The weekday and weekend specific features were extracted as well since most features in the sleep and physical activity showed significant difference between weekdays (Monday to Friday) and weekends (Saturday and Sunday). Specially, the sleep features on weekends are calculated based on sleep periods from Friday night to Saurday morning, Saturday night to Sunday morning, and last night to a holiday morning. Furthermore, holdiays could be counted as weekends if user provide the holiday information through the argument of holidayFN in the afterggir() function. In brief, sleep domain referred to sleep duration, midpoint, efficiency, etc. Physical activity referred to daily motor activity such as sedentary behavior, light, and moderate-to-vigorous physical activity (MVPA). Circadian rhythms were natural rhythms that regulates the sleep-wake cycle within every 24 hours. For example, the cosinor curve and FPCA analysis were used in modeling of biological rhythms. A comprehensive list of all features could be found in the supplementary table and more detailed definition could be found in the mMARCH.AC paper, GGIR manual and Di et al.’s publication in 2019.

All features listed are calculated using minute-level activity data by default. Several duplicate features were also extracted from GGIR results as an additional data source. As an integral module of mMARCH.AC package, we proposed to use a multi-modal integrative technique, Joint and Individual Variance Explained (JIVE), to efficiently and simultaneously deal with multiple features representing physical activity (PA), sleep (SL), and circadian rhythmicity (CR) (Di et al., 2019; Lock et al., 2013; O’Connell and Lock, 2016). The integrative analysis of multiple derived variables in each of these three domains assessed by accelerometry is applied to explain the joint and individual variation in these three domains. Conceptually, JIVE assumes four groups of latent variables that generate: i) individual variation captured by PA, SL, CR; and ii) joint variation by PA, SL, CR. These latent variables (JIVE scores) can be estimated and then used as novel measures capturing both individual and joint variability of measures of sleep, physical activity and circadian rhythmicity. JIVE has been applied to multivariate actigraphy derived features to address the interdependence between domains of physical activity, sleep, and circadian rhythm (Di et al. 2019). JIVE decomposition utilized the subject level means of all 88 features including those duplicated features from GGIR outputs. Heatmaps of the estimated joint structure of the full JIVE decomposition are created, with examples shown in Figure 1. JIVE scores are estimated for both joint and individual variation structures. Building the activity features could enrich the markers for understanding genetic contribution of human diseases.

Input and output of Module 0

• Command = mMARCH.AC.shell(mode=0)
  
• Output folder = ./

Input and output of Module 1

• Main input files: csv files under /meta/csv folders of GGIR output
• Command = mMARCH.AC.shell(mode=1)
• Output folder = ./data

f0 and f1 are the file index to start and finish with
Xs is the epoch size to which acceleration was averaged (seconds) in GGIR output

Input and output of Module 2

• Main input files
  + ./data/All_studyname_ENMO.data.csv
  + GGIR results: part2, part4 and part5 (please specify \(part5FN\) in the main function)
  + GGIR raw data when bindir was specified

• Command = mMARCH.AC.shell(mode=2)
  

• Output folder = ./summary

Input and output of Module 3

• Main input file: ./data/All_studyname_ENMO.data.csv
  
• Command = mMARCH.AC.shell(mode=3)
• Output folder = ./data

*Xs is the epoch size to which acceleration was averaged (seconds) in GGIR output

Input and output of Module 4

• Main input file: ./data/flag_All_ studyname _ENMO.data.5s.csv
• Command = mMARCH.AC.shell(mode=4)
• Output folder = ./data

Description of flag variables in the output data

Input and output of Module 5

• Main input files
  • ./summary/studyname_ggir_output_summary.xlsx
  • ./summary/part24daysummary.info.csv
  • ./data/plot.nonwearVSnvalidhours.csv
  • ./data/impu.flag_All_studyname_ENMO.data.flag.epochOuts.csv
• Command: run module5_studyname_Data_process_report.Rmd
• Output folder = ./

Input and output of Module 6

• Main input file: nonwearscore_studyname_01_xx_900s.csv
• Command = run module6_studyname_NonWear.report.Rmd
• Output folder = ./

f0 and f1 are the file index to start and finish with
Xs is the epoch size to which acceleration was averaged (seconds) in GGIR output

Input and output of Module 7a

• Main input
  • ./data/impu.flag_All_studyname_ENMO.data.flag.epochOuts.csv
  • GGIR: /results/QC/part4_nightsummary_sleep_full.csv
• Command = run module7a_studyname_calculate_newfeatures.Rmd
• Output folder = ./

Input and output of Module 7b

• Main inputs
  • GGIR: part2_summary.csv
  • GGIR: part2_daysummary.csv
  • GGIR: part4_nightsummary_sleep_cleaned.csv
  • GGIR: /results/QC/part4_nightsummary_sleep_full.csv
  • GGIR: part5_daysummary_part5FN.csv
    
  • module7_studyname_all_features_dictionary.xlsx
• Command = run module7b_studyname_merge_GGIRfeatures.Rmd
• Output folder = ./

Input and output of Module 7c

• Main inputs
  • module7b_studyname_all_features_3_subject.csv
  • module7b_studyname_all_features_4_subjectSD.csv
• Command = run module7b_studyname_merge_GGIRfeatures.Rmd
• Output folder = ./

Input and output of Module 7d

• Main input: ./data/impu.flag_All_studyname_ENMO.data.flag.epochOuts.csv
• Command = run module7d_studyname_calculate_WD_WE_avg_features.Rmd
• Output folder = ./

Description of features of domains of physical activity, sleep and circadian rhythmicity

Sleep Domain

Physical Activity Domain

Circadian Rhythmicity Domain

Note
(1) The column of level indicates the feature was calculated as subject level or day level.
(2) The column of source indicates the feature source. R indiciates the feature was calculated based on activity data. Part2, part4 and part5 in ‘Source’ column indicate the feature was extracted from summary results from GGIR outputs.