Validity and Reliability of Chinese Physical Activity Questionnaire for Children Aged 10-17 Years

Physical activity is important for physical, psychological and cognitive health of children^[1]. Systematic reviews reinforce global public health concerns that physical inactivity and sedentary behavior are associated with adiposity, cardiovascular disease, cognitive decline and social behavioral problems^{[2, 3]}. Recent findings suggested that there might be synergistic health benefits by obtaining optimal combinations of movement behaviors (e.g. more physical activity, less sedentary behavior and high quality sleep)^{[4, 5]}. With more than 316 million children and adolescents classified as overweight or obese globally^[6], the development of strategies for increasing daily physical activity and minimizing sedentary time, and promote healthy lifestyle that will continue to adulthood is a public health priority.

Physical activity pattern is a multi-dimensional construct and represents the combined effects of the frequency, intensity, time, type and context of physical activity and sedentary behavior. Recent literature suggests that the description of physical activity should be reframed as a pattern comprising multiple domains, dimensions, or correlates^[7]. For children, the nature of physical behaviors (such as short and intermittent bouts of activities), the characteristics of patterns (such as various types of activities) and their cognitive capacity (such as difficulty in performing detail) make it challenge to measure physical activity pattern^[8].

Accurate physical activity measures are essential for public health research, policy making and practice^[9]. A variety of devices which can record human physical activity level effectively, such as pedometers, multi-sensors, and smart watches, have been developed^[10]. However, these devices provide less information on the type of activity behaviors or where and in what context the physical activity was performed, which is critical for understanding the underlying modifiable factors promoting physical activity. In addition, self-report methods might be more feasible to estimate physical activity level in large surveys due to their cost effectiveness and easy availability. Questionnaires are important in assessing the mode and domain of physical activity that are not available to be measured objectively, making physical activity pattern assessment possible^{[11, 12]}. To maximize utility questionnaires is need to fit the needs of the country in which they are based.

Numerous physical activity questionnaires have been developed, especially for Caucasian youth, with variation in measurement protocol, physical activity dimensions assessed, recall period and other aspects^[13]. A paucity of studies have applied the commonly used physical activity questionnaires into the Chinese counterparts and found that the translation of available questionnaires performed less well in Chinese youth^{[14, 15]}. It is noticeable, reliable and validated physical activity questionnaire for use in Chinese young population is limited^[16]. This is a major omission from the literature as China is currently undergoing an extensive economic development and rapid urbanization. As such, there is a growing need to develop a new questionnaire for Chinese children in assessing physical activity patterns across a whole day, providing the basis for children’s 24-hour movement measurement and its related health effect research. Understanding how it changes as a result of economic development is a research gap that urgently needs to be addressed.

The Chinese Children Physical Activity Questionnaire (CCPAQ) was designed to address the full complements of physical activity pattern across 24-hour period and provide the estimation of physical activity energy expenditure in Chinese children. The aims of the current study were to report on the development of the CCPAQ and to evaluate its validity as well as the test-re-test reliability in children aged 10-17 years.

The CCPAQ was developed in five stages: 1) review of the literature to identify key elements of self-report measures; 2) assessment of reliability and validity study of the existing questionnaires; 3) designing the questionnaire format, content and flow; 4) consulting the experts of epidemiology, physical activity and other related fields and then revising the questionnaire; and 5) pilot testing.

The CCPAQ is a 7-day recall physical activity questionnaire for children to collect the information on physical activity pattern comprising of 23 questions (Additional file ‘CCPAQ Guide’ are available from the author on request). It uses a time-based structure which has been used in other surveys, such as Physical Activity Questionnaire for Older Children (PAQ-C)^[17] and a checklist of responses that is comparable to Self-Administered Physical Activity Checklist (SAPAC)^[18]. In CCPAQ, the activities across whole days, including weekday and weekend day, in the past week are asked to recall sequentially. The codes table of children’s physical activity types, which was a list of 32 activity codes in 3 categories, was developed based on Chinese children’s activity. Physical activity pattern was evaluated by asking the students to select the kind of activities from the codes table, report the intensity of the physical activity according to the revised Rate of Perceived Exertion Scale for Exercise^[19], and to best approximate the frequency and the amount of time spent in that activity last week. The energy expenditure of each activity was then estimated based on the latest Youth Compendium of Physical Activity (Activity Codes and Metabolic Intensity, 2017)^[20]. Finally, the number of days per week that the students engaged in moderate-to-vigorous physical activity (MVPA) accumulated for at least 60 min a day was asked. The CCPAQ takes on average 12-15 min to complete.

Physical activity was assessed using ActiGraph wGT3X-BT accelerometer (ActiGraph, LLC, Pensacola, Florida, USA) as criterion method on the basis of its established reliability and accuracy^[21]. For this study, the accelerometer was set to record data at a sampling rate of 30 Hertz. ActiGraph files were downloaded and then transformed into 10 s epoch files in the commercial software (Acti life 13.3), using standard procedures for identifying non-wear time and interpolating gaps with missing data. The sleeping time of each individual was marked as non-wear time. The minimum wear-time for inclusion in the analysis was at least 8 h per day for at least 5 days (including at least 3 weekdays and 1 weekend days). The reason was that the monitoring for 8 h in 4-5 days and 8 h daily of monitoring have shown reliable activity data in a week^[22].

The intensity of physical activity was defined using the cut points derived by Vanhelst et al. (2011)^[23], with sedentary behavior below 400 counts/min, light physical activity (LPA) between 401 and 1,900 counts/min, moderate physical activity (MPA) between 1,901 and 3,918 counts/min, vigorous physical activity (VPA) greater than 3,919 counts/min. The threshold of Vanhelst et al. was chosen for this study as it has shown good classification accuracy in Chinese children^[24]. In addition, accelerometer-measured energy expenditure was calculated using the equation of Freedson VM3 Combination (2011)^[25].

In the spring and summer of 2018, a total of 120 children in grade 4, 7, 10 in two schools (one elementary school and one middle school) in two districts (Yanqing of Beijing in northern China and Wanzhou of Chongqing in southern China) were recruited by Center for Disease Control and Prevention. The children who were taking medications or had medical illness affecting growth, injured or had other conditions limiting participation in physical activity, or had problems reducing adherence to the study protocol were excluded from the study.

Participants were recruited after a brief presentation and instruction in a parents meeting, in which written and oral information were provided for the parents of all interested individuals. Written informed parental consent was obtained for all participants. The design of the study was according to the Helsinki Declaration and approved by National Institute of Nutrition and Health, Chinese Center for Disease Control and Prevention (Ethic committee approval code: 2013-018).

Participants’ involvement was over 10 days as shown in Figure 1. Participants were asked to wear an ActiGraph accelerometer for 7 consecutive days in May in 2018. The participants received the accelerometer at the first visit and were required to engage in their normal activities. Participants were instructed to wear the accelerometer on the right side of the body at hip level. The wear time of the accelerometer was 24 h for 7 days. In the monitoring, investigators contacted the participants’ parents by phone to remind of wearing and checking on compliance with the accelerometer, and answer any questions about the study. Participants were advised to only remove the monitor for swimming or bathing and record non-wear time and reasons for removal. At the end of the 7-day monitoring, the participants returned the accelerometer to school. Meantime, the CCPAQ for the first time (CCPAQ 1) was administrated in the face-to-face interviews by trained observers during a class. One day later, the participants were asked to complete the CCPAQ for the second time (CCPAQ 2) at schools with data collected by the same interviewers. Anthropometric measures, including body height and weight measured by using an electronic stadiometer (Hochoice, EF07), were collected from all the participants. The COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) Checklist was used as a guidance for the design of CCPAQ and the implementation of this study^[26].

Figure 1.  Timeline of the study protocol. CCPAQ 1 = the first-time administration of Chinese Children Physical Activity Questionnaire; CCPAQ 2 = the second-time administration of Chinese Children Physical Activity Questionnaire.

Descriptive analyses included calculating the median and standard deviation, quartiles or percentages for all variables. Univariate ANOVA procedures or χ² test were used to examine differences in the characteristics of the sample. The physical activity variables for the evaluation of reliability and validity of the CCPAQ were total time spent on physical activity, sedentary behavior, and on two intensities levels namely MVPA and LPA, as well as energy expenditure. Statistical analyses were performed with SAS 9.4 (SAS Institute Inc., Cary, NC, USA).

Reliability Analyses　　To examine the reliability, the single measure, parametric interclass correlation coefficients (ICC) with 95% confidence intervals (CI) were calculated to evaluate the extent of CCPAQ in the test-retest analysis. ICC ≥ 0.70 was indicative of acceptable reliability^[27].

Validity Analyses　　The validity analysis involves two types of measurement properties in this study: content validity and construct validity. The content validity of the CCPAQ was assessed by consulting experts in the fields of sport, nutrition and related areas. For construct validity, non-parametric Spearman’s rank correlation coefficient (rho) was used to examine the correlations between minutes per day in each intensity level and physical activity energy expenditure as kilocalorie per day from the CCPAQ data and similar data from the accelerometer. The answers from CCPAQ 1 were used in the validity analyses. Based on the COSMIN Checklist, rho ≥ 0.50 indicates that the validity is acceptable. The Bland and Altman method was used to provide an indication of the heteroscedasticity and the systematic random error of the data with 95% limits of agreement (mean difference ± 1.96 standard deviation) between the CCPAQ and the accelerometer. The variables used for the Bland and Altman analysis were physical activity energy expenditure, time spent on total physical activity, MVPA and sedentary behavior.

Sensitivity Analyses　　We recognize that there is debate in the field of accelerometer-measured physical activity on which intensity threshold should be used^[28]. To test the robustness of different cut-points of physical activity intensity for validity study, we made a number of sensitivity analyses repeating the main analyses based on other 3 different intensity thresholds (cut-off of Evenson et al.^[29], cut-off of Puyan et al.^[30], cut-off of Freedson et al.^[31]). Cut-off of Evenson et al. (2008) has been widely used in Western country. Cut-off of Puyan et al. (2002) is one of the highest intensity thresholds of physical activity, while Freedson et al. (1998) has a low intensity threshold for MVPA. As these three thresholds were conducted with Western samples it is unclear whether they would be applicable for Chinese children.

A total of 120 participants were enrolled in the study. Finally 119 participants (99%) were included in the reliability analysis and 106 participants (88%) were included in the analysis of criterion validity (see ‘Flow diagram’ in Supplementary Figure S1, available in www.besjournal.com). Characteristics of the participants in reliability study and validity study stratified by sex are shown in Table 1. Boys spent more time in MVPA per day and had more physical activity energy expenditure than girls. Physical activity energy expenditure and daily time spent on total physical activity, MVPA and sedentary behaviors were longer on weekday than on weekend day (see Supplementary Table S1, available in www.besjournal.com).

Figure S1.  Flow diagram outlining the number of included and excluded. CCPAQ 1&2, the first-time and the second-time administration of Chinese Children Physical Activity Questionnaire.

Reliability coefficient for time spent on total physical activity, MVPA, sedentary behavior and sleeping ranged from 0.7 to 0.9, while the reliability coefficient for LPA and screen time was slightly lower (0.4 to 0.7) (Table 2). Overall, the reliability coefficient was higher for physical behavior variables on weekday than on weekend day. Similarly, the reproducibility seemed to be higher for variables on boys than girls. The reliability coefficient for types of physical activity ranged from 0.57 to 0.88 (Table 3).

Content Validity　　The CCPAQ exhibits content validity as it provides information on the natural and intuitive of physical activity pattern, especially different intensity and types of physical activities as well as sedentary behaviors and sleep time across a whole day in the past week. Furthermore, the CCPAQ is capable of investigating other subset of activity, such as screen-based activities, activities in school, out of school, in household and in transportation.

Criterion Validity　　Physical activity energy expenditure displayed moderate validity on a week (rho = 0.58, P < 0.001) or on weekday (rho = 0.57, P < 0.001). Correlations between the CCPAQ and the accelerometer tended to be higher for total time spent on physical activity and sedentary behaviors compared with correlations for MVPA and LPA. Correlations were stronger for boys than for girls, except daily time spent on MVPA and sedentary behavior during weekend. Mean differences for time spent on all activity variables were negative which means CCPAQ values on average were lower than accelerometer-measured values. There was no correlation between the two methods for the average days in which at least 60 minutes of MVPA were accumulated in each day last week (Table 4).

Bland-Altman plots showed the degree of error in CCPAQ depended on the level of physical activity energy expenditure, MVPA or sedentary behaviors (Figure 2). Higher level of energy expenditure and MVPA were more likely to be overestimated and the difference between both methods was more obvious, whereas more time spent on sedentary behaviors was more likely to be underestimated and the difference between both methods was less obvious.

Figure 2.  Bland and Altman plots in physical activity energy expenditure (A), total PA (B), MVPA (C), and SB (D). CCPAQ, Chinese children’s physical activity questionnaire; PA, Physical Activity; MVPA, moderate-to-vigorous physical activity; SB, sedentary behaviors.

Using the cut-off of Evenson et al., similar validity correlations of physical activity variables between the CCPAQ and the accelerometer were found. Accelerometer data were re-categorized using 0-799 counts as the threshold for sedentary behavior in the cut-off of Puyan et al., and then the correlation was only 0.14 and insignificant (P = 0.163). Using the cut-off of Freedson et al., lower than that of Vanhelst et al., there are no significant correlations in any physical activity variables (Table 5).

This study demonstrates that estimations of daily physical activity energy expenditure, time spent on physical activity and sedentary behaviors in children aged 10-17-years by using CCPAQ were highly reliable and have moderate validity. Boys spent more time on high-intensity activities and had higher energy expenditure compared with girls in this study, consistent with numerous investigations^{[32, 33]}. Physical activity level tended to be higher on weekday than on weekend day, similar to other study^[34], indicating that it would be more feasible for self-report tool to divide a week into weekday and weekend day to collect physical activity data.

The overall reliability of CCPAQ was strong and the reliability coefficient was ranging from 0.70 to 0.92. A systematic review illustrated that the median reliability correlation for newly developed questionnaires in youth was 0.68 (ICC) and for existing questionnaires was 0.64 (ICC)^[35]. We found better agreement in CCPAQ than the average of other questionnaires. It might be due to the 1-day interval between the two administrations of CCPAQ, as previous studies examining the reliability of questionnaire for measuring physical activity during the past week usually used a time interval of 1 day to 3 months. The CCPAQ can also provide the information on the types of physical activity and showed good reliability. Reproducibility was higher for in-school activity than sports/exercise outside school and household activity. Few studies have examined the reliability for the types of children’s physical activity^[36].

For physical activity energy expenditure and time spent on total physical activity, we found that the CCPAQ tended to be more reliable for boys and on weekday. A reliability study by Rangul et al.^[37] found that the WHO HBSC questionnaire was more reliable for girls, and another study by Treuth et al. showed no sex differences^[38]. The sex differences in our study might be due to the higher intensity of activity and more time spent on competitive sports in boys than in girls. High-intensity exercises tended to be more structured and memorable. The organized activity in physical education on weekday has also been found to be easier to recall and has higher repeatability than free-living activity on weekend day.

The validity correlations between the CCPAQ and the accelerometer for the time spent on total physical activity, sedentary behaviors and physical activity energy expenditure were generally moderate (rho = 0.32, rho = 0.32, and rho = 0.58). Systematic review evidence has shown that the median validity correlations of physical activity for youth was 0.22 (rho) and none of physical activity questionnaires for children showed high validity^[35]. The CCPAQ has been shown to be higher validity compared with other 7 day self-report questionnaires, and the difference might be explained by recalling physical activity in a day sequence and classifying physical activity types into categories. However, using one question to assess the achievement of physical activity guidelines (60-minute MVPA per day) in this study might be inaccurate. Conversely, Single-item Physical Activity Measure to this guidelines was found to have moderate validity (rho = 0.44)^[39]. Moreover, the CCPAQ seemed to be more valid for boys than girls, which is the same in terms of sex difference with SAPAC measure^[18].

Previous validation study of PAQ-C in Chinese suggested limited validity for MVPA compared with accelerometer data (rho = 0.24, P < 0.01)^[14]. This is similar to our results for time spent on MVPA (rho = 0.20, P = 0.040). Times spent on MVPA and LPA were less consistent with accelerometer compared with time spent on total physical activity. This might be due to the lack of consensus on homogenizing cut-off points when translating accelerometer intensity into physiologic intensity. The accelerometer cut points for the intensity of physical activity and sedentary behavior among children in the previous studies varied, leading to different estimates of activity intensity^[28]. In this study we used an accelerometer threshold that was most appropriate for Chinese youth, but it is broadly in agreement with other thresholds that have been used^[24]. As the threshold for when accelerometer data are used to indicate MVPA increases (i.e. a higher cut-point) the number of participants that underreported time spent in physical activity by CCPAQ declined but the validity correlation seemed to be lower. Using of the lowest cut points (Freedson et al).^[31] would have resulted in overestimates of accumulated MVPA, which showed no criterion validity in the CCPAQ compared with the accelerometer. As such, it seemed that the threshold used in this study provided a good approximation of physical activity in Chinese children and adolescents. Bland-Altman analyses revealed relatively wide variation in limits of agreement, suggesting that the CCPAQ is more reliable for group physical activity estimates than individual investigations and therefore might be particularly useful as surveillance measure. Compared with the accelerometer data, we also found that the CCPAQ underestimated all the physical activity variables besides physical activity energy expenditure. The mean differences between the two measures were -78.0 min/day for total physical activity, -127.7 min/day for sedentary behavior, and 226.8 kcal/day for physical activity energy expenditure. These differences might be caused by recall bias, especially the recall of a whole day, which might influence the retrospective response. Unlike the other studies which over-reported physical activity levels, our study seemed to be less likely to record the time spent on LPA. The amount of LPA that children participate in represents a very large and trivial fraction of their overall activity, which may be difficult to recall completely and limits the validity of subjective instrument.

Responses to the CCPAQ were compared between boys and girls to determine whether the measure reflected sex specific differences in physical activity and whether the reliability and validity differed by sex. Although sample size deceased due to the stratification analyses, recent study showed that 50 to 99 participants could provide stable agreement estimates between subjective and objective measurement of physical activity^[40]. Besides, the response rate in this study was high (88% for validity study) and the general wearing time of accelerometer was long due to the effective study management and implementation using the COSMIN checklist as a standardized tool for research. Thus, this study could provide support for investigating the reliability and validity coefficients of the CCPAQ in both boys and girls.

A feature of the CCPAQ is the integration of a compendium of energy costs designed for the specific types and intensity of physical activities in children. Since youth’s basal metabolic rates decline gradually as they grow and become mature, the CCPAQ converts reported physical activity data to energy expenditure using determinate Metabolic Equivalent (METs) in different age groups. Previous studies examining the validity of energy expenditure in young people indicated limited validity compared with accelerometer data^{[41, 42]}. The higher correlation coefficients found in this study might be due to the use of youth-derived standard METs of specific activities. To our knowledge, this is the first study based on standardized METs reference in children to assess the validity of estimated physical activity energy expenditure from questionnaire compared with the accelerometer. Assessment of physical activity and estimation of its energy expenditure in large scale has many implications for public health, applied research and clinical practice in young population.

The two administrations of the CCPAQ were completed in reference to the same 7-day recall so that the differences between the two administrations only consisted of reporting error with no variation due to the real activity differences over time. However, the short time interval between two administrations in the reliability analyses was a potential limitation as it was possible for participants to recall answers in the first administration. For measuring physical activity during the past week, a time interval of 1 day to two weeks might be appropriate and if there are more questions (e.g. more than 25 questions) and the questionnaires is complicated, the time interval could be shorten^[43]. It is also important to recognize that data were collected in two areas in China and the results may not be generalized to other areas of China and other settings.

The CCPAQ is a unique questionnaire developed to focus on physical activity pattern and energy expenditure measurements across the whole day during last week for Chinese children. The CCPAQ has been found to be a reliable tool that exhibits acceptable content and construct validity of physical activity pattern. It might be an easy and feasible tool used in children with the highlight of the importance of 24-hour movement behaviors and thus could be particularly useful for large scale surveys and surveillance measures. This study provided insight into a surveillance method for physical activity pattern to fill the gaps in developing countries. Future research is needed to focus on activity components and context to promote physical activity.

All authors contributed to the conception of the review and the revision of the manuscript. Prof. ZHAO Wen Hua directed all aspects of the study, including the design of the study protocol, performance of the research and interpretation of the results. Ms. YANG Xi contributed to the conception, design, acquisition, analysis and interpretation of data. Prof. JAGO Russell contributed intellectual input into the main ideas of this paper and assisted the writing of the manuscript. Dr. ZHAI Yi and Dr. YANG Zhen Yu contributed to study design and assisted with writing of the manuscript. Dr. WANG Yu Ying and Ms. SI Xiang contributed to study design. WANG Jun, GAO Jian Fen, CHEN Jing Ron, and YU Ying Jie coordinated the data-collection. All the authors read and approved the final manuscript.

All authors would like to thank all the students, parents, teachers and schools who supported this study. The authors gratefully acknowledge the contribution of all project staff from National Institute for Nutrition and Health, Chinese Center for Disease Control and Prevention, Centers for Disease Control and Prevention of Beijing, Chongqing, Yan Qing District and Wanzhou District, especially Dr. ZHANG Qian, Dr. ZHANG Jian, Ms. LIU Dang, Ms. JU La Hong, Mr. YU Wen Tao for the collection of data, and Dr. QIU Jun Qiang from Department of Sports Biochemistry, Beijing Sport University for the loan and help with the accelerometers for data collection.

The authors declare that they have no competing interests.