Developing a Subjective Evaluation Scale for Assessing the Built Environments of China’s Hygienic City Initiative

China’s Hygienic City Initiative was implemented across the country in 1989 with the aim of improving urban health and enhancing people’s awareness of hygiene and improving their health^[1]. This initiative is one of the most important public health policies that have been implemented in China.

Studies have demonstrated that creating hygienic cities in China plays a positive role in promoting the construction of urban health infrastructure, enhancing urban health management, improving the urban environment, and preventing and controlling the effects of vectors and infectious diseases, thereby improving residents’ health^[2-4]. In 2013, the World Health Organization gave special recognition to the Chinese government for its Healthy City (Hygienic City) Initiative, praising China’s outstanding achievements in creating healthy cities nationally.

The development of the Hygienic City Initiative refers to comprehensive social management, which mainly focuses on the factors influencing the population’s health. Currently, Chinese scholars have investigated hygienic cities mainly by summarizing their implementation and studying the methods through which such initiatives were launched in each city. They have evaluated the objective indicators of environment creation from the viewpoints of policy makers and policy implementers. Few scholars have assessed subjective perceptions of the built environments of hygienic cities from the perspective of the residents affected by the policy. Foreign studies on the evaluation of built environments began in the 1960s. At present, many scholars believe that subjective perceptions of built environments are the most direct factor determining their level of success. Therefore, it is inadequate to study built environments using only an objective evaluation method.

In this study, we developed a standardized scale to measure subjective perceptions of the built environment of a hygienic city from the perspective of residents. This increased the reliability and validity of the measurement tool, making it effective for evaluating subjective perceptions of the built environments of hygienic cities.

The residents of Chaoyang District, Beijing, were selected as respondents. The construction of a hygienic city was initiated in the district in 2017, and its construction was completed within three years. Its residents were invited to participate in an online survey through the Wenjunxing network platform. Our respondents were required to meet the following conditions: (1) they were residents who had been living in Chaoyang District, Beijing, for at least a year and (2) they were at least 18 years old. A total of 1,047 valid completed questionnaires were collected from Chaoyang District residents from March to April 2020. The first 499 questionnaires were used to evaluate and screen the items. The respondents included 220 (44.1%) men and 279 (55.9%) women, who were aged 35.24 (± 10.51) on average. The additional 548 questionnaires were used to assess the reliability and validity of the scale and included responses from 236 (43.1%) men and 312 (56.9%) women with an average age of 34.71 (± 10.28).

Our study was conducted in four phases.

The major factors involved in creating a hygienic city in China were identified through policy analysis, a literature review, and expert consultations. Twenty-nine items were initially chosen for inclusion as factors in the scale, whose aim was to assess the subjective perceptions of residents.

A total of 11 residents of Chaoyang District, Beijing, were invited to evaluate whether the content included in the scale correctly represented the lives of common residents. We asked whether there was any ambiguity in the 29 items. Subsequently, we excluded seven items that were unrelated to residents’ lives, such as ‘Is medical waste disposed of according to regulations?’ and ‘Are patients with severe mental disorders handled effectively?’ We also excluded or modified other items that were identified as unclear or ambiguous. Our final, revised scale contained a total of 22 items.

A total of 499 valid questionnaires were collected from the network platform WJX.cn. We conducted a statistical analysis of the 22 reserved items using the discrete trend method, critical ratio method, correlation coefficient method, and factor analysis method. The items were evaluated based on sensitivity, distinction, and representativeness. An item was deleted if any two of the following criteria were satisfied.

(1) Discrete trend method: The mean value, standard deviation, and coefficient of variation (CV) of each item were calculated based on descriptive analysis, and the items with CV < 0.2 were deleted^[5].

(2) Critical ratio method: With the 27^th and 73^rd percentiles of the total score as the cut-off points, all the research subjects were assigned to a low-score group (total score ≤ P₂₇), medium-score group (P₂₇ < total score < P₇₃), and high-score group (total score ≥ P₇₃). The difference in the average score of each item between the low-score group and the high-score one was examined using a t-test, and the items with no statistically significant difference (α > 0.05) were deleted^[6].

(3) Correlation coefficient method: The correlation coefficient between the score of each item and the total score of the scale was calculated to reflect the correlation of each item. The higher the correlation coefficient, the more consistent the detected property of the items. The items with a corresponding correlation coefficient < 0.4 were deleted^[7].

(4) Factor analysis method: The main indicators determining the factors were considered according to the degree of correlation between principal components and indicators based on characteristic root > 1 and cumulative contribution rate > 50%. The indicators with a load on all common factors < 0.4 were deleted^[7].

An additional 548 valid questionnaires were collected using the network platform WJX.cn to assess the reliability and validity of the scale.

(1) Reliability evaluation: The internal consistency reliability and split-half reliability of the scale were evaluated using Cronbach’s α coefficient and Spearman-Brown’s split-half reliability coefficient. The value of the Cronbach’s α coefficient ranges from 0 to 1, and α ≤ 0.6 generally indicates inadequate internal consistency and reliability^[8]. The value of the split-half reliability coefficient is usually required to be > 0.7^[7].

(2) The dimensions of the scale were constructed using EFA. We then calculated the factor loading and goodness-of-fit index (GFI) of each item using confirmatory factor analysis. Meanwhile, the construct validity of the scale was verified by determining the correlations of factors with the total scale.

a. Factor loading

Hair et al. (2006)^[9] argued that an adequate factor loading indicates that an item has construct validity. Tabachnica and Fidell (2007)^[10] proposed that a factor loading larger than 0.71 (i.e., the latent variable can explain 50% of the variations in the observed variable) is optimal. Moreover, a factor loading > 0.55 is preferable, and a factor loading < 0.32 is unsatisfactory (i.e., the latent variable cannot explain 10% of the variations in the observed variable). Generally, such items may form latent variables, but they contribute little to the analysis, so they may be deleted to improve the consistency of all the factors.

b. Goodness-of-fix index (GFI) of the model

The GFI of the model is a statistical indicator applied to assess the degree of fitness between the theoretical model and the data. Various types of GFI can be used to measure the theoretical model from the perspectives of model complexity, sample size, relativity, and absoluteness. In this study, χ²/df < 3, root-mean-square error of approximation (RMSEA) < 0.08, adjusted GFI (AGFI) > 0.9, GFI > 0.9, incremental fit index (IFI) > 0.9, comparative fit index (CFI) > 0.9, parsimony normed fit index (PNFI) > 0.5, and parsimony-adjusted CFI (PCFI) > 0.5 were used as the standards for measuring the fitness of the model’s structure.

c. Analysis of correlation among dimensions

The correlation coefficient between each dimension and other ones, Cronbach’s α coefficient of the dimension, and the correlation coefficient between the dimension and the total scale were utilized to evaluate the discriminant validity of the scale. If the correlation coefficient between each dimension and the others was smaller than the Cronbach’s α coefficient of the dimension and the correlation coefficient between the dimension and total scale, the scale was considered to have discriminant and convergent validity.

Results of the Discrete Trend Method　The CV of every item was > 0.2, indicating good sensitivity among the items. Therefore, all the items were retained.

Results of the Critical Ratio Method　At the level of α = 0.05, there were no statistically significant differences in the average scores of items “Access to public toilets” (Can you find public toilets nearby at an outdoor location?) and “Measures to promote a healthy diet ” (Can you obtain knowledge about healthy diets at your residence?) between the high-score group and the low-score one.

Results of the Correlation Coefficient Method　As the score of each item is a graded variable that cannot be directly calculated by the linear correlation coefficient, it was judged using Spearman’s rank correlation coefficient. Through this method, items “City functional lighting” (Does the night time lighting in the area of residence satisfy the demands of night time travel?) and “Daily disposal of the community’s garbage” (Is the garbage in the area of residence cleared on time and without being left overnight?) were not within the standard limit.

Results of Exploratory Factor Analysis　We conducted the Kaiser-Meyer-Olkin (KMO) and Bartlett tests. The KMO value was close to 1 (KMO value = 0.903). Meanwhile, P < 0.01 was detected in the Bartlett test of sphericity, and the null hypothesis was rejected, implying that there were correlations among the variables. All these results indicated that the data were applicable for factor analysis. The results showed that five common factors (characteristic root > 1) were produced, and the loading for none of the items of the five common factors was under 0.4, so no indicator was deleted.

Based on these results, none of the items met two criteria for deletion at the same time. Hence, all 22 items were retained.

Since all 22 items were retained, the five common factors resulting from the analysis described above were taken as the dimensions of the scale to construct the scale’s structure. The five common factors represented specific meanings combined with professional knowledge. Factor 1 stood for evaluation of urban lifestyle, including eight items (garbage collection, smoking control and bans in public places, vaccination availability, voluntary blood donations, measures to promote a healthy diet, measures to promote personal hygiene, vector control, and services from community health service centers). There were five items related to Factor 2 regarding the evaluation of various aspects of urban governance, including air quality, drinking-water safety, food safety, management of fair-trade markets, and management of street vendors. Factor 3 represented the evaluation of urban basic functions, including three items: urban public fitness facilities, urban greening, and healthy places. Factor 4 for the evaluation of the urban environment included four items, including garbage-collection facilities, “No Smoking” signs, daily disposal of the community’s garbage, and urban sanitation. There were two items for Factor 5 that pertained to urban amenities, and these included City functional lighting and access to public toilets (Table 1).

Reliability Evaluation　Evaluations of internal consistency reflect the consistency and stability of a scale’s items. In this study, the total scale and all the factors were subjected to Cronbach’s α coefficient analysis. We found the total Cronbach’s α coefficient of the scale for subjective perceptions of the built environments of hygienic cities in China to be 0.876 (> 0.6), suggesting that the scale has an adequate level of internal consistency. Furthermore, the Cronbach’s α coefficient of the five dimensions was 0.879 (urban lifestyle), 0.706 (urban governance), 0.593 (urban basic functions), 0.533 (urban environmental sanitation), and 0.402 (urban amenities). According to the literature, the Cronbach’s α coefficient of a scale containing fewer than four items may be below 0.6 or 0.5^[11]. This indicates that the internal consistency of urban environmental sanitation (four items) and urban amenities (two items) is acceptable.

In addition, the items of the scale were split into two equivalent parts, and the correlation coefficient between the scores of the two parts was calculated using the Spearman-Brown formula. The split-half reliability coefficient of the scale was 0.796.

Construct Validity Evaluation　A significance test of factor loading, the model’s goodness-of-fit test, and an analysis of the correlations among the dimensions were used to evaluate the construct validity.

(1) Significance test of factor loading

The differences in all parameters were statistically significant (P < 0.01), suggesting that the measurement model confirmed our hypothesis. None of the items had a factor loading lower than 0.32, implying that the latent variables adequately explained the items and that the theoretical model was accurate (Table 2).

(2) The model’s goodness-of-fit test

The results of the goodness-of-fit test for our model displayed χ²/df = 2.46 < 3 (critical value), RMSEA = 0.052 < 0.08 (critical value), AGFI = 0.902 > 0.9 (critical value), GFI > 0.9 (critical value), IFI > 0.9 (critical value), CFI > 0.9 (critical value), PNFI > 0.5 (critical value), and PCFI > 0.5 (critical value). This indicates that our model possesses adequate fitness (Table 3).

(3) Analysis of correlations among dimensions

The Pearson correlation coefficients between the dimensions and the total scale for subjective perceptions of the built environments of China’s hygienic cities had statistical significance. Specifically, the correlation coefficient between urban amenities and the total scale was 0.454, while the correlation coefficients of the remaining dimensions with the total scale were > 0.60. The correlation coefficients between each dimension and others were all smaller than the correlation coefficients between the dimension and the total scale, indicating that the dimension’s structure demonstrated discriminant validity. Furthermore, the Cronbach’s α coefficient of each dimension was greater than the correlation coefficients between the dimension under examination and the others, suggesting a favorable convergent validity of the dimension’s structure (Table 4).

A city’s built environment is one of the most determinative factors of its residents’ health^[12]. The term "built environment" refers to all kinds of artificially constructed and renovated buildings and places as well as the environments that can be changed through policies and human behaviors^[13]. The measurement methods for assessing a built environment can be characterized as a subjective evaluation of it (referred to as the "subjective built environment") or an objective one (referred to as the "objective built environment")^[14-16]. An assessment of the subjective built environment is based on the environmental perceptions of the respondents, and the respondents’ satisfaction with and perception of the environment are reflected in scales and questionnaires.

During the implementation of public policies, the attitude of the target group influences whether they meet expectations. Some researchers^[17] have proposed a customer-oriented pattern for assessing the effects of public policies and summarized the attitudes of ‘customers’ (target groups of policies) towards these policies. The subjective evaluation scale for the built environment of China’s hygienic cities targets permanent residents of them and aims to objectively measure the effect of the Hygienic City Initiative from the subjective viewpoints of their residents. The scale also aims to assess the perceptions of the population affected by the policies, thus providing a measurement tool for policy makers to evaluate the subjective built environments of hygienic cities of China. It is important for policy makers to understand the implementation of policies and propose improvement measures.

We applied the theories of scale development^[18] to construct a measurement scale for evaluating residents’ perceptions of the environment created by the Hygienic City Initiative. The evaluation’s results can be regarded as an effective supplement to objective measurements of the built environment. During the scale’s development, the items were initially subjectively screened through consultation with experts and interviews with residents. The final 22 items formed a scale for assessing subjective perceptions of the built environments created by China’s Hygienic City Initiative, which includes five dimensions (urban lifestyle, governance, basic functions, environmental sanitation, and amenities). We evaluated these dimensions using screening methods and statistical analysis.

The results of the scale’s reliability evaluation revealed that the Cronbach’s α coefficient and split-half reliability coefficient of the total scale for subjective perceptions of the built environments of China’s hygienic cities were in line with the criteria for the reliability coefficients of the scale (Cronbach’s α coefficient > 0.6 and split-half reliability coefficient > 0.7). These results demonstrate that the scale is reliable. According to the validity analysis results, the GFI of the model in the confirmatory factor analysis was within the ideal value range. The Pearson correlation coefficient of each factor with the total scale was positive, and the correlation coefficients between each dimension and the others were all smaller than the correlation coefficients between the dimensions and the total scale. These results indicate that the scale has a clear and reasonable structure as well as validity.

A test-retest reliability evaluation was not conducted because the relevant information was collected from the networks anonymously. For this reason, the reliability of the scale should be investigated further. At the same time, since there is no recognized or valid scale setting the standard for evaluations of the built environments of hygienic cities, the correlation between the new scale and a standard one cannot be tested. In addition, the method of expert consultation was adopted in this study to judge the representativeness of the content, and quantitative evaluations of the content validity indices are bound to be better than those of qualitative methods.

In conclusion, the scale for assessing subjective perceptions of the built environments resulting from China’s Hygienic City Initiative in this study exhibits strong reliability and validity. Future empirical studies of the scale should be carried out using data from Chinese residents to further verify the value of the scale in terms of practical applications.