Introduction
Environmental pollution, waste of resources, economic recession, political games, and other problems force human beings to take the path of sustainable development. Education is an important force in promoting sustainable social development, helping to narrow individual cognitive differences, seeking common grounds while reserving differences, and establishing values for sustainable development. The United Nations Educational, Scientific and Cultural Organization (UNESCO) is the first to actively promote the concept of sustainable education. In 2022, UNESCO published “Education Policy Outlook 2022”, emphasizing the importance of educational change for green transformation and encouraging the integration of sustainable development education into mainstream education systems. At the same time, the United States launched the Education Sustainability Movement earlier. In 1990, Jean Mayer, the president of Tufts University in the United States, initiated a conference on sustainable education in universities. In 1994, the Campus Earth Summit held by Yale University approved the “Blueprint for a Green Campus” report. In recent years, the European Union has also issued a series of relevant policy documents. For example, in 2019, the “European Green Deal” was released, and in 2022, “Proposal for A Council Recommendation on Learning for Environmental Sustainability” and “Proposal for A Council Recommendation on Learning for the Green Transition and Sustainable Development” were released [1]. Therefore, promoting sustainable development in education has reached a consensus in developed countries.
China has always attached great importance to sustainable development in education, forming a “student-centered” training concept, and emphasizing the comprehensive improvement of talent quality from the aspects of “moral education”, “intellectual education”, “physical education”, “aesthetic education”, and “labor education”. General Secretary Xi Jinping has underscored the importance of emphasizing the subjectivity of students in the work of ideological and political education. It is essential to respect the needs and individuality of students, guide them in establishing correct values, and create conditions conducive to the comprehensive development of students [2-4].. Therefore, improving the effectiveness of ideological and political education is an important aspect of the sustainable development of higher education in China. This article will objectively evaluate the current level of graduate education based on the background of sustainable education, and provide the reference for decision-making departments.
The literature on graduate education in China and abroad mainly focuses on training models, innovative abilities, and ideological and political education [4- 8]. LI et al. (2021) studied the collaborative promotion of discipline construction and postgraduate education [9]. ZHAO et al. (2021) constructed the “1335” model of professional graduate training [10]. FAN and XIAO (2022) put forward the model of cross-training of liberal arts graduate students [11]. LI (2022) designed a collaborative training mechanism for professional degree graduates [12]. FU and HAN (2022) studied the factors affecting the employment of doctoral students [13]. SUN et al. (2023) analyzed the interdisciplinary talent cultivation model of Cornell University [14]. LIU and ZHANG (2023) believe that graduate education requires multi-objective integration and multi-departmental collaboration [15]. LI and MEMGN (2023) hold that the cultivation of professional graduate students needs to highlight the unity of theory and practice [16]. LI et al. (2023) opine that it is necessary to emphasize the “interdisciplinary” characteristic for cultivating doctoral students. [17]. ZHOU et al. (2022) believe that the cultivation of professional graduate students needs to focus on the cultivation of innovative abilities [18]. LI et al. (2022) insist that a first-class discipline group needs to improve the quality of graduate education through holistic thinking [19]. YANG et al. (2023) analyzed the innovative training model for graduate students in economics and management at German universities of science and technology [20]. TU et al. (2022) proposed a “1+6” model for the training of science and engineering graduate students [21]. These studies explore how to improve the quality and innovation ability of graduate education from the perspectives of schools, students, and society. In addition, the issue of ideological and political education for graduate students has also attracted attention. LI et al. (2023) believe that universities need to explore the role of full media in graduate ideological and political education in terms of teams, platforms, and mechanisms [22]. CHENG et al. (2023) hold that the “walking red classroom” should be used to improve the quality of research-based education [23]. ZHENG (2023) believes that it is necessary to build a graduate training model that combines explicit and implicit education [24]. ZHANG and ZHANG (2021) believe that graduate supervisors and counselors should work together to improve the quality of graduate education [25]. The above research analyzes the process of integrating ideological and political education into graduate training from the perspectives of form, content, carrier, and function, which have important value.
To sum up, the existing results mainly focus on the model, path and method of graduate cultivation, and the importance of graduate ideological and political education has aroused attention. The above studies have amply demonstrated the importance of graduate education, which is also a difficult problem in current research. However, there are still obvious shortcomings in the existing research: (1) Most of the studies are mainly qualitative analyses. The existing research has carried out a lot of work from policy, mechanism and ideas, but lacks empirical data support, which would lead to the subjectivity of conclusions. (2) There are few studies on how to evaluate the education level of postgraduates. If the problem of graduate education cannot be accurately evaluated, then the basis of relevant research conclusions will be insufficient. To solve these problems, this paper will combine qualitative and quantitative methods, theoretical and empirical research, to study the evaluation of graduate education effect.
This paper realizes innovations in the following aspects: First, based on the theory of educational evaluation and starting from the result, this paper discusses the effects and problems of graduate cultivation.; Second, the quantitative analysis method is adopted to comprehensively apply the analytic hierarchy process, questionnaire survey, comprehensive index method and other techniques to evaluate the cultivation effect of graduate students, and convincing conclusions are obtained through data analysis. Therefore, this paper will provide an important reference for the cultivation of graduate students in universities and the formulation of educational policies by government departments.
Material and Methods
Construction of Evaluation Index System Based on Expert Group Decision-Making Method
The evaluation of the effect of integrating red genes into graduate ideological and political education is a part of curriculum ideological and political education, which is an emerging direction in the field of higher education in China and a key area of teaching reform [26]. At present, there is relatively little empirical research in the relevant field, most of which focuses on qualitative analysis, so the unified evaluation index system has not been formed.
The construction of an evaluation index system for integrating red genes into graduate ideological and political education, which is a group multi-attribute decision-making problem, is difficult to complete by a single expert and then collaborative decision-making by multiple experts is requested. Therefore, this article will adopt the method of expert group decision-making to construct an index system. Referring to existing research findings [27-29], combined with the problems studied in this paper, the construction method of the evaluation index system based on the expert group decision-making method is as follows:
Step 1: Establish a basic indicator library. Firstly, using the literature review method and tools such as grounded theory, the relevant research questions are systematically sorted out and an initial indicator library is constructed [30].
Step 2: Select suitable experts. In order to obtain a scientific and reasonable indicator system, selected experts need to fulfill certain conditions:
(1) Professionalism. Experts are required to engage in work related to graduate education management.
(2) Authority. The selected experts need to have a certain academic influence and be familiar with the graduate teaching and research work of university courses. (3) Rationality. Experts need to be well acquainted with ideological and political education and management in graduate courses.
Step 3: Multiple rounds of expert consultation. This work includes at least three rounds of consultation. In the first round, experts are invited to score the indicators through electronic questionnaires. In the second round, according to the experts’ scores in the first round, the indicators are ranked in descending order, and those that meet the requirements will be selected accordingly. At the same time, the results are fed back to experts for their comments. In the third round, the indicator system will be further adjusted and optimized based on the experts’ opinions through telephone or face-to-face communication.
Step 4: Determine the evaluation index system. Considering expert opinions, the final evaluation index system will be released.
Weight Construction of Evaluation Indicators for Graduate Education Effectiveness Based on the Analytic Hierarchy Process
In order to scientifically and reasonably evaluate the effectiveness of integrating red genes into graduate ideological and political education, in addition to constructing an evaluation index system, it is also necessary to clarify the weights of the indicators. In the field of education and teaching, the most commonly used method is the Analytic Hierarchy Process (AHP), which is an effective method for solving the weights of multi-objective and multi-attribute decision-making indicators [31-33]. Firstly, proposed by Professor T. L. Saaty in the United States, AHP is particularly suitable for disposing of fuzzy attribute problems such as educational evaluation [34].
Referring to relevant studies [35-37], the weight determination process for evaluating the effectiveness of integrating red genes into graduate ideological and political education based on the Analytic Hierarchy Process is as follows:
Step 1: Determine the judgment matrix. Different indicators are compared one by one to obtain their relative scales. The main formula is as follows:
Table 1.
Nine Scale Method.
In the above equation, bij represents the importance of different indicators and n represents the number of indicators. For calculation bij, the following 9-scale method can be used:
Step 2: Normalize the judgment matrix.
Step 3: Determine the weight vector of the standard layer γ. The specific calculation formula is as follows:
Step 4: Test of weight vectors. The weight vector γ is mainly determined by B(bij)n×n. The commonly used consistency indicator is DR. The calculation formula of DR is as follows:
Among them, γmax represents the maximum value of the feature vector. In the above equation, m represents the number of decision options, and the average consistency index R1 can be obtained by looking up Table 2.
If R1≤0.1, the judgment matrix Gx passed the consistency test and the indicator weights could be used. Otherwise, if R1>0.1, the judgment matrix did not pass the consistency test, and a secondary evaluation is needed to correct the weight of the indicators until all indicators pass the test.
Construction of a Comprehensive Index Model for Evaluating the Effectiveness of Graduate Education
Due to the fuzzy attribute of the evaluation of graduate education effectiveness, it is necessary to compare the specific differences in educational effectiveness through quantitative means. This paper introduces comprehensive indexing for comparison. Based on statistical thinking, the comprehensive index method standardizes different evaluation indicators and then obtains corresponding indices through weighting [38-40].
Referring to relevant studies [41-43], the modeling process of the comprehensive index method is mainly divided into two steps:
Step 1 Standardization of data. The specific formula is as follows:
Xij represents the standardized value of the data, X'ij represents the original value of the data, Xjmax and Xjmin respectively represent the maximum and minimum values.
Step 2: Calculate the comprehensive index. Use standardized data and indicator weights to obtain evaluation indices ρ.
Among them, γi is the weight of the i-th indicator calculated by the Analytic Hierarchy Process, and X'ij is the standardized value of the i-th data. By comparing the ρi sizes, the effectiveness of graduate education can be compared.
Step 3: Evaluation result level.
As the results obtained by using the comprehensive evaluation method are still difficult to directly reflect the “good” and “bad” evaluation results, it is possible to draw on the ideas of existing research and correspond the evaluation results to the corresponding levels. By referring to relevant practices [44-46], guidelines can be established between evaluation results and effectiveness levels.
Data Sources
The relevant data in this research is obtained through the questionnaire survey. In the construction stage of the indicator system, selected experts mainly consist of teachers engaged in graduate teaching management at a certain university in Tianjin, including master’s supervisors and graduate management personnel. There are 5 experts, all of whom have rich experience in graduate education management. In the stage of evaluating the effectiveness of graduate education, a random sampling survey is mainly conducted on graduate students in school. In order to analyze the effectiveness of the study, 120 graduate students from 2 colleges of a certain university in Tianjin are randomly selected for a survey. A total of 110 questionnaires are collected, with a response rate of 91.67%. Nine invalid questionnaires are deleted, and 101 are ultimately valid, with a validity rate of 91.8%.
Results and Discussion
Evaluation Index Results Based on Expert Group Decision-Making Method
In response to issues related to the effectiveness of graduate education, the research group invited 5 doctoral, professor, and master’s supervisors from universities in Tianjin to form an expert evaluation group. The experts conducted five rounds of evaluation on the evaluation indicators. In the first round, various methods such as either/or, exclusion, and pairwise comparison were used to preliminarily rank the basic indicator library. Then, the expert group formed a unified opinion through online discussions. Finally, the indicator system as shown in Table 3. was obtained.
From the results, the expert group has mainly formed an indicator system of five dimensions: academic discipline construction, daily management, party and youth league construction, academic integrity, and curriculum ideological and political education. And following the principles of “measurable” and “concise”, corresponding three-level indicators have been set. Considering the principle of “student-centered”, in terms of the three-level indicators, data is mainly obtained through graduate student ratings to assess students’ satisfaction with educational outcomes.
For example, regarding the construction of academic discipline, a three-level indicator is set, “A11. How do you rate the integration of red genes into graduate student academic discipline in your university?”; In terms of daily management, set a three-level indicator “B11. How do you rate the integration of red genes into the daily management of graduate students in your university?”; In terms of party and youth organization construction, set a three-level indicator “C11. How do you rate the integration of red genes into graduate party and youth organization construction in your university?”; In terms of academic integrity, set a three-level indicator “D11. How do you rate the integration of red genes into graduate academic integrity education in your university?”; In terms of ideological and political education in the curriculum, a three-level indicator is set, “E11. How do you rate the integration of red genes into graduate courses in your university?”. The construction of the indicator system provides an important basis for scientific and reasonable empirical research in the next stage.
Weight Results of Evaluation Indicators for Graduate Education Effectiveness Based on Analytic Hierarchy Process
Applying the Analytic Hierarchy Process (AHP) model, the article obtains students’ data on the effectiveness of integrating red genes into graduate education through a questionnaire survey. The results are shown in Fig. 1. Furthermore, the evaluation results of graduate education effectiveness are calculated through statistical software, and the weight values of different indicators are shown in Table 4.
The results show that the feature vectors of the five indicators A11, B11, C11, D11, and E11 are 0.908, 0.974, 0.937, 0.968, and 1.213, respectively. The maximum feature vector of the E11 indicator is 1.213, indicating that teachers and students pay the highest attention to the effectiveness of curriculum ideology and politics. Therefore, as the educational management department of the school, it is necessary to attach importance to the construction of ideological and political education in the curriculum and provide training, supervision, guidance, motivation, and support to teachers who carry out graduate courses in ideological and political education.
Table 3.
Indicator System.
Furthermore, from the weight results of the indicators, the weight values of A11, B11, C11, D11, and E11 are 18.16%, 19.47%, 18.75%, 19.36%, and 24.25%, respectively. A11 has the lowest weight, while E11 has the highest weight at 24.25%. The result is consistent with the feature vector, which is evidence that the data results are robust. All weight values remain within the range of [18% to 25%], indicating significant differences in the importance of the indicators.
When using the Analytic Hierarchy Process to calculate indicator weights, it is also necessary to conduct consistency checks on the results. As shown in Table 4., the value of CR is -0.701, which is less than 0.1, indicating that the judgment matrix has passed the test. The weight of the indicators calculated using AHP is significant.
Analysis of Evaluation Results of Graduate Education Effectiveness
According to the indicator system of the expert group decision-making method, combined with the Analytic Hierarchy Process to determine the weight of indicators, the comprehensive index method is used to obtain the evaluation scores of all evaluation objects regarding the effectiveness of graduate education. The specific results are shown in Fig. 2.:
Table 4.
Indicator Weight Results
| Term | Feature vector | Weight value | Maximum eigenvalue | CI value |
|---|---|---|---|---|
| A11 | 0.908 | 18.16% | 1.858 | -0.785 |
| B11 | 0.974 | 19.47% | ||
| C11 | 0.937 | 18.75% | ||
| D11 | 0.968 | 19.36% | ||
| E11 | 1.213 | 24.25% |
Table 5.
Consistency Test Results
| Maximum eigenvalue | CI value | RI value | CR value | Consistency test results |
|---|---|---|---|---|
| 1.858 | -0.785 | 1.12 | -0.701 | adopt |
From the results, the average comprehensive index of graduate education effectiveness for evaluation objects 1-25 is 0.99492, the average comprehensive index of graduate education effectiveness for evaluation objects 26-50 is 1.00996, the average comprehensive index of graduate education effectiveness for evaluation objects 51-75 is 1.0008, and the average comprehensive index of graduate education effectiveness for evaluation objects 76-101 is 1.124. The comprehensive evaluation score of 101 evaluation subjects on the effectiveness of research education is 1.033326733, which is greater than 1.0. This indicates that the university attaches great importance to the ideological and political education of graduate students, which has a significant effect. The university has carried out a lot of work around the construction of A. academic discipline, B. daily management, C. party and youth league construction, D. academic integrity, and E. course ideological and political education for graduate students, which is conducive to cultivating high-quality and top-notch talents.
Furthermore, in order to more intuitively demonstrate the level of graduate education effectiveness in the university where the survey samples are located, a grading standard for graduate education effectiveness is constructed based on relevant practices (see Table 6). According to the score of the comprehensive evaluation index, the educational effect is divided into 7 levels, namely I, II, III, IV, V, VI, and VII, corresponding to the level of educational effect as “excellent, very good, good, moderately good, comparatively good, qualified, and poor”. Therefore, the selected sample in this article gets an average comprehensive evaluation score of 1.00996 on the effectiveness of graduate education, which is in the range [1.0,1.1], corresponding to level II, indicating very good educational effects. Therefore, it can be considered that the sample university has achieved significant results in cultivating graduate students to establish correct values.
Discussion on the Evaluation Results of Graduate Education
From the foregoing research, it can be seen that the comprehensive use of the expert group decision-making method, analytic hierarchy process, and comprehensive index method are effective in evaluating the effect of graduate education. This model changes the traditional model of simple, qualitative assessment. From the results, the effect of postgraduate education is divided into seven grades: Ⅰ, Ⅱ, Ⅲ, Ⅳ, Ⅴ, Ⅵ and Ⅶ. The comprehensive evaluation score of the study sample is 1.00996, which is greater than 1.0, indicating a very good educational effect.
From the specific subgroups, the average evaluation score of subjects 1-25 is 0.99492, and its effect level is Ⅲ. The average evaluation score of subjects 26-50 is 1.00996 (effect level II), and the average evaluation score of subjects 51-75 is 1.0008 (effect level II). The average evaluation score of subjects 76-101 is 1.033326733, and the effect grade is Ⅱ. Therefore, both the whole and the group show that the effect of postgraduate education is at a good level.
Conclusions
In the context of sustainable development, in order to achieve the sustainable goals of education and cultivate high-quality talents, the evaluation of the effectiveness of graduate education has important practical value. This article constructs an education effectiveness evaluation index system based on an expert group decision-making method from the perspective of integrating red genes into graduate education effectiveness evaluation. The weights of the indicators are determined through the Analytic Hierarchy Process, and finally, the evaluation results are formed using the comprehensive index method. The theoretical and practical value of this paper is as follows: it expands the theory of evaluation of the effect of postgraduate education through a comprehensive method; Through the empirical analysis method, the actual level of graduate education effect is accurately estimated. This paper draws the following conclusions:
(1) The evaluation index system for the effectiveness of graduate education can be carried out around five aspects: A. academic discipline construction, B. daily management, C. party and youth league construction, D. academic integrity, and E. curriculum ideological and political education. This article uses the expert group decision-making method and after 5 rounds of repeated inquiries, obtains a scientific and reasonable indicator system.
(2) The weights of indicators for evaluating the effectiveness of graduate students exhibit typical heterogeneity. The weight values of the five indicators A11, B11, C11, D11, and E11 are 18.16%, 19.47%, 18.75%, 19.36%, and 24.25%, respectively. A11 has the lowest weight, with a value of 18.16%; E11 has the highest weight, with a value of 24.25%. The difference between the highest and lowest is 6.09%.
(3) The average score for evaluating the effectiveness of graduate education in the university where the survey samples are located is 1.00996, which is in the range [1.0,1.1], corresponding to level II, indicating very good educational effectiveness. This indicates that the university has carried out a lot of work around the construction of graduate students’ academic discipline, daily management, Party and Youth League building, academic integrity, and ideological and political education in courses. Paying attention to ideological and political education is conducive to improving the quality of talent cultivation.
This paper makes a systematic analysis of the comprehensive evaluation of graduate education from both theoretical and empirical levels. Although the research is somewhat innovative, there are still the following deficiencies, which are worth further study in the future:
(1) This paper uses the expert group decision method, analytic hierarchy process and comprehensive index method. Although these methods have reached good results, some other more effective methods, such as fuzzy comprehensive evaluation, can be explored.
(2) The data of a university in Tianjin is selected as a typical research sample. Subsequently, further empirical research can be conducted by expanding the sample.
