The evaluation index improvement for the relocation of research equipment

In this paper, evaluation index improvement for research equipment relocation is presented and discussed to boost the effectiveness of research equipment management. The analytic hierarchy process (AHP) model was designed for the evaluation index improvement for research equipment relocation, and the pairwise comparison scale was set up based on the importance of each evaluation criterion. The consistency rate (CR) was measured, and it was confirmed that the decision-making was reasonable. The improvement of the evaluation index was necessary for the objective and fair relocation of research equipment. Therefore, the evaluation index for the relocation of research equipment was designed for an objective and fair evaluation. It is hoped that the study findings will be very useful and will contribute greatly to the professors, researchers, and policymakers involved in science and technology policymaking and R&D.


Introduction
The Ministry of Science and ICT (MIST) of South Korea introduced a comprehensive review system for research facilities and equipment in 2016 to prevent overlapping investment in research equipment infrastructure and promoted the research institute's own review system.
MIST has been conducting a research equipment status survey every year since 2012 to manage research equipment. A roadmap for large research facilities was established in 2013 to strategically introduce large research facilities with high construction costs. MIST has also enacted a law for the national research facilities and equipment standard guideline (2016). It has operated research equipment and support services and has trained research equipment manpower through education programs. The allowable rate for the mutual utilization of research equipment was raised to enhance the utilization of research equipment, and the support for promoting the recycling of idle and underutilized research equipment was strengthened. The allowable rate for the mutual utilization of research equipment in South Korea was initially 60% but was 60.3% in December 2017.
In this paper, evaluation index improvement for research equipment relocation is presented and discussed to boost the effectiveness of research equipment management. MIST has transferred 153 idle and underutilized research equipment free of charge to universities, laboratories, and nonprofit organizations, among other institutions. The government allowed researchers to freely apply for research equipment to be transferred to them free of charge.
There were so many researchers who had applied for research equipment utilization that it was necessary to develop an evaluation index for the objective and fair relocation of research equipment. In this paper, evaluation index improvement for research equipment relocation is presented and discussed to boost the effectiveness of research equipment management. MIST has transferred 153 idle and underutilized research equipment free of charge to universities, laboratories, nonprofit institutions, etc. The government allowed researchers to freely apply for research equipment to be transferred to them free of charge. There were so many researchers who had applied for research equipment utilization that it was necessary to develop an evaluation index for the objective and fair relocation of research equipment. The objective and fair relocation of research equipment is very important for R&D effectiveness and research results. The evaluation index for research equipment relocation needs for the objective and fair relocation of research equipment. Therefore, an evaluation index for the relocation of research equipment was designed for an objective and fair evaluation. In this paper, the analytic hierarchy process (AHP) model is utilized as a methodology. The pairwise comparison scale was set up based on the importance of each evaluation criterion and the consistency rate (CR) was measured. This paper consists of introduction, literature review, science and technology environment change, global R&D investment trend, government R&D investment in South Korea, relocation of research equipment, methodology, results and discussion, evaluation index improvement research design, and conclusions and policy implication.

Literature review
Using the particular case of a large-scale R&D programme concerned with learning technologies within the European Community's Third Framework Programme for precompetitive industrial R&D, a participative evaluation method is outlined (Stern, 1993). The research evaluation must allow a more systematic evaluation approach to permeate the organisation whereby a common frame of reference regarding the choice of evaluation subjects, choice of strategic focus, and use of fundamental concepts is established (Brofoss, 1998). The technometrics approach is a method of evaluation of biotechnology R&D which contributes to ex ante evaluations (Meyer-Krahmer & Reiss, 1992). Research evaluation in a context of delegation and as a self-organising system for research actors guaranteed by the state has been strongly developed in the last few years (Sanz-Menéndez, 1995). In the field of R&D policy at least, reality, theory and therefore the needs of evaluation users seem to have moved well ahead of evaluators' conceptual apparatus (Arnold, 2004). Because of emerging funding volumes and increasing expectations in results, concepts for performance measurement and management gain importance (Schröder et al., 2014). The evaluation path is better characterised by an experimental attitude of constant change with each new programme, rather than a consolidation of practices learned and tested in previous programmes (Silva & Henriques, 1995). The Research Assessment Exercise (RAE) represents one of the most institutionalised forms of research evaluation in the OECD economies (Barker, 2007).
Widespread and increasing public subsidy for research and development (R&D) has given rise to a large and growing number of evaluation studies (Dimos & Pugh, 2016). Interest in evaluating non-economic social outcomes of science and technology research has risen in policy circles in recent years. The interest in social impacts of research has not yet given rise to a great proliferation of useful, valid techniques for evaluating such impacts (Bozeman & Youtie, 2017). Over recent years there has been a consistent shift in the way in which R&D policy is viewed. At a simplistic level, this has resulted in a convergence of the previously separate domains of S&T and industrial policy into a more coherent innovation policy perspective . Grand challenges stress the importance of multi-disciplinary research, a multi-actor approach in examining the current state of affairs and exploring possible solutions, multi-level governance and policy coordination across geographical boundaries and policy areas, and a policy environment for enabling change both in science and technology and in society (Amanatidou et al., 2014). Innovation underpins competitiveness, is crucial to addressing societal challenges, and its support has become a major public policy goal . Government policy objectives can include the development of technology for private users where there is a public policy rationale (Rigby et al., 2016). Clearly, in its simplest sense, an innovation support measure must be defined as a policy instrument designed to support the process of innovation, at the national, regional or other levels .

Science and technology environment change
New industries and markets based on the new-value-creation-type product service are created through industry-ICT convergence. The development of the industry infra, such as the smart factory, is accelerated. High-value-added industries will be created by upgrading the mainstream technology related to the industry. Digital technology scope expansion is applied to mobile devices and various other products for a society based on the Internet (Chang et al., 2016;Kauffman al., 2015). The development of digital-technology-based products such as 3D printers, autonomous vehicles, and drones will enhance business efficiency. The method change for R&D performance improvement enhances the research trend in a timely manner due to the greater collaboration, higher cost, shortened product technology life cycle, etc.
Commercialization of technologies on demand should be promoted based on R&D (Cerqueti et al., 2016;Lynskey, 2016). Also, the demand for a technology entrepreneurial ecosystem is increased. The strengthening of scientific and technical responsibilities highlights the role of science and technology in addressing global and social issues such as climate change, aging population, and the emergence and spread of intractable diseases. The responsibility of science and technology in resolving the technological side effects, such as information infringement and security issues, through the development of science and technology is enhanced. Labor force replacement is required due to the development of smart technologies and ICT, which can extend or replace human intelligence. New industries and services are required to be created to overcome the problems of job polarization and high unemployment rate.

Global R&D investment trend
Despite the global economic downturn, including the low interest rate and the increasing instability in the international financial and commodity market, the R&D investments of several countries are steadily increasing (Lewis & Tan, 2016;Schatz & Bashroush, 2017). Developed countries (USA, Japan, EU, etc.) are increasing the share of R&D investment in their GDP while maintaining their dominance in R&D investment. In particular, the R&D investment in China is rapidly expanding in both the government and the private sector, amounting to USD294.6 billion in 2013. Its proportion of the GDP was 1.39% (USD117,386 million) in 2007 and 2.02% (USD294,621 million) in 2013 for the total R&D investment of China. The proportion of government funding in the total R&D investment (USA, Japan, EU, etc.) is slightly fluctuating but is expanding continuously.

USA
In the USA, the federal government R&D budget in 2015 was USD136.5 billion, representing a 0.7% increase from the previous year. The non-defense funds (47%) in the 2015 government budget increased by 1.5%. The reduced budget pressure promotes investment in selected sectors under limited financing conditions. The R&D budget growth rate (2014-2015) was 11.7% for energy and 3.5% for the environment. The basic and applied research budgets were reduced by 1.8% for energy and 1.6% for the environment from the previous year. That for energy was USD31.7 billion, and that for the environment was USD34.2 billion. The

Japan
In Japan, the science and technology budget in 2015 was JPY4.03 trillion, representing a 10.4% increase from the previous year. The science and technology budget was stagnated for 10 years. It was the original budget + the revised budget + the science and technology promotion budget + the local public entity budget. The science and technology budget trend was JPY40,490 billion in 2003, JPY42,405 billion in 2008, and JPY44,393 billion in 2013. To promote economic and social development through science and technology innovation and the technological innovation system, the 2014 science and technology innovation strategy was presented. It included the five policy tasks (innovation of a clean and economical energy system, the realization of a healthy and long-living society for leading the international society, construction for pioneering the next-generation infrastructure, new-industry promotion for utilizing the local resources, and early recovery from the great East Japan earthquake). To realize the science and technology innovation strategy, the Council for Science and Technology Policy (CSTP) promoted linkage and cooperation among the ministries through a top-down budget policy. CSTP carried out ten major research projects linked to the policy tasks through the Strategic Innovation Creation Program (SIP; 2015 budget: JPY50 billion). It emphasized the creation of a science and technology innovation environment through R&D enhancement, regional innovation hub establishment, and environment improvement for venture activation.
The Ministry of Education, Culture, Sports, Science, and Technology (MEXT) is forming an innovative hub business centered on the R&D corporations (2015, JPY5 billion).

China
In 2015, the science and technology budget was CNY277.7 billion, representing a 3.1% increase from the previous year. The Rule of Science and Technology Development (2011) suggested R&D investment expansion, the combined improvement of science and technology and the market economy, and the development of human resources for science and technology.
Among the national R&D investments, the central government funding doubled, from CNY108. Also, the government investment had been strengthened for basic science, frontier science, and economic and social problem-solving. The eight missions are the top-level design promotion of an innovation-driven development strategy, new science and technology plan management model establishment, science and technology system reformation, special project implementation, science and technology innovation capability enhancement, localinnovation-development-level improvement through the regional development strategy, promotion of science and technology achievement and service industry development, and international open cooperation establishment. The basic and frontier sciences are nano, protein, climate change, space, sea, etc. while the economic and social problem-solving is on clean energy, alternative energy vehicles, the information and communication network, remote sensing, and biotechnology.

EU
The total budget of Horizon 2020 was EUR76.9 billion (2013). The science and technology budget was EUR7.32 billion in 2014 and 7.06 billion in 2015, representing a 3.7% decrease.
The budget was reduced by 6.7% for excellent science, and by 14.4% for social challenges. It was increased, however, by 6.2% for industrial leadership, and by 162.1% for the others. The Fast Track to Innovation (FTI) was established and supported by EUR100 million in 2015. It was based on openness and carried out a cooperation project that went beyond the boundaries (technology, industry-academe-research institutes, etc.). Germany, the core of the EU economy, is expanding its investment in the energy and biotechnology sectors, with the federal R&D budget growing at an average annual rate of 2.7% (2011)(2012)(2013)(2014). The R&D investment growth rate of the German government (2011-2014) was 13.1% for energy and 7.6% for biotechnology.

Government R&D investment in South Korea
The government R&D investment relative to the gross domestic product (GDP) in 2013 was 4.15%. It was no. 1 in the world. The R&D cost was KRW54.2 billion, the sixth largest in the world. The total R&D cost increased by 11.8% per annum over the past 5 years (2009)(2010)(2011)(2012)(2013).
The proportion of investment to the GDP rose by 0.86%, from 3.29% in 2009 to 4.15% in 2013.
The government R&D budget growth is slowing down (11.4% in 2009 → 6.4% in 2015), but the expansion of R&D investment is strong and is focused on the key areas. This study focused on the expected effect of R&D investment on research equipment in South Korea. When the government R&D budget is invested, it is essential to monitor and verify the expected effects. Also, this research verified the effectiveness of R&D investment through the calculation of the expected effects of R&D investment in research equipment.  Table 1.

Methodology
In this study, the analytic hierarchy process (  Table 3. Third, weight is estimated. The weight estimate calculates the geometric average of the pairwise compression matrix. The matrix weight estimation for pairwise compression is shown in Table 4. Fourth, the consistency is judged. The values obtained from the pairwise compression should be considered for the overall consistency of the evaluation index improvement for the relocation of research equipment.

Results and Discussion
The AHP model was designed by the Research Equipment Relocation Review Committee. The AHP model for evaluation index improvement for the relocation of research equipment is shown in Figure   2. Pairwise compression was used to evaluate the relative importance in terms of the upper evaluation criteria, through a survey for the evaluation elements. A pairwise compression scale was designed by measuring the importance of each evaluation criterion. The results of the importance evaluation of the evaluation criteria are shown in Table 5. The relative importance of each of the four evaluation criteria was studied by the Research Equipment Relocation Review Committee. The relative importance of each evaluation criterion is shown in Table 6. The importance of contribution to R&D activity, utilization, contribution to mutual utilization, and appropriateness of the installation environment of expensive and low-cost research equipment was evaluated. The results of the importance evaluation of contribution to R&D activity are shown in Table 7, those of research equipment utilization are shown in Table 8, those of contribution to mutual utilization are shown in Table 9, and those of appropriateness of the installation environment are shown in Table 10. Based on Table 4, the importance of each evaluation criterion was calculated. The total relative importance of the evaluation criteria is shown in Table 11. The normalized matrix was used through the adjustment of the importance and pairwise compression scale. The weight was calculated by averaging the row. The weight of each evaluation criterion is shown in Table 12. It was used to calculate the relative importance (0.24, 0.28, 0.27, and 0.21) of the effects on the evaluation criteria in Table 11. The weights of the evaluation elements were calculated in the same way. The elements of the evaluation criteria are shown in Table 13. The superior standard was evaluated through the evaluation criteria. The importance was calculated in Table 12 and 13. Below is the detailed formula that was used for the calculation.
The importance of the evaluation criteria for expensive equipment was 0.638, and that for low-cost equipment was 0.458. The AHP application model for evaluation index improvement for research equipment relocation is shown in Figure 3. The consistency rate (CR) was measured to determine the level of consistency of the research results. When the evaluation was completed, CR became 0 (zero) and was consistent if the CR was less than 0.1. Below is the detailed formula that was used.
The random index (RI) value according to n change is shown in             As the research results showed a 0.007 consistency ratio, they have validity in the decision.

Evaluation Index Improvement Research Design
Based on the research results, an evaluation index for research equipment relocation was designed for objective and fair evaluation, for a number of researchers who applied for the transfer of research equipment to them free of charge. The evaluation index was designed to allow research equipment to be relocated to researchers who can manage and operate them and enable researchers to conduct excellent researches. The existing evaluation index was determined through surveys among equipment experts, and the Research Equipment Relocation Review Committee. The evaluation index for the relocation of existing research equipment is shown in Table 16. In the past, one to three researchers applied for one research equipment. The evaluation index was designed for use in the evaluation of the researcher to be selected among the three or more researchers who applied for the transfer of one research equipment to them, as shown in Table 16. The basic principle of the research design is that the design should benefit more institutions and that the selection of expensive equipment should not be restricted while low-cost equipment should be limited to a maximum of 5 points (USD18,475) for research institutes or researchers. Also selected are the institutions that can utilize and maintain research equipment well after their relocation. The operation control performance rate in the 3 rd -and 5 th -year plans is checked after an agreement is reached for the relocation of research equipment, and the relocation to another research institution or researcher is executed when it is not sufficient for operation and management. If the 3 rd -year operation control performance rate is 70% and the 5 th -year rate is below 90%, the research equipment will be relocated to other research institutes or researchers. Young researchers, however, are given a grace period for the third year.
Considering that the research equipment that was used in this study was general-purpose, young researchers were selected rather than higher-performing researchers. The target research equipment was divided into expensive and low-cost research equipment. The expensive research equipment was worth more than USD18,475, and the low-cost research equipment was worth less than   Table 18. A summary of the salient points of the discussion on the relocation of research equipment is shown in Table 19. Are repair and relocation possible?
Is normal operation possible?
Is it possible to operate after repair?
Can parts be secured for equipment repair?

Research equipment operating life
How many years is the research equipment expected to run?

Relocation period
How long will the relocation take?

Research equipment education method
Is it necessary to ask the equipment supplier for training on consignment?
Is it necessary to ask an expert for training on consignment?
Does the researcher educate himself?
Is maintenance training necessary?

Installation environment review
Is the space suitable for the size of the research equipment?
Is electrical power available for the research equipment?
Is water available for the research equipment?
Is vibration safety secured for the research equipment?
Is noise safety secured for the research equipment?
Is dust safety secured for research equipment?
Are heating and cooling ready?
Is a fire installation system in place?
Are there any precautions against dangerous goods?
Is the locking mechanism in place?
Is there an Internet communication facility?
Are there any obstacles or risks to the research equipment?

Operation environment review
How many people are operating the equipment?
Do the operating manpower need to be professionals?
How much is the operating cost? Contribution to mutual utilization

Mutual utilization system
Implementation plan and schedule of mutual utilization service Performance of mutual utilization service (internal, external)

Appropriateness of installation environment
Operation cost of research equipment (operation cost, repair and maintenance cost) Manpower of research equipment (regular employee, non-regular employee)

Others
Preferential treatment for young scientist (low-cost research equipment) Preferential treatment for mutually utilized integrated facilities (expensive research equipment)

R&D activity
Is the use of research equipment necessary to achieve the research goal?
Is the research history sufficient, and is the research plan appropriate?
Should new researchers be selected first?

Utilization of research equipment
Is research equipment available according to the plan?
Is performance management related to research equipment utilization possible?
Is it continuously available for long periods of time?

Contribution to mutual utilization
Is the actual mutual utilization service possible?
Is it possible to contribute to the promotion of research equipment mutual utilization?

Appropriateness of installation environment
Does the research institute have adequate space and environment for research equipment installation?
Is the research equipment operation cost and manpower secured?

Conclusions and Policy Implication
In the study, the basic principle and detailed evaluation criteria for the relocation process of research equipment were established. The basic principle of the research design was to benefit more institutions and researchers. Moreover, the institutions that could utilize and maintain the research equipment well after their relocation were selected. Considering that the target research equipment was the general-purpose one, young researchers were selected rather than higher-performing researchers. Duplication was prevented through the advance review of many research equipment.
Based on the research results, an evaluation index for research equipment relocation was designed for the objective and fair evaluation of the researchers who applied for the transfer of research equipment to them free of charge. As the basic principle of the research design, as mentioned above, was to benefit more institutions, expensive equipment was not restricted from selection, and low-cost equipment was limited to a maximum of 5 points (USD18,475) for research institutes or researchers.
Moreover, the institutions that could utilize and maintain the research equipment well after their relocation were selected. The target research equipment was divided into expensive and low-cost research equipment, with the former worth more than USD18,475 and the latter worth less than USD18,475. For the expensive research equipment, the priority levels were based on the rate of research equipment utilization, the contribution to mutual utilization, the contribution to R&D activity, and the appropriateness of the installation environment. For the low-cost research equipment, on the other hand, the priority levels were based on the appropriateness of the installation environment, the contribution to R&D activity, the rate of research equipment utilization, and the contribution to mutual utilization. An attempt was made to relocate research equipment to research institutions and researchers by reflecting the characteristics of the research equipment. It was found that the rate of research equipment utilization is the most important element for the expensive research equipment and that the appropriateness of the installation environment is the most important element for the lowcost research equipment.
In this paper, evaluation index improvement for research equipment relocation is presented and discussed to boost the effectiveness of research equipment management. We focused on an efficient management method of research equipment through evaluation index improvement for research equipment relocation. The efficient management of research equipment can improve the efficient management of the R&D budget. It can promote government R&D innovation through government R&D investments in various research fields. The Ministry of Science and ICT (MIST) of South Korea has transferred 153 idle and underutilized research equipment free of charge to universities, laboratories, nonprofit institutions, etc. The government also allowed researchers to freely apply for research equipment to be transferred to them free of charge. There were so many researchers who applied for research equipment to be transferred to them free of charge that it was necessary to develop an evaluation index for the objective and fair relocation of research equipment. Therefore, an evaluation index for research equipment relocation was designed for objective and fair evaluation. For the advancement of South Korea's science and technology infrastructure, the management system through life cycle system installation and operation of research equipment is essential. The relocation of research equipment can increase the efficiency of the national R&D investment. The evaluation method for equipment relocation can replicable and applicable to various research equipment in many countries. It is hoped that the study findings will be very useful and will contribute greatly to the professors, researchers, and policymakers involved in science and technology policymaking and R&D.