Washington, DC, USA. A memo from the US Federal government says science agencies and research institutions should build infrastructures to evaluate the quality, impact, and results of scientific research. This can be sensitive since congressional budgets target results and shy from multi-year appropriations, while basic research can be lengthy and exploratory in nature.

The administrative memo was written by Julia Lane, program director of Science of Science and Innovation Policy (SciSIP) at the National Science Foundation (NSF) and her co-author Stefano Bertuzzi, Office of Science Policy at the National Institutes of Health (NIH).

Issued late in 2010, the memo calls on all federal agencies and executive departments to develop tools to "better assess the impact of [...] science and technology investments." It says "There is increasing pressure to document the results of [...] research investments in a scientific manner". They make the observation in a Policy Forum paper that appears in the journal Science.

Lane and Bertuzzi examine critical questions for measuring the effects of scientific research.

• Is it possible to create such a system?
  
  
• What would be the inputs, outputs and structure of the system?
  
  
• What scientific disciplines should inform the formulation of such a model?

"It is important that the scientific community, together with the federal government, engage in describing the results," said Lane of the special burdens placed on measuring the fundamental and exploratory research that NSF is responsible for funding, but does not always show immediate results.

"Fundamental research is a long way from the market place, so there are long lags before any results are seen," she said emphasizing the need to be diligent and sensitive to the exploratory nature of basic research.

The coauthors note that other countries already have begun developing systematic ways to describe the results of science investments.

• The Higher Education Funding Council for England (HEFCE) has been conducting Research Assessment Exercises to gauge the quality and impact of funded research since 1986.
  
  
• In addition, last year, the Japanese government began creating a program to advance the science of science and innovation.
  
  
• In the United States, a program attempts to address some of these same challenges. Called STAR METRICS (the Science and Technology for America's Reinvestment: Measuring the Effects of Research on Innovation, Competitiveness, and Science, the program is being developed by NIH and NSF with the backing of the White House Office of Science Technology and Policy. The Department of Energy (DOE) and the Environmental Protection Agency (EPA) are joining the consortium.

"This is a collaborative approach," said Lane. "The goal is to work together with research institutions to build a scientific data infrastructure that gathers inputs, outputs and outcomes from a variety of sources in an open a fashion as possible."

Lane and Bertuzzi say they hope the active engagement of the federal science policy community through STAR METRICS will help ensure scientific advances in science measurement.

Another major aim of STAR METRICS is to reduce, as much as possible, manual reporting by research institutions and principal investigators. "Researchers currently spend as much as 42 percent of their time in administrative reporting," said Lane. "That's not a good use of their time."

Name

Email

Website

Title

Comment

smaller | bigger

I have read and agree to the Terms of Usage.

Add Comment