Assessing Research Climates
Our Assessment Tools
Effectively managing the academic environment requires data. A professional, principled, inclusive and psychologically safe climate is crucial to for the facilitation of work flow and interactions between academic leaders, faculty, staff, and students. The National Center for Principled Leadership and Research Ethics has helped to develop tools which make it easier to efficiently measure work place climate. Tools include SOURCE, which measures research integrity, and CARES, which measures interpersonal engagement. We use a purpose-built Results Analysis Engine to manage the process and present findings. Both SOURCE and CARES comprise survey questions based on a Likert Scale as well as background questions. The Engine allows leaders to see a full summary on what areas of the environment are succeeding or failing. These tools are useful for addressing long term and short term issues within the professional academic environment, to ensure that each member of the community can thrive.
About The SOURCE
The Survey of Organizational Research Climate (SOURCE) is a validated instrument specifically designed to measure the climate of research integrity in academic organizations. It collects confidential responses from participants and provides a snapshot of program and unit microclimates through the aggregated perspectives of their members. Its use also contributes to the overall climate for research integrity by contributing de-identified data to a national comparison database. For an insight into how SOURCE results relate to institutional research integrity, please see this Nature article on the subject. For our newest white paper on developing academic leaders, see Academic Leadership Development, Critical Capabilities at the Department Level.
Results can help individual academic and research institutions:
- Characterize and compare departmental research integrity climates,
- Assess the efficacy of educational and mentoring approaches,
- Detect where research policies or practices (e.g., concerning data management, etc.) might need attention through responsible conduct of research (RCR) instruction
- Document and demonstrate for research sponsors institutional commitment to research integrity and RCR education,
- Justify enhanced efforts to foster a climate of integrity education and quality research.
Running the SOURCE at intervals of two to four years will demonstrate a commitment to improving the research integrity climate and provide data to assess its effectiveness over time.
The survey contains 32 items (5 point Likert scales) plus classification information about respondents (rank, departmental affiliation, type of highest degree). It takes 10-15 minutes to complete online.
In addition to enabling confidential institution-specific reports on units and the campus as a whole, de-identified data is compiled in a national comparison database so that the community as a whole may benefit. The norming initiative permits home institution departments to be appropriately assessed with respect to other comparable units in the same discipline.
CARES was developed specifically to assess the quality of interpersonal climates in research settings. We know from the validation of this new instrument that results from the CARES correlate with research integrity climate, organizational commitment, and a sense of integration with one’s work unit.
Although existing work has been done to develop assessments of research integrity climate (via the SOURCE, the Survey of Organizational Research Climate), up to now, no assessment tool has existed to measure interpersonal climate, particularly as it exists within academic research-focused environments. Over the past several years, as part of a project funded by the Alfred P. Sloan Foundation (Sloan Grant #2019-12294), our team has rigorously developed and validated such a tool in CARES.
CARES includes 24 question items (α=0.95), all asked on a Likert scale ranging from 1 to 5, with the following response categories: Not at All, Somewhat, Moderately, Very, and Completely. A “No basis for judging” option is also provided. These questions yield four means-scored summary dimensions, one tapping Institutional Practices influencing climate, and three tapping various aspects of the work-unit climate.
How can you and your university run the SOURCE or CARES?
An organization that wishes to run the SOURCE or CARES can make arrangements to do so with the University of Illinois’ National Center for Professional and Research Ethics (NCPRE). The license fee is based on the size of the project and the level of support desired in interpretation and analysis.
Once the license with NCPRE is in place, an institution provides emails and names of participants with departmental affiliations and status (graduate student, post-doc, faculty member). NCPRE manages the survey administration using the Results Analysis Engine (RAE)
The RAE sends emails via institutional email addresses seeking survey participation and follows up with reminders over a 4 week period. Response rates during previous studies have ranged from 15-50%.
The RAE provides analyses used to generate the summary reports for the contracting institution. As part of building a national “norming” database for providing comparative data, NCPRE retains de-identified information with limited demographics for comparison and benchmark purposes.
More detailed information on the procedures involved in implementing these instruments at your institution can be found here.
If we can answer any questions for you, please contact us at: firstname.lastname@example.org or 217-333-1416.