ScienceConsult is a research and consulting company that was formed to provide objective measures and feedback to both universities and scientific institutes in Germany on the effectiveness of their teaching programs. The company has been engaged by the University of Heidelberg’s medical faculty, one of the premier medical schools in Germany, to provide a continuous program of evaluation of its teaching across a range of subjects studied by medical students.
ScienceConsult sought a Web-based interviewing package that would allow students to complete surveys either on their own PCs or on the many PCs available to students at the university campus. By adopting Quancept Web, very complex questionnaires could be designed in a format that was easy to complete, as well as accessible and appealing to the group being researched.
Interested in ScienceConsult? Download the PDF
At most of the world’s leading universities, reputations are built on the quality and strength of published research. Rating the quality of teaching at these institutions, however, is rarely discussed among academics. Many consider this to be something that could not be measured in a systematic way. This may be about to change due to some groundbreaking research among student cohorts at the University of Heidelberg’s medical school using Quancept Web. Dr. Gisela Gieselmann heads ScienceConsult, a specialist research and consulting firm that is developing objective methods to measure the quality of teaching in universities. A political discussion in Germany has resulted in various initiatives that will link the state funding awarded to universities to the quality of their research and teaching for the first time. ScienceConsult has been assisting the University of Heidelberg with a program to measure the quality of teaching within its medical school, one of the leading schools in Germany.
For Dr. Gieselmann, it was imperative that the research methodology should be beyond criticism. If the method included only a sample, or suffered from a low response rate, there was a real danger that the methodology would be criticized and the results ignored. By administering all her surveys across the Web to entire student populations, Dr. Gieselmann has achieved very high response rates of between 70 percent and 90 percent.
These questionnaires successfully exploit the dynamic interviewing environment of Quancept Web. Dr. Gieselmann has developed a complex questionnaire which, to the respondent, appears very simple. A multi-factorial model for measuring the effectiveness of teaching is based around eight basic components of quality in teaching and the factors that can influence the quality rating, such as the perceived importance of a subject. Free text fields are also used to capture descriptive comments from the students. This can help to answer the question “why” when a score is unexpectedly low.
For us, the investment has been a big one. But the advantage is that I can do the work with fewer personnel and concentrate only on the things that are very important. I do not need people to enter the data and this also reduces the hardware requirement.
— Dr. Gisela Giselmann
ScienceConsult
Dr. Gieselmann recalled, “Recently, one professor had learned from us that teaching in his department was not satisfactory. He had read the report and his first comment was that it was an intelligent and elegant method that had been used to find things out. He did not put into question the method or the result. He just accepted it.” These results came after only six months of using Quancept Web.
ScienceConsult initially considered outsourcing the technical work to Web specialists, but Dr. Gieselmann was determined to maintain control over the entire project. Using Quancept Web in-house has allowed ScienceConsult to be completely self-sufficient. Results are analyzed in PASW Statistics Base, which integrates closely with Quancept Web and is another advantage, given PASW Statistics Base’s high prevalence in academic institutions. “I do not have many people I can involve. For us, the investment has been a big one. But the advantage is that I can do the work with fewer personnel and concentrate only on the things that are very important. I do not need people to enter the data and this also reduces the hardware requirement.”
For her clients, the administrative burden is also quite limited. A letter explaining the survey is included in a pack of course materials. Students may then use either their own PCs or a campus PC to access the questionnaire.
Careful design of the questionnaire has been a major factor in ensuring high completion rates, and here too, the flexibility of control in Quancept Web has played a big part. As Dr. Gieselmann admits, presentation is an important part of the design. “Students these days like working on their computers. We have to appeal to the ‘Game Boy generation’.”
Dr. Gieselmann stressed the importance of the backup behind the software from SPSS Inc. “Our questionnaires are very complicated. We had a crash course and this was very well done. It was the main reason why we were able to start so easily. We also have a very good hotline. It is not just a matter of the software—the support and the training are essential.”
While the research project is still at an early stage, Dr. Gieselmann is observing that it is already starting to make a difference. It has put the issue of the quality of training on the agenda. Professional esteem and peer pressure is ensuring that academic staff take both the good and the bad news from their students very seriously. Most encouraging of all, for some departments where teaching has been criticized, a second wave of research has already detected some improvement.
Prof. Dr. Hans-Günther Sonntag, the dean of the medical faculty of the University of Heidelberg, is enthusiastic about Dr. Gieselmann’s work: “With the commitment of Dr. Gieselmann to develop new possibilities for an external evaluation of medical education, an instrument will be established which will help to find gaps in education procedures and processes, as well as support the establishment of high-quality medical education. It will also give a highly sophisticated background for the benchmarking of medical education between medical faculties in Germany.”
Worldwide Offices | Communications | Site Map | Contact Us | Terms and Conditions | Privacy Policy
© 2010 SPSS Inc., an IBM Company All rights reserved.
SPSS Inc., an IBM Company Headquarters, 233 S. Wacker Drive, 11th floor
