|
Figure 1. Quality Improvement Priority Matrix
|
|
HOW A QUALITY IMPROVEMENT PRIORITY MATRIX REVEALS CHANGE IN A UNIVERSITY DEPARTMENT
By Stuart Umpleby and Aram Karapetyan with assistance from Saadia Khilji, Amir Raminfar, Maria Romanova, Tatyana Sedach, and Lilia Timofeeva
Research Program in Social and Organizational Learning The George Washington University Washington, DC 20052 USA Email: umpleby@gwu.edu URL: http://www.gwu.edu/~rpsol
May 16, 2003
A shorter version of this paper was presented at the annual conference of the Alliance of Universities for Democracy, Timisoara, Romania, November 9-13, 2002
|
|
Table of Contents
Abstract
A Quality Improvement Priority Matrix is a useful method for achieving data-driven decision-making. Regular information from employees and customers about the features of the organization that most need improvement allows managers to focus attention and resources where they can best contribute to improving employee and customer satisfaction. In May 2001 and May 2002, the members of the Department of Management Science at The George Washington University used a Quality Improvement Priority Matrix to identify those features of the Department that they felt were high on importance but low on performance. The changes in how the features of the Department were rated for importance and performance clearly reveal where progress was made in the intervening year and where attention next needs to be focused. This paper reports on experiments with how best to present the data.
Introduction
In February 1995, a Quality Improvement Priority Matrix was described by the people from GTE Directories in their presentation. This described how they won the Baldrige Award (Carlson, 1995). A similar matrix, called a “strategic improvement matrix,” was used by the people from Armstrong Building Products Operations in their presentation to the February 1996 Baldrige Award conference (Wellendorf, 1996). The matrix was used in several GWU student group projects in the late 1990s. A matrix was used by visiting scholars at GWU in December 2000 to identify how the US Department of State’s Junior Faculty Development Program might be improved (Naoumova and Umpleby, 2001). Later in May 2001, the members of GWU Department of Management Science also used a matrix (Umpleby and Melnychenko, 2001).
A Quality Improvement Priority Matrix asks customers or employees to rate several features of an organization on two scales – importance and performance. That is, how important to them is that particular feature, and how effectively is the organization currently performing on that feature. For this exercise we asked the faculty in the Department of Management Science at GW to evaluate various features of the Department and the School of Business and Public Management. Although the Department is functioning well, the quality improvement literature claims that improvement is always possible. If so, where is improvement most needed? With this method one looks at the quadrant that corresponds to high importance and low performance. What features of the organization fall into this quadrant? Those are the features where improvement will lead to the greatest increase in customer and/or employee satisfaction.
This is the second study of how a Quality Improvement Priority Matrix was used among faculty members in the Department of Management Science at The George Washington University. The first questionnaire was distributed in May 2001. The second questionnaire was distributed in May 2002. The 2001 questionnaire contained 51 features related to the Department and five questions about the matrix itself. These questions asked whether the members of the Department found the exercise to be useful and whether they thought it would be helpful to other departments in the University. A large majority thought the results were useful and that similar exercises in other departments would be helpful to them as well.
The 2002 survey listed 52 features of the Department and included some questions seeking additional information on the features rated high on importance and low on performance in the 2001 survey. The 2002 questionnaire contained no questions about the questionnaire itself.
Results of May 2002 Survey
Table 1 presents the mean ratings on importance and performance for the features in the Quality Improvement Priority Matrix that was distributed to members of the Department of Management Science at their May 2002 annual retreat. 19 questionnaires were returned. In some cases people did not rate all features. In these cases the mean is based on the number who replied to that feature, not 19. Figure 1 is the Quality Improvement Priority Matrix that presents graphically the data in Table 1.
The features of greatest interest are those that fall in the “southeast” quadrant, that is, those rated high on importance and low on performance. 16 of the 52 features lie in the SE quadrant. Using Excel it is difficult to attach either numbers or names to the points in the matrix. So, the features in the SE quadrant are listed in descending order of importance in Table 2 and increasing order of performance in Table 3.
Comparing the 2001 and 2002 Surveys
We also attempted to present the matrix data collected from the two years and hence to show how opinions had changed between the two surveys. When we plotted both 2001 and 2002 data on one matrix in order to see how the ratings had changed, we found that the matrix was hard to interpret, because the data were too crowded. So, we divided the features of the Department into three groups – office equipment, activities, and support. Table 4 contains the data from both 2001 and 2002. Figures 2, 3, and 4 present arrows showing how the positions of the features on the matrix changed in one year. The arrows were drawn by hand.
Figures 5, 6, and 7 present Pareto Charts of the differences in importance between the two years for the three groups of features: office equipment, activities, and support. Figures 8, 9, and 10 present Pareto Charts of the differences in performance between the two years for the three groups of features.
What we see from Figures 2 through 10 is that in general evaluations of the features of the Department have improved. The mean of all performance scores increased from 5.25 to 5.45. The mean of all importance scores declined from 7.85 to 7.52. There are several reasons for the improved scores. A new parking garage was built, so performance scores increased for both student and faculty parking while importance scores decreased. In addition a new office and classroom building was built, increasing the space available for faculty offices, classrooms, and conference rooms. These physical changes are reflected in higher performance scores on features such as office space, classrooms, and conference rooms. The importance scores on these features also declined. The data for the two years revealed one surprise. The importance scores for faculty, department, and school websites increased noticeably, while performance declined very slightly. Apparently faculty members are using websites more and their standards for what is a good website have risen slightly. Hence, figures 2, 3, and 4 demonstrate that when changes are made, satisfaction improves. Furthermore, the matrix can identify features that need increased attention.
The quality improvement priority matrix is very helpful in identifying issues needing attention. But often additional questions need to be asked. Table 5 shows the results of nine questions asked to clarify the results of the questionnaire the year before. Figures 11 and 12 clarify the issue of secretarial support. We wanted to know whether the faculty felt we needed more secretaries or better-trained secretaries. Apparently the concern is not that the Department needs more secretaries, but rather better trained secretaries. Figures 13, 14, and 15 clarify the issue of teaching assistants. Here the faculty said that more teaching assistants are needed, but also better trained teaching assistants. Teaching assistants with better English language skills is important, but not as important as training in general.
Conclusions
We believe that this brief report is useful for the Department of Management Science not only because of the results, which can be used to guide improvement efforts in the coming year but also because the report illustrates the utility of a management method for helping a group of people to focus attention where improvement efforts can be most productive. Future studies will probably focus on several issues:
References
Carlson, Marilyn. “GTE Directories: Customer Focus and Satisfaction,” The Quest for Excellence VII, The Official Conference of the Malcolm Baldrige National Quality Award, February 6-8, 1995, Washington, DC.
Naoumova, Irina and Stuart Umpleby, “Two Methods Useful for Starting A Quality Improvement Program,” Research Program in Social and Organizational Learning, The George Washington University, Washington, DC, April 15, 2001, 14 pages.
Umpleby, Stuart and Oleksandr Melnychenko, “Using a Quality Improvement Priority Matrix in a University Department,” Research Program in Social and Organizational Learning, The George Washington University, Washington, DC, September, 2001.
Wellendorf, James A. “Armstrong Building Products Operations: Information and Analysis,” The Quest for Excellence VIII, The Official Conference of the Malcolm Baldrige National Quality Award, February 5-7, 1995, Washington, DC. |
|
Table 1. Data for the 2002 Quality Improvement Priority Matrix
|
|
Table 2. SE Quadrant Sorted by Importance
Table 3. SE Quadrant Sorted by Performance
Table 4. Comparison of Data from 2001 and 2002
|
|
Figure 2. 2001-2002 Office Equipment
Figure 3. 2001-2002 Activities
Figure 4. 2001-2002 Support
|
|
Figure 5. Pareto Chart of Importance Differences: Office Equipment
Figure 6. Pareto Chart of Importance Differences: Activities
Figure 7. Pareto Chart of Importance Differences: Support
Figure 8. Pareto Chart of Performance Differences: Office Equipment
Figure 9. Pareto Chart of Performance Differences: Activities
Figure 10. Pareto Chart of Performance Differences: Support |
|
Figure 11. Secretarial Support
Figure 12. Training for Secretaries
Figure 13. Teaching Assistants
Figure 14. Training for Teaching Assistants
Figure 15. Language Skills
|
|
Table 5. Results of Supplementary Questions
Appendix A
Questionnaire for a Quality Improvement Priority MatrixBelow is a list of features of the Department of Management Science or SBPM. We would like to create a Quality Improvement Priority Matrix (2002) using these features. So, please rate each feature on a scale from 0 to 10. That is, on the importance scale, 0 would mean the feature is not important at all. 5 would mean the feature is moderately important. 10 would mean the feature is very, very important. On the performance scale, 0 would mean that current performance is very, very poor. 5 would mean that current performance is neither bad nor good. 10 would mean that current performance is excellent. Please fill out the form and give it to Stuart Umpleby today (Monday). Results will be presented (briefly) tomorrow.
Appendix B
Additional QuestionsLast year some items were ranked low in performance. These questions seek additional information on those items.
1. SBPM working paper series a. Do you read the working papers? Yes No If not, why not?
b. Do you contribute working papers to the series? Yes No If not, why not?
c. What would make the working paper series more useful to you?
2. Help with writing research proposals a. Where would help be most useful? (Check all that apply.) __ Generating ideas for research __ Finding likely funding sources __ Preparing the description of the research, including research methods __ Preparing the budget, institutional descriptions, vitae, attachments __ Other
b. What kinds of help would you like to have? (Check all that apply.) __ Research assistant __ Professional proposal writer __ Other
3. Use of continuous improvement methods in the Dept.
a. Should we create several process improvement teams? Yes No b. If so, what processes should be worked on?
c. Should the members of process improvement teams receive training in process improvement methods?
4. Accounts payable a. Has Accounts Payable improved in the past year? Yes No If so, in what way?
b. What can the Dept. do to improve the performance of Accounts Payable?
5. Dept. organization to implement its strategic plan
a. Should the Dept. have a written strategic plan? Yes No
b. Should there be a team to implement the strategic plan? Yes No
6. Building/ physical environment What could be done to improve the building/ physical environment (before moving to the new building)?
7. Secretarial support
a. We need more of them.
strongly agree slightly neutral slightly disagree strongly agree agree disagree disagree
b. They need more training/ preparation.
strongly agree slightly neutral slightly disagree strongly agree agree disagree disagree
c. Other
8. Coordination with other departments
a. In what ways is coordination with other departments not working well?
b. What are the causes of poor coordination?
c. What could be done to improve coordination?
9. Opportunities to meet local businessmen and government managers a. Do you attend meetings of the SBPM Alumni Association? Yes No
b. What could the Dept. do to increase your opportunities to meet local businessmen and government managers?
10. Consulting opportunities in the DC area
a. Do you currently do consulting? Yes No
b. What could the Dept. do to increase your consulting opportunities?
11. Teaching assistants a. We need more of them.
strongly agree slightly neutral slightly disagree strongly agree agree disagree disagree
b. They need more training/ preparation.
strongly agree slightly neutral slightly disagree strongly agree agree disagree disagree
c. Their language skills should be better.
strongly agree slightly neutral slightly disagree strongly agree agree disagree disagree
|
