USING A QUALITY IMPROVEMENT PRIORITY MATRIX
IN A UNIVERSITY DEPARTMENT

By Stuart Umpleby and Oleksandr Melnychenko
with assistance from Nino Okribelashvili, Nadya Krylova, and Mohamad Lazkani

Research Program in Social and Organizational Learning
The George Washington University
Washington, DC  20052 USA
Email:  umpleby@gwu.edu
URL: http://www.gwu.edu/~rpsol

September 17, 2001


 

ABSTRACT

             A Quality Improvement Priority Matrix is a useful method for achieving data-driven decision-making.  Regular information from employees and customers about the features of the organization that most need improvement allows managers to focus attention and resources where they can contribute most to improving employee and customer satisfaction.  In May 2001 the members of the Department of Management Science at The George Washington University used a Quality Improvement Priority Matrix to identify those features of the Department that they felt were high on importance but low on performance.  This quadrant of the matrix contained 17 of the 51 features listed. Several questions were also asked concerning whether the members of the Department found the exercise to be useful and whether they thought it would be helpful to other departments in the University.  A large majority thought the results were useful and that similar exercises in other departments would be helpful to them as well.


INTRODUCTION

This study was undertaken to achieve several purposes:

1. To introduce the method of a quality improvement priority matrix to the members of the GW Department of Management Science,
2. To generate data that would be helpful in focusing department improvement efforts during the coming academic year,
3. To test the utility and acceptance of a quality improvement priority matrix in an academic department before proceeding to encourage its adoption by other departments within The George Washington University, and
4. To test its utility and acceptance within an American university before encouraging other universities in other countries to use it in their planning and decision-making activities.

Hence, this exercise not only generated data for use by the members of the Department of Management Science.  It also asked for their assessment of the method and its utility.  In addition the paper makes some suggestions on how the method might be improved in the future. 


A QUALITY IMPROVEMENT PRIORITY MATRIX

            A Quality Improvement Priority Matrix was described by the people from GTE Directories in their presentation in February 1995 describing how they won the Baldrige Award (Carlson, 1995).  A similar matrix, called a “strategic improvement matrix,” was used by the people from Armstrong Building Products Operations in their presentation to the February 1996 Baldrige Award conference (Wellendorf, 1996).  Stuart Umpleby’s students have used similar matrices in several group projects ().  A matrix was used by visiting scholars at GW in December 2000 to identify how the US Department of State’s Junior Faculty Development Program might be improved (Naoumova and Umpleby, 2001).

            A Quality Improvement Priority Matrix asks customers or employees to rate several features of an organization on two scales – importance and performance.  That is, how important to them is that particular feature, and how effectively is the organization currently performing on that feature.  For this exercise we asked the faculty in the Department of Management Science at GW to evaluate various features of the Department and the School of Business and Public Management. Although the Department is functioning very well, the quality improvement literature claims that improvement is always possible.  If so, where is improvement most needed?  With this method one looks at the quadrant that corresponds to high importance and low performance.  What features of the organization fall into this quadrant? Those are the features where improvement will lead to the greatest increase in customer and/or employee satisfaction.

The quality improvement priority matrix is a simple but powerful management tool.  It makes possible data-driven decision-making.  Often in organizations resources are allocated to favored groups.  For example, a dean who is a chemist might choose to allocate more funds to the chemistry department.  Or additional resources are allocated to the group that complains the loudest.  This matrix presents an alternative.  By gathering data from customers and/or all employees a manager can find out where improvement is most needed and where additional attention will produce the greatest returns in customer and employee satisfaction.  The matrix can also be used to explain why funds are allocated as they are.  The matrix thereby tends to silence critics of the way funds are allocated.


RESULTS FOR THE DEPARTMENT OF MANAGEMENT SCIENCE

            Table 1 is the questionnaire that was distributed to members of the Department of Management Science at their annual retreat at The Woods, a conference center in Hedgesville, WV.  Eighteen questionnaires were returned eventually, but one came late.  So only seventeen returns were used for the data in this report.  Table 2 lists the mean responses for importance and performance.  In some cases people did not rate all features.  In these cases the mean is based on the number who replied, not seventeen.  Figure 1 is the Quality Improvement Priority Matrix that presents graphically the data in Table 2.  On the second day of the retreat the Quality Improvement Priority Matrix was presented to the members of the Department and the results were discussed.   

            If one looks at the Quality Improvement Priority Matrix (QIPM) for the Department of Management Science, one sees that 16 of 51 features lie in the lower right quadrant.  This quadrant contains the features that were rated high in importance but low in performance.  Table 3 lists the features in this quadrant that are most important in descending order, that is with the most important at the top and less important items below.  The five items in the lower right quadrant with the highest importance are salaries, classroom facilities, office space for faculty, building/ physical environment, and parking for faculty and staff.  Table 4 lists the features in the lower right quadrant that have the lowest performance ratings.  The five items with the lowest performance ratings are parking for students, SBPM working papers series, help with writing research proposals, conference room and other space, and use of continuous improvement methods in the Department.

Following the discussion, a second questionnaire was distributed (see Table 5).  This second questionnaire asked the members of the Department to evaluate the usefulness of the Matrix.  The distribution of responses to the five questions in the second questionnaire are presented in Figures 2, 3, 4, 5, and 6.  Eighteen of the second questionnaires were returned.


A METHODOLOGICAL ISSUE

            One member of the Department of Management Science noted that it is incorrect to average ordinal data.  This is a valid methodological point.  Our reply is that we find the matrix useful.  An issue that we believe is at least as important is the fact that human beings have difficulty interpreting large amounts of quantitative information.  We believe the Matrix helps people to find informative patterns in a large amount of data.  Somewhat like a road map, the Matrix helps people find the points of interest and then plan more detailed exploration.


CONCLUSIONS AND SUGGESTIONS

            We believe that this brief report is useful for the Department of Management Science not only because of the results but also because it illustrates the utility of a simple management method in helping a group of people to focus attention where improvement efforts can be most productive.  We believe that if such management methods were routinely used by departments at GW, faculty, staff, and student satisfaction and productivity would rise, and the university would continue to improve at a faster rate.

            Using this method has led us to suggest some possible improvements for the future.

1.      To increase feedback, some blank lines could be left at the bottom of the original questionnaire.  Respondents could be asked to suggest additional features to be included in future studies using the matrix.  In addition respondents could be asked for their comments – either questions about the meaning of some features or other comments on the method. 

2.      For those items that are of particular interest, a follow-up questionnaire could ask for additional information on what aspects of that feature are of concern. 

REFERENCES

Carlson, Marilyn.  “GTE Directories:  Customer Focus and Satisfaction,”  The Quest for Excellence VII, The Official Conference of the Malcolm Baldrige National Quality Award, February 6-8, 1995, Washington, DC.

Wellendorf, James A.  “Armstrong Building Products Operations:  Information and Analysis,” The Quest for Excellence VIII, The Official Conference of the Malcolm Baldrige National Quality Award, February 5-7, 1995, Washington, DC. 

Naoumova, Irina and Stuart Umpleby, “Two Methods Useful for Starting A Quality Improvement Program,”  Research Program in Social and Organizational Learning, The George Washington University, Washington, DC, April 15, 2001, 14 pages.

Table 1

 QUESTIONNAIRE FOR A QUALITY IMPROVEMENT PRIORITY MATRIX
 

Below is a list of features of the Department of Management Science or SBPM. We would like to create a Quality Improvement Priority Matrix using these features.  So, please rate each feature on a scale from 0 to 10.  That is, on the importance scale, 0 would mean the feature is not important at all.  5 would mean the feature is moderately important.  10 would mean the feature is very, very important.  On the performance scale, 0 would mean that current performance is very, very poor.  5 would mean that current performance is neither bad nor good. 10 would mean that current performance is excellent.
 

Table 2

 DATA FOR A QUALITY IMPROVEMENT PRIORITY MATRIX

Table 3

SORTED BY IMPORTANCE 

Table 4

SORTED BY PERFORMANCE
 

Table 5

EVALUATION OF THE QUALITY IMPROVEMENT PRIORITY MATRIX

Please circle one response to each statement.

1.   The results of the Quality Improvement Priority Matrix for the Department of Management Science were interesting.

strongly            agree        slightly        neutral         slightly            disagree         strongly
 agree                                agree                          disagree                                 disagree

2.   The results will be useful in planning activities in the coming academic year.

strongly            agree        slightly        neutral         slightly        disagree         strongly
 agree                               agree                           disagree                             disagree 

3.   We should do this exercise again next year.

strongly            agree        slightly        neutral         slightly        disagree         strongly
 agree                               agree                           disagree                            disagree 

4.   Other departments could benefit by using such matrices.

strongly            agree        slightly        neutral         slightly        disagree         strongly
 agree                                agree                          disagree                             disagree 

5.   Regular use of such matrices through out the university would help the university move forward more rapidly. 

strongly            agree        slightly        neutral         slightly        disagree         strongly
  agree                               agree                          disagree                            disagree

Table 6

RESULTS OF THE EVALUATION OF THE QUALITY IMPROVEMENT PRIORITY MATRIX
 

Figure 2 

THE RESULTS OF THE QUALITY IMPROVEMENT PRIORITY MATRIX FOR
THE DEPARTMENT OF MANAGEMENT SCIENCE WERE INTERESTING

Figure 3

THE RESULTS WILL BE USEFUL IN PLANNING ACTIVITIES IN THE COMING ACADEMIC


 




 

Figure 4

WE SHOULD DO THIS EXERCISE AGAIN NEXT YEAR

Figure 5

OTHER DEPARTMENTS COULD BENEFITY BY USING SUCH MATRICES

Figure 6

 REGULAR USE OF SUCH MATRICES THROUGHOUT THE UNIVERSITY
WOULD HELP THE UNIVERSITY MOVE FORWARD MORE RAPIDLY