USING A QUALITY IMPROVEMENT PRIORITY MATRIX
IN A UNIVERSITY DEPARTMENT

 

 

By Stuart Umpleby and Oleksandr Melnychenko
with assistance from Nino Okribelashvili, Nadya Krylova, and Mohamad Lazkani

 

 

 

Research Program in Social and Organizational Learning
The George Washington University
Washington, DC  20052 USA
Email:  umpleby@gwu.edu
URL: http://www.gwu.edu/~rpsol

  

 

September 17, 2001


 

ABSTRACT

             A Quality Improvement Priority Matrix is a useful method for achieving data-driven decision-making.  Regular information from employees and customers about the features of the organization that most need improvement allows managers to focus attention and resources where they can contribute most to improving employee and customer satisfaction.  In May 2001 the members of the Department of Management Science at The George Washington University used a Quality Improvement Priority Matrix to identify those features of the Department that they felt were high on importance but low on performance.  This quadrant of the matrix contained 17 of the 51 features listed. Several questions were also asked concerning whether the members of the Department found the exercise to be useful and whether they thought it would be helpful to other departments in the University.  A large majority thought the results were useful and that similar exercises in other departments would be helpful to them as well.


INTRODUCTION

This study was undertaken to achieve several purposes:

  1. To introduce the method of a quality improvement priority matrix to the members of the GW Department of Management Science,
  2. To generate data that would be helpful in focusing department improvement efforts during the coming academic year,
  3. To test the utility and acceptance of a quality improvement priority matrix in an academic department before proceeding to encourage its adoption by other departments within The George Washington University, and
  4. To test its utility and acceptance within an American university before encouraging other universities in other countries to use it in their planning and decision-making activities.

Hence, this exercise not only generated data for use by the members of the Department of Management Science.  It also asked for their assessment of the method and its utility.  In addition the paper makes some suggestions on how the method might be improved in the future. 


A QUALITY IMPROVEMENT PRIORITY MATRIX

            A Quality Improvement Priority Matrix was described by the people from GTE Directories in their presentation in February 1995 describing how they won the Baldrige Award (Carlson, 1995).  A similar matrix, called a “strategic improvement matrix,” was used by the people from Armstrong Building Products Operations in their presentation to the February 1996 Baldrige Award conference (Wellendorf, 1996).  Stuart Umpleby’s students have used similar matrices in several group projects ().  A matrix was used by visiting scholars at GW in December 2000 to identify how the US Department of State’s Junior Faculty Development Program might be improved (Naoumova and Umpleby, 2001).

            A Quality Improvement Priority Matrix asks customers or employees to rate several features of an organization on two scales – importance and performance.  That is, how important to them is that particular feature, and how effectively is the organization currently performing on that feature.  For this exercise we asked the faculty in the Department of Management Science at GW to evaluate various features of the Department and the School of Business and Public Management. Although the Department is functioning very well, the quality improvement literature claims that improvement is always possible.  If so, where is improvement most needed?  With this method one looks at the quadrant that corresponds to high importance and low performance.  What features of the organization fall into this quadrant? Those are the features where improvement will lead to the greatest increase in customer and/or employee satisfaction.

The quality improvement priority matrix is a simple but powerful management tool.  It makes possible data-driven decision-making.  Often in organizations resources are allocated to favored groups.  For example, a dean who is a chemist might choose to allocate more funds to the chemistry department.  Or additional resources are allocated to the group that complains the loudest.  This matrix presents an alternative.  By gathering data from customers and/or all employees a manager can find out where improvement is most needed and where additional attention will produce the greatest returns in customer and employee satisfaction.  The matrix can also be used to explain why funds are allocated as they are.  The matrix thereby tends to silence critics of the way funds are allocated.


RESULTS FOR THE DEPARTMENT OF MANAGEMENT SCIENCE

            Table 1 is the questionnaire that was distributed to members of the Department of Management Science at their annual retreat at The Woods, a conference center in Hedgesville, WV.  Eighteen questionnaires were returned eventually, but one came late.  So only seventeen returns were used for the data in this report.  Table 2 lists the mean responses for importance and performance.  In some cases people did not rate all features.  In these cases the mean is based on the number who replied, not seventeen.  Figure 1 is the Quality Improvement Priority Matrix that presents graphically the data in Table 2.  On the second day of the retreat the Quality Improvement Priority Matrix was presented to the members of the Department and the results were discussed.   

            If one looks at the Quality Improvement Priority Matrix (QIPM) for the Department of Management Science, one sees that 16 of 51 features lie in the lower right quadrant.  This quadrant contains the features that were rated high in importance but low in performance.  Table 3 lists the features in this quadrant that are most important in descending order, that is with the most important at the top and less important items below.  The five items in the lower right quadrant with the highest importance are salaries, classroom facilities, office space for faculty, building/ physical environment, and parking for faculty and staff.  Table 4 lists the features in the lower right quadrant that have the lowest performance ratings.  The five items with the lowest performance ratings are parking for students, SBPM working papers series, help with writing research proposals, conference room and other space, and use of continuous improvement methods in the Department.

Following the discussion, a second questionnaire was distributed (see Table 5).  This second questionnaire asked the members of the Department to evaluate the usefulness of the Matrix.  The distribution of responses to the five questions in the second questionnaire are presented in Figures 2, 3, 4, 5, and 6.  Eighteen of the second questionnaires were returned.


A METHODOLOGICAL ISSUE

            One member of the Department of Management Science noted that it is incorrect to average ordinal data.  This is a valid methodological point.  Our reply is that we find the matrix useful.  An issue that we believe is at least as important is the fact that human beings have difficulty interpreting large amounts of quantitative information.  We believe the Matrix helps people to find informative patterns in a large amount of data.  Somewhat like a road map, the Matrix helps people find the points of interest and then plan more detailed exploration.


CONCLUSIONS AND SUGGESTIONS

            We believe that this brief report is useful for the Department of Management Science not only because of the results but also because it illustrates the utility of a simple management method in helping a group of people to focus attention where improvement efforts can be most productive.  We believe that if such management methods were routinely used by departments at GW, faculty, staff, and student satisfaction and productivity would rise, and the university would continue to improve at a faster rate.

            Using this method has led us to suggest some possible improvements for the future.

1.      To increase feedback, some blank lines could be left at the bottom of the original questionnaire.  Respondents could be asked to suggest additional features to be included in future studies using the matrix.  In addition respondents could be asked for their comments – either questions about the meaning of some features or other comments on the method. 

2.      For those items that are of particular interest, a follow-up questionnaire could ask for additional information on what aspects of that feature are of concern. 


REFERENCES

Carlson, Marilyn.  “GTE Directories:  Customer Focus and Satisfaction,”  The Quest for Excellence VII, The Official Conference of the Malcolm Baldrige National Quality Award, February 6-8, 1995, Washington, DC.

Wellendorf, James A.  “Armstrong Building Products Operations:  Information and Analysis,” The Quest for Excellence VIII, The Official Conference of the Malcolm Baldrige National Quality Award, February 5-7, 1995, Washington, DC. 

Naoumova, Irina and Stuart Umpleby, “Two Methods Useful for Starting A Quality Improvement Program,”  Research Program in Social and Organizational Learning, The George Washington University, Washington, DC, April 15, 2001, 14 pages.






Table 1

 QUESTIONNAIRE FOR A QUALITY IMPROVEMENT PRIORITY MATRIX
 

Below is a list of features of the Department of Management Science or SBPM. We would like to create a Quality Improvement Priority Matrix using these features.  So, please rate each feature on a scale from 0 to 10.  That is, on the importance scale, 0 would mean the feature is not important at all.  5 would mean the feature is moderately important.  10 would mean the feature is very, very important.  On the performance scale, 0 would mean that current performance is very, very poor.  5 would mean that current performance is neither bad nor good. 10 would mean that current performance is excellent.
 

 

Feature

Importance

Performance

1

Computer hardware

 

 

2

Computer software

 

 

3

Office space for faculty

 

 

4

Conference room and other space

 

 

5

Computer labs

 

 

6

Copiers

 

 

7

Fax machines

 

 

8

Office security

 

 

9

Secretarial support

 

 

10

Teaching assistants

 

 

11

Annual retreat

 

 

12

Social activities

 

 

13

Recreational activities

 

 

14

Building/ physical environment

 

 

15

Accounts payable

 

 

16

Classroom scheduling

 

 

17

Classroom facilities

 

 

18

Projection equipment

 

 

19

Course catalogue

 

 

20

Faculty websites

 

 

21

Dept. websites

 

 

22

SBPM websites

 

 

23

Campus grounds

 

 

24

Parking for faculty and staff

 

 

25

Parking for students

 

 

26

Library journal collection

 

 

27

Library book collection

 

 

28

Interlibrary loan

 

 

29

Coordination with other depts.

 

 

30

A supportive climate in the dept.

 

 

31

Dept. head protects faculty from admin. interference

 

 

32

Transparency of APT process

 

 

33

Travel support

 

 

34

Funds to support research

 

 

35

SBPM working papers series

 

 

36

Help with writing research proposals

 

 

37

English skills of students

 

 

38

General ability of students

 

 

39

Course evaluations

 

 

40

Faculty annual reports

 

 

41

Salaries

 

 

42

Health care benefits

 

 

43

Retirement benefits

 

 

44

Opportunities for academic work with Dept. faculty

 

 

45

Opportunities for academic work with other GW faculty

 

 

46

Assistance with learning IT, e.g., Prometheus

 

 

47

Dept. strategic plan

 

 

48

Dept. organization to implement its strategic plan

 

 

49

Use of continuous improvement methods in the Dept.

 

 

50

Consulting opportunities in DC area

 

 

51

Opportunities to meet local businessmen and govt managers

 

 

 


 

Table 2

 
DATA FOR A QUALITY IMPROVEMENT PRIORITY MATRIX

 

Feature

IMPORTANCE

Performance

1

Computer hardware

9.4375

6.4375

2

Computer software

9.466667

6

3

Office space for faculty

9

4.055556

4

Conference room and other space

7.611111

3.176471

5

Computer labs

8.9375

5.0625

6

Copiers

8.235294

5.764706

7

Fax machines

7.4375

6

8

Office security

8.875

5.3125

9

Secretarial support

7.5

4.1875

10

Teaching assistants

8.5

4.75

11

Annual retreat

7

6.941176

12

Social activities

5.1875

5.266667

13

Recreational activities

4.375

4.333333

14

Building/ physical environment

8.6875

3.75

15

Accounts payable

8

3.583333

16

Classroom scheduling

8.2

5.466667

17

Classroom facilities

9

4.0625

18

Projection equipment

8.647059

5.882353

19

Course catalogue

7.125

6.375

20

Faculty websites

6.375

5.125

21

Dept. websites

7

5.5

22

SBPM websites

6.9375

5.5

23

Campus grounds

7.2

6

24

Parking for faculty and staff

8.125

4.625

25

Parking for students

7

2.923077

26

Library journal collection

8.588235

5.235294

27

Library book collection

8.5625

5.5

28

Interlibrary loan

8.4375

6.357143

29

Coordination with other depts.

7.266667

4.428571

30

A supportive climate in the dept.

8.882353

7.764706

31

Dept. head protects faculty from admin. interference

8.25

8.625

32

Transparency of APT process

8.928571

6.785714

33

Travel support

8.533333

8.333333

34

Funds to support research

8.625

5.071429

35

SBPM working papers series

6.1875

3

36

Help with writing research proposals

6.764706

3.071429

37

English skills of students

8.529412

5.058824

38

General ability of students

8.764706

6.882353

39

Course evaluations

7

5

40

Faculty annual reports

6.375

5.8125

41

Salaries

9.444444

4.277778

42

Health care benefits

9.5

5.266667

43

Retirement benefits

9.5

6.4375

44

Opportunities for academic work with Dept. faculty

8.533333

5.6

45

Opportunities for academic work with other GW faculty

8.357143

5.214286

46

Assistance with learning IT, e.g., Prometheus

7.75

7.266667

47

Dept. strategic plan

7.3125

4.4375

48

Dept. organization to implement its strategic plan

7.058824

3.625

49

Use of continuous improvement methods in the Dept.

5.75

3.4375

50

Consulting opportunities in DC area

6.6875

4.5625

51

Opportunities to meet local businessmen and govt managers

6.875

4.5625







 





Table 3

SORTED BY IMPORTANCE
 

 

Feature

Importance

Performance

41

Salaries

9.44

4.28

3

Office space for faculty

9.00

4.06

17

Classroom facilities

9.00

4.06

14

Building/physical environment

8.69

3.75

10

Teaching assistants

8.50

4.75

24

Parking for faculty and staff

8.13

4.63

15

Accounts payable

8.00

3.58

4

Conference room and other space

7.61

3.18

9

Secretarial support

7.50

4.19

47

Dept. strategic plan

7.31

4.44

29

Coordination with other depts.

7.27

4.43

48

Dept. organization to implement its strategic plan

7.06

3.63

25

Parking for students

7.00

2.92

51

Opportunities to meet local businessmen and govt. managers

6.88

4.56

36

Help with writing research proposals

6.76

3.07

50

Consulting opportunities in DC area

6.69

4.56

35

SBPM working papers series

6.19

3.00

49

Use of continuous improvement methods in the Dept.

5.75

3.44






Table 4

SORTED BY PERFORMANCE
 

 

Feature

Importance

Performance

25

Parking for students

7.00

2.92

35

SBPM working papers series

6.19

3.00

36

Help with writing research proposals

6.76

3.07

4

Conference room and other space

7.61

3.18

49

Use of continuous improvement methods in the Dept.

5.75

3.44

15

Accounts payable

8.00

3.58

48

Dept. organization to implement its strategic plan

7.06

3.63

14

Building/physical environment

8.69

3.75

3

Office space for faculty

9.00

4.06

17

Classroom facilities

9.00

4.06

9

Secretarial support

7.50

4.19

41

Salaries

9.44

4.28

29

Coordination with other depts.

7.27

4.43

47

Dept. strategic plan

7.31

4.44

51

Opportunities to meet local businessmen and govt. managers

6.88

4.56

50

Consulting opportunities in DC area

6.69

4.56

24

Parking for faculty and staff

8.13

4.63

10

Teaching assistants

8.50

4.75





 

Table 5

EVALUATION OF THE QUALITY IMPROVEMENT PRIORITY MATRIX


Please circle one response to each statement.

1.   The results of the Quality Improvement Priority Matrix for the Department of Management Science were interesting.

strongly            agree        slightly        neutral         slightly            disagree         strongly
 agree                                agree                          disagree                                 disagree

2.   The results will be useful in planning activities in the coming academic year.

strongly            agree        slightly        neutral         slightly        disagree         strongly
 agree                               agree                           disagree                             disagree 

3.   We should do this exercise again next year.

strongly            agree        slightly        neutral         slightly        disagree         strongly
 agree                               agree                           disagree                            disagree 

4.   Other departments could benefit by using such matrices.

strongly            agree        slightly        neutral         slightly        disagree         strongly
 agree                                agree                          disagree                             disagree 

5.   Regular use of such matrices through out the university would help the university move forward more rapidly. 

strongly            agree        slightly        neutral         slightly        disagree         strongly
  agree                               agree                          disagree                            disagree







Table 6

RESULTS OF THE EVALUATION OF THE QUALITY IMPROVEMENT PRIORITY MATRIX
 

1. The results of the Quality Improvement Priority Matrix for the Dept. of Management Science were interesting.

 

 

 

 

 

 

 

 

 

 

strongly agree

4

 

 

 

 

 

 

 

 

agree

8

 

 

 

 

 

 

 

 

slightly agree

5

 

 

 

 

 

 

 

 

neutral

0

 

 

 

 

 

 

 

 

slightly disagree

0

 

 

 

 

 

 

 

 

disagree

1

 

 

 

 

 

 

 

 

strongly disagree

0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2. The results will be useful in planning activities in the coming academic year.

 

 

 

 

 

 

 

 

 

 

 

 

 

strongly agree

2

 

 

 

 

 

 

 

 

agree

8

 

 

 

 

 

 

 

 

slightly agree

3

 

 

 

 

 

 

 

 

neutral

2

 

 

 

 

 

 

 

 

slightly disagree

1

 

 

 

 

 

 

 

 

disagree

2

 

 

 

 

 

 

 

 

strongly disagree

0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3. We should do this exercise again next year.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

strongly agree

3

 

 

 

 

 

 

 

 

agree

7

 

 

 

 

 

 

 

 

slightly agree

3

 

 

 

 

 

 

 

 

neutral

3

 

 

 

 

 

 

 

 

slightly disagree

1

 

 

 

 

 

 

 

 

disagree

1

 

 

 

 

 

 

 

 

strongly disagree

0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4. Other departments could benefit by using such matrices.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

strongly agree

2

 

 

 

 

 

 

 

 

agree

8

 

 

 

 

 

 

 

 

slightly agree

1

 

 

 

 

 

 

 

 

neutral

3

 

 

 

 

 

 

 

 

slightly disagree

1

 

 

 

 

 

 

 

 

disagree

1

 

 

 

 

 

 

 

 

strongly disagree

0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5. Regular use of such matrices throughout the university would help the university move forward more rapidly.

 

 

 

 

 

 

 

 

 

 

strongly agree

2

 

 

 

 

 

 

 

 

agree

7

 

 

 

 

 

 

 

 

slightly agree

2

 

 

 

 

 

 

 

 

neutral

3

 

 

 

 

 

 

 

 

slightly disagree

1

 

 

 

 

 

 

 

 

disagree

2

 

 

 

 

 

 

 

 

strongly disagree

0

 

 

 

 

 

 

 

 






Figure 2 

THE RESULTS OF THE QUALITY IMPROVEMENT PRIORITY MATRIX FOR
THE DEPARTMENT OF MANAGEMENT SCIENCE WERE INTERESTING






 

Figure 3

THE RESULTS WILL BE USEFUL IN PLANNING ACTIVITIES IN THE COMING ACADEMIC


 





 

Figure 4

WE SHOULD DO THIS EXERCISE AGAIN NEXT YEAR

 




 

Figure 5

OTHER DEPARTMENTS COULD BENEFITY BY USING SUCH MATRICES







 

Figure 6

 REGULAR USE OF SUCH MATRICES THROUGHOUT THE UNIVERSITY
WOULD HELP THE UNIVERSITY MOVE FORWARD MORE RAPIDLY

 
 

top of this page