HOW A QUALITY IMPROVEMENT PRIORITY MATRIX
REVEALS CHANGE IN A UNIVERSITY DEPARTMENT

By Stuart Umpleby and Aram Karapetyan

with assistance from Amir Raminfar, Maria Romanova, Tatyana Sedach, and Lilia Timofeeva

Research Program in Social and Organizational Learning

The George Washington University

Washington, DC  20052 USA

Email:  umpleby@gwu.edu

February 3, 2003

Published in Perspectives in Higher Education Reform, Volume 12

 Blagoevgrad, Bulgaria: Alliance of Universities for Democracy, 2003

 HOW A QUALITY IMPROVEMENT PRIORITY MATRIX

REVEALS CHANGE IN A UNIVERSITY DEPARTMENT

By Stuart Umpleby and Aram Karapetyan

with assistance from Amir Raminfar, Maria Romanova, Tatyana Sedach, and Lilia Timofeeva

The George Washington University

            A Quality Improvement Priority Matrix is a useful method for achieving data-driven decision-making.  Regular information from employees and customers about the features of the organization that most need improvement allows managers to focus attention and resources where they can best contribute to improving employee and customer satisfaction.  In May 2001 and May 2002 the members of the Department of Management Science at The George Washington University used a Quality Improvement Priority Matrix to identify those features of the Department that they felt were high on importance but low on performance.  The changes in how the features of the Department were rated for importance and performance clearly reveal where progress was made in the intervening year and where attention next needs to be focused.

            A Quality Improvement Priority Matrix was described by the people from GTE Directories in their presentation in February 1995 describing how they won the Baldrige Award (Carlson, 1995).  A similar matrix, called a “strategic improvement matrix,” was used by the people from Armstrong Building Products Operations in their presentation to the February 1996 Baldrige Award conference (Wellendorf, 1996).  The matrix was used in several GWU student group projects in the late 1990s.  A matrix was used by visiting scholars at GWU in December 2000 to identify how the US Department of State’s Junior Faculty Development Program might be improved (Naoumova and Umpleby, 2001).  And a matrix was used by members of the GWU Department of Management Science in May 2001 (Umpleby and Melnychenko, 2001).

            A Quality Improvement Priority Matrix asks customers or employees to rate several features of an organization on two scales ­ importance and performance.  That is, how important to them is that particular feature, and how effectively is the organization currently performing on that feature.  For this exercise we asked the faculty in the Department of Management Science at GW to evaluate various features of the Department and the School of Business and Public Management. Although the Department is functioning very well, the quality improvement literature claims that improvement is always possible.  If so, where is improvement most needed?  With this method one looks at the quadrant that corresponds to high importance and low performance.  What features of the organization fall into this quadrant? Those are the features where improvement will lead to the greatest increase in customer and/or employee satisfaction.

            This is the second study of how a Quality Improvement Priority Matrix was used among faculty members in the Department of Management Science at The George Washington University.  The first questionnaire was distributed in May 2001.  The second questionnaire was distributed in May 2002.  The 2001 questionnaire contained 51 features related to the Department and five questions about the matrix itself.  These questions asked whether the members of the Department found the exercise to be useful and whether they thought it would be helpful to other departments in the University.  A large majority thought the results were useful and that similar exercises in other departments would be helpful to them as well.

            The 2002 survey listed 52 features of the Department and included some questions seeking additional information on the features rated high on importance and low on performance in 2001.  The 2002 questionnaire contained no questions about the questionnaire itself.

Results Of The May 2002 Survey

            Table 1 presents the mean ratings on importance and performance for the features in the Quality Improvement Priority Matrix that was distributed to members of the Department of Management Science at their May 2002 annual retreat.  19 questionnaires were returned.  In some cases people did not rate all features.  In these cases the mean is based on the number who replied to that feature, not 19.

            The features of greatest interest are those that fall in the “southeast” quadrant, that is, those rated greater than 5 on importance and less than 5 on performance.  16 of the 52 features lie in the SE quadrant.   The features in the SE quadrant are listed in descending order of importance in Table 2 and increasing order of performance in Table 3.  

Comparing 2001 and 2002

            We wanted to compare the data from 2001 and 2002 to see how opinions had changed between the two surveys.  When we plotted both 2001 and 2002 data on one matrix in order to see how the ratings had changed, we found that the matrix was hard to interpret, because the data were too crowded.  So, we divided the features of the Department into three groups ­ office equipment, activities, and support.  Figures 1 presents arrows showing how the positions of the “office equipment” features changed in one year. 

            What we see from Figure 1 is that in general evaluations of the features of the Department have improved.  The mean of all performance scores increased from 5.25 to 5.45.  The mean of all importance scores declined from 7.85 to 7.52.  There are several reasons for the improved scores.  A new office and classroom building was built, increasing the space available for faculty offices, classrooms, and conference rooms.  These physical changes are reflected in higher performance scores on features such as office space, classrooms, and conference rooms.  The importance scores on these features also declined.  In addition a new parking garage was built, so performance scores increased for both student and faculty parking while importance scores decreased.  The data for the two years revealed one surprise.  The importance of websites for faculty members, the department, and the school increased noticeably, while performance declined very slightly.  Apparently faculty members are using websites more and their standards for what is a good website have risen slightly.  Hence, Figure 1 demonstrates that when changes are made, satisfaction improves.  Furthermore, the matrix can identify features that need increased attention.

            The quality improvement priority matrix is very helpful in identifying issues needing attention.  But often additional questions need to be asked.  Figures 2 and 3 clarify the issue of secretarial support.  We wanted to know whether the faculty felt we needed more secretaries or better trained secretaries.  Apparently the concern is not that the Department needs more secretaries, but rather better trained secretaries. 

Conclusions

            A quality improvement priority matrix is a useful tool for helping a group of people focus attention where improvement efforts can be most productive.  Future studies will probably focus on several issues:

,       More work can be done on finding the best ways to display data in order to show changes in     ratings by customers or employees.

,       It would be useful to establish a criterion for deciding when changes in ratings are statistically      significant. 

,       Finally, additional work could be done on finding ways to encourage people in organizations to use        such matrices to guide improvement efforts. 

References

Carlson, Marilyn.  “GTE Directories:  Customer Focus and Satisfaction,”  The Quest for Excellence VII, The Official Conference of the Malcolm Baldrige National Quality

Award, February 6-8, 1995, Washington, DC.

Naoumova, Irina and Stuart Umpleby, “Two Methods Useful for Starting A Quality Improvement Program,” in Perspectives in Higher Education Reform, Volume II, Russell J. Meyer and David Keplinger (eds.), Alliance of Universities for Democracy, Blogoevgrod, Bulgaria, 2002, pp. 185-193.

Umpleby, Stuart and Oleksandr Melnychenko,  “Using a Quality Improvement Priority Matrix in a University Department,” in Customer Satisfaction Management Frontier ­ VI, Johnson A. Edosomwon (eds.), Fairfax, VA: Quality University Press, 2002, pp.6.1-6.12.

Wellendorf, James A.  “Armstrong Building Products Operations:  Information and Analysis,” The Quest for Excellence VIII, The Official Conference of the Malcolm Baldrige National Quality Award, February 5-7, 1995, Washington, DC.

Table 1

Data for the 2002 Quality Improvement Priority Matrix

Table 2

SE Quadrant Sorted by Importance

Table 3

SE Quadrant Sorted by Performance

Figure 2

Figure 3

Reprinted by permission of the author Stuart Umpleby