Instituto Cultural Minerva

Institute of Brazilian Issues

The George Washington University

Washington, DC

 

Developing a Performance Measurement System for a Public Organization: A Case Study of the Rio de Janeiro
City Controller’s Office

 

By Luiz Carlos dos Santos Azevedo

Minerva Program, Spring 1999

 

 

Table of Contents

Abstract

Introduction

  1. Overview
    1. – Theoretical Background
    2. 1.1.1 – Measurement Concept

      1.1.2 – Indicators Concept

    3. – What is Performance Measurement in Government?
    4. – The American Experience in Performance Measurement
    5. – The International Experience in Performance Measurement
    6. – The Structural Support of Performance Measurement Systems
  2. Measurement Methodologies used in Governments
    1. – Balanced Scorecard in Government
    2. – Performance Budgeting
    3. – Benchmarking

III. Case Study of the Rio de Janeiro Controller’s Office (CGM)

3.1 – Introduction

3.2 – Methodology Application

3.2.1 - Organization as System

3.2.2 – Elements involved in the Development of Indicators

3.2.2.1- Stakeholders

3.2.2.2- Strategic Management

3.2.2.3 – Operational Management

3.2.3 – Alignment of the Stakeholders Dimensions, Critical Success Factors and Strategic Goals to the Strategic Indicators

3.2.4 – Selecting the Key Processes

3.2.5 – Strategic Indicators

3.2.6 – Operational Indicators

IV – Results and Conclusions

4.1 – Strategic Indicators

4.2 - Difficulties in Public Organizations

4.3 - Conclusions

 

 

ABSTRACT

Governments from developed to developing countries, motivated by different reasons, have been asked to document results of their performance. There is a great movement throughout the globe pushing them to the use of performance measurement. This paper displays what is the movement of performance measurement in governments, an American experience as well as International and the methodology applied in the Rio de Janeiro City Controllers Office to give support for the decision making process. Besides it presents some other methodologies applied to government, the characteristics of a good measurement system, the difficulties in public organizations and ultimately the role of measurement in the improvement of performance.

 

Introduction

The process of globalization in course, the information and communication revolutions as well as the democratization (whether that applies to citizens, shareholders, employees or customers) have created circumstances in which the traditional approaches to public sector are no longer working. The gap between what countries need and what they have, has created moments of crisis. These have opened opportunities for the inflow of new ideas into the public administration. (Mathiasen,1997).

The role of government is one major topic that has been talked all over the globe in recent years .The feeling that government has not been doing its job becomes even bigger when one compares the tax money given to the government to the quality of service that he/she receives. The perception of wasted resources is aggravated when investments that were supposed to go to either infrastructure or significant services are spent on bureaucracy, frauds or failed projects and as a result the government cannot be counted on to promote growth of the nation or even to support the current aspirations of ordinary citizens.

This dissatisfaction with public sector management and the current debate about its role has provoked a great movement in some countries of the so-called first world. The taxpayers have found a way to put pressure on elected officials to change performance in government, usually frightening politicians with votes during elections. Furthermore, the media also has been playing an important role exposing the critical performance of some services run by government, including questioning if some services should or not be privatized in order to bring better quality return and, obviously, cost-benefits improvement to citizens.

Even though the taxpayers of developing countries do not put the same amount of pressure on their government officials, it does not seem they are happy with their governments. The burden of impost on taxpayer shoulders is generally greatly heavy vis-a-vis the quality of services provided. In deed, the pressure for governmental reforms come from other facts: some of these countries disclose serious vulnerabilities in their economies, mainly caused by huge fiscal deficits and attending the new order of globalization, which requires a national economy to adjust and be competitive in the international scenario, the emerging countries have been rearranging the state functions and a structural reform has been taking place to promote sustainable growth to the nations’ future.

Governments from developed to developing countries have been structuring their administration and their way of doing business, or according to Robert Glaeber and David Osborne "Reinventing Government". This new paradigm for public administration encourages the use of the private sectors’ management best practices. Many management tools such as Total Quality Management, Benchmarking, Reengineering, Balanced Scorecard, Contracting Out applied to the private sector has been implemented in the public sector. Despite attempts to reinvent themselves, implement quality programs and privatize functions governments are still seeking a way to become truly performance-oriented. While Reducing costs and improving quality, efficiency and effectiveness, they must actively and continuously pursue effective ways to measure and track performance, and always be learning new ways to make quantum leaps in improvements.

It is unrealistic to expect improvement without first defining the expectations, communicating these goals(sharing the vision), providing the needed tools to achieve and measure the desired results (Wilson & Pearson,1995). The role of measurement in the improvement of an organization; public, private or non profit can be summarized in this simple management dictum " you can not manage what you do not measure".

The public administration indeed benefits from a management tool such as Performance Measurement. It, besides being a powerful management instrument, reveals whether the government is achieving or not success in its role, it also enhances accountability and displays transparency in policies and actions, improves government services, increases public trust in government performance, and ultimately portrays the improvement of the population’s quality of life and well being.

1. - Theoretical Background

  1. Measurement Concept

" To measure is to understand, to understand is to gain knowledge, to have knowledge is to have power. Since the beginning of time, the thing that sets humans apart from the other animals is our ability to observe, measure , analyze, and use this information to bring about change." Dr. H. JAMES HARRINGTON

" People are generally wary, if not distrustful, when numbers are introduced into the traditional process of decision making. But appropriately chosen numbers can represent variations in feelings more faithfully than can words or rhetoric. In the face of complexity, we run out of words to express adequately our full awareness of what sense to be taking place. Words limit the perspectives of our feelings " Saati (1990). Developing a Measurement System requires understanding the concept of measurement, which consists of the process to assigning numbers to objects to represent quantities of measures, attributes, variables of a tangible or intangible object . Measurement involves the use of rules and create data . Measurement is part of our daily life. Sink (1985)

But we do not spend much time, effort, or resources in measurement, unless the particular phenomenon is of great interest, then typically we attempt to be precise and accurate in our efforts to measure or specify its characteristics. If an individual wants to know something about a particular phenomenon, it is possible to measure certain attributes: its size, color, shape, temperature, magnitude, weight, state, quality, and so forth (Sink, 1985), in accordance with some rules knowing as scaling which could be as simple as 0 to 1 (bad or good), 0 to 10 (as in athletic competitions), or a logarithmic scale like the Richer Scale used to measure the magnitude of earthquakes. Stevens (1959) cited by Sink (1985) describes four basic types or levels of scales: Nominal ( e.g., identity numbers), Ordinal (e.g., preference), Interval ( e.g., temperature) and ratio (e.g., length).

One important characteristic to note about measurement is that all measures are relative. A measure that is not referenced to something else has no meaning. But if something is to be measured, one of the first considerations is what to use as a basis of comparison. This basis can be called standardized measurement if it is internationally recognized, such as grams, meters, second or volts or countable items such as money, defects and so on, which everyone agrees what a given number represents. If no accepted standard exists, the measure will be called a relative measure. A relative measure can be compared to itself at some other point in time or to the same measure in another system. As an example, the customer satisfaction index. (Kaydos, 1999).

Measurement requires collection of data. In general, there are three basic ways to collect data about a given phenomenon or organizational system: Inquiry,(e.g. surveys and interviews), Observation (e.g. time studies) and Collection data or documentation (e.g. work samples) as examples of these three categories. Surveys, time studies, interviews, and work samples (Sink, 1985).

In organizations, managers must measure in order to manage and improve productivity and accountability. Measurement can be casual and intuitive, or it can be specific, disciplined, and systematic. One is not necessarily better than the other, although explicit measurement is viewed as critical to improved decision making. It can be demonstrated that explicit measurement leads to more consistent decision makers will outperform inconsistent ones in the long run.(Morris,1979).

1.1.2 - Indicators Concept

Indicators are basic tools for Organization management Systems. The Information which they provide are crucial to the decision making process. (Figueiredo,1996,p.43)

Indicator – " Function that allows to get information about characteristics, attributes and outputs and outcomes of products, Systems or Process, throughout the time". The indicator is comparable to a movies it shows the trajectory of a variable throughout the time. They can be a mathematical relation between measures; Provide information about variables related to product state, process or System; and be associated to goals, previously established."(FPNQ,1998).

Usually the quality of indicators can be evaluated using the following criteria, which give them practicality as well as viability, according to Sink ( 1995), Tadashi (1995) and MEFP(1994):

Selectivity or Importance – Provide information about key-variable of product, process or system.

Simplicity and Clarity or comprehensibility – Indicators should be as simple as understandable as possible and still convey the message and meaning intended . The names and expressions should be easy to understand and known by all users.

Representibility – Capability of demonstration on a broad view the more important and critical stages of processes. Unnecessary data should not be collected. On the other hand, important data must be precise, responding to the objectives and searched in the right source.

Accessibility – Easy of access , being available at the right time, to the right people and without distortions .

"Researchbility" – Easy researching of data either for following, register or maintenance.

Comparability – Indicators should be easily comparability with proper benchmarks, such as : the best competitor and the average of the sector

Stability – Generated systematically procedure and constantly.

Cost effectiveness – Designed to be cost effective . The benefit-to-burden ration should satisfy aspiration levels. Some indicators are simply too difficulty to operationalize to justify their development.

The components of Indicators are:

Measure – "Qualitative and quantitative greatness which allows to grade characteristics, attributes and outputs and outcomes of Products, Processes or Systems".

Index – " Value of an indicator at a certain moment."

The Index is comparable to a photograph; it is demonstrated in the exact moment of its measurement.

Benchmark – " Arbitrated index as satisfactory for an indicator" and a "comparative standard for evaluating accomplishments.

"Goal – " Aimed index for an indicator in relation to the benchmark, to be achieved during a certain time."

Figure 3.6 Indicators

(Adapted from Figueiredo, 1996)

1.2 - What is Performance Measurement for government?

Performance Measurement is an objective and systematic process for collecting, analyzing and using information to determine how efficiently and effectively government services, products, or processes are being delivered and objectives are being achieved (Matzer,1997).

Performance measurement is the public sector’s way of determining whether it is providing a quality service or product at a reasonable cost.

Although performance measurement is a new demand for documentation of results for public organizations (Wholey and Hatry,1992), the assessment of service delivery at the local level of government is not recent, but the vinculum of indicators to program mission, setting performance targets, and regularly reporting on the achievement of target levels of performance are new marks in the performance measurement trend. (Newcomer,1997).

The terminology of performance measurement comprises different elements to be measured. The type of measures and concepts generally varies from government to government . In one sense or another, all of these terms represent measures of performance.. The following definitions were taken from the Government Performance and Results Act:

Outcomes may be usefully differentiated by whether or not they are intermediate or end outcomes. Intermediate outcomes include immediate reaction of clients. End outcomes or impacts are the intended results of the program. (Newcomer,1997).

The major purposes for performance measurement in the public sector turns around the concept of improved accountability. The accounting firm of Price Waterhouse has identified three key advantages of using performance measurement. These are:

    1. Measurement clarifies and focuses long term goals and strategic objectives. Performance measurement involves comparing actual performance against expectations and setting up targets by which progress toward objectives can be measured.
    2. Measurement provides performance information to stakeholders. Performance measures are the most effective method for communicating to legislatures and citizens about the success of programs and services.
    3. Measures encourage delegation rather than "micro-Management". Hierarchical structures and extensive oversight requirements can obstruct organizational effectiveness. Performance measures free senior executives for more strategic decision-making and selective intervention, while clarifying the responsibilities and authority of managers.

As government begins the process of developing and implementing performance measurement, the benefits of accountability, focus on results, and a better defined relationship between resources and mission can be achieved. But is equally important to recognize that performance measurement is only one of a number of tools available to governments managers and policy makers. The use of all the tools in harmony with one another will result in the reinvention and posturing of government institutions that is needed to reach a new public management.

1.3 - American Experience in Performance Measurement

The Total Quality Management program, beginning in the 1980s in state and local governments in the United States, has encouraged an explicit focus on customers and feedback from customers on services.

The Governmental Accounting Standards Board (GASB), beginning in the early 1990s , has pushed for experimentation on what it calls "service efforts and accomplishments" reporting by state and local governments (Governmental Accounting Standards Board,1994). GASB is a private nonprofit organization that develops accounting standards for state and local governments to use as a basis for external reporting. The Service Efforts and Accomplishments Movement’s intention new tentative is on reporting outcomes (accomplishments). This effort has encouraged a number of governments to begin outcomes reporting. For example, since 1991, the city of Portland, Oregon has been issuing a document called " Service efforts and Accomplishment: Annual Report on City Government and Performance."

At the beginning of the 1990s, a number of state legislatures, guided by Oregon and Texas, introduced annual requirements for performance measurement by state agencies. In Texas, performance indicators have been included in the state’s legislature’s annual appropriations bill. Oregon’s legislatively generated benchmark effort has been the precursor of a number of subsequent public-sector efforts by other state and local governments, including the states of Minnesota and Florida. These efforts have identified outcome-oriented indicators for broad statewide indicators.

The performance management process of Sunnyvale, California, that begun in the early 1970s, gained notoriety when one of its ex-mayors became a key staff member on the Senate Government Affairs Committee. He helped develop the Government Performance and Result Act of 1993. Sunnyvale’s initial focus was primarily on indicators of productivity (indices of the relationship of the amount of output to the amount of resources applied). However, in recent years, Sunnyvale has added outcomes to its data collection efforts.

Many other local governments have introduced annual requirements for performance information with a n explicit requirement for performance data. These include such varied local governments as Charlotte, North Carolina, and New York City. Charlotte has also introduced a new management trend originated by Kaplan and Norton , named Balanced Scorecard.

At the federal level, the Government Performance and Results Act (1993) required that every federal department and agency, by September 30, 1997, developed five-year strategic plans linked to measurable outcomes, which was required government-wide beginning with fiscal year 1999. The performance plans must cover any agency program activity set forth in the agency budget, with specific performance indicators, and " objective, quantifiable, and measurable " goals. In the development of measures, the emphasis is to be placed on programs outputs and outcomes.

Beginning no later than March 31, 2000, and filed by March 31st of each succeeding year, annual program performance reports are required, setting forth clearly the performance indicators, goals, and actual performance for at least the three preceding years. The performance reports would: review successful performance, describe unsuccessful performance, detail any remedial action that may be required, and recommend any necessary changes to performance goals for subsequent fiscal years.

1.4 - International Experiences in Performance Measurement

Governments from diverse nations have perceived the benefits of performance measurement. Agency performance is measured in the United States, Great Britain, Canada, Australia, New Zealand, Sweden, Finland, Norway, Denmark, Switzerland and the Netherlands. Norway, Switzerland and the Netherlands stress accountability and control. Great Britain, which names Value for Money, focuses on public report of performance and also incorporated Performance Contracting. The United States, Australia, Canada, Finland and Sweden include performance information in the budget process. Australia, the United States and Norway integrate performance measurement into their planning.

The World bank and Inter-American Development Bank (IDB) have increased their interest in evaluation, including regular tracking of performance information. Some World Bank staff have been encouraging and assisting developing countries to undertake service delivery surveys that use sample surveys of citizens to assess the quality of services they receive from their governments. This endeavor is an ongoing program of the World Bank’s Economic Development Institute. Such international efforts tend to reinforce each other. For example, developments such as New Zealand and Australia giving more authority to agencies in return for more accountability on what the agencies produce, have begun to increase similar interest in the U.S public sector.

In Brazil, the use of Performance measurement is in its infancy, aside from few federal agencies which are required by the new administrative reform, few initiatives have been taking throughout the three levels of government, some of these beginners are motivated by the requirement of performance measurement systems in the implementation of Total Quality Management(TQM).

1.5 -Structural Support for the Development of measurement Systems

Measurement seems like a simple concept. Somehow, when we attempt to design a performance measurement system, confusion and ambiguities tend to appear. Bain,1982; Demin,1986, 1993; Dixon, Nanni & Vollman, 1990 ; Kaplan & Norton, 1992; Mali,1978;. Sink ,1985, 1990; Sink & Tutle, 1989 points out 26 criteria that in general indicate the quality of a good measurement system:

"The flow of accountability for performance measurement starts with top management, through to the line managers and continues to the employees on the floor". (Clark,1995).

In Khaden & Lorber (1986) point of view there are five crucial requirements of a performance measurement system:

In studies for implementation of GPRA in federal agencies, Newcomer (1999) identified five columns that supports a Performance-Based Management System :

Internal Capabilities

Cultural Receptivity to use of Performance Measures

Intra-Organizational Incentives

Inter-Organizational Incentives

Socio-Economics Context

2.1 – Balanced Scorecard in Government

Balanced Scorecard (BSC) is a new approach to management introduced in the private sector by Robert Kaplan and David Norton in the 1990’s. After many years of experiences with performance measurement systems in companies worldwide, they realized that the traditional financial indicators well utilized during the industrial era could no more guide the organizations into the future. According to their theory, the financial indicator aim at just one aspect of the organization. Despite its importance, the total concentration on just this aspect, causes "myopia" to the management, considering the skills and competencies that organizations are attempting to lead nowadays.

The BSC promotes the translation of an organization’s mission, vision and strategy into a few performance measures that enable the managers to have a fast but comprehensive view of the organization position toward their desired end-state ( vision). As Kaplan says, " The BSC is like the dials in a n airplane cockpit: it give managers complex information at a glance" and , at the same time, this management tool permits communication improvement and mobilizes all individuals into actions directed at attaining organizational objectives. Priorities are set within the major categories, first at the corporate level and then at division, department, team, and even individual levels.

In essence, BSC focuses performance measurement in four perspectives: customer, financial accountability, internal processes, and learning and growth. The traditional financial measure is balanced with the three sets of operational measures :customer , internal processes, and learning and growth.

The City of Charlotte, North Carolina, borrowed the model from the private sector, and adapted to the City government. It was the first municipality to implement the Balance Scorecard. For more than 25 years, Charlotte measured government efficiency and effectiveness by setting objectives and tracking performance against them. Although the method served the city well, it focused primary on the past. Therefore the city began searching for a performance measurement system that emphasized strategic planning for the future.

Charlotte City Council focused on five areas that must improve in order for the council to meet its strategic goals : community safety, transportation, economic development, neighborhoods, and restructuring government according to City’s strategic planning. Each focus area has goals addressing the four perspectives. Those priorities and goals were later modeled (Figure 2.1), representing the "corporate" level of the city’s scorecard. Kaplan (1996) defends the use of this new management tool in government " While the initial focus of the Balanced Scorecard has been in the for-profit (private) sector, the opportunity for the scorecard to improve the management of governmental organizations is even greater." Further, Kaplan says "A Balanced Scorecard provides substantial focus, motivation, and accountability in government...the scorecard provides the rationale for their existence and communicates to external constituents and internal employees the outcomes and performance drivers by which the organization will achieve its mission and strategic objectives."

Aside from Charlotte City, various federal government agencies in the U.S.A adopted the Balanced Scorecard, such as Department of Transportation, US Postal Service, Federal Aviation Administration, NASA, Veterans Affairs and so on .

Except to " learning and growth" the other three perspectives seem similar to what other governments are doing with regard to performance measurement. But the omitted component is somehow implicit in any measurement effort. Training personnel and upgrading equipment is necessary to improve performance. However the Balanced Scorecard makes it explicit and helps managers to track whether staff has been trained regularly and technology has been upgraded.

2.2 - Performance Budgeting

Performance budgeting is the term referring generally to the process of linking expected results to budget levels; in other words it is the government-wide initiative designed to better align spending decisions with expected performance (GAO, 1997).

Performance budgeting is not a new effort of the U.S. government. Since World War II, numerous initiatives have emerged: the first one, also called performance budgeting, rose from the first Hoover Commission in its efforts to downsize the post-World War II government in 1950 by President Truman(burkhead,1956). Others examples include: Planning-Programming-Budgeting–System (PPBS) begun in 1965 by President Johnson, Management by Objectives(MBO) initiated in 1973 by President Nixon, and Zero-Based Budgeting (ZBB) initiated in 1977 by President Carter.(GAO,1997)

In recent years, public discontent with government and the belief in the superiority of the private sector management have led various government to make another attempt at linking performance with funding (Mascarenhas, 1996). At the federal level of U.S government, the new performance budgeting was introduced by the passage of the Government Performance and Results Act (GPRA). Several states followed the trend. Many local governments have also installed performance budgeting systems. Governments in other countries, such as Australia, Great Britain, the Netherlands, and New Zealand, also have embarked on such an endeavor ( Mascarenhas,1996)

A major difference between the old and new performance budgeting is their respective emphases on different types of measures. In general, the old performance measurement focused on input and output measures, while the new performance budgeting attempts to link efficiency, outcome, and effectiveness measures with funding reasons.

While certain localities such as Prince William County, Virginia, or Sunnivale, California are making progress in the area of performance-based budgeting, the state government as well as the federal agencies does not seem to be there yet.

The assessment of various governments shows that performance budgeting has not been able to change budgetary decision making and outcomes. This is due to the difficulties of measuring outcome and effectiveness in the public sector, and the rare utilization of outcome and effectiveness measures by budget makers. Aside from those facts, Lu (1998) cites various reasons why performance budgeting is not functioning in governments: The dominance of Input and Output Indicators; the lack of advanced measurements; unfeasibility of collecting outcome and effectiveness measures on a government –wide basis and questionable utility to budget makers.

2.3 - Benchmarking

Benchmarking is the systematic comparison of one organization to another with the aim of mutual improvement. Through benchmarking, an organization can set improvement goals and thereby judge the adequacy of both improvement goals and improvement rates in the context of what needs to be done (Thor, 1993; Camp, 1989).

Benchmarking traditionally has been associated with cost analysis, focusing on what competitors do and how much it costs them to do it; this includes machines, materials, and manpower, as well as nonproduction costs such as distribution.

The new quality benchmarking approach generalizes the method and is easily applied to nonfinancial indicators. It is a new form of sharing a process information.

For many years, in many nations, intercompany comparisons have been conducted by productivity centers or specialized consultants to allow participants to see where they fit in the performance of their industry group and to give the sponsoring organization some insight into the condition of the industry as a whole. Today, benchmarking practice is also adopted by governments. In the U.S.A, the broad performance measurement efforts nationwide enable benchmarking comparisons not only among government organizations, but also departments or services within a n organization, which can study the methods of similar units in other places and find a best practice. By the same token, the organization next steps are to analyze what the best practitioners are doing that one’s organization is not, and to reengineer best practices for importing, once these practices have already been identified.

The International City/County Management Association (ICMA) launched its performance measurement Consortium, which the main function of is to help the cities and counties develop performance measures across key functions. One product of the effort is the publication of an annual data report that compares the participating jurisdictions across those functional areas. The endeavor is based on the very solid theory that comparing the performance of like jurisdictions has tremendous value, particularly when it comes to learning about how other jurisdictions do various things well. However, ICMA has to face some obstacles in the exercise of benchmarking among cities and counties. First of all is the comparison of places that are just plain socio-economically or demographically different. Secondly is the comparison the performance of places that are different for reasons of geography, and thirdly, the different methodology of measurement as well as the interpretation of data.

Despite all these problems city officials and citizens want to know how their governments are doing in comparison to other governments and specially if government managers are giving the best value for their tax-money.

III – Case study of The Rio de Janeiro City Controller’s Office (CGM)

3.1 - Introduction

Created at the end of 1993, the General Controller’s Office (CGM) of Rio de Janeiro City is the pioneer public institution exclusively in charge of internal control in a government city in Brazil. The Office has the status of Secretary, being the Controller appointed by the Mayor usually with a four year term. Today, it consists of 240 technicians and 145 support.

The CGM’s chief objectives are to supervise the internal control systems, to monitor the government expenses and propose to the municipal Secretaries and Agencies efficient mechanisms for reducing public expenses along with fostering the effectiveness and efficiency of all the activities in the local administration. In addition it oversees the public administration policies and supports the battle against corruption and fraud. Among its master tasks the CGM has to support the Court of Audit and carry out the accounting, financial, budget and patrimonial control of the whole municipal administration.

Proud of its internal control leadership in the City category in the country, the CGM is making a great effort to become a " learning organization" (Senge, 1990) and thus the Office has been involved in many different projects in order to become modernized as well as to get prepared to face challenges of a new administrative reform that has been taking place in Brazil and particularly in Rio de Janeiro City Administration. Based on these assumptions the CGM in the middle of 1997 launched its Total Quality Management program , with the assistance of a consultant firm.

The program was initiated with a broad diagnosis of the whole organization system that consisted of two major phases. The first one was the constructing of the employees (internal customers) survey taking into consideration the Strategic, Tactic and Operational levels and secondly, the stakeholders survey in the direction of encouraging them to expose their points of friction and accordance with the Office . As a result of both surveys, a Document was produced, indicating the Excellence Stakeholder Dimensions. These results conferred a powerful base for the CGM Strategic Planning and it is also a cornerstone of the development of strategic indicators. In December, 1997, 20 representatives from the Strategic Management and Tactic levels participated in the birth of first CGM Strategic Plan.

One of the actions of the Committee responsible for the implementation of TQM in the Organization was to constitute five teams for each TQM area .The Organizational Performance Measurement Team (EMDO) is the one in charge of development of indicators. It is basically composed of the consultant and one representative of each major area of the Organization. The team meets once a week and does not spend more than two hours per week with this task. The idea is to discontinue the team once the methodology is applied and all the indicators have their "owners". Based on the information extracted for the Diagnosis Document and the Strategic Plan and the constant interaction with the management team the methodology of measurement systems has been applied.

In fact, the development of a performance measurement system at CGM is a consequence of its involvement in a TQM program, which requires a management based more in facts than judgements. Although there were some strong ideas in favor of the creation of such system in the organization before, it only became reality with the implementation of TQM program.

The tool is still a new concept for the Office and naturally it faces some resistance of employees and managers, but the organization presents some great advantages in the enforcement of this new management instrument. Apart from being a governmental institution, which has serious problems in management field, it is a young organization, and reasonably open and flexible, in comparison to others public institutions. In addition, it also benefits from the great number of employees quite skilled and technically proficient and the top management willingness in sustaining such project.

After almost a year working with this project, the EMDO has already disclosed the Strategic Indicators and it has been working together with the divisions to raise the operational indicators. By this time, the Auditing division as well as the Coordination of Internal Control, which has a branch in each Secretary and encompasses the most of CGM employees, have developed their operational indicators.

The methodology was created by Moacyr Figueiredo who has a Master of Science of the Institute of Military Engineering (IME). Besides the Rio de Janeiro City Controller’s Office , This methodology has been implemented in the Army General Hospital of Rio de Janeiro and the Hematology Institute of Rio de Janeiro State.

The methodology consists of the development of strategic and operational indicators regarding the strategy, the key processes and the requirements of the stakeholders (customers, employees, society and suppliers of the organization, and so on), utilizing the Quality Function Deployment(QFD) Methodology as a support to the alignment of Strategic Management and development of the Product, and the Analytic Hierarchy Process (AHP) applied to the selection of the key/critical processes.

As a precondition, the methodology requires a change of paradigm of the management team. The team has to see the organization from a different managerial viewpoint. The organization has to be seen as an adaptive system mutating from a vertical to a horizontal view of the organization. Moreover, in the face of the complexity that involves systems, this methodology regards strategic management as a tool that gives a sense of direction in these turbulent times. In addition, its instruments such as its Vision, Success Critical Factors, and Strategic Goals, are definitely crucial to the development of operational and strategic indicators.

In order to have an idea of the scope of the methodology, it is relevant to understand in which phase of the process of the measurement system

Sink & Tuttle (1990) based on the Kursdet model, set out a process for development of measurement and evaluation systems of the organizational performance, which embodies five phases: In the first phase, the management team amplifies the understanding of the organizational system; defining the Organizational System; analyzing the upper levels strategic plan and, analyzing the Organizational System strategic plan; in the second phase, the improvement actions are raised to serve as a basis for the indicators development; next comes the definition of what to measure. After that , the needed data , sources and means of collection are identified to compound the selected indicators defined in the phase 3; and, ultimately, the conversion of data in information. Figueredo’s methodology basically concentrates in the phase 3 of Kurdest model: that is the definition of what to measure.

3.2 – The methodology Application

3.2.1 – Organization as a System

" The primary tool for communicating direction, for establishing accountability, for defining roles, for allocating resources, for monitoring/evaluating performance, for linking the Three Levels (Strategic, Tactic and Operational), and for taking improvement action is Measurement"

For Measurement to play this role, an organization has to be seen from a different angle. Shifting from a traditional, organizational chart (vertical) view to a systemic (horizontal ) view of the organization is definitely the starting point. With the CGM, the experience is not different from other complex organizations. Employeees and also managers tend to concentrate their activities on what they see as important to their division or sector, rather than see the big processes in which they are involved.

Organizational charts do not display the customers, suppliers and other stakeholders that are related to the organization. They do not give a sense of the workflow (process) trough which we develop, produce, and deliver the product or service. They do not show the environment in which the organization is inserted. Sights that can be seen exclusively from a horizontal view of the organization or " organization as a system".

Most organizations nowadays share their activities and tasks, according to the model of division and specialization, and indeed each team from a specific division, sector, department attempts to optimize their results, paradoxically contributing to a sub-optimization of the whole organization (Figueiredo, 1996).

In order to apply the Figueredo’s measurement methodology is a fundamental condition to have the system vision of the organization. Rummler & Brache points out the elements involved in the organization seen as a system and the figure 3.1 displays a general panorama of CGM as a system.

Rummler & Brache ,1995 states (figure 1) : an organization is a processing system (1) that converts various resource input (2) into product and service outputs (3), which it provides to receiving systems (4). The organization is guided by its own internal criteria and feedback (5) but is ultimately driven by the feedback from its customer (6). The competition (7) is also drawing on those resources and providing its products and services to the customer. This entire business scenario is played out in the social, economic, and political environment (8). Looking inside the organization, we see functions, or subsystems, which exist to convert the various inputs into products and services (9). These internal functions or departments, have the same systems characteristics as the total organization. Finally, the organization has a control mechanism – management (11) – that interprets and reacts to the internal and external feedback, so that the organization keeps in balance with the external environment.

3.2.2 – Elements involved in the development of indicators

The figure 3.2 explicates how the strategic management, operational management and stakeholders dimensions elements interrelate in the process of creating a measurement system.

Figure 3.2 – Elements involved in the Development of Indicators

(Adapted from Figueiredo, 1996)

3.2.2.1 - Stakeholders

"Stakeholders are individuals, groups, or organizations affected by the processes, products, or services of an organization."(HRONEC, 1993)

The success of the Organization system is directly linked with the stakeholder satisfaction and the performance must be measured in relation to the degree in which the Organizational System can accomplish the stakeholders need.

Hronec, 1994, identified five different groups: External Customers, Internal Customers (employees), Shareholders, Society and Suppliers.

As shown in the figure 3.1, the CGM stakeholders disclose a curious situation. The external customers and suppliers are essentially the same actors. It occurs basically because the Office plays the role of a means organization in the municipal chart, usually receiving and delivering products/services from/to the same agent. Another strong reason is that CGM provides many advisory or consultative services, which requires constant interaction with the stakeholders.

3.2.2.2 - Strategic Management

Strategic Management is a long-term, future oriented process of assessment, goal setting, and decision-making that maps an explicit path between the present and a vision of the future, that relies on careful consideration of an organization’s capabilities and environment, and leads to priority-based resource allocation.

Rowe, Mason & Dickel, 1982 state "The strategic management reflects the capability of the organization to balance the demands imposed by external and internal forces and to integrate the overall functioning of the organization so as to allocate resources in a manner best designed to meet goals and objectives."

Neves, cited by Figueiredo, 1996, has proposed five phases of strategic planning linked to the development of indicators.

Following Neves steps, in view of initiating the CGM strategic management, the guidance team in the end of 1997 created the first CGM’s strategic plan, taking into consideration the municipality general strategies. The plan covers an action program that will be accomplished by December 31, 1999.

In order to undertake such a process of identification, analysis, assessment, and action-oriented decision-making, the Office established its Mission (see below), where it want to be in some years ahead (Vision) and its supports: the Critical Success Factors; what circumstances will either facilitate or inhibit its movement in the desired direction (strengths and weaknesses). Its shared values, general objectives, and goals, as shown below. These elements are crucial in the development of the measurement methodology.

Mission

The purpose of CGM is: To "Oversee the management control of Rio de Janeiro administrative resources"

Vision

The CGM’S vision is To: " Gain recognition for the excellence of service delivery as well as the for the generation of savings to Rio de Janeiro municipality by means of controlling actions"

Critical Success Factors

  • Infrastructure
  • Management
  • Training
  • Service Quality
  • Planning
  • Integration

Shared Values

The CGM shared values are : ethics, quality service, competence, commitment, results, cooperation, constant learning and integrity.

General Objectives

  • Arrange the needed resources for activities execution
  • Encourage and motivate the workforce to carry out the Mission and accomplish the Vision
  • Plan the actions
  • Instruct the workforce
  • Execute the planned actions with quality
  • Promote the partnership and the good relationship among CGM areas

Strategic Goals

  • Arrange necessary infrastructure by September, 1999
  • Introducing training program by December, 1999
  • Develop a biannual operational planning of each area by May, 1998
  • Meet over 80% of external customer satisfaction by December, 1999
  • Integrate the sectors by December, 1999

3.2.2.3 - Operational Management

There is no doubt that the vision of an Organization as a system is the first step in order to develop indicators; however, it is necessary to know the processes, through which the tasks are performed.(Figueiredo,1996)

Harrington (1993) defines process as a group of interrelated activities that consume resources and produce products and services.

There are a number of components involved in a Process(Figueiredo,1996):

All processes can be deployed in layers of sub-processes, activities and individual tasks.

Rummler,cited by Hronec(1994), classifies three types of processes:

After the processes of an organization are identified, they are sorted into primary and support categories, in order to enable prioritization among tasks as well as functions and to set goals to support the organization’s strategies. Then the Critical or key processes that directly impact the achievement of strategies and goals – a combination of primary and support processes - have to be selected (Hronec, 1993).

The instrument utilized by Figueiredo’s methodology is the Analytical Hierarchy Process(AHP) exposed in the item 3.2.4.

The figure 3.3 illustrates the CGM Primary and Support processes.

 

Figure 3.3 - CGM’s Primary and Support processes

3.2.3 - Alignment of the Excellence Stakeholders Dimensions, Critical Success Factors and Strategic Goals to the Strategic Indicators

The methodology employed to the alignment of the Stakeholders Dimensions, Critical Success Factors and Strategic Goals to the Strategic Indicators is the Quality Deployment Function (QFD). The concept of quality deployment was first proposed by Akao in 1966 and according to Bossert, it is a process that provides structure to the development cycle. The idea is complemented by the American Supplier Institute (cited by Figueiredo, 1996,p.32) that defines QFD as a system which translates the customer’s requirements in the organization’s requirements, phase by phase, from the research and development of product, service or software, to the engineering, manufacturing, marketing/sales and distribution. Luz, (cited by Figueiredo, 1996,p.36), highlights the Methodology as an important support of planning, communication and documentation of new product development and existing product improvement. QFD’s its utilization also benefits from bringing the reduction of development risks, time and, costs and, above all it introduces the customer’s voice in the process of elaboration. This tool was initially created to assist in focusing the design process to develop products that satisfy customers, then the service industry adopted the tool for designing services.

King (cited by Figueiredo, 1996) considers two existing approaches for the application of QFD. The one adopted here is designated the Four Phase Approach to QFD ,largely employed in measurement methodologies due to a more focus approach.

The QFD methodology helps in the alignment of Stakeholders Excellence Dimensions, Critical Success Factors and Strategic Goals in the process of Strategic Indicators development (figure 3.4). It is also useful in the process of Operational Indicators development, in which it is used to obtain the product and process requirements, that are the starting point in the creation of Operational Indicators.(Figueiredo,1996,p.35)

Figure 3.4 Alignment of the Excellence Stakeholders Dimensions, Critical Success Factors and Strategic Goals to the Strategic Indicators

(Adapted from Figueiredo, 1996)

Each phase of the figure 3.4 takes place in a QFD matrix of relationship, where the confrontation between items occurs pair by pair, e.g. infrastructure (critical success factor) X training (strategic goal). Each confrontation indicates whether the relationship between the items is strong, medium, weak or has no relation at all. Receiving weights according to the intensity. In the CGM’s case the strong = 9, medium = 3 and weak = 1.Theses weights, in the matrix, are multiplied by the Inputs (importance rate) and as result the outputs are displayed in the bottom row of the matrix e.g. Figure 3.5.

IMPORTANCE RATE

STRATEGIC GOALS

INFRASTRUC

TURE

TRAINING

OPERATIONAL PLANNING

CUSTOMER SATISFACTION

SECTORS INTEGRATION

CRITICAL SUCCESS FACTORS

INFRASTRUCTURE

0.20

MANAGEMENT

0.16

TRAINING

0.18

SERVICE QUALITY

0.32

PLANNING

0.06

INTEGRATION

0.08

STRATEGIC GOALS IMPORTANCE RATE

3.8

5.6

2.92

3.36

3.30

Figure 3.5 – relationship matrix between critical success factors and strategic goals

The Figure3.5 illustrates the CGM’s relationship matrix, which occurs in the phase 2 of the figure 3.4. (critical success factors X strategic goals).

The Inputs (importance rate) are the results from the previous matrix. Only the excellence stakeholders dimensions inputs are raised according to the Hayes (1992) methodology, by means of surveys. Following the Figueiredo’s methodology the next step would be the alignment of strategic goals and strategic indicators, pointing to the leading strategic indicators.

3.2.4 - Selecting the key processes

The Analytic Hierarchy Process (AHP) is the method utilized to select the key processes. The AHP is a systematic procedure for representing the elements of any problem, hierarchically. It organizes the basic rationality by breaking down a problem into its smaller and smaller constituents . (Saaty & Kearns, 1985). This method, created by Saaty, enables the decision makers to tell how intensively the elements of lower levels influences the governing element, helping managers to prioritize among criteria and alternative solutions. After defining the problem and determining the purpose, the hierarchy should be structured from the top through the intermediate levels to the lowest level. (figure.3.6)

Figure 3.6 – The Analytic Hierarchy Process

Having constructed the pyramid the comparison of sets of pairwise follows (e.g. E1 with E2, figure 3.6). The elements are compared in pairs with respect to their relative impact(weight or intensity) on a property they share in common. Theses comparisons are translated in numbers according to the " relative judgment scale" (Table 3.1) and listed in a pairwise evaluation matrix, e.g. figure 3.8. In other words, the AHP method demonstrates the opinion or preference of the lower level elements compared to each other, with respect to a particular criterion in the level immediately above. for each of the lower levels – one matrix for each element in the level immediately above.

Table 3.1 - Scale of Relative Importance Pag. 27(SAATY, 1990)

Intensity of relative importance

Definition

Explanation

1

Equal importance

Two activities contribute equally to the objective

3

Moderate importance of one over another.

Experience and judgment slightly favor one activity over another

5

Essential or strong importance

Experience and judgment strongly favor one activity over another

7

Demonstrated importance

An activity is strongly favored and its dominance is demonstrated in practice.

9

Extreme Importance

The evidence favoring one activity over another is of the highest possible order of affirmation

2,4,6,8

Intermediate values between two adjacent judgments.

When comprised is needed.

Reciprocals of above non-zero numbers.

If a n activity has one of the above numbers(e.g. 3) compared with a second activity has the reciprocal value(i.e.,1/3) when compared to the first.

 

This method allows prioritization of Key processes due to a hierarchy through criteria levels, the comparability par to par and the use of ratio-scale numbers, which fits well to the problem of selection of key processes.(Figueiredo,1996,p.41)

The key processes are the ones that contributes to the accomplishment of the Vision, and are strongly related to the Stakeholder Excellence Dimensions and secondly to the Critical Success Factors . Applying the AHP to this problem the Hierarchy would be (figure 3.7): On the Top, the Vision of the Organization, the principal goal of an Organization, on the first level, the Excellence Stakeholders dimensions, on the second level the Success Critical Factors, and ultimately the Processes.

Figure 3.7 – Selecting the key processes

(Adapted from Figueiredo, 1996)

In the first level the management team weights the Excellence Stakeholders Dimensions with respect to the Vision and in the second level , the Critical Success Factors are weighed with respect to each Excellence Stakeholder Dimension and Finally the processes that achieved the higher weighs are the key processes of the organization system.

The Figure 3.8 shows the CGM’s leading processes, with respect to the Infrastructure (critical success factor) criterion. In this case the auditing process is the leading process. But the key processes only will be revealed after the comparisons with all Critical Success Factors. This matrix illustrates the prioritization of processes displayed in the 2ş and base level of the figure 3.7

CRITERION: infrastructure (critical success factor)

PROCESSES

AU

IM

AC

IC

HR

BM

TE

AI

AS

Priorities

Auditing (AU)

1

1

1

7

3

5

5

5

5

0.1951

Information Technology Management (IM)

1/5

1

1

5

3

5

5

5

5

0.1785

Accounting (AC)

1/5

1

1

5

3

5

5

5

5

0.1785

Internal Control Advisory (IC)

1/7

1

1/5

1

3

5

1/5

5

5

0.1216

Human Resources Management (HR)

1/3

1/3

1/3

1/3

1

3

1/3

3

3

0.1025

Budget Monitoring (BM)

1/5

1/5

1/5

1/5

1/3

1

1/3

1

1

0.0370

Technical Standard Elaboration (TE)

1/5

1/5

1/5

5

3

3

1

3

3

0.1100

Managerial Accounting (MA)

1/5

1/5

1/5

1/5

1/5

1

1/3

1

5

0.0492

Administration Support (AS)

1/5

1/5

1/5

1/5

1/3

1

1/3

1/5

1

0.0280

Figure 3.8 – Selecting CGM Key processes

 

Figure3.9 displays the CGM’s key processes

Figure 3.9 The CGM’s key Processes

3.2.5 - Strategic Indicators

They are related to the Strategic management of the Organization, supply information about the Organizational performance in correlation to stakeholders’ requirements and are developed, based on Excellence Stakeholders Dimensions, critical success factors and Strategic goals of the Organization.

The methodology focuses on " how much" the Organizational System has been accomplishing for the Stakeholders (External customers, employees, politicians, society and suppliers) expectations. Therefore, the Strategic Indicators are developed attached to Strategic Management, with the purpose of showing in a quantifiable model the deviation between the actual direction and the desired direction in accordance with the shared Vision.

The methodology embraces three key dimensions or perspectives:

Effectiveness – "The Organizations’ capability to meet the stakeholder expectations."

Efficiency – " The Organizations’ capability to maintain effectiveness, optimizing their resources."

Growth and Innovation – " The Organization’s capability to maintain an infra-structure of personnel, equipment and technology, which supports a long term Organization development."

The Figure 3.10 displays the types of strategic indicators and its dimensions

Type

Dimension

Effectiveness

  • customer satisfaction
  • dissatisfaction
  • conformance

Efficiency

  • productivity
  • cycle Time
  • cycle efficiency
  • cost

Growth and Innovation

  • education and training
  • employee satisfaction
  • technology investment
  • new implemented products or ideas

Figure 3.10 CGM’s strategic indicators types and dimensions

The methodology is developed carrying out the alignment among Excellence Stakeholder Dimensions, Critical Success Factors and Strategic Goals, applying the QFD methodology (item 3.2.3 of this paper), adapted to this case. Subsequent to the alignment, the strategic indicators are created, in order to monitor the organization system performance.

The strategic indicators development basically follows two stages. First stage, which are the alignment among Excellence Stakeholder Dimensions, Critical Success Factors and Strategic Goals, has the following 4 steps: Setting the relationship between critical success factors and excellence stakeholders dimensions, Rating the hierarchy among stakeholders dimensions, rating the hierarchy among critical success factors, setting the relationship between strategic goals and critical success factors and finally rating the hierarchy among strategic goals.

The second stage is the creation of indicators, which has also 4 steps: Selection of strategic indicators, alignment of selected indicators with strategic goals, rating the strategic indicators and ultimately the description of strategic indicators.

The figure 3.11 exhibits the description of one CGM strategic indicator.

STRATEGIC INDICATOR FORM

NAME: Internal control activity hours per civil servant

 

ACRONYM: HACS

No. 12

TYPE: efficiency

 

Dimension: Office Skills

OBJECTIVE: Measure the CGM workforce skills in function of the number of hours working with internal control per civil servant.

 

DESTINATION:

Controller and

Deputy Controller

INDEX CONCEPTION

Person Responsible:

Administration Director

Human Resources Coordinator

PERIODICITY: Each semester

MATHEMATICAL REPRESENTATION :

HACS = NHEAC/TSEAC

 

 

DESCRIPTION OF ELEMENTS:

HACS – Hours working exclusively with intern control activities.

NHEAC – Number of hours working exclusively with

Internal control

TSEAC – Total of CGM civil servants executing Internal control activities

DATA COLLECTION

INPUT DATA:

Input date of expenditure files destined to processing.

Number of hours in the internal control activity in the Office

Total of civil servant working with internal control activities

PERIODICITY:

monthly

DATA SOURCE:

Record extracted from timesheet system

PERSON RESPONSIBLE FOR COLLECTION:

Employee designated by the administrative director or Human Resources Coordinator

Figure 3.11 Strategic Indicator Form

The EMDO team initially raised 61 strategic indicators, but only the 18 most important (Figure 3.12) has been monitored by CGM.

Types

Strategic Indicators

Effectiveness

Related to Customers

  • Customer satisfaction

Related to the capacity of generating economies or savings

  • Avoidance of waste and expenditures
  • Detection of waste and expenditures
  • Workforce returns rate

Related to the capacity of customers’ time response

  • Average time of documents permanence

Related to employee recognition, well - being and participation

Employee satisfaction

Efficiency

Related to the Mission and Vision accomplishment

  • Activity turn around time
  • Technicians turnover
  • Control activity hours per civil servant

Related to employee interaction and sectors integration

  • Training participation
  • Cultural and Social events participation

Figure 3.12 CGM’s Strategic Indicators

Innovation & Growth

Related to the intellectual capital and learning capacity

  • Average training time per employee
  • Investment in Training
  • Percentage of graduates

Related to infrastructure (personnel, hardware, software)

  • Investment in Infrastructure(percentage of the budget)
  • Investment in Information Technology
  • Technicians in charge
  • PC per employee

Figure 3.12 CGM’s Strategic Indicators

3.3.5 - Operational Indicators

They are related to the Operational management of the Organization, provide information about the performance of key processes and are developed based on customers’ requirements raised in the application of QFD methodology.

The measurement process requires the development of Operational Indicators that can provide the management team the information for performance improvement. But the difficulty emerges in the need of a definition of clear and broad criteria that evaluate the Processes’ improvement.

The methodology defines two basic criteria for evaluation of Process’s performance:

Effectiveness - "The extent to which the outputs of a process or activity meet the customer’s needs and expectations."(Harrington, 1995). Although some concepts of effectiveness state that the aim of this criteria is the accomplishment of what was set out to be accomplished, the methodology adopts Harrington concepts, which include the customers’ satisfaction. It measures how much of the outputs accomplish the customer’s need. The effectiveness indicators, in this sense, provide useful information for the Process’s outputs improvement.

The figure 3.13 points out to where the effectiveness indicators are measured.

Figure 3.13 Effectiveness Indicators

(Adapted from Figueiredo, 1996)

 

Efficiency Criteria "the extent to which the Process optimizes their limited resources in order to maintain effective" (Harrington,1995)

The figure 3.14 points out to where the efficiency indicators are measured.

Figure 3.14 Efficiency Indicators

(Adapted from Figueiredo, 1996) 

The figure 3.15 displays the types of operational indicators and its dimensions.

Types

Dimensions

Effectiveness

  • customer satisfaction
  • outputs conformance

Efficiency

  • productivity
  • cycle time
  • cycle efficiency
  • cost
  • inputs conformance
  • suppliers satisfaction

Figure 3.15 Operational Indicators Types and Dimensions

The creation of operational indicators is developed in four stages. In the first, after the comprehension of the organizational system’s processes the management team selects the key processes in function of its alignment with the Vision, Excellence Stakeholder Dimensions and Critical Success Factors, applying the Analytical Hierarchy Process (AHP).(item 3.3.3 of this paper).

In the second phase the management team selects teams to manage the key-processes and these teams exploit the key processes details. At this point the methodology pursues the following steps: definition of processes missions and goals, definition of key process context and scope, elaboration of key process macro flux, identification of customers and suppliers and finally is raised the customers requirements.

In the third and fourth stage the development of effectiveness and efficiency indicators are created and described.

The Figure 3.16 exhibits one example of a CGM’s operational indicator.

OPERATIONAL INDICATOR FORM

NAME: Average time permanence of expenditure files

 

ACRONYM: TMPPE

No. 03

TYPE: effectiveness

 

Dimension: Speed

OBJECTIVE: Measure the speed of expenditures execution in function of the average time of expenditure file permanence in the internal control branches.

 

DESTINATION:

Internal Control Branches Coordinator and Managers

INDEX CONCEPTION

Responsible:

Financial Administration Assistant

PERIODICITY: Monthly

MATHEMATICAL REPRESENTATION :

TMPPE = STPPE / TPE

 

 

DESCRIPTION OF ELEMENTS:

TMPPE – Average time permanence of expenditure files

STPPE – Total of expenditure files time permanence in the month

TPE – Total of Expenditures files within the month

DATA COLLECTION

INPUT DATA:

Input date of expenditure files destined to processing.

Number of expenditure files processed within the month

PERIODICITY:

daily.

DATA SOURCE:

Record extracted from SICOP

COLLECTION RESPONSIBLE:

Employee designated by the branch manager

Figure 3.16 Operational Indicator Form

The EMDO team has already raised the operational indicators of the General Audit Department (figure 3.17) and the Audit and Accounting Supervision Branches (figure 3.18).

The General Audit Department takes charge of the internal control of the Executive. Its major task is to preserve the municipal patrimony by means of financial and accounting procedures control.

Type

Operational Indicators

Effectiveness

 

 

Efficiency

  • Percentage of implemented recommendations
  • Percentage of accepted recommendations
  • Percentage of reports with errors
  • Percentage of audits accomplished in the planned time
  • Percentage of special audits
  • Total of audits
  • Total of audit findings per area
  • Total of audit findings in formal files(processes)
  • Economies/Savings generated by Audit Division

Figure 3.17 Audit Department Operational Indicators

The Audit and Accounting Supervision Branches help the municipal divisions with their budgets, giving support about the legality and the economical aspects. Besides that, they are in charge of the municipal patrimonial control.

Type

Operational Indicators

  • efficiency

  • Total of pendings on expenditures formal file
  • Average time permanence of expenditures file
  • Total of expenditures files (processes)
  • Total of hours spent in assets control
  • Percentage of time in operational inspection
  • Total of Assets files received
  • Figure 3.18 Audit and Accounting Supervision Branches Operational Indicators

    IV- Results and Conclusions

    4.1- Strategic Indicators Results

    4.2-Difficulties of performance measurement in Public Organizations

    Although generally the development of a performance measurement system can expose similar problems, such as lack of resources or even data to feed the system, public sector requires an additional concern. It displays some peculiar characteristics. The environment is different, usually without competition, giving no alternatives to customers (citizens) in the use of public services. As a consequence, the pressure for changes can be manifested, but is not a guarantee of improvement. Moreover, the relations within the government are complex and most cases the signs for change does not come from the politicians, but by individual management efforts.

    In the CGM case, the creation of a performance measurement system by initiative of the top management can be a gesture of its engagement in the process of the organization development. But on the other hand, it also brings an anxiety about the future of the system. Usually every four years the City has a new controller appointed by new elected mayor, which necessary has no commitment to the continuance of this management tool. Distinct from the federal U.S. government agencies, which have the Government and Performance Results Act requiring the use of performance measurement, the Rio municipal agencies do not have laws enforcing its employment.

    Another major difficulty in maintaining the system can be the lack of comprehension and recognition of the importance of such a system by some employees and managers. Once the performance measures are normally useful to the management or decision making processes, where they usually do not take part, it is difficulty to have the attention and effort of personnel in strategic affairs. Generally the staff is focused in what they perceive as important for their bosses, divisions and sectors.

    A substantial obstacle may be the employee resistance. Employees are threatened by measurement. They see negative results and are skeptical about its use and value. Measurement is sometimes viewed by employees as management’s way of punishing them because they are not doing their jobs. It is a hard task to convince them that the results will be used for the organization improvement. Worsening the circumstances, the mechanism of motivation and recognition in the public sector are very inflexible and limited.

    Because some results are easier to measures than others, there is a temptation to measure and report on some activities but not on others. It demands a great effort of the management team to have the performance measures in time to decide or to use it in the elaboration of the strategic plan.

    Unrealistic expectations can also be a problem. One can not expect the systems to produce results overnight and solve all problems. In addition measures are presented with no explanation of trends or radical changes in performance. But it indicates where improvement is necessary.

    Measurement supports or enables improvement and portrays improvement, but not directly causes improvement. In the management system model, measurement is a mediator between the current state of the organization system and decisions and actions aimed at improving that state. Decisions and actions cause changes in the organizational system that may lead to improvement or not, which is detected through measurement to influence news decisions and actions.

    VI - Conclusion

    Unquestionably, Sink & Smith (1993) are right when they state that methodologies for measurement systems are still in their infancy. In the process of creation and maintenance of a good measurement system there are a larger number of requisites(item 1.5) to be addressed . Furthermore , the environmental factors outside the control of the organization that impact performance are still a challenge to be forecasted and measured. Plus, a complete and effective system requires years of consistent, incremental work. All of these reasons, turn the development of a performance measurement system until a hard and complex task..

    Those difficulties become even bigger when the ambience is a government institution, in which the diversity of political interests makes the consensus difficult and managers, politicians and citizens skeptical about results. Consequently, no clear goals for some programs and agencies are established, and the poor performance of programs does not really constitute a motive for budget cuts. On the contrary, it could be a sign for resources allocation. These problems are added to the inability of government to motivate workers, since it can not give benefits for those who perform well.

    However, motivated by different reasons, there is a wave that has been sweeping governments from developed to developing countries asking for documenting results. Lessons learned in each country have been used in others as a good practice case and researchers have figured out that measurement of any kind affects the behavior of individuals within the organization, for better or worse. It has capacity to focus attention. Management needs to recognize their obligation to monitor and direct the resulting changes. Reporting performance measures will also affect the behavior of Top management, and it can also influence the voters, especially those who are in constant contact with government services.

    Total Quality Management is not a condition for the execution of Performance Measurement. But coincidentally the state and local governments in the United States and The CGM office ,in different eras, have been pushed to the use of performance measurement, in the course of TQM application. In the CGM, performance measurement has been helping to clarify goals and focus on what is relevant towards its Strategic Management. The tool is a new concept to the users and it leads to some resistance, but it is faced as part of a cultural changing process.

    Although the Office does not provide direct service to the citizen of Rio de Janeiro, the use of performance measurement brings a great benefit to the City. Once everyone is focused on the accomplishment of the organization Vision which calls for " the generation of savings to the municipality" the economies can be used in the improvement of the population quality of life. Whether the use of performance measures by CGM achieves success, it could be spilled over throughout the municipality. In fact, the Office could use its expertise and help the other Secretaries to become performance-oriented.

     

    References

    Bain, D.(1982) . The Productivity Presciption : The Manager’s Guide to Improving Productivity and Profits . New York: M C Graw Hill

    Clark, L.A. & Sink, D.S. (1995) Visible Measurement Systems Improve Performance. ASQC Annual Quality Congress Proceedings. Cincinnati, OH: ASQC Press

    Camp, R.C. (1989). Benchmarking: The Search for Best Practices that Lead to Superior Performance Quality Progress, 19(1-5)

    Demin, W.E. (1986) Out of Crisis. Boston, MA: MIT Press.

    Demin, W.E. (1993) The New Economics, Boston, MA: MIT Press

    Dixon, J.R., Nanni, A. J. & Vollmann T. E.(1993). Improving Strategic Performance through Better Performance Measurement. Quality and Productivity Management 10(3), 7-11

    Epstein, P. D. (1984) Using Performance Measurement in Local Governments, New York : Van Nostrand Reinhold Company

    Figueiredo, M. A. (1996) Metodologia Para o Desenvolvimento de Indicadores Estrategicos e Operacionais, Rio de Janeiro :Instituto Militar de Engenharia

    FNPQ, Fundação para o Prêmio Nacional da Qualidade (1998). Critérios de Excelência, São Paulo.

    Harrington, H. J. & Harrington, J.S.(1995), Total Improvement Management – The Next Generation in Performance Improvement, New York: John Wiley & Sons.

    Harrington, H. J.(1991), Business Process Improvement: The Breakthrough Strategy for Total Quality, Productivity, Competitiveness. New York: Mc Graw-Hill.

    Hatry, H.(1997) Where the Rubber Meets the Road: Performance Measurement for State and Local Government – Using Performance measurement to Improve Public and Nonprofit Programs .Evaluation Review, San Francisco: Jossey Bass.

    Hronec. S. M. (1993) Vital Signs : Using Quality, time and cost to chart your company’s future , New York: Amacom, American Management Association.

    International City/County Management Association, An Overview of Performance Measurement, Whashington DC, http://www.icma.org/abouticma/performance/PM-pmoverview.9-94.cfm , (Feb 1999).

    Kaydos, W. J. (1999) Operational Performance Measurement : Increasing Total Productivity , Boca Ratton, FL: CRC Press.

    Kaplan, R.S., & Norton, D.P. (1992). The Balanced Scorecard – Measures that Drive Performance. Harvard Business Review 70(1) 71-79.

    Kaplan, R.S., & Norton, D.P. (1993). Putting the Balanced Scorecard to Work.. Harvard Business Review 71(5) 134- 147.

    Kaplan, R.S., & Norton, D.P. (1996). The Balanced Scorecard : Translating strategy into action.Boston MA: Harvard Business Scholl Press.

    Kadhem, R., & Lorber, R. (1986). One Page Management: How to Use the Information to Achieve your Goals. New York: Morrow.

    Kursdedt, H.A. (1990). Catering to Crises – How to Escape. Quality and Productivity Management, 8(2), 5-13.

    Lu, H.(1998). Performance Budgeting Resuscitated: Why is it still Inviable?. Journal of Public Budgeting, Accounting & Financial Management 2(10), 151-172.

    Mali, P. (1978) Improving Total Productivity: MBO Strategies for Business, Governments and Not-for-profit Organizations, New York: John Wiley & Sons.

    Mascarenhas, R.C.,(1996). Searching for Efficiency in the Public Sector: Interim Evaluation of Performance Budgeting in New Zealand, Public Budget & Finance, 16(3): 13 –27.

    Matzer, J. (1997) Performance Measures, Book 12, Local Government Financial Management Training Series for Slovakia.: International City/County Management Association (ICMA) , Washington, DC.

    McKevitt, D. & Lawton, A. (1994) Public Sector Management : Theory, Critique & Practice, London: Sage Publications.

    MEFP/IPEA. (1994) Critérios para a Geração de Indicadores de Qualidade e Produtividade no Serviço Publico, Brasília, DF

    Morris, W (1979), Implementing Strategies for Industrial Enginneers, Reston, VA: Reston Publishing Company

    Newcomer, K. (1997) Using Performance Measurement to Improve Programs, Evaluation Review. San Francisco, Jossey-Bass

    Newcomer, K.(1999). Program Evaluation for Public and Non Profit Organizations, class notes PAD264- George Washington University-fall1999.

    Osborne, D.E.& Gaebler, T. (1992). Reinventing Government : How the Entrepreneurial Spirit is Transforming the Public Sector, Reading, MA : Addison-Wesley Pub. Co.

    Revelle, J. B., Moran, J. W.& Cox, C. A.(1998). The QFD Handbook, New York: John Wiley & Sons

    Risher, F. (1995). The Performance Imperative , San Francisco: Jossey-Bass

    Rummler & Brache (1995), Improve Performance , San Francisco: Jossey-Bass

    Saaty, T.L. & Kearns K. P. (1985) Analytical Planning , New York: Pergamon Press

    Saaty T.L.(1990). Decision Making for Leaders , Pittsburgh : RWS Publications

    Sink, D. S. (1985). Productivity Management : Planning, Measurement and Evaluation, Control and Improvement, New York: John Wiley & Sons

    Sink, D. S. & Tuttle T.C. (1989). Planning and Measurement in Your Organization of the Future , Norcross, GA: Industrial Engineering and Management Press

    Sink, D. S. (1990). Total Quality management is…, Quality and Productivity Management 8(2), 14-25

    Sink, D. S. & Smith, G. L. (1993). Performance Linkages: Understanting the Role of Planning, Measurement, and Evaluation in Large Scale Organizational Change. In Productivity & Quality Management Frontiers – IV, Norkcross, GA: IE Press

    Sink, D. S. & Tuttle T.C. (1989). Planning and Measurement in Your Organization of the Future , Norcross, GA: Industrial Engineering and Management Press.

    Thor, C. (1993). Ten Rules for Building a Measurement System. Quality and Productivity Management, 9(1), 7-10

    U.S. General Accounting Office (1996), Efffectively Implementing The Government Performance and Results Act, Washington, DC

    U.S. General Accounting Office (1997), Performance Budgeting: Past Initiatives Offer Insights for GPRA Implementation, Washington, DC

    Waterhouse P.(1993), Performance Measurement: The key to Accelerating Organizational Improvement, Washington, D.C

    Appendix:

    Charlotte’s Corporate Scorecard

    Customer perspective

    Reduce

    Crime

    Increase

    Perception

    Of Safety

    Strengthen

    Neighborhoods

    Improve

    Service

    Quality

    Availability of

    Safe, Convenient

    Transportation

    Maintain

    Competitive

    Tax

    Rates

    Promote

    Economic

    Opportunity

    Financial Accountability Perspective

    Maximize

    Benefit/

    Cost

    Expand

    Non-City

    Funding

    Grow the

    Tax

    Base

    Maintain

    AAA

    Rating

    Internal

    Process

    Perspective

    Increase

    Positive

    Contacts

    Promote

    Community

    Based Problem

    Solving

    Secure

    Funding/

    Service

    Partners

    Improve

    Productivity

    Streamline

    Customer

    Interactions

    Increase

    Infrastructure

    Capacity

    Promote

    Business

    Mix

    Learning and

    Growth

    Perspective

    Enhance

    Knowledge

    Management

    Capabilities

    Close the Skills

    Gap

    Achieve

    Positive

    Employee

    Climate

    Figure 2.1