ABSTRACT

Professional service professionals (PSPs)--such as management consultants, public accountants, and investment bankers--must satisfy the increasing demands of their clients, the changing demands of their leaders, and the developmental needs of staff. Professional service firms

(PSFs) are both creating and responding to the pressures faced by their professionals through enhanced performance management systems that use multisource feedback for assessment and professional growth. The development of the Relationship Management Survey (RMS) for PSPs is presented. The RMS dimensions are then used to generate hypotheses regarding how different stakeholder assessments of professionals' work-related performances can assist in performance management and contribute to their subsequent career success.

INTRODUCTION

A career in a professional service firm (PSF) can be very rewarding--personally, financially, and professionally--particularly if one survives to become a partner or managing director (Maister, 1997; Stumpf, 2002). Yet, PSFs are facing significant profitability challenges due to the costs associated with globalization, consolidation within their industries, and client demands for more differentiated and value-added services. The increased pressure to competently serve many diverse stakeholders has led to 70-80 hour work weeks, a sense of irresolvable conflicts among some stakeholders, and feelings of being overwhelmed. With up-or-out selection-to-partner rates below 10% at many of the most prestigious PSFs (Stumpf, 2002), young professionals must continuously examine their career potential and current success--accepting and understanding the perspectives of their clients, partners who are often the owners, peers, and direct reports. Since these stakeholders often have differing goals, and frequently have different expectations for the performance of young professionals, ones career development must be actively managed and shared by the PSF and the individual. The performance management system can assist in this regard by facilitating more effective career choices by both the individual and the organization.

Performance management in PSFs involves significantly different roles and competencies than are found in traditional work organizations such as banks, manufacturing firms, or retail organizations (Stumpf, Doh, & Clark, 2002). PSF work is that of a consultant to a client. The competencies necessary to be successful in professional services, beyond the expertise for which the PSF is retained, are closely associated with relationship building, more so than the traditional competencies associated with line management and task supervision. Before one can development measures of performance excellence for PSPs, it is necessary to understand the nature of PSF work.

PROFESSIONAL SERVICE WORK

Services generally possess four defining attributes relative to products: services are more heterogeneous, less tangible, more perishable, and more likely to be purchased and consumed simultaneously (Bessom & Jackson, 1975). Professional services--such as business consulting, public accounting, investment banking, providing legal advice, and public relations/advertising--share the attributes of services in general, and have additional defining attributes that make them distinct from other services such as retail, consumer banking, telecommunications, utilities, travel and entertainment. Professional services tend to involve the delivery of scarce expertise that is applied to a specific task. The work takes place over a limited time period (days, weeks or months) and involves extensive investigation and analysis (Maister, 1993, 1997). PSPs often have a college or graduate education and receive specialized training for their profession. PSFs generally agree to provide their services according to accepted professional practices and a code of professional ethics, sometimes codified through certification profession (e.g. Certified Public Accounting exam for accountants). Some professions seek to restrict entry through requirements for advanced education and state licensing. Most provide ongoing development for their professionals, encourage self-monitoring, require continuing educational activity, and espouse the use of sanctions for unethical behavior.

The work in PSFs is project oriented, serving the needs of the external client organization (or customer) as well as internal management (Maister, 1993). Terms used to describe this project work are engagements, deals, cases, and campaigns. Because project work is often interdisciplinary and complex, it may require a several PSPs to work as a team, often in collaboration with the client, for several weeks. The work is often done at a client's site for easy access to key information sources--the client's employees and physical records. The research and analysis associated with completing the work creates opportunities to identify additional opportunities related to current work, or to extend that relationship to another project based on facts and situations discovered during the initial project. Upon completion of a project, the engagement or case team is disbanded and its members are considered for other projects. PSPs may be reassigned immediately or left unassigned (not billable) until suitable work is found.

Most junior professionals do not report to a single supervisor but to team leaders, partners, and mentors on a project-by-project basis. One might have from 1 to 10 upwardly reporting relationships in a given year, and peer relationships with from 2 to 20 depending on the number of different projects work on. A client manager or team leader may also provide direction and oversee the work of some PSPs--increasing the likelihood of changing and/or conflicting demands associated with one having more than one 'boss'.

The number and complexity of PSF reporting relationships suggests the need for and use of a performance management system that is able to capture the diversity of views and changing nature of these relationships. However collecting assessment information from many stakeholders does not address the issue of how to use the information when different stakeholders assess the same person differently--as they undoubtedly will as the 'stakes' associated with the stakeholders vary. Whose assessments should be used? Whose are most valid? Should assessments of different stakeholders averaged? If an average or weighted average is calculated, is the intended feedback by the different stakeholders being obfuscated? Taking a stakeholder approach suggests that one needs to consider the differences among stakeholder assessments as meaningful differences, not measurement error or situational biases.

A STAKEHOLDER FRAMEWORK FOR PSFS

Those interested in stakeholder theory often seek to understand how stakeholder relationships affect individual and organizational performance (Doh & Stumpf, 2005a, b; Freeman, 1984; Freeman & Gilbert, 1988; Post, Preston, & Sachs, 2002; Waddock, 2005). This paper examines how the relationships among the performance assessments made by different stakeholder constituencies that are part of the network surrounding one's work role--i.e., self, direct report, peer, partner/owner, and client assessments--may provide valuable information for use in performance management and be differentially related to subsequent career success.

The importance of stakeholders to individual and organizational success (or failure) is well established (Freeman & Gilbert, 1988; Doh & Stumpf, 2005a). People that have a stake in what others do may act in a supportive or in a destructive manner towards these individuals (Mitchell, Agle, & Wood, 1997). The relationships that individuals have with their salient stakeholders may be career enhancing or damaging, depending upon how these stakeholders view and value their work-related performances (Ashford & Tsui, 1991; London, 1995). Performance is viewed as the level of effectiveness and skill exhibited on work-related activities as perceived and judged by different stakeholders. Career success is the extent to which (1) organizationally controlled rewards and benefits are provided to the individual in light of his or her performance (e.g., salary increases, bonuses, promotions), and (2) one is satisfied with the career progress and competency development experienced. Career success has elements that are objective (e.g., promotions, salary increases) and subjective (e.g., one's perceived satisfaction with his or her progress, perceived salary increase relative to peers).

Most leadership theories support the need for organizational leaders to obtain feedback from others on their skills, decision effectiveness, and the outcomes resulting from those decisions as perceived by these others (Doh & Stumpf, 2005a; McCall, 2002). Obtaining such assessments and feedback in a timely and productive manner is not easily accomplished. More often than not, only limited stakeholder feedback is solicited by leaders or their organizations and much of this is not fed back to those who might learn from it. The un-requested feedback provided is often viewed as biased, political, and critical of the decisions that were made. Such feedback tends to lead to defensiveness and resistance, rather than learning and professional development as might be the intention of the feedback giver. More proactive and systematic involvement by organizational leaders of key stakeholders in the discussion of key issues and subsequent decision making is a common suggestion found in the management literature. A process is needed for collecting assessments from key stakeholders--whether in person, by third-party interview, or by questionnaire. Once collected, the information needs to be synthesized and provided as performance feedback in a manner that maintains the integrity of the assessment, and often the confidentiality of the assessors as well.

MULTISOURCE FEEDBACK AS INPUT FROM STAKEHOLDERS

One methodology used to obtain feedback from stakeholders at the individual level of analysis is multisource feedback (MSF) (Bracken, Timmerck, & Church, 2001)--also known as multirater assessments (Nadler, 1977) and 360-degree feedback (Tornow, London, & CCL Associates, 1998; Waldman & Atwater, 1998). MSF entered the world of psychological assessment and performance management in the late 1960's, becoming popular as computer and web technology made the collection and analysis of complex sets of data more manageable (Goldsmith & Walt, 1999; Lepsinger & Lucia, 1997; London, 1995, 1997). While the literature on MSF abounds, few researchers have published articles which postulate linkages or directly link the psychological literature on MSF to the strategy and stakeholder literature. The roots of MSF have been in industrial psychology and performance management. Stakeholder theory has its roots in the management, strategy, and corporate social responsibility literatures. These fields of inquiry cited below as they relate to stakeholder views of one's performance, the performance management system, and subsequent career success.

MSF AS PART OF A PERFORMANCE MANAGEMENT SYSTEM

The multisource feedback literature has largely focused on whether feedback recipients improve over time (as reflected by better ratings over time) or on cross-sectional analyses (e.g., whether multisource ratings are related to supervisors' appraisals or assessment center performance, where all measures are collected concurrently). Linking MSF via a performance management system to subsequent rewards and career success has yet to be explored.

In developing a performance management (PM) system, one starts with a discussion of what should be measured (Campbell, Dunnette, Lawler, & Weick, 1970; Howard, Byham, & Hauenstein, 1994; Landy & Farr, 1983; London, 1997; Smither, 1998). Most PM systems assess one or more of the following: (1) outcomes and goal progress or accomplishments that are related to the organization or unit's mission, values, and strategy, (2) competencies and capabilities found to be essential to attain these goals and other desired outcomes, (3) behaviors and attitudes that reflect the organization's mission and values, and (4) personal attributes and developmental needs that link to individual and organizational success. Most MSF instruments tend to focus on points 1 and 2--the assessment of progress on goals and the exhibition of competencies found to be essential for individual and organizational success (Bracken, et al., 2001; Tornow et al., 1998).

The more reliable, valid, and useful MSF systems also benefit from measures (items) that have the following characteristics (Ashford, 1989; Cleveland, Murphy, & Williams, 1989; London, Smither, & Adsit, 1998; Peters & O'Connor, 1985). They are ...

* more objective in nature (e.g., observable behaviors rather than perceptions of someone's intent).

* independent and converge or diverge consistent with the theory behind their use.

* meaningful, valued, practical, and convenient to collect.

* free of known and knowable biases (blind reviews, also helps if anonymous or confidentially collected).

* reliable over time and similar situations.

* valid with low measurement error--content, construct, predictive, concurrent, face.

* under the control of the individuals/units being assessed.

A MSF system, designed for use in PSFs, is described below. In the development of this PM tool, care was taken to address each of the above characteristics.

DEVELOPING A MSF TOOL FOR PSF USE

As Campbell writes in the foreword to The Handbook of Multisource Feedback (Bracken et al., 2001: xiii):

"Multisource feedback, in its simplest form (which, as we shall see, is not so simple), has these basic components:

* Some purpose for doing an assessment: "the project"

* Some person to be assessed: "the ratee"

* Some collection of people to do the assessing: "the raters"

* Some specific characteristics to be assessed: "the items"

* Some technique, either paper-and-pencil or electronic, for collecting the data: "the survey"

* Some method for aggregating the rater's responses: "the data"

* Some form for reporting the results: "the results"

* Some process for providing the report to somebody: "the feedback"

* Some actions to be taken, developmental or administrative or both: "the decisions"

* Some procedure for determining if all of this was worthwhile: "the follow-up""

Each of the above components was addressed in the design and use of a new MSF survey. This development process was undertaken by a management consulting firm for its performance management system. The instrument, called the Relationship Management Survey (RMS), has been in use for five years and is available to organizations and researchers through LearningBridge.com. The survey development process is discussed below.

DEVELOPMENT OF THE RELATIONSHIP MANAGEMENT SURVEY

The Relationship Management Survey (RMS) was developed over two years by a top-five management consulting firm in conjunction with faculty members at two universities and two industrial/organizational (I/O) psychology consulting firms. The impetus for the RMS came out of a change in the sponsoring firm's strategy, followed by a revision of its performance management system. The firm was facing significant profitability challenges due to the costs associated with its near-random globalization efforts, consolidation within the management consulting industry yielding larger competitors who could offer a wider range of services on a global scale, and client demands for higher value-added services. The strategy change moved the firm away from project work that often had a regional focus with mid-sized clients to doing major re-engineering and information technology transformation programs in the global market of multibillion dollar in revenue multinationals that were willing to spend $10's of millions of dollars for consulting expertise each year. The PSF's goal--bill and deliver an average of $30 million/year dollars of consulting service globally to each of 100 firms within 5 years.

The need to revise the performance management system was related to both the change in strategy and the escalating costs of attracting, developing, and retaining the best talent. The increased pressure to competently serve larger clients, globally, with major transformational programs involving even more stakeholders was leading the PSPs to 80 hour work weeks away from their home office, a sense of irresolvable conflicts among some stakeholders, and feelings of being overwhelmed. The majority of new PSPs were voluntarily leaving within 5 years, and nearly all females and minority professionals were leaving within 7 years. This PSF's human resource function planned to respond to these pressures which the firm's partners and clients were placing on the younger professionals, in part, through enhancing their performance management system. One of the enhancements was to be a more rigorous assessment and feedback process which could reflect the conflicting demands of different stakeholders in both performance assessment and professional development.

The consulting skills necessary to effectively implement and deliver on this global-transformational strategy were believed to be somewhat different than those currently possessed by many of the firm's senior PSPs, and significantly different that those then being taught in the firm's professional and partner development programs. Research was undertaken by a team of five senior partners, led by the president, to identify which competencies were critical for the implementation of the new strategy. These partners conducted hour-long interviews with all other partners (about 160 of them), the current clients in the target market (22 CEOs, 17 CFOs, and 11 CIOs), and a sample of CEOs/CFOs/CIOs from firms that were in the target market but not currently clients (n = 38 interviews). A content analysis of these interviews was performed by a separate team of four high potential, non-partner consultants. Based on the 240+ structured and documented interviews reviewed, the project team identified 16 competencies as necessary to sustain the new strategy. Each competency was defined in a short paragraph and given a label. These evolved into 17 of the subsequent 29 RMS competencies.

The internal human resource professionals, working with the external I/O psychology consultants, conducted an interview study of 164 of the highest performing non-partner PSPs (top 10% of associates, senior associates, and principals) based on the most recent two years of annual performance reviews. As it was already the practice within this PSF to conduct multirater annual reviews involving interviews with 8-15 stakeholders in each PSP's performance, it was not difficult to engage PSPs at all levels in the creation of a new performance measurement system. These interviews and their subsequent analysis led to six additional competencies that were believed to be relevant to these non-partner PSPs (three for coaching, two for collaboration, and one more for leadership).

The academic and I/O psychologist partners in the RMS development conducted a literature review to ground the competencies identified in the leadership and consulting literatures (e.g., Maister, 1993, 1997; DePree, 1997; Kouzes & Posner, 1987; O'Shea & Madigan, 1997; Rackham, Friedman, & Ruff, 1996; Schaffer, 1997). This led to a revision of language around some of the competencies.

A review by members of the leadership team (14 senior and lead partners) of the PSF resulted in the creation of six relationship outcome items. These items focused on the results desired, rather than the competencies likely to produce those results. Each competency was cast into a survey format, with high and low anchors for each. For example:

                Circle the number below based on the following scale.
                  1         2          3              4        5
            Needs Skill Development    Moderate Skill    Highly Skilled
                                                Trust

Generates   Highly Confident: People feel comfortable allowing me to
Confidence  handle problems that are critical to them. They feel
            confident that results will follow from their discussions
            with me.

1 2 3 4 5   Not Very Confident: People rarely allow me to handle
            problems that are critical to them. They refuse to let
            me have influence on issues that are important to them.
            They seem to doubt whether any results will follow from
            their discussions with me. They sometimes monitor
            my actions or check-up on my team.

Multifaceted anchors were preferred, in contrast to Likert-type anchors, as the competencies sought were complex and required stakeholder judgments regarding the entire competency, not separate judgments on discrete facets. While this decision was questioned by the I/O psychologist consultants, it was viewed as necessary to both avoid subsequent management decisions around how to weight or average the discrete facets, and to increase the likely response rates by partners and clients who were believed to be 'put off' by 100+ item surveys.

The RMS was web-enabled and subsequently powered by a contractor (LearningBridge.com) so as to secure the data off site and provide responders with anonymity. A web-based instrument was viewed as essential in order to reach the many different stakeholders likely to be invited to participate in the RMS process. People needed to be contacted, and be able to respond, 24/7/365 from almost anywhere in the world.

Individuals are invited to participate by the PSP being assessed via an email sent from LearningBridge.com. The PSPs are provided a user name (their email address), a security password, and directed to the LearningBridge.com web site to complete the survey. Upon logging in they are provided instructions regarding the survey, how to input stakeholder names and emails, and asked to complete the RMS themselves. LearningBridge.com has an automated process that handles all subsequent contacts with the stakeholders, and provides personalized customer service regarding any inquiries. PSPs are first asked to complete a few demographic items, after which the 29 RMS competencies are randomly presented to the user via 29 separate web screens. All screens permit open-ended write-in responses in addition to a quantitative response. The wording of each competency reflects either a self assessment or third-party assessment. One can stop, exit, and return to the RMS at any time with all responses saved. The randomization of the remaining items continues upon return.

The feedback report provides instructions on how to interpret the stakeholder assessments. It clusters the 29 items into 6 dimensions as noted in Table 1 and provides all write-in comments without disclosing the identity of the stakeholders. The six competency dimensions reflect the sponsoring organization's view of what are meaningful clusters. Each RMS report is made available to the participant as a confidential .pdf file.

The stakeholder classification used in the RMS parallels the one that the sponsoring PSF used in its internal performance and career management system--incorporating the opportunity for multiple partners and multiple client assessments as well as the traditional peer (team members) and direct report assessments. The RMS was piloted tested with 38 principals as part of two professional development programs. The currently reported response scales and items reflect the language and understanding that the partners, principals, senior associates, associates, and clients have regarding the development of consulting skills.

RMS PSYCHOMETRIC PROPERTIES

While a detailed discussion of all the psychometric properties of the RMS is beyond the scope of this paper, summary statistics are reported below to reflect the analysis undertaken on a sample of 184 principals and new partners that participated in the RMS as a required part of a voluntary professional development program. A total of 1,451 RMSs were completed, with an average of 8 responders (9 including self assessments) per participant. The response rates of the different stakeholder classifications were 67% for partner responders, 73% for client responders, 84% for peer responders, and 88% for direct report responders. Data was missing from 1.3% of the responders possible responses, resulting in a decision to use pair-wise deletion of missing data. The sample demographics were: 58% U.S., 24% European, and the remaining Latin American/other; 90% male; mean age 38.3 years; and a mean firm tenure of 6.5 years.

1. Intra-rater correlations were calculated for direct report, peer, partner, and client ratings to determine the degree to which these stakeholders agreed in their assessments on each of the 29 competencies. A minimum of three stakeholders per category was used in this analysis. Item agreement was modest to good, ranging from intra-class correlations of .23 to .74, with a median r of .48. While the lower correlations may be interpreted as an indication of measurement error, it is also possible that different individuals within a stakeholder classification assessed PSPs differently based on accurate observations of different behaviors in different engagements with different clients, team members, and leadership. The way engagements are staffed, and the short term nature of many projects, raises ambiguity as to whether different assessments by members of a stakeholder group are meaningful indicators of different behaviors or measurement error (Yammarino & Markham, 1992).

2. While the RMS was developed as a 29-competency MSF tool, the competencies are clustered into six dimensions for feedback purposes. The logic of this clustering reflects the literature on the dimension competencies (e.g., leadership, trust, etc.) as well as the views of the sponsoring PSF. Coefficient alpha estimates of internal consistency were calculated for each stakeholder classification (n=5) for each dimension (n=6), yielding 30 coefficient alphas. These ranged from r =.26 to .69, with a median correlation of .47. This suggests caution in using the summary dimensions in lieu of the actual items for some stakeholder-dimension combinations.

3. Convergent validity estimates were calculated by examining the extent to which different stakeholder assessments on each item correlated highly with other stakeholders (a multitrait-multirater matrix). An average assessment was calculated for each stakeholder group. These average assessments were then correlated with each other across stakeholders and competencies. Of 116 correlations, 24 were below r =.25 (none were negative), 56 were above r = .40, and seven were above .60. The evidence for convergent validity is mixed. This, in part, led to the consideration of reasons for different stakeholder assessments discussed below.

4. In addition to convergent validity, discriminate validity estimates were calculated. Discriminate validity examines the extent to which items purported to be different have near zero correlations with each other. The evidence here was also mixed with some items relating modestly with other items that conceptually were viewed as distinct. While none of these relationships were large (e.g., none were higher than r = .40), some were statistically significant at the .05 level, with 80% ranging from r's of .10 to .24.

5. The concurrent validity of the items was assessed by calculating the correlations of the RMS competencies with the most recent year's annual performance assessments which had summary ratings of "1"--below average performance, "2"--average performance, "3" above average performance, and "4"--exceptional performance. For the year in question, those rated "1" received no bonus and were often 'councilled-out'. Those rated "2" receive a minimum bonus, typically 5%. Those rated "3" received a 15% bonus. Those rated "4" received a 30% bonus. The r's ranged from .32 to .57 (p < .01). The RMS stakeholders were in significant agreement with those who conducted the annual performance reviews. The overlap of these groups, based on a random sample of 30 participants where the names of the annual reviewers were compared to those who provided RMS responses, was only 37%. The RMS stakeholders often included clients and partners not interviewed as part of the annual review multirater process.

Following traditional psychometric theory guidelines, one might conclude that the RMS is a marginally adequate assessment tool--reliability, internal consistency, and validity indices are modest, and in some cases weak (Conway & Huffcutt, 1997; Murphy, Cleveland, & Mohler, 2001). However, such guidelines are primarily based on a model of work performance in manufacturing or financial service organizations. This work model assumes much more consistency in work and stability of assessors than is present in many PSFs--particularly the sponsoring PSF. The value and relevance of the RMS within the performance management system, and to potentially predict subsequent career success, needs to be explored before conclusions about the RMS's psychometric quality are reached.

RMS ASSESSMENT USE IN PERFORMANCE MANAGEMENT

In order to assess the value and relevance of the RMS, the sponsoring PSF was in consultation with nine other non-competing PSFs to discussion its generalizability and application. This consultation took the form of a forum of PSF senior human resource professionals who met quarterly for 1 to 2 days each time over two years to "share ideas in order to develop the best professional workforce possible." Participants included a senior HR professional from A. D. Little International, A. T. Kearney, Bain & Company, Booz|Allen|Hamilton, The Boston Consulting Group, Deloitte Consulting, Ernst & Young/Cap Gemini, Korn Ferry, KPMG Consulting, and Mercer-Delta Consulting.

These forums led to the sharing of human resource practices believed to be germane to the development and assessment PSPs by each PSF. They provided a sounding board for new ideas and an opportunity to develop shared understandings of important issues within the PSF industry. What follows is the 'collective wisdom' of the forum members that was summarized in meeting notes, as such wisdom relates to MSF and its use in performance management.

Face validity of RMS items and dimensions: There was strong agreement that the dimensions and items were germane to PSF work. While some PSFs would place a greater emphasis on some dimensions over others, or cluster the items differently into dimensions, all felt that the RMS reflected many of the competencies believe to be relevant to PSF work.

Convergent assessments among stakeholders: When different stakeholders agree in their assessment of an individual on a dimension (which averages the item responses) or a competency, using this assessment as a performance measure for reward allocation or professional development is uncomplicated. If most stakeholders generally agree that a competency is strong, the organization should reward it, encourage the individual to leverage the talent, and possibly even arrange for the individual to mentor or train others in this area. If the assessments converge around the belief that competency development is need, training and/or staffing to develop the competency should be undertaken. A mentor that exhibits this competency strongly might be identified and paired with the person needing development.

Different assessments by different stakeholders: An appropriate course of action is less clear when different stakeholders hold different assessments--some believing that they are observing a strength, others a weakness, and still others may not observe the behavior at all. In such situations it is necessary to understand the nature of the competency and the likelihood of it being exhibited and relevant to different stakeholders. For example, the leadership competencies--challenges the situation, inspires a shared vision, empowers others, models effective behavior, and celebrates achievements)--that direct reports look for in their bosses may be different competencies than looked for by partners in their subordinates or by clients in an engagement team leader. Similarly, an individual may exhibit different leadership behaviors to direct reports than he or she exhibits to partners or clients. Which ratings are most relevant to the performance management system must reflect the views and values of the PSF leadership. We present the forum members' perspectives on this issue--research examining the efficacy of these views is needed.

IMPORTANCE OF DIFFERENT COMPETENCIES TO DIFFERENT STAKEHOLDERS

The members of the forum provided their subjective assessments of which RMS competency dimensions would be most relevant to which stakeholder groups within their PSF. Table 2 summarizes the results using a low (L), medium (M), and high (H) index of value. A low rating means that the majority of the forum professionals rated this constituency's assessment of low value to their organization--they would not use this information in annual performance reviews. A high rating means that the majority of the forum professionals rated this constituency's assessment of high value--they would use this information in annual performance reviews. Medium ratings of value indicate that forum members were equivocal or near equally split in their beliefs that their firms would use or not use the dimension-by-stakeholder assessment.

Self ratings: Self ratings on all competencies were viewed as valuable only in that they could help an individual understand his or her performance relative to the assessments by others. This was viewed by the forum members as more of a self-awareness measure than a performance measure (Ashford, 1989). Self ratings would rarely be used in a quantitative manner in annual reviews or for annual bonuses.

Direct report ratings: The assessments of direct reports on leadership and coaching competencies were highly valued. Direct reports were viewed as most affected by these behaviors and were in the best position to assess the value to them and the firm (or team). The rationale was that most direct report observations take place at the client's facilities within a team room, where few if any senior PSPs are routinely present.

Peer ratings: Peer assessments of building relationships, trust, and collaboration were highly valued and would be used in performance management. Building relationships and trust with peers was viewed as an early indicator of being able to do so with clients. It was the peer assessment of collaboration that was most valuable to several of the PSFs as collaboration was considered 'voluntary' and beyond the demands of an engagement--unless it was so directed by a senior PSF--then it was no longer viewed as collaboration but as part of the revised engagement assignment. Peers could avoid collaboration if they wanted to do so by noting that he/she was already over-committed and working 80 hours per week.

Partner and client ratings: Partners and clients were viewed as the most important assessors of building relationships, trust, and relationship outcomes--subject to the individuals being assessed having ample contact with these stakeholders. If was recognized that junior PSPs might not have sufficient exposure to either partners or clients for such assessments to be made. For the senior PSPs, including partners, the other partner and client assessments were the highly valued and would be a prime source of assessment information for annual reviews and bonuses.

HYPOTHESIZED PERFORMANCE DIMENSION: STAKEHOLDER LINKAGES TO CAREER SUCCESS

In addition to considering stakeholder assessments of competencies for use in annual reviews and performance management, these assessments are potentially related to future career success. Based on over 240 partner and clients interviews and 164 associate/senior associate/principle interviews done in the creation of the RMS, it was possible to define career success for PSFs and summarize the relationships proposed during the interview process.

A "career success team" of 6 PSPs reviewed and discussed the interview reports for indications of how the RMS competency dimensions might predict career success. Career success was viewed as having both objective and subjective aspects--ranging from increases in salary and promotions, to satisfaction with one's career progress and competency development. While not a comprehensive list, the items in Table 3 reflect the conceptualization of career success considered by this team in its analysis of the host PSF interviews. Each of the items in Table 3 could become a measure used in subsequent research.

Based on the RMS dimensions and stakeholder constituencies identified, the suggested or implied linkages (hypotheses) reported by the career success team are summarized in Table 4. A low (L) strength of relationship is defined as a simple correlation between the dimension by stakeholder constituency aggregate measure and a measure of career success (e.g., annual salary increase, promotion, etc.) of .150 or below. A medium (M) strength of relationship is a correlation of .151 to .350. A strong (S) correlation is one of .351 or above. These subjective definitions parallel the level of relationships typically found with performance measures at one time with outcome measures two or more years later. As all dimensions are worded and scaled such that a higher response is 'better', all expected relationships would be positive.

RATIONALE FOR THE HYPOTHESIZED RELATIONSHIPS

All stakeholder groups spoke of the importance of leadership. However, it was the leadership competencies of their immediate boss that were most salient to them. Given the significant use of project teams that involved many peer relationships, the leadership competencies of one's peers was also noted as key to career success. Based on these interview notes, it is hypothesized that one's direct report assessments of leadership competencies will have the strongest relationship with subsequent career success, with peer assessments having the next strongest relationship.

The need to build relationships is essential to one's success in a PSF environment. This need becomes increasingly important as one is advanced--ultimately becoming a client service partner and leaving the more technical and analytical duties for less senior professionals. Based on the nature of the work at different career levels (Maister, 1997; Stumpf, 1999), it is hypothesized that the perspective of the more senior roles (peer, partner) and the clients' perspective will be the best predictor of subsequent career success. Since relationships are 'mutual', the individual self-report might also be a reasonable predictor of the quality of the relationship, and hence career success.

Trust is one of the most valued attributes within a PSF relationship. If the leaders (partners) or clients do not perceive one as trustworthy, career success is severely limited. Similarly, but somewhat less importantly, a perception of mistrust among one's peers can be damaging. And, as with relationship building skills, trust is something around which one should have some self-understanding. The trust-career success relationship is hypothesized to be high for partners and clients and moderate for both peer and self-report assessments.

Due to the project nature of the work, often involving different people on each project, and the frequent time-related pressures to produce results within agreed upon billable hours, collaboration with others over which one has no direct control or authority is important. Peers are most likely to be the best judge of how well one collaborates--are others there when needed with useful ideas and assistance? The peer assessed collaboration-career success relationship is hypothesized to be the strongest. The effects of collaboration may be seen by direct reports, partners, and clients--suggesting that a moderate strength collaboration-career success relationship.

Coaching is a skill predominantly experience by oneself along with one's direct reports (those being coached). The importance of coaching stems from the apprenticeship model of career development common to many PSFs. Learning the craft involves getting advice and guidance from a more senior person. It is the coachee's view of that competency that is hypothesized to be most strongly related to the career success of the coach.

The relationship outcomes items address the results of one's work. The clients and partners are in the best position to both observe these outcomes and use this information for career management purposes--hence the hypothesis of a high correlation. More moderate relationships with career success are hypothesized with peer and direct report assessments as both these stakeholder groups would also have some knowledge of the competencies exhibited.

No self-rating assessment is hypothesized to be a strong predictor of career success (Yammarino & Atwater, 1997). This reflects the findings in the MSF literature cited above, and was confirmed by the interview reports done as part of RMS development. Only the dimensions that involve a mutual experience between the individual and different stakeholders were considered likely to have a moderate relationship with career success--i.e., relationship building, trust, and coaching. The subjective perceptual bias of one's self-reported leadership, collaboration, and relationship outcome competencies seemed probable--leading to the hypothesized weak relationships of these perceptions to subsequent career success.

DISCUSSION

The key ideas proposed are that stakeholder assessments can be used as valued input into a PSFs performance management system, and might also be used to predict subsequent career success. Stakeholder assessments are not, a priori, expected to agree--different stakeholders constituencies may value different behaviors and have different goals that they are pursuing via the relationship. This suggests that in using stakeholder assessments for performance management, the users take time to understand the nature of the stakeholder relationship to the focal PSP, and incorporate this understanding in to the PM process. Similarly, if PSPs use stakeholder assessments as input into their career development activities, they should do so with an understanding of the different goals and expectations of the different stakeholder constituencies.

Both uses of the RMS assessments have implications for incorporating feedback from stakeholders in one's career management. Research supporting these hypotheses would be particularly relevant for identifying high potential professionals and determining whether stakeholder judgments are indicators of advancement. Research exploring these hypotheses will determine differences among stakeholder groups in predicting career success and whether selected performance dimensions from different stakeholders are more or less important to future career success. This has both theoretical and practical relevance. It will advance our understanding of how stakeholders' perceptions affect later judgments of those making personnel decisions. Such research can inform the design of succession management and career development programs. They may lead to stakeholder training in evaluating high potential professionals and to target PSPs improved use of stakeholder ratings in shaping their career plans.

Research exploring these hypotheses should contribute to the stakeholder research literature and the professional development literature on MSF practices in several ways. It can:

(1) provide an enhanced understanding of what different stakeholder groups value in the PSP's behaviors, thereby supporting the importance of involving stakeholders in organizational issues.

(2) add an alternative perspective with respect to measurement issues in MSF whereby different stakeholder assessments are expected to correlate (converge) as a sign of construct validity. Statistical independence (low correlations) among different stakeholder assessments may be a more meaningful sign of validity than their covariation.

(3) provide individuals receiving MSF with guidance on what different assessments by different stakeholders may mean with respect to their career success, and suggest how to interpret differences in these assessments as function of the likely importance of a competency to the stakeholder.

(4) provide work organizations, particularly the human resource function, with more information on which to base their design of MSF surveys and from whom to solicit feedback on which MSF items/dimensions.

(5) provide PSFs with a better idea of what performance competencies really matter to their stakeholders as these competencies relate to the future success of their leaders and other professionals.

ENDNOTE

The author is the Fred J. Springer Chair in Business Leadership at Villanova University. While employed at a professional service firm (PSF), the author led the team that developed the Relationship Management Survey. He expresses his appreciation to the professional service professionals who shared their insights and ideas, with special thanks to John Helding, Mark Nevins, and John Savage as the thought leaders in this regard. This paper has benefited from the input of many colleagues, including extensive work on the design and statistical analysis of the RMS by Michele Williams, and thoughtful ideas regarding PSFs by Jonathan Doh and Kevin Clark

REFERENCES

Ashford, S. J. (1989). Self-assessments in organizations: A literature review and integrative model. In L. L. Cummings & B. M. Staw (Eds.). Research in Organizational Behavior. Vol. 11. Greenwich, CT: JAI Press.

Ashford, S. J. & A. S. Tsui (1991). Self-regulation for managerial effectiveness: The role of active feedback seeking. Academy of Management Journal, 34, 251-280.

Bessom, R.M. & D. W. Jackson (1975). Service retailing: A strategic marketing approach. Journal of Retailing, 8, 137-49.

Bracken, D. W., C. W. Timmerck & A. H. Church (2001). (Eds.) The handbook of multisource feedback. San Francisco, CA: Jossey-Bass.

Campbell, J. P., M. D. Dunnette, E. E. Lawler, III & K. E. Weick (1970). Managerial behavior, performance, and effectiveness. NY: McGraw-Hill.

Cleveland, J. N., K. R. Murphy & R. Williams (1989). Multiple uses of performance appraisal: Prevalence and correlates. Journal of Applied Psychology, 74, 130-135.

Conway, J. M. & A. I. Huffcutt (1997). Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. Human Performance, 10, 331-360.

DePree, M. (1997). Leading without power. San Francisco, CA: Jossey-Bass, Inc.

Doh, J. P. & S. A. Stumpf (2005a). (Eds.). Handbook on responsible leadership and governance in global business. Cheltenham, UK: Edward Elgar.

Doh, J. P. & S. A. Stumpf (2005b). Towards a framework of responsible leadership and governance. In J. P. Doh and S. A. Stumpf (Eds.), Handbook of responsible leadership and governance in global business: 3-18. Cheltenham, UK: Edward Elgar.

Freeman, R. E. (1984). Strategic management: A stakeholder approach. Boston, USA: Pitman.

Freeman, R. E. & D. R. Gilbert Jr. (1988). Corporate strategy and the search for ethics. Englewood Cliffs, NJ: Prentice-Hall.

Goldsmith, M. & C. Walt (1999). Developing technological savvy: Leader to leader. San Francisco, CA: Jossey-Bass.

Howard, A., W. Byham & P. Hauenstein (1994). Multirater assessment and feedback. Pittsburgh, PA: DDI.

Kouzes, J. & B. Posner (1987). The leadership challenge. San Francisco: Jossey-Bass.

Landy, F. & J. Farr (1983). The measurement of work performance: Methods, theory, and applications. San Diego: Academic Press.

Lepsinger, R. & A. Lucia (1997). The art and science of 360-degree feedback. San Francisco, CA: Jossey-Bass.

London, M. (1995). Self and interpersonal insight. How people gain understanding of themselves and others in organizations. NY: Oxford University Press.

London, M. (1997). Job feedback: Giving, seeking, and using feedback for performance improvement. Mahwah, NJ: Lawrence Erlbaum Assoc.

London, M., J. W. Smither & D. L. Adsit (1998). Accountability: The Achilles' heel of multisource feedback. Group and Organization Management, 22, 162-184.

Maister, D. H. (1993). Managing the professional services firm. New York: Free Press.

Maister, D. H. (1997). True professionalism. New York: Free Press.

McCall, J. J. (2002). Leadership and ethics: Corporate accountability to whom, for what and by what means. Journal of Business Ethics, 38(1), 133-139.

Mitchell, R. K., B. R. Agle & D. J. Wood (1997). Toward a theory of stakeholder identification and salience: Defining the principle of whom and what really counts. Academy of Management Review, 22(4), 853-886.

Murphy, K. R., J. N. Cleveland & C. J. Mohler (2001). Reliability, validity, and meaningfulness of multisource ratings. In D. W. Bracken, C. W. Timmerck, & A. H. Church (Eds.), The handbook of multisource feedback: 130-148. San Francisco, CA: Jossey-Bass.

Nadler, D. (1977). Feedback and organizational development: Using data based methods. Reading, MA: Addison-Wesley.

O'Shea, J. & C. Madigan (1997). Dangerous company. NY: Random House.

Peters, L. J. & E. J. O'Connor (1985). Situational constraints and work outcomes: The influences of a frequently overlooked construct. Academy of Management Review, 5, 391-398.

Post, J. E., L. E. Preston & S. Sachs (2002). Managing the extended enterprise: The new stakeholder view. California Management Review, 45(1), 6-29.

Rackham, N., L. Friedman & R. Ruff (1996). Getting partnering right. NY: McGraw-Hill.

Schaffer, R. H. (1997). High impact consulting. San Francisco, CA: Jossey-Bass.

Smither, J. M. (1998). Performance appraisal: State of the art in practice. San Francisco, CA: Jossey-Bass.

Stumpf, S. A. (1999). Phases of professional development in consulting. Career Development International, 4(7), 392-399.

Stumpf, S. A. (2002). Becoming a partner in a professional services firm. Career Development International, 7(2), 115-121.

Stumpf, S. A., J. P. Doh & K. D. Clark (2002). Professional services firms in transition: Challenges and opportunities for improving performance. Organizational Dynamics, 31(3), 259-279.

Stumpf, S. A. & R. Longman (2000). The ultimate consultant: Building long term, exceptional value client relationships. Career Development International, 5(3), 124-134.

Stumpf, S. A. & W. Tymon (2001). Consultant or entrepreneur?: Demystifying the 'War for Talent'". Career Development International, 6(1), 48-55.

Tymon, W. & S. A. Stumpf (2003). Social capital in the success of knowledge workers. Career Development International, 8(1), 12-20.

Tornow, W., M. London & CCL Associates. (1998). Maximizing the value of 360-degree feedback: A process for individual and organizational development. San Francisco, CA: Jossey-Bass.

Waddock, S. (2005). Corporate responsibility, accountability and stakeholder relationships: Will voluntary action suffice? In J. P. Doh & S. A. Stumpf (Eds.), Handbook of responsible leadership and governance in global business: 180-194. Cheltenham, UK: Edward Elgar.

Waldman, D. & L. E. Atwater (1998). The power of 360-degree feedback. Houston, TX: Gulf Publishing.

Yammarino, F. J. & L. E. Atwater (1997). Do managers see themselves as others see them? Implications of self-other rating agreement for human resource management. Organizational Dynamics, 25(4), 35-44.

Yammarino, F. J. & S. E. Markham (1992). On the application of within and between analysis: Are absence and affect really group-based phenomena? Journal of Applied Psychology, 77, 168-176.

Stephen A. Stumpf, Villanova University

Table 1:  Relationship Management Survey Items Clustered into
Six Dimensions

  1.0       Leadership
  1.1       Challenges the Situation
  1.2       Inspires a Shared Vision
  1.3       Empowers Others
  1.4       Models Effective Behavior
  1.5       Celebrates Achievements
  2.0       Building Relationships
  2.1       Understands Others
  2.2       Can Take Others' Perspective
  2.3       Puts Others at Ease
  2.4       Shows Concern for Others
  2.5       Respects Others
  3.0       Trust
  3.1       Generates Confidence
  3.2       Engenders Mutual Trust
  3.3       Is Dependable
  3.4       Is Open to and with Others
  4.0       Collaboration
  4.1       Communicates Well Across Boundaries
  4.2       Learns from Others
  4.3       Creates Synergies Across Boundaries
  4.4       Seeks Win-Win Outcomes
  4.5       Shares Useful Expertise
  5.0       Coaching
  5.1       Doesn't Dwell on Failures
  5.2       Encourages Others to Take Reasonable Risks
  5.4       Rewards Achievement
  6.0       Relationship Outcomes
  6.1       Achieves Impact
  6.2       Builds Client Relationships
  6.3       Develops Team Relationships
  6.4       Collaborates Well with Clients
  6.5       Fosters Information Sharing that Leads to Results
  6.6       Develops Personal Relationships

Table 2:  Expert Opinion of the Relative Value of Skill Dimensions by
Stakeholder Constituency for Use in Performance Management

        Skill               Self       Direct      Peers
      Dimension                       Reports

Leadership                   L           H           M
Building Relationships       L           M           H
Trust                        L           M           H
Collaboration                L           M           H
Coaching                     L           H           M
Relationship Outcomes        L           L           M

        Skill             Partners    Clients
      Dimension

Leadership                   M           L
Building Relationships       H           H
Trust                        H           H
Collaboration                M           M
Coaching                     L           L
Relationship Outcomes        H           H

Table 3: Career Success Measures

  1       How has your skill proficiency changed over the past three
          years? (1=skills are atrophying, 2-3=no real change, 4-5,
          =much more skilled)

  2       How many promotions have you earned in the past 3 years
          including promotions associated with a change in employer?
          (number)

  3       How satisfied are you with your career progress over the
          past 3 years? (1=not satisfied, 2-3,= moderately satisfied,
          4-5 =, very satisfied)

  4       How satisfied are you with your career success as defined
          by you over the past 3 years? (1=not satisfied, 2-3 =
          moderately satisfied, 4-5 = very satisfied

  5       How satisfied are you with your personal success over the
          past 3 years? (1=not satisfied 2-3 = moderately satisfied,
          4-5 = very satisfied)

  6       How do you view your career progress relative to your peers
          over the past 3 years? (1=less progress, 2-3 = about the same
          progress, 4-5 = much greater progress)

  7       How do you view your career success relative to your peers
          over the past 3 years? (1=less success, 2-3=about the same
          success, 4-5 = much greater success)

  8       How satisfied are you with your salary progression over the
          past 3 years? (1=not satisfied, 2-3 = moderately satisfied,
          4-5 = very satisfied)

  9       Assess your salary progression relative to your peers over
          the past 3 years? (1=less, 2-3 = about the same, 4-5 = much
          greater)

  10      How satisfied are you with the employer perks provided and
          your quality of work life over the past 3 years? (1=not
          satisfied, 2-3 = moderately satisfied, 4-5 = very satisfied)

  11      Assess the employer perks provided and your quality of work
          relative to your peers over the past 3 years? (1=less 2-3 =
          about the same, 4-5 = much greater)

  12      How much has your total annual compensation changed over the
          past 3 years? (numbers in $5,000 increments starting with
          $0 up to $50,000 or more)

  13      On a percentage basis how much has your total salary and
          bonuses increased cumulatively over the past three years?
          (0% to 40% or more in 2% increments)

  14      Relative to your peers how has your industry marketability
          --ease of getting a position in another firm within your
          industry--changed over the past three years? (1=less
          marketable, 2-3 = about the same, 4-5 = much more marketable)

  15      Relative to your peers how has your overall marketability
          --ease of getting a position in any firm--changed over the
          past three years? (1=less marketable, 2-3 = about the same,
          4-5 =,much more marketable)

Table 4: Hypothesized Strength of Relationship with Career Success
Measures By Stakeholder Constituency

        Skill               Self       Direct      Peers
      Dimension                       Reports

Leadership                   L           H           M
Building Relationships       M           L           H
Trust                        M           L           M
Collaboration                L           M           H
Coaching                     M           M           L
Relationship Outcomes        L           M           M

        Skill             Partners    Clients
      Dimension

Leadership                   L           L
Building Relationships       H           H
Trust                        H           H
Collaboration                M           M
Coaching                     L           L
Relationship Outcomes        H           H