Keys To Effective Evaluation

Alliance of Work-Life Professionals
1997 San Diego Conference
February 6-7, 1997

Summary prepared by David H. Rodbourne ~ Director of Programs
Minnesota Center for Corporate Responsibility

Dupont

Cindi Johnson, Competency Leader, Work/Life Human Resources Department, Dupont, presented a powerful argument for the importance of solid data and evaluation in securing the commitment of senior executives. Dupont's Work/Life group recognized the critical importance of winning the support from the vice presidents and general managers responsible for its business units. First, they undertook a thorough demographic, issues and trend analysis of Dupont's workforce. Results were organized so that each business unit could determine its own situation. Second, the Work/Life group secured the endorsement of Dupont's most respective executive on research. He told the VP/GMs that "this is valid data" and added that he had tried every way to tear it apart before reaching that conclusion. Third, when the data was present to them, the VP/GMs reacted with this conclusion - "this data is so compelling that if we don't pay attention our businesses won't be successful." Johnson said that winning endorsement from the highly respected senior research executive gave them instant credibility with operating executives, instant access to the CEO, and a powerful champion. The attention triggered by Dupont's research on its U.S. workforce has triggered action by its European operations.

One interesting finding on the Dupont workforce which is 76% male: Among "exempt" men, the number of families with dual careers is decreasing. Among "non-exempt" employees, the number of families with dual careers is increasing. Conclusion: many senior managers do not experience and do not understand what other managers and employees are going through in trying to balance work and family.

Motorola, Inc.

Kathy Zweber, Director, Work/Life and Wellness Initiatives, Motorola, said that a simplistic analysis of Work/Life program results is not enough. She argued that companies need to put Work-Life evaluations in a context that enables them to answer several key questions: Are these results good enough? Is this program or service appropriate and effective? How do we define excellence?

How can we improve?

• Identify the specific product or service that is being offered (in Work/Life).

• Identify the customer and determine what is important to them (in any Work/Life practice).

• Identify what it takes for the service provider to be able to provide high or appropriate levels or quality of service.

• Flow chart the Work/Life service processes in detail in order to identify where service breakdowns could occur. By predicting where mistakes might occur you are better positioned to prevent them.

• Measure results and continuously work to improve these service processes.

Zweber stressed the importance of including both users and non-users of programs in any evaluation study. You need to think carefully, for example, about how do you (and your customers) define quality in a childcare center or in a resource and referral service. Without such criteria evaluation is of doubtful value.

One alternative to surveying workers - many of whom feel they've filled out one too many surveys already - is to look for alternative sources of data. Summaries of health claims can provide clues. For example, one company found that 53% of claims paid were for mental health services.

Alfred P. Sloan Foundation

Kathleen Christensen, Program Officer, Alfred P. Sloan Foundation, said that shoddy research on Work/Life results undercuts the message. She cited a number of flaws:

• Relying on "self-reports" by those using programs opens the door to bias. Participants themselves may not have a clear understanding of what to expect and/or may use criteria different from those relevant to the business.

• Timing of an evaluation can bias results. Early evaluations may suffer from bias present in a honeymoon period. Voluntary participants in telecommuting pilot program, as one example, may approach it quite differently than employees who telecommute when the program is scaled up.

• Measure what is relevant; not just what is easy to measure.

• Measure the whole Work-Life complement of strategies; not just one program in isolation.

• Measure multiple dimensions; not just a single aspect of a program or policy's impact.

She, and others speaking, challenged Work-Life professionals to clearly define what they mean by productivity. Managers who equate productivity with simply being present (as opposed to absent) may be satisfied by a measure that shows reduced absenteeism. But, managers who think of productivity as being "psychologically engaged" will want to know about creativity, team performance, etc.

Basing decisions and full-scale programs on faulty research or flawed assumptions will lead to failure in the long run.

Families and Work Institute

Arlene Johnson, Vice President, Families and Work Institute, argued that studies done by companies, nonprofit centers and universities have taught us a great deal and provide reliable guides for action.

• One conclusion she cited: programs are not sufficient. It is important to get into changing an organizational culture.

• Second, she cited research finding a powerful symmetry between what makes good management and organizational effectiveness and people's sense of balance and well-being. When you find one, you find the other.

• Third, national data on the workforce can be helpful. A powerful tidbit: "more than half of women who work supply more than half of their family income."

Still, she concurred that, while national data may get an executives attention, you need data on your own industry and workforce to stimulate executives to take action. In other words, executives are asking how will this effect my business and my bottom line.

Observations: Everyone agreed that better evaluation of Work-Life was extremely important. On the other hand, good evaluation requires time, expertise and can sometimes be expensive. Under pressure to control costs, it can be difficult to make the case for investing in solid research on workforce demographics, Work-Life issues, and program evaluation. No one had any solution to this dilemma, but Dupont's example of how solid research can leverage executive action tells us why it is important to do better.

Collaboration - with other companies and with academic partners - is one research strategy that deserves greater attention. And it is a good way to share the load.

Using "anecdotes" to tell the Work-Life story is problemmatic. Some panelists argued forcefully that stories do not persuade executives to take action. Others were equally certain that stories can play a key role, principally in bringing data to life. On the whole, most agreed anecdotes are a poor basis for making policy.

Summary prepared by David Rodbourne, director of programs, Minnesota Center for Corporate Responsibility for MCCR's Work/Life Task Force and The Work«Life Network. This summary is based on notes taken during conference presentations.

Center for Ethical Business Cultures

1000 LaSalle Avenue, TMH 331 ▪ Minneapolis, MN 55403-2005 ▪ USA

Phone: 651 962 4120 or 800 328 6819 Ext. 2-4120 ▪ Facsimile: 651 962 4042

Email: mail@cebcglobal.org

© 1978-2008 Center for Ethical Business Cultures. All Rights Reserved.

Business Partnering with the University of St. Thomas - Minnesota