5 Questions to Ask Before Hiring an Evaluator

Making the decision to hire an evaluator is a big step. Almost as big as deciding to get married! After all, you and your evaluator will be glued-to-the-hip for an extended period of time—at least a year, if not for the full term of your project: three to five years, give or take a no-cost extension or two! Here are five minimum questions you should ask a prospective evaluator before making your final decision.

Question 1: What are your credentials?

Credentials are qualifications and consist of:

• Academic education, degrees and professional training

• Years experience as an evaluator

• Experience with programs and populations similar to yours

• Experience with various forms of evaluation design and statistical analysis

To answer your question, your evaluation candidate should be able to provide evidence of his/her credentials, including but not limited to:

• Curriculum Vitae (CV) or Summary of Qualifications

• Description of Sample Projects (discipline, population, evaluation design and methods of analyses employed)

• One or two sample reports or published articles authored by the evaluator

Question 2: How familiar are you with the population and community we serve?

Many evaluators work across the nation and internationally. However, just because an evaluator is not a member of your community or neighborhood, doesn’t mean he/she can’t effectively serve your project, especially using technology. However, evaluators should know something about you, your organization and community, the population and yes, even the policies and culture of your geographic service area. He/she should be able at interview to show some familiarity with:

• Your organization, the community where you are located, and the population you serve

• Basic demographics of your population, for example: gender, age, developmental age, race/ethnicity, economic levels, health conditions, or languages spoken

• Recent policy or cultural issues that could negatively impact or positively benefit your project

Question 3: Are you willing to train our staff on evaluation as part of your services?

It is no secret that project personnel, who have never worked with an evaluator, are afraid of finding themselves or their projects 'under the microscope'. They might fear they will lose their positions if an evaluator thinks they aren’t doing a good enough job or that their workload and paperwork will increase or materially change. Some believe evaluation is a waste of much needed resources and that the dedicated budget should go to services and constituent needs.

More importantly, not all project personnel understand the what, why, how, and how-to of program evaluation, its benefits, its methods or its terminology. Successful projects blend program with evaluation—the two must work hand-in-hand. So, find out if your prospective evaluator will:

• Train staff on the purposes and components of your project evaluation

• Take time to show them how each part of the program fits with the various evaluation components

• Explain how the data will be collected, and how it will be used

• Discuss the benefits of evaluation for your project as well as your organization. Well-leveraged evaluation results can grow your organization, expand your service capacity, increase your organizational capability, and increase your funding success!

• Provide on-going technical assistance

Question 4: Are you registered with an Institutional Review Board?

While the Secretary of Health and Human Services (HHS), in conjunction with the Office of Science and Technology Policy (OSTP) is considering revisions to the rules governing Participant Protection and Confidentiality is some areas affecting evaluation (American Evaluation Association, info@eval.org, September 6, 2011), it is important to understand your project may be subject to review and approval by an Institutional Review Board (IRB). This is particularly true if your project serves ‘vulnerable populations’ such as minor children, minorities, and individuals involved with the criminal justice system, among others.

Obtaining an IRB can be a labor intensive and at times, expensive process. Find out if your prospective evaluator is currently registered with an IRB. If not, ask how he/she would go about securing one for your project.

Question 5: How do you structure your fees?

Whether your prospective evaluator is an individual, a member of a for-profit or non-profit corporation, or a university faculty member, he or she will expect payment for the services they provide. Some evaluators take projects on a flat-fee basis, others charge by the hour and still others charge based on ‘deliverables’. You also want to know when and how you will pay your evaluator. Some prefer to be paid monthly, others at the time of delivery of a project deliverable, and still others ask for ‘retainers’ paid up-front, followed by remaining payments structured over the balance of the contract period (e.g. quarterly).

Consider these questions. Take time to interview your prospective evaluator. Get to know him or her and make sure the two of you 'fit'. If you do, it will be match made in heaven!