RAINN
Phase 4

Evaluate Your Work

Evaluation of the online hotline focuses on both process evaluation — how and what services are delivered — and outcomes evaluation — how useful or effective were the services. Combining these two evaluation approaches can paint the richest portrait of RAINN’s successes, as well as the changes that might better serve its clients.

Process Evaluation

A process (formative) evaluation of RAINN’s online hotline includes collecting information about the number of visitors, length of sessions, technical difficulties, types of situations discussed, types of services provided, and demographic information about visitors. In addition, qualitative information from Focus Groups provides information from volunteer staff about their perceptions of training, training needs, experience with use of the technology, and general feedback about the program.

Outcome Evaluation

Outcome (summative) evaluation includes visitors’, staffs’, and supervisors’ perceptions of the usefulness of the online hotline; user satisfaction with the services; and users’ intention to use services to which they were referred. Although improvements in visitors’ emotional well-being and perceived safety would also be valuable outcomes, the current RAINN model—which provides mostly one-time crisis intervention—affords little opportunity to gather these data.

Answer the following questions

  • What are two process evaluation questions that can be answered from RAINN system data?
  • What are two outcome evaluation questions that can be answered from the Online User Survey?
  • Why is it important to have both process and outcomes data in evaluating a program? Why might this be particularly crucial in the event of a relatively novel service delivery model, as RAINN was in its inception?
  • If you were a policymaker deciding how to spend scarce intervention dollars, what could you learn from other experiments with online service delivery to inform your evaluation? For example, in some states, agencies serving Medicaid recipients are putting more of their applications and resources primarily online when many of the Medicaid population do not have access to technology. What questions should advocates be asking about the technological advances on reducing costs and affecting access?

My Evaluation Tasks

Task 1

Use the Excel or SPSS data files to develop and answer research questions based on the Visitor and Volunteer data.

Task 2

Describe the limitations of the RAINN data. What questions cannot be answered from the available data? Can RAINN be considered successful given these limitations? Why or why not?

Task 3

Provide a summary of the evaluation. Given the evaluation data, write a brief summary about the effectiveness of the online hotline and what can be done to improve it.