What is your understanding of a program logic model? Explain using your own words. Discuss the limits of logic modeling as an approach to describing programs. What roles do program logic models play in the process of evaluating programs? Please explain so we can learn from your response. For program evaluators? For other stakeholders in the evaluation process? Select a policy or program and create a logic model – see examples provided in the PowerPoint. A logic model template is included in the module for this week. Upload a copy of your original logic model for your chosen program/policy. USE POWERPOINT ATTACHED TO FILE FOR REFERENCE AND EXAMPLES PAD 4320 PROGRAM EVALUTION IN PUBLIC MANAGEMENT WEEK 3 – AGENDA • Reflection on what we have covered so far. • This Week: Introduction and Understanding of Logic Models • How to Create a Logic Model BEFORE WE BEGIN — THIS SLIDE WILL APPEAR ON ALL P OW E R P O I N T P R E S E N TAT I O N S • Please note that some of the material presented herein has been adopted or adapted from various sources. Since this is solely for educational purposes, direct citation is not mandatory for today’s classroom. However, if you plan on using some of the information presented herein, please make sure to properly cite. WHAT IS PROGRAM EVALUATION? • Evaluation in this course refers to social programs and policy evaluations. • Why is evaluation carried out and for what reasons. There are distinct differences in the various types of evaluations and the context of the evaluations. • The text identified 6 P’s — programs, policies, products, personnel, performance, and proposals • People evaluate different things in their personal, academic, or professional lives. People can evaluate the books they read, the cars they drive, or the employees they supervise. In your academic life, you also evaluate professors each semester. The why, how, and for whom should be considered in each evaluation. There are differences in the important details for each type of evaluation. WHAT HAVE WE LEARNED SO FAR? Program evaluation consists of a set of activities developed to help correct, support, and extend the way that people, individually, collectively, or naturally make sense of policies and programs implemented to meet human needs. • We developed an understanding of the foundation for program evaluation. • Four purposes of evaluation (p. 13) — evaluators need to clearly distinguish these four purposes. • Assessment of merit and worth • Program and organizational improvement • Oversight and compliance • Knowledge development • Four inquiry modes for evaluation practice (p. 15) • The path from human needs to social betterment (p. 25) 5 W H Y I S P RO G R A M E VA LUAT I O N I M P O RTA N T ? We examined different types of program evaluation, and the different reasons to evaluate programs. We also reviewed ethics and values in program evaluation, and we discussed appropriate methods for program evaluation. Evaluations exists to improve programs and policies • Social Betterment • Meeting Human Needs • Legitimacy and Credibility • Improve Social Conditions • Oversight and Compliance • Assessing the extent to which policies and programs follow the directives of statutes, regulations, and other mandates. • Seeking Better Ways • Improving, overseeing, selecting or understanding policies and programs • Supporting decisions, deliberations, and actions WHAT ARE PROGRAM LOGIC MODELS? Logic models are useful for visually summarizing the structure of a program. Program logic models are widely used to develop to show the intended causal linkages in a program. There are many different styles of logic models, but they identify the major set of activities in the program, their intended outputs, and outcomes. • Logic model serves as a roadmap for a program to produce its intended outcomes. • Program logic models also help evaluators better understand the roles that different types of stakeholders play within the program. • The logic model is a beneficial evaluation tool that facilitates improvement in program planning, implementation & evaluation. 7 • WHAT IS A LOGIC MODEL? A logic model: • provides a way of visually representing the essentials of a program, policy, or even a whole organization, as an open system • categorizes key activities and outlines the intended flow of these activities, linking outputs to outcomes • depicts networks of constructs (ideas/concepts) that can indicate intended cause-and-effect linkages 8 DESCRIBE THE PROGRAM VIA LOGIC MODELS AND PROGRAM THEORY A program logic model is a picture of how your program works –the theory and assumptions underlying the program.This model provides a road map of your program,highlighting how it is expected to work,what activities need to come before others,and how desired outcomes are achieved (Kellogg Foundation,p.35). Using evaluation and the logic model results in effective programming and offers greater learning opportunities, better documentation of outcomes,and shared knowledge about what works and why. 9 • Program logic models are often developed with five categories, illustrating means such as: 1. Inputs and Resources 2. Activities 3. Outputs 4. Outcomes 5. Impacts. • In other words, inputs/resources and activities of a program are planned and implemented in order to produce its desirable outputs, outcomes, and impacts. CATEGORIES OF PROGRAM LOGIC MODELS 10 D E V E L O P I N G T H E E VA LUAT I O N P L A N • Program context • What is the problem or need for the program? • What are the goals and objectives of your program? • • Who is involved? • What activities will take place? How will you measure progress and impact? Create a’Logic’ Model Inputs: Resources necessary for program implementation Illustrate the relationship among your program/project elements: Activities: Interventions that will be implemented to achieve outcomes Outputs: Direct products obtained as a result of program activities CREATE A ‘LOGIC’ MODEL Outcomes: The impacts, changes, or results of the program activities and outputs; link to your objectives and your goals Short-term: Intermediate-term: Long-term: ? knowledge ? behaviors ? environmental ? skills ? practices ? social conditions ? attitudes ? policies ? economic ? motivation ? procedures ? conditions ? awareness ? political ? conditions ESSENTIALS OF A LOGIC MODEL Inputs Components Outputs Outcomes (short, medium, long term) Required resources, for example, money people equipment technology facilities Major clusters of activities, for example, intake skill development job placement Typically, units of work done or activities completed, for example, Number of clients served Number of job placements Intended outcomes related to the design of the program, policy, or organization, for example, Number of people employed 15 MCDAVID, PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT, 3E. © SAGE PUBLICATIONS, 2019 KEY CONCEPT S AND TERMS • Logic models categorize key program activities • inputs–the resources needed for the program or organizational components • components–clusters of activities that produce outputs, for example, a job training program has three components: intake, skills development, job placement • outputs–typically the units of work done, for example, number of clients served; number of training hours • Logic models link outputs to intended outcomes • the component activities and outputs are intended to result in specified outcomes, for example, number of certified skilled workers; number of people with jobs 16 CREATEA’LOGIC’MODEL Programs have boundaries (usually invisible) Programs have purposes (goals,objectives) ASSUMPTIONS UNDERLYING LOGIC MODELS The structure and function of programs can be seen as supporting a series of means-ends relationships The program model can be specified, that is, it contains the essential constructs and their intended linkages The level of complexity of the program will need to be considered (more on this later) 18 ESSENTIALS OF A LOGIC MODEL Inputs Components Outputs Outcomes (short,medium,long term) Required resources, for example, money people equipment technology facilities Major clusters of activities, for example, intake skill development job placement Typically,units of work done or activities completed,for example, Number of clients served Number of job placements Intended outcomes related to the design of the program,policy, or organization,for example, Number of people employed 19 A B A S I C L O G I C M O D E L F O R P O L I C E B O DY-WO R N C A M E R A PROGRAMS MCDAVID, PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT, 3E. © SAGE PUBLICATIONS, 2019 20 S I M P L E , C O M P L I C A T E D , AND C O M P L E X PROBLE MS MC D A V I D , PR O G R AM E V A L U A T I O N AN D P E R FO R M AN C E M E A S U R EM EN T, 3 E . © SAGE PUBLICATIONS, 2019 Glouberman and Zimmerman (2002, p. 2) 21 MCDAVID, PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT, 3E. © SAGE PUBLICATIONS, 2019 I N T E RV E N T I O N S AS SIMPLE, C O M P L I C AT E D , OR COMPLEX P RO G R A M S • Complex policy or program problems are often tackled by agencies or governments that address parts of the problem in complementary ways • Phased or iterative programs can be practical ways of addressing a complex problem in a multi-organizational or multi-jurisdictional context • Nested logic models can be developed, and lines of evidence gathered for each one • Put the whole program effectiveness picture together from the phased or partial perspective 22 SPECIFYING PROGRAM OBJECTIVES MCDAVID, PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT, 3E. © SAGE PUBLICATIONS, 2019 The four characteristics of ideal objectives: • Specify target population/domain • Specify the direction of intended effects–positive or negative change • Specify the time frame Specify the magnitude of the expected change, where possible These four criteria, if all realized, will greatly facilitate the work of evaluators. The evidence for program outcomes can then be analyzed considering the population, direction, and time frame factors that were specified. • 23 L O G I C M O D E L S T H AT C AT E G O R I Z E A N D S P E C I F Y I N T E N D E D CAUSAL LINK AGES MC D A V I D , PR O G R AM E V A L U A T I O N AN D P E R FO R M AN C E M E A S U R EM EN T, 3 E . © SAGE PUBLICATIONS, 2019 24 L O G I C M O D E L S T H AT C AT E G O R I Z E A N D S P E C I F Y I N T E N D E D CAUSAL LINK AGES MC D A V I D , PR O G R AM E V A L U A T I O N AN D P E R FO R M AN C E M E A S U R EM EN T, 3 E . © SAGE PUBLICATIONS, 2019 25 L O G I C M O D E L S T H AT C AT E G O R I Z E A N D S P E C I F Y IN T E N D E D C AU SA L L IN K AG E S MC D A V I D , PR O G R AM E V A L U A T I O N AN D P E R FO R M AN C E M E A S U R EM EN T, 3 E . © SAGE PUBLICATIONS, 2019 Family Preservation a n d Strengthening Program: Ministry of Children and Family Development Inputs Components I n t e r ve n t i o n •C o n tra ct amount: 112,500 f or D e c 1 , 2002 to August 31, 2003 • Human re s o u r c e s: 2 . 5 fa m ily p r e s e r va t i o n workers; .25 a d m inist ra to rs Assessment and planning Outputs R e s p o n s e t i m e f r o m ref erral t o first f a m i l y m e e t i n g Service plans developed N u m b e r o f f a m ilie s r e c e i v i n g services (monthly count) F a m ily-fo c u s e d home-based assistance and t ra in in g • Technology • page N u m b e r o f direct client h o u r s of service (per family per month) L e n g t h o f p r o g r a m f or e a c h f a m i l y a n d o ve ra ll a v e r a g e Outcomes Reduced incidence o f r e p o r t s o f c h ild abuse P a r e n ting skills improve D e c r e a s e in n u m b e r of foster care re -e n t rie s Reduced days of out-of-home care D e c r e a s e in n u m b e r of youth out-of-home placements Improved family I fu n c t io n in g Children develop in healthy h o m e s C h i l d r e n l e s s likely to be c o n s id e re d h i g h risk N u m b e r and types of h o m e visits p e r f a m ily rs f or o n -ca ll w o rk e rs N u m b e r a n d t y p e s o f t ra in in g and education materials reviewed/discussed Family support networks Improved parenting behaviours D e c r e a s e in y o u t h ju st ice a n d mental health uses N u m b e r of parent/contract w o r k e r visits t o c o m m u n i t y centres N u m b e r of hours at com m unity centres N u m b e r o f h o u r s wit h s u p p o r t groups 26 CONSTRUCTING A LOGIC MODEL FOR PROGRAM EVALUATIONS MC D A V I D , PR O G R AM E V A L U A T I O N AN D P E R FO R M AN C E M E A S U R EM EN T, 3 E . © SAGE PUBLICATIONS, 2019 • Build a working understanding of the program structure • • Do a first draft of the logic model • • based on your understanding Validate the draft logic model • • • use existing information sources, such as websites, brochures, annual reports, and work plans consult with key stakeholders seek consensus but be prepared for different views Revise the logic model • based on a fuller understanding and input 27 WHO CONSTRUCT S LOGIC MODELS? • Professional Evaluators Purpose: Usually a program evaluation or a • Program Managers Purpose: Usually performance measurement performance measurement system Sources of Information: Sources of Information: Program managers Program staff Other programs Knowledge and experience Existing documentation Stakeholders inside and outside the program Other programs Evidence-based program theory Timing: Usually after the program design Timing: Sometimes concurrent with program design but usually included in an overall accountability framework has been completed 28 P O T E N T I A L O P P O RT U N I T I E S A N D C O N S T R A I N T S T O T H E I M P L E M E N TAT I O N A N D S U C C E S S O F P RO G R A M S MC D A V I D , PR O G R AM E V A L U A T I O N AN D P E R FO R M AN C E M E A S U R EM EN T, 3 E . © SAGE PUBLICATIONS, 2019 29 • Logic models are used to plan for program evaluations and/or performance measurement • Start thinking about a program or policy that you would like to evaluate — we will discuss more in the next few weeks. Use this week as an exercise to create a logic module related to your program evaluation class project. Below are some examples and ideas to guide you: • PROGRAM EVALUATI ON • • • • • • Covid-19 Mask Policy Mandates Covid-19 Schools Opening Policies and Programs Covid-19 Screening and Airlines/Travel Legalizing Marijuana — University Policies Governing Use Police Body Cameras Other Policies or Programs that are of interest to you. If you are unsure — send me a few options (I recommend that you pick a program that is of interest to you that will be useful for your class project). Logic models change in complexities based on the program and/or performance measurement being evaluated. 30 BRIEF REVIEW OF LOGIC MODELS • • • A program logic model is a picture of how your program works — the theory and assumptions underlying the program. This model provides a road map of your program, highlighting how it is expected to work, what activities need to come before others, and how desired outcomes are achieved (Kellogg Foundation, p. 35). The logic model is a beneficial evaluation tool that facilitates improvement in program planning, implementation & evaluation. Using evaluation and the logic model results in effective programming and offers greater learning opportunities, better documentation of outcomes, and shared knowledge about what works and why. Next week we review Research Designs for Program Evaluations. LOOKING AHEAD WEEK4 Quiz 1 — Covers everything we have discussed Weeks 1 to 4. This will be a good exercise to apply the materials we have covered so far. Remember to post your assignment by the due date to the Canvas Discussion Board. Discussion Boards close at 11:59 p.m. on Sunday for each assignment. This is your last slide! Thank you for hanging in there. Do not hesitate to email me if you have any questions. I am also available to set up Zoom Meetings by appointments or during my office hours (Wed 4-6 pm). 32 Purchase answer to see full attachment Explanation & Answer: 500 words Tags: program logic model create a logic model program evaluators Student has agreed that all tutoring, explanations, and answers provided by the tutor will be used to help in the learning process and in accordance with Studypool’s honor code & terms of service.