Measuring effectiveness

Inputs, outputs and outcomes

When preparing to measure the effectiveness of a scheme it is important to be clear about the inputs, outputs and outcomes of the programme. These should be agreed by steering group members and be clearly documented. The Local Programme Lead will be responsible for setting up a system for measuring these outputs, and demonstrating how these contribute to a change in the outcomes.

The Logic modelling below is helpful in distinguishing between inputs outcomes and outputs:

The logic model describes the sequence of events thought to bring about benefits or change over time. It portrays the chain of reasoning that links investments to results.

An input is a resource that is invested and imported into the youth justice liaison and diversion (YJLD) service to help the work of that serviceand allow desired outputs to be achieved. These resources include time, people (staff, volunteers), funds, materials, equipment, partnerships, research base, and technology.

Outputs are activities conducted or products created that reach targeted participants, populations, specific audiences, decision makers or groups of individuals. Outputs are 'what we do' or 'what we offer'. These outputs are intended to lead to specific outcomes.

The logic model includes a specific focus on participation or reach. This is who we target. Good programme design depends upon a clear articulation and understanding of the target audience. Activities are designed based on participant characteristics and expected outcomes then flow from those. A focus on participation also helps us be accountable for the effective and efficient use of resources. We often must track and report participation data, for example, who are we working with and how many? The assumption is that the activity needs to be delivered as intended before the expected outcomes can occur.

Impact in this model refers to the ultimate, longer-term consequence or effects of the programme, for example a reduction in entrants to the Youth Justice System. In the model, impact is synonymous with the long-term outcome or the goal.

What are outcomes?

An outcome is what happens to a service user as a consequence of what we do, for example being diverted from the Youth Justice System or being diverted to local health services. It is possible that an outcome may have a positive impact, have no impact at all or even a negative impact.

As a YJLD service develops, it is good to set measurable outcomes to be achieved, however it is never too late to change practice as a YJLD site develops and partners may be interested in key strategic outcomes that the service can contribute to. Outcomes agreed for the YJLD service must have clear relevance to the terms of any service level agreements with local commissioners.

An outcome focused approach can help a YJLD scheme:

  • Explain the purpose of the service to commissioners, local partners and service users and their families
  • Recruit and retain staff to the service
  • Gain public recognition for the service.

It is important to measure what progress is being made and to understand how this can be done. It is also important to consider how these outcomes fit with other priorities including those beyond YJLD schemes. The list below is not definitive but is useful to illustrate the importance of outcomes:

  • They help establish the changes that need to be made to address the needs that have been identified.
  • They help ensure everyone is clear about, and has the same understanding of, the changes needed. They are in effect the bridge from needs identified and the work to plan and do.
  • They provide a start point for working out the steps needed to result in the change wanted and for recognising that the changes have been made.
  • They can help to measure what those changes are.
  • They provide information to help review the work carried out with children, young people and families, including changing outcomes or changing the way they are trying to be achieved.
  • They provide information to help build up a picture of how well services are doing for all the children and young people they are working with (if information is aggregated from individual cases). This could demonstrate how services are contributing to achievement of the local partnerships.
  • They provide information that can be useful for a wide range of local activities:
    • explaining what services can carryout and achieve because they have the proof it works
    • explaining what users think about the service if feedback is obtained
    • showing how YJLD fits with other services so is part of the local activity to help address some of the needs of children and young people while other services tackle other needs as rarely can a single service address multiple or all the needs of service users
  • They provide information to help build the business case for the national development of liaison and diversion services. The information obtained is just as important locally for the implementation and sustainability of local schemes.

Outcomes may be long term, medium term or short term. Although long term outcomes should be the aim, for example children and young people developing well, in order to enable the service to demonstrate the contribution it is making, it is best to break down the outcomes that are planned into smaller, more practical and achievable steps. These are known as SMART outcomes (specific, measurable, achievable, realistic, time-limited):

Specific Improve behaviour Improve health and well-being
Measurable Can be given a numerical value Expressed by narrative only
Achievable Fewer children missing immunisations Improving health and well-being of all children and young people
Realistic A 5% increase in immunisation uptake A 75% increase in immunisation uptake
Time-limited Outcome to be achieved in next 2 years No timeline given

Validates against XHTML 1.0  Cascading Style Sheet compliance  Single A Accessibility compliance