Steps to measuring effectiveness
- Step 1: Identifying the key desired outcomes for the service
- Step 2: Check that some outcomes can help the scheme contribute to the broader strategic outcomes in an area
- Step 3: Plan how to measure progress being made (or not) towards these outcomes
- Step 4: Work out how you will get feedback will be obtained, from the outset from young people and parents about their satisfaction with the service and the way it is delivered
- Step 5: Plan how, when and to whom outcomes of the service will be reported
- Step 6: In conjunction with commissioners, draw up a plan for feeding into the commissioning cycle
Step 1: Identifying the key desired outcomes for the service
As part of agreeing the scope of Youth Justice Liaison and Diversion (YJLD) work it will be important to establish the key desired outcomes. The Local Programme Lead will need to agree these with the YJLD steering group and other key partners.
Focus on outcomes that can be directly attributable to what your service does with outputs from your service for example numbers diverted via YJLD:
- From the youth justice system
- From custody.
Step 2: Check that some outcomes can help the scheme contribute to the broader strategic outcomes in an area
In a locality, as well as outcomes for the YJLD service itself, there may be broader strategic outcomes that have been highlighted and that all partners are contributing to eg Improving education attainment. If the YJLD has an impact on reducing school exclusions, this activity will contribute to the broader strategic outcome.
YJLD pilot site case example:
Dealing with family conflict and self harm
Taneesha is a 15 year old girl. She was arrested for common assault and was 'triaged' under the YCAP triage scheme. During screening with the triage worker, Taneesha mentioned that she had once self-harmed in the past and that she was feeling low in mood. This promoted a referral to YJLD.
The YJLD worker conducted an assessment, which revealed that Taneesha was having difficulties with managing her anger. Her relationship with her mother and her younger brother at home was poor. The YJLD worker offered brief therapeutic sessions with Taneesha to explore her mood and feelings. She learnt non-violent strategies to manage her anger and worked on how to improve the relationship with her mother. Unfortunately, Taneesha's mother was not able to attend the scheduled appointments due to the nature of her work but with Taneesha's consent, the YJLD worker liaised with Taneesha's mother over the phone, exploring some of the issues discussed at sessions with Taneesha.
Taneesha has kept herself out of trouble following the YJLD intervention. She completed her work experience last summer and is now back in full time education. There has been an improvement in Taneesha's relationship with her mother and the YJLD worker continues to meet with her fortnightly to monitor the situation.
(from Centre for Mental Health website)
All names have been changed.
The service needs to be clear about the fit between what activity it is doing and what other services are doing in the locality to contribute to an overarching priority outcome in the locality.
If the service is to be sustainable it is important to demonstrate how the service can also contribute to broader strategic outcomes for commissioners, partners and children, young people and families.
YJLD pilot site case example:
Improving immunisation rates for cervical cancer
A young woman is identified during a health screen by the YJLD worker as being at risk of teenage pregnancy, and one of the actions is to refer her to a sexual health clinic. This action enables her to gain both sexual health advice and specifically receive an immunisation for cervical cancer. This vaccine is usually given at school but because she has not been attending school she has missed the immunisation programme.
In this case, referring this young women to a school nursing service contributes to the YJLD priority of meeting previously unmet health needs, but also contributes to the local priority to improve immunisation rates for cervical cancer.
Referring to Joint Strategic Needs Assessments enables YJLD schemes to be aware of broader strategic outcomes. A Joint Strategic Needs Assessment identifies areas for priority action, to help commissioners and key partners to specify outcomes that encourage local innovation and help providers shape services to address needs. Understanding what are the areas for priority action and how the YJLD service can contribute to these broader outcomes is important for YJLD sites to understand and to also understand the key priorities for action dependent upon need.
Find out more about:
Step 3: Plan how to measure progress being made (or not) towards these outcomes.
In order for commissioners to assess how well a service is performing they will require good quality performance information and analysis to help them to judge both the efficiency, effectiveness and value for money.
By using locally determined indicators, usually quantitative, the service is able to demonstrate the efficiency and effectiveness of the service. Indicators can also be qualitative and it is important to use both quantitative and qualitative indicators to demonstrate progress towards key service outcomes.
Whatever the indicators chosen for the service they must be quantifiable. The indicator needs to be clearly defined and measurable and to indicate what good performance will look like. This might be a reduction in first time entrants to the Youth Justice System or an increase in the uptake of immunisations.
Step 4: Work out how feedback will be obtained, from the outset from young people and parents about their satisfaction with the service and the way it is delivered.
The feedback from young people and families is key to demonstrate satisfaction with the service and how they are delivered. Feedback ensures that qualitative information is provided to the commissioner and allows the service delivery model to be adjusted to take into account the voice of young people and families. However care is needed here as user feedback should not be conflated with measuring service effectiveness. Children, young people and families may like a service that is nevertheless failing to achieve its intended outcomes.
In the initial stages of implementation a key focus of service delivery is an assertive outreach model. This information is also key to provide to commissioners of the service and to be able to demonstrate a robust element of qualitative data to support the service model.
It is important to work out how feedback is obtained from the outset from young people and parents about satisfaction with the service and the way it is delivered.
Step 5: Plan how, when and to whom outcomes of the service are reported.
Links to key strategic boards and forums and partnership arrangements will vary in each locality, however as robust governance arrangements are put in place, key reporting arrangements can be agreed. The reporting arrangements need to be transparent and agreed over an annual programme. This ensures that arrangements are robust which is particularly critical during periods of transition when reporting arrangements need to be clear and agreed with strategic boards and forums.
It is helpful to provide service outcomes to local champions of the service and to agree any potential opportunities to share these more widely. There may also be an opportunity to use local case studies to provide a ‘story’ behind the data and to ensure that the qualitative aspects of the service are highlighted.
Reporting the outcomes of the service is also important to gain public recognition of the service and to raise the profile of the service in a locality including how the service can be accessed. Reporting arrangements can also be part of a wider communication plan and links to the local media can be a useful partner as the service is developing as well as to support sustainability.
Agreement about what is to be measured, how this will be measured and when success will be measured must be agreed with the local steering group and documented on a development plan.
Step 6: In conjunction with commissioners, draw up a plan for feeding into the commissioning cycle.
As relationships with commissioners of the YJLD service develop it is important to contribute to a plan for feeding into the commissioning cycle. Local commissioners can alert you to the timings and information required to support the commissioning of the service and future potential investment in a locality.
YJLD schemes may also provide information that would be useful to partners and commissioners wider than just YJLD. They may help identify gaps in other services or trends that may be developing that they can highlight to partners and commissioners. For example, if a number of young people being seen by a YJLD scheme have been in contact with a service but have not engaged, commissioners may wish to understand what can be done to change this. It may be that the young person does not attend the clinic/setting as they find it unwelcoming or overwhelming.
Commissioners may wish to consider You’re Welcome standards. These are no longer set by the Department of Health but the review process is led locally and commissioners and local areas will still find the previous Department of Health quality criteria for young people friendly health services relevant.