Improving the Health Promoting Schools self-assessment tool

Client: Ministry of Health

Evaluation is about finding out what works well and what doesn’t. An effective evaluation will help you celebrate your success and help you improve your programme. This begins with collecting the right data and providing accurate information – and this means having the right measurement tool.

In collaboration with our partners Cognition Education, Standard of Proof tested whether the Health Promoting Schools (HPS) school self-assessment tool was providing accurate indications of school health and wellbeing so that the school workforce can trust the results and make decisions about the next steps in their delivery approach.

We applied the Rasch measurement model to the evidence collected from the self-assessment tool. This technique provided a robust reflection of the tool, and provided useful modifications to the self-assessment – making sure it provided reliable evidence.

The analysis examined:

  • Does the assessment tool have clear questions and response categories so it can provide reliable measures of all groups?
  • Are the range of questions and response categories both easy enough to accommodate schools with low ability, and hard enough to accommodate schools with high ability?
  • Are the scaled response categories able to provide accurate measures of growth over time?
  • Does the overall assessment tool provides a single measure of school health and wellbeing, which can reliably test any relationship with student achievement?
  • The results were presented to a national hui with the HPS workforce, and the findings are being used by the Working Group to further improve the value of the self-assessment tool for school communities.

Bringing in the Boys: Research to support the expansion of the New Zealand HPV Immunisation Programme

Client: Ministry of Health

Delivering the best communication programme possible means understanding your priority audience and their influencers.

The Ministry of Health asked Standard of Proof to undertake some exploratory research to inform the Human papillomavirus (HPV) immunisation campaign for 2017.

HPV is among the most common sexually transmitted infections. 1 in 20 cancers are HPV related and the proportion of HPV related cancers is on the rise in the Western world.

Recent changes to the HPV immunisation programme allow for an expanded and refreshed communications programme. The Ministry of Health aims to increase the immunisation coverage rate for girls, and ensure uptake of the immunisation by boys.  Knowing how to engage your audience is key to achieve these outcomes.

Standard of Proof engaged Māori, Pacific, Asian and NZ European parents to find out what would work best when communicating sensitive health information about their children. Our team also engaged with school principals, public health nurses and medical officers, as key influencers.

The group discussions identified communications channels, messages, concepts and approaches that engage key target audiences. The participants also generated ideas on communication approaches that would be most relevant and engaging for them.

We were pleased to deliver a cost-effective approach to explore the most relevant communication channels with these groups. Our client has been able to use this information and develop a strategic approach to support uptake of the HPV immunisation by boys, particularly among rangatahi Māori and Pacific young people.

Understanding opportunities and needs for library services in New Zealand.

Client: Public Libraries New Zealand

“Understanding your audience priority needs helps determine what an effective client-focused service delivery model could look like. This means listening to the community and summarising their responses in ways that best represent their views. 

Standard of Proof helped the Public Libraries of New Zealand to better understand communities’ priority needs across New Zealand.

We used a sample survey approach; working in partnership with Buzz Channel to collect data from large numbers of our target community, we measured individuals’ priorities for library services using an effective and efficient data collection approach.

We used a range of techniques to:

  • identify and present focused and meaningful data rather than presenting too much detail (factor analytic techniques)
  • compare clusters of citizens according to their priority service needs (parametric statistical approaches)
  • estimate the results to reflect the wider communities efficiently (weighting and statistical estimation)

The analysis provided robust evidence about citizens’ need-to-have and nice-to-have services, enabling the local libraries to adapt their amenities to best meet their local communities’ needs.

We were pleased to deliver a cost-effective approach to gathering evidence. Our client has been able to use the evidence to inform future library service delivery in New Zealand.

Research to inform the measles immunisation approach

Client: Ministry of Health

Segmentation research can help identify various risks and potential mitigations to achieve your goals.

The Ministry of Health asked Standard of Proof to undertake research to understand potential reactions to a specific approach to administering health solutions to young people. Through a series of interviews, Standard of Proof identified segments of potentially distinct reactions to the new approach, measuring the risk involved for the client while setting out how these different reactions could be managed to support the successful implementation of an health-focused implementation approach.

The results provided the client with a clear pathway and confidence to achieve their goals.

Measures for He Poutama Rangatahi initiative

Client: Ministry of Business, Innovation and Employment

Psychometric analysis ensures young people can get the support they need

“Standard of Proof helped a government agency develop a questionnaire which would help organisations understand the strengths, needs and progress relevant to each young people sustaining employment over time. The factors that could help these organisations identify needs and progress towards their long term goal – sustained employment – included aspects of individual’s foundational needs, employability and opportunity.

The intended use of the questionnaire highlights the importance of providing accurate information. If the data misrepresent a young person’s ability to sustain work, inappropriate decisions might be made that could have negative effects on the person. For example, the young person may be placed into work too early, before they are able to sustain a job, which may have a lasting negative impact on their future employment opportunities. Inaccurate information may be worse than having no information at all.

Standard of Proof, with the support of the University of Western Australia, applied the Rasch unidimensional measurement model to determine if the organisations and young people could use and interpret the data as intended, and if government organisations could trust the results. Of even greater importance at this early stage, the technique produced diagnostic information necessary to improve the questionnaire, making it easier for young people to respond to and providing information that everyone can rely on in order to inform the ‘right’ next steps.