Our program evaluation services
ACIL Allen has advanced expertise in the following areas to help you:
- Strategic reviews and meta-evaluations – assess suites of related programs for both their individual and combined impact.
- Suitability assessments – determine at the outset whether particular programs are suitable to be evaluated, and if so, which form of evaluation is most suitable
- Evaluation design and co-design - establish evaluation frameworks to guide evaluation activities, including through elaboration of program theory and development of program logic models to determine the pathway from objectives to outcomes
- Evidence-based research – undertake literature reviews, research syntheses, benchmarking studies or needs analyses to identify service gaps and leading approaches elsewhere
- Field research – undertake stakeholder consultations and surveys to assess program implementation and outcomes and gauge the before and after changes associated with interventions
- Cost-effectiveness analysis – assess the cost-effectiveness or return on investment of interventions or options.
- Capacity building – assist client program implementation and evaluation teams to develop internal capabilities to undertake evaluation activities as appropriate to any need for independence.
We use leading qualitative and quantitative data analysis technology and techniques. Staff members at ACIL Allen have expertise in a wide range of techniques, including:
- most significant change (MSC), to gauge the before and after changes associated with interventions
- appreciative inquiry, to take a strengths-based approach that focuses on the characteristics of interventions and essential drivers of their success
- outcomes mapping, to gauge behavioural changes achieved.
All our evaluations are conducted with acknowledgment for the Australasian Evaluation Society’s Guidelines for the Ethical Conduct of Evaluations and in line with the requirements Of the National Statement on Ethical Conduct in Human Research (2007) and the AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research.
Policy and program evaluation involve the design, collection and analysis of evidence regarding how change may have an impact. Evaluation requires advanced skills in both qualitative and quantitative analysis to establish appropriate samples and eliminate evaluator bias. This ensures that the resulting analysis is thorough and sound.
Why use program evaluation?
While evaluation can be useful for assessing outcomes achieved (impact evaluation), it is equally important to improve program rollout during the implementation phase (process evaluation).
As a formative tool, evaluation can be used to support program improvement. Evaluation seeks to assess the impacts of programs, as programs near their conclusion. Evaluation can also be used to monitor changes over time (longitudinal evaluation).
Our approach at ACIL Allen
All our evaluations are guided by helping our clients understand whether their policies and programs:
- achieve their stated objectives
- are appropriate for addressing the identified issue or problem
- have been implemented efficiently
- deliver value for money
- can be improved.
There is a growing body of highly specialised language and techniques being applied around evaluation (such as realist evaluations, lapsing program evaluations, theory of change, theory of action, program logic models, investment logic models, to name but a few).
Our starting point is to work with our clients to co-design an evaluation framework and approach that best answers the questions posed above in the context of their needs.
We are also conscious that evaluations come at a cost, not just financially but in staff and stakeholder time, effort that can sometimes divert resources and attention away from the program or initiative being evaluated. We aim to deliver evaluations that are organisationally sustainable for the long term, not just for the period of the evaluation itself. This can include:
- building program evaluation into the program design and embedding data collection into project activities to make them easier and to the extent possible self-evaluating
- working with and augmenting existing program data to leave our clients with an approach that enables ongoing tracking and performance monitoring.
Where we work
We have undertaken evaluations for many departments at national and state levels and for private sector or not for profit sector organisations seeking to understand the impacts of interventions across many key social and economic sectors, such as:
- Education - including early childhood, school education, higher education/skills, adult education, health and wellbeing.
- Health and Community Service - including mental health, disability, aged care, health, and medical research.
- First Nations peoples and communities – including social and economic development, education, Indigenous business, and employment pathways.
- Justice - including family violence prevention, crime prevention, counter-terrorism, alcohol, and other drugs.
- Energy - including clean energy, circular economy, climate change mitigation and adaptation, supporting vulnerable consumers
- Primary industries - including agriculture, natural resources.
We also provide evaluation services across sectors in areas such as:
- Sector reform – structural transition, examples being the closure of car manufacturing and mining industries.
- Organisational performance – particularly entities or units established to introduce programs and policies, as well as providing evaluation capacity-building activities.