4.3 Pros and cons of scoring systems & importance of context in welfare assessments
This section provides two case studies of approaches used in welfare assessment schemes (WASs), briefly comparing them and how they differ by their underlying purposes and considering their pros and cons. This allows further examination of what OWIs are included in the WAS, and why/how; The structure of the WAS, such as how OWIs are being used together / categorised; and whether a scoring system is included, and if so, why/how.
Exemplar WAS #1 - Welfare certification schemes (standards):
Main purpose: Most certification schemes (i.e., welfare standards) are designed largely as a risk-mitigation approach, and for providing assurances to other stakeholders that the farmed animals' welfare are being protected. Examples of these standards are the 'Code of Good Practice' (CoGP), the 'RSPCA welfare standards for farmed Atlantic salmon', and the 'Aquaculture Stewardship Council' (ASC) Farm standards.
Structure: In contrast to typical diagnostic, on-farm monitoring routines, these welfare standards focus on:
- Identifying hazards, their consequences, and likelihood of them occurring.
- Identifying critical control points for these hazards in the production process.
- Orient standards and clauses (i.e., requirements that are later assessed for compliance) around these critical control points to avoid welfare risks from occurring the first place.
Set criteria for these standards prioritise managing these control points in the form of clauses, focusing on specific farming conditions and best practices as outlined in Module 3. As a result, indirect, risk-based OWIs (i.e., input OWIs) tend to form the majority of these assessments. These clauses are then organised into different sections based on their relevance to specific welfare factors (e.g., environmental conditions, transportation, interventions, staff management etc.).
Model of assessment: Farms certified under their respective welfare standards are typically assessed by third party auditors on at least an annual basis. During these audits, each clause that is relevant to the site is assessed based on their level of compliance to them. Instead of scoring OWIs on an interval scale, outcomes are typically reached based on the number and severity of violations, or 'non-compliances'. Non-compliances can range from less severe (i.e., unlikely to pose any significant risk or suffering to the salmon, and usually resulting in no more than a formal warning to the farm site staff), to more severe or repeated non-compliances (i.e., auditors have identified the salmon have likely suffered directly as a result of significant or repetitive neglect / malpractice). If the violations are severe enough, or have occurred multiple times over demonstrating the farm's inability to adjust and protect their farmed animals, suspension or withdrawal of the certification is likely the final outcome. This is typically down to the discretion of the certification scheme that awards the label to the farms under audit.
Pros:
Broad scope - Owing to the necessity of assurances requiring a comprehensive coverage of important welfare factors, the scope of such welfare assessment schemes is typically very broad, covering the majority if not all of the welfare factors relevant to the farmed aquatic animal(s) involved.
Preventative approach - Prioritising the broad use of indirect OWIs helps identify and mitigate the range of risks posed to the animals before they can actually negatively impact welfare. Both in terms of practicality and for the sake of the animals' QoL, prevention is superior to treatment when it comes to welfare issues.
Minimal disturbance to animals - Majority of such assessments will be carried out without ever directly involving the animals.
Practical - Such assessments are typically made of mostly risk-based, input indicators, making them simple and easy to carry out.
Cons:
Lack of direct welfare assessment - Minimal to zero real-time, outcome based data on the welfare status of the animals means that assessors are still unaware of what actually transpired for the animals. Covering all risk factors does not guarantee that animal welfare is still protected, and severe issues may continue to be missed with such an approach. Focusing on systemic conditions can also easily overlook individual suffering that may be deteriorating without any notice.
Risks of "box-ticking" compliance - Farm sites may adapt and focus simply on meeting paperwork and procedural requirements rather than ensuring real welfare improvements if the WAS involved does not include the appropriate coverage with the OWIs involved. Simple "yes" or "no" records on whether certain risk mitigation practices are in place does not mean that these practices are constantly being maintained.
Risks of ambiguity for clause-based OWIs - Even where clauses pose specific requirements, it is often very easy to refer to "adequate" or "appropriate" conditions, without specifying exactly what these are; this often is a result of there being too many variables involved in determining exactly what each requirement "should" be (as outlined numerous times in the previous modules). The reality is many OWIs are required to be context dependent, and without that information such assessments are often forced to be ambiguous.
Exemplar WAS #2 - Salmon Welfare Index Model (SWIM 1.0):
Main purpose: Designed to enable farmers to conduct on-farm, standardised assessments of salmon welfare (specifically in sea cages) using a set list of OWIs. Its main function was a diagnostic tool, identifying what OWIs are demonstrating reduced welfare for the farmed salmon.
Structure: 17 OWIs selected off the following criteria:
1) Focusing on 'quality of life', as perceived by the animals themselves; certain OWIs would need to cover welfare needs from the animal's own point of view (i.e., emotional affective states).
2) OWIs were to be specifically used by farmers on sea cages; high degrees of practicality/feasibility required.
3) OWIs would need to be linked to either a) physical welfare needs (respiration, osmotic balance, nutrition, thermoregulation etc.), or b) behavioural welfare needs (control over their own behaviour, safety & protection from others, social contact, rest, body care, sexual behaviour etc.).
Model of assessment: Assumes salmon experience welfare states on a continuum (poor to excellent). Each OWI divided into different levels of ranks, with each rank assigned an 'indicator score'. For example, the environmental OWI of temperature: #1 = 10-15 degrees, #2 = 7-10 degrees, #3 = 16-17 degrees. All OWIs are eventually scored and their scores are then combined to calculate a single score (i.e., 'overall welfare index'). It is this index score that represents the "overall welfare status" achieved within the sea cage under assessment.
Additional notes: Weighted scores calculated and used for each OWI to determine how much each indicator impacts the final index score (i.e., it is not evenly spread across the 17 OWIs). Weightings are based on the supposed intensity, duration, and incidence of the welfare impact linked to each OWI and its different levels; this value judgement was achieved through a systematic literature review of each OWI and the corresponding welfare factors associated with them.
Pros:
- Provides actionable insights and facilitates comparisons: Standardised, single scores facilitates comparisons to be made across different farm sites of the same species, provided they are at the same life stage and under similar types of enclosure. This allows for monitoring changes to overall welfare states over time on a broader scale, making it easier for stakeholders and / or consumers to potentially make judgements / comparisons of their own. Summarised index scores are also significantly easier to convey to stakeholders (farmers, policymakers, or the public).
Cons:
- Loss of dimensionality allows for problems to be masked: Relying on the report of a single welfare score more than likely results in the "story" of the animal's welfare to be poorly represented. Situations could easily arise where the overall index indicates an 'acceptable' welfare status has been achieved on site, but specific OWIs (that have been scored as poorly) are masked by other OWIs that have achieved far better scores.
- Inherent subjectivity in conclusions made: Evaluating the scores of these OWIs, particularly in terms of what weight they have to the final overall index score, inevitably depends on the subjective lines that are drawn between what is "acceptable" or "unacceptable", as well as what OWIs matter more over others. Such indicators are also typically more difficult to standardise and ensure consistent assessments for.
Concluding remarks on WASs -
The difference in the purpose of WASs, as outlined above, will likely result in distinct differences to the selection, structure, and evaluation process of the OWIs involved, even if the same production system and species is involved. These stark contrasts highlight the importance in clearly outlining what purpose the WAS has before working to develop, evaluate or improve upon specific welfare assessments.
A successful WAS must correctly balance generating actionable insights (required from the purpose of carrying out the WAS in the first place), while minimising subjective value-judgements in weighting different aspects of welfare against each other. While assessing key welfare components separately as an alternative may help enhance the precision and objectivity of a WAS, care must be taken to not lose the broader, more holistic picture of welfare captured.
A balanced approach is often most appropriate, where different key aspects (e.g., behavioural vs. physical needs) are assessed separately but where some integrated analysis is still involved so that interactions between welfare factors are not overlooked.
Setting thresholds for "acceptable" welfare -
When carrying out any WAS, there is typically a particular condition, context, or assessment of the animal involved that must meet the welfare needs of the animal in question. Agreeing on where this "acceptable" level lies for each WAS is no simple task, and requires consultation with multiple stakeholders, industry professionals, and welfare scientists. As previously mentioned, how that threshold is exactly defined will depend on the type of WAS and how it is evaluated (i.e., compliance levels vs. an index score). In general, however, the standard for minimum acceptability should be "mildly positive", in such ways that the animals are not only not suffering at all on a day-to-day basis, but experiencing at least some form of positive welfare states.
