Lesson 5.1. Citizen science in policy evaluation
With
the right research framework and a demonstrable link to the policy
context, citizen science projects can start making inroads into
traditionally sceptical policy circles, slowly but steadily facilitating
change in views towards unorthodox data sources i.e. sensor networks
managed by actors other than government agencies or professional
scientists. COMPAIR’s static campaign in Berlin demonstrates a clear
pathway - the deployment of a mixed-sensor network of low-cost monitors
strategically placed to evaluate change before and after an intervention
- through which citizen science can support policy evaluation.
Case studies
Berlin
After joining COMPAIR, residents of Berlin's Kreuzberg district were able to participate in Project Graefekiez as citizen scientists. Redevelopment of car parks in the district began in July 2023. To understand how this work affected air quality and traffic in the area, a network of citizen science sensors (bcMeter, sensor.community kit, Telraam) was deployed a month in advance (June 2023), with a view to performing a before/after analysis at the end of the campaign which lasted until October 2023.
Monitored were not just the streets where reconstruction was happening (red lines in the figure below) but also the wider neighbourhood (purple markers). With this approach, the research team was able to compare how levels of black carbon varied between the affected streets (where parking lots were removed) and a control group (areas further afield without any changes) over the same period.
Measurement locations in the Graefekiez campaign
The results show that black carbon concentration in Graefestraße (target area) was lower than in Schönleinstraße (control area). While the difference can be attributed to many factors (e.g. weather, traffic patterns), based on the statistical analysis performed, and the fact that two areas are within close proximity of each other and were monitored simultaneously, the influence of car-park redevelopment is likely to have played a major role.
Difference in BC pollution between the target area (red plots) and the control area (blue plots)
In addition, since black carbon is a component of fine particulate matter (PM2.5), citizen science measurements were tested for accuracy by comparing them with readings from two BLUME reference stations located in Nansenstraße and Müggelsee. The average PM2.5 pollution recorded by citizen scientists was below 5µg/m3 over the entire period, slightly lower than 7-8 µg/m3 recorded by BLUME stations. Although the difference exists, it is not statistically significant due to low dispersion of maximum and minimum values.
PM2.5 data: citizen science sensors vs reference-grade BLUME stations
As regards traffic, car-park redevelopment resulted in considerably more pedestrians in the area. The change was accompanied by a drastic reduction of cars and two-wheelers (bicycles, bikes), and a small decrease in heavy vehicles. The before/after comparison was performed on Telraam data and presented through a Policy Monitoring Dashboard.
Traffic changes before and after the car-park redevelopment in Graefekiez
Findings of the Berlin campaign were shared with representatives of Berlin's Senate Department for Mobility, Transport, Climate Protection and the Environment. Speaking at the final workshop, policy makers said measurement results confirmed their assumptions about the expected impact of redevelopment measures. They added that the experiment showed the potential of citizen science to support policy, especially at the M&E phase (monitoring and evaluation). However, to be fully integrated in the decision making process, citizen science must be consistently implemented over a longer period of time, guided by standards and protocols approved by the administration.
Although not achieved in the Berlin campaign, a fully-fledged integration of citizen science (data) into the government machinery is possible. For that to happen, a complete makeover of not just the campaign but the way in which an entire project is managed may be necessary. Citizen science results that are palatable to the government, especially in politically sensitive areas like air quality, may require a setup in which a public authority has a prominent - perhaps even leading - voice at the table. Such a governance arrangement would certainly satisfy policy makers, but it's far from certain that something like this would go down well with other actors, notably citizens with an entrenched distrust of all things government. In a broader community, some might find imposed protocols too strict and, as a result, would be less inclined to participate under rigid conditions. Others might wonder whether an initiative with the government in the driver's seat is at all worthy to be called citizen science. All these concerns and considerations are yet another reminder that a perfect model for a citizen science-policy interface is yet to be discovered. Depending on priorities, projects may be fully or partially government-led, if policy impact is the ultimate goal, or citizen-led, if flexibility and community empowerment are more important.