Skip to content
Science, Maths & Technology

Diary of a data sleuth: Data-scraping the sick bucket

Updated Thursday 20th December 2012

Are more of us getting sick in winter or is it just hype? Our resident data sleuth set out to unpick the stories of winter bugs in the run up to Christmas.

It's that time of year again when the dreaded "norovirus" hits the news, with tales of wards closing and hospital visitors being encouraged to stay away from their loved ones if at all possible. Norovirus, aka the "winter sickness bug", is back, and with a vengeance, it seems (you can read up about it on the NHS Choices website: norovirus).

A quick look at Google Trends, a tool for reviewing the relative volume of search terms entered on the Google web search engine, shows how searches for "norovirus" grows around this time each year...

Copyrighted image Icon Copyright:

(You can also use Google Trends to look up search activity around other terms. Try presents, for example, or trifle, or try them both together).

Of course, this may in part be a reflection of the extent to which people are looking up "norovirus" to see what it is, having heard it referenced in the news. For example, if we look at the coverage of "norovirus"-related stories in the Guardian over the last couple of years, we see how stories mentioning the disease feature prominently at the end of the year, and particularly in the run up to Christmas...

Data snapshots relating to the Norovirus

The story this year appears to be that Winter bug cases [are] '83% up on 2011' based on estimated occurrences since the summer compared to the same period last year, as reported by the Health Protection Agency. In addition, "[t]he figures also show there were 61 outbreaks of norovirus in hospitals in the fortnight up to December 16 - almost double the number in the same period last year when there were 35."

So can we find any data to explore these claims in a little more detail? Well it so happens we can... sort of. Each week, the NHS publishes a winter pressures daily situation report that records, on a weekly basis, a variety of Health Trust-related status reports collected each weekday. This data includes bed closures resulting from "D & V / Norovirus", which I take to mean diarrhoea and vomiting. A couple of weeks ago, I started looking at ways of aggregating this data in a convenient online database, rather than having to download and then try to cope with the officially released spreadsheet. The technique I used to grab the data is often referred to as data-scraping and the tool I use is called Scraperwiki. (You can read about some of the trials and tribulations associated with getting the data out of the released spreadsheets, and tidied up so it can be put into a database on my blog.)

With the news about norovirus cases being up on last year, I had a quick look to see if there was winter sitrep data available from last year, and indeed there is: NHS winter sitrep data 2011/12. Rather conveniently, the spreadsheet format used this year to release the data is the same that was used last year, so I pointed a copy of my datascraper at last year's data, and got that into a database, too.

A little bit of tinkering, and I managed to plot a handful of charts showing the number of hospital bed closures due to norovirus outbreaks over the winter period 2011/2012, and since early November of this year (2012). You can see the live charts here, and a snapshot is shown below. The chart on the left shows figures for 2011, the one on the right for 2012. If you click through to the actual charts, you will find they are interactive. The slider at the bottom of the chart allows you to zoom in to a particular date range.

In each chart, the upper blue line shows reported figures at 8am or 9am on the day of reporting for Beds closed norovirus ("The number of beds closed due to D&V or norovirus-like symptoms"); and the lower red line shows Beds closed unocc ("Of the number of beds closed due to D&V or norovirus-like symptoms, the number of beds that are unoccupied"). These terms are described in the data release notes.
Data snapshots relating to the Norovirus

As you can see, norovirus does seem to be having more of an effect than it did last year.

For more examples of how to view the winter sitrep data, see NHS Winter Situation Reports: Shiny Viewer v2.

For more information on the measures hospitals are likely to put in place around a norovrius outbreak, see the NHS Guidelines for the management of norovirus outbreaks in acute and community health and social care settings.

For further information, take a look at our frequently asked questions which may give you the support you need.

Have a question?

Other content you may like

OpenLearn Live: Christmas Special 2016 Copyright free image Icon Copyright free: Pexels article icon

TV, Radio & Events 

OpenLearn Live: Christmas Special 2016

We're sending you this Christmas card - some of the best of 2016, and some Christmassy things

Thirty-five winters of discontent Creative commons image Icon Highways Agency under CC-BY licence under Creative-Commons license article icon

History & The Arts 

Thirty-five winters of discontent

Since the original 1978/79 Winter Of Discontent, any setback at any time between late summer and Easter has been garlanded with the same grim title.

Nature of Britain Calendar: November Copyrighted image Icon Copyright: Other - from calendar, cleared for use online article icon

TV, Radio & Events 

Nature of Britain Calendar: November

Follow the Nature of Britain's seasonal hints and tips - this month we look at November, when the countryside will be draped in gossamer web from linyphiid spiders, and enlivened by the sound of returning swans

Numbers, units and arithmetic free course icon Level 1 icon

Science, Maths & Technology 

Numbers, units and arithmetic

Do fractions and decimals make you apprehensive about maths? Do you lack confidence in dealing with numbers? If so, then this free course, Numbers, courses and arithmetic, is for you. The course will explain the basics of working with positive and negative numbers and how to multiply and divide with fractions and decimals.

Free course
5 hrs
Numbers: An introduction to subtraction Copyrighted image Icon Copyright: Used with permission free course icon Level 1 icon

Science, Maths & Technology 

Numbers: An introduction to subtraction

Do you want to improve your ability to subtract one number from another, especially if decimals are involved, without having to rely on a calculator? Numbers: An introduction to subtraction, is a free course that will help you get to grips with subtraction and give you some practice in doing it.

Free course
1 hr
We can read your mind Copyrighted image Icon Copyright: The Open University video icon

Science, Maths & Technology 

We can read your mind

In this video it seems as though your mind had been read – but how is that possible? Discover the magic of maths

5 mins
Analysing skid marks Copyrighted image Icon Copyright: Used with permission free course icon Level 1 icon

Science, Maths & Technology 

Analysing skid marks

This free course, Analysing skid marks, is the second in the series of five courses on mathematical modelling. In it you are asked to relate the stages of the mathematical modelling process to a previously formulated mathematical model. This example, that of skid mark produced by vehicle tyres, is typical of accounts of modelling that you may see in books, or produced in the workplace. The aim of this course is to help you to draw out and to clarify mathematical modelling ideas by considering the example. It assumes that you have studied the course Modelling pollution in the Great Lakes.

Free course
4 hrs
Modelling static problems Copyrighted image Icon Copyright: Used with permission free course icon Level 2 icon

Science, Maths & Technology 

Modelling static problems

This free course, Modelling static problems, lays the foundation of the subject of mechanics. Mechanics is concerned with how and why objects stay put, and how and why they move. In particular, the course considers why objects stay put. And it assumes that you have a good working knowledge of vectors.

Free course
16 hrs
Complex numbers Copyrighted image Icon Copyright: Used with permission free course icon Level 3 icon

Science, Maths & Technology 

Complex numbers

You may have met complex numbers before, but not had experience in manipulating them. This free course gives an accessible introduction to complex numbers, which are very important in science and technology, as well as mathematics. The course includes definitions, concepts and techniques which will be very helpful and interesting to a wide variety of people with a reasonable background in algebra and trigonometry.

Free course
20 hrs