Technology Evaluation
Technology Evaluation

This free course is available to start right now. Review the full course description and key learning outcomes and create an account and enrol if you want a free statement of participation.

Free course

Technology Evaluation

3.1 What do we mean by programme evaluation?

Use of the term ‘programme evaluation’ has nowadays become confusing. Historically, and indeed in much of the literature on evaluation, programme evaluation refers specifically to the type of evaluation that focuses on government policies and interventions. This formal type of programme evaluation can trace its history back to seventeenth- and eighteenth-century social reformers and their search for solutions to social problems (Weiss, 1998). A widely accepted view of the period was that there were social ‘laws’ comparable to those that were increasingly being shown to exist in the physical and natural sciences. This belief provided impetus to the evaluation of social interventions, such as in education and health services, because it implied that if the properties of a ‘law’ could be identified, and their impact measured and compared to the impact of other ‘laws’, then those ‘laws’ that had a beneficial impact on social problems, such as crime, poverty and ill health, could be used to combat those problems.

I think it safe to conclude that the role of technology in the social intervention programmes of the nineteenth and early to mid-twentieth centuries was always seen as relatively minor, even if this was not so in some cases (for example, technology has long been important in medicine). However, as the twentieth century progressed, governments and public sector organisations in general became profoundly implicated in the development and use of technology in an increasingly wide range of contexts. Transport, energy and telecommunications are just three of the fields in which governments often took a major role, not only indirectly through policy but also as developers, users and owners of industries and services, and there are many more examples where technologies have become more central to, and embedded in, social policies (e.g. CCTV and electronic tagging as core features of policies to combat crime).

It is important to note, however, that the degree of direct intervention by government in these and other fields differed between the United States and Europe, and countries where the US or European countries held influence. Typically, European governments were far more prone to take direct ownership of activities that were regarded as strategically significant to the state and therefore that ownership was in the national interest. Rail, airlines, energy generation and infrastructure, telecommunications, and significant amounts of scientific and technological research and development are all examples. This situation began to change from the 1980s with the increasing global acceptance of free-market ideas. Countries that embraced these ideas more fully, such as the UK and New Zealand, have tended to pursue the widespread privatisation and marketisation of an extensive array of state-owned enterprises and organisations. Governments in other countries, particularly in western Europe, have tended to be more cautious in divesting themselves of state-owned assets.

From a technology evaluation perspective this shift of ownership of technology-intensive industries and research facilities has been offset to a large extent by a massive increase in the investment by government and public services in the ‘family’ of ICTs. The scale and scope of this is vast (see Box 3), but the significant point I want to highlight is that the advent of new ICTs means that nowadays technologies no longer play a minor role in social intervention policies and programmes – they are often a central feature of them, to the extent that policies are designed around technologies, not vice versa. Obvious examples would be:

  • the use of CCTV and electronic tagging as primary features of policies to control crime;
  • biometric ID cards and other forms of access control;
  • the vast range of technologies now used in medicine and healthcare;
  • traffic management and road pricing;
  • online delivery of services, such as vehicle licensing, taxation and planning applications.

The significance for programme evaluation of technologies increasingly becoming central to government and public services is straightforward: programme evaluation would clearly benefit from recognising the importance of technology evaluation and assessment. However, as I noted earlier, just as technology evaluation has been slow to recognise the ‘social’, so the ‘social’ has been slow to recognise the technological. In other words, as I noted in the introduction, a melding of the two fields of evaluation would almost certainly lead to more rigorous and realistic evaluations, which produce more comprehensive and accurate results for stakeholders.

Box 3 The IT ‘revolution’ and government investment in new technology

Through the 1960s and 1970s the development, implementation and management of new ICTs became central to the efficient operation of many of the core administrative processes of government, such as taxation, benefits, pensions and vehicle registration. The same technological developments were also used as an instrument to deliver employment policies. For example, the EC’s use of regeneration funds for infrastructure development in the outlying regions of the European Union (EU), such as Greece, Portugal and Ireland, and more recently, the Eastern European states. In the UK similar policies led to the location of large-scale batch processing centres in areas of high unemployment and/or lower wages, such as vehicle licensing to Swansea in South Wales, National Girobank to Merseyside, and pensions administration to the north-east of England. It was argued that technology-enabled improvements in back-office functions led to significant improvements in the delivery of public services at the front office.

1979 signalled the beginning of a period of organisational and institutional change across government and public services in the UK that continues to the present day. The efficiency reviews of government bureaucracy of the 1980s, the reinvention and governance agendas of the 1990s, and the ‘transformation’ agenda promoted since 1997, are all premised on the intensive use of ICTs.

One of the most significant features of these developments is that by the 1990s the focus of development and deployment had switched from automating back-office functions to how new ICTs could be used to replace or enhance front office functions, i.e. the interaction between the state and public services and citizens. Hence a significant aim of a 2005 change initiative, Transformational Government, is ‘... to seize the opportunity provided by technology to transform the business of government.’ (Cabinet Office, 2005, p. 2). All this, while making substantial cost savings in the process.

In large part, contemporary policies for e-government in the EC and globally continue to be built on an updated version of the argument above. That is, that the deployment and management of technology can both replace existing front-office mechanisms for service delivery and reconfigure or re-engineer back-office functions and processes to increase their economy, efficiency and effectiveness. Although specific developments may differ, all member states of the EU are committed to the principle of ‘transformation’. Members of the Organisation for Economic Co-operation and Development (OECD) are also enthusiastic advocates.


Take your learning further

Making the decision to study can be a big step, which is why you'll want a trusted University. The Open University has over 40 years’ experience delivering flexible learning and 170,000 students are studying with us right now. Take a look at all Open University courses.

If you are new to university level study, find out more about the types of qualifications we offer, including our entry level Access courses and Certificates.

Not ready for University study then browse over 900 free courses on OpenLearn and sign up to our newsletter to hear about new free courses as they are released.

Every year, thousands of students decide to study with The Open University. With over 120 qualifications, we’ve got the right course for you.

Request an Open University prospectus