C) Data Collection for Needs Analysis

This section outlines the different methods of data collection used to conduct needs analysis. It is an optional section and you may determine whether to complete the section or not depending on the pertinence of the section to your project.

This section outlines the different methods of data collection used to conduct needs analysis. It is an optional section and you may determine whether to complete the section or not depending on the pertinence of the section to your project.

OAD projects will not typically engage in extensive data collection on their own. Collecting social data is full of unexpected challenges; it is easy to inadvertently collect misleading data or to spend a lot of time, energy and expense collecting data which later proves useless because of some small oversight. If you are interested in collecting data, however, we hope to support you in doing so—ideally in partnership with professionals who specialise in this area (e.g. economists, statisticians, and/or development organisations). 

Points to Note About Survey Data

Data collection requires careful planning regarding sampling strategies and the development of data collection tools. For quantitative data (e.g. surveys), questionnaires must be carefully designed (see guide with template questions on the OAD website. There is extensive research showing how small changes in wording or presentation can significantly alter the results you get. We recommend that you acquire advice and expert peer-review when designing the survey.

It is ideal to have someone who has been trained and is familiar with the questionnaire to administer surveys in person or over the telephone. In many places, there are professionals or research firms who will provide this service for a reasonable fee. If a survey is important to you, consider including this in your project budget.

Points to Note About Interviews and Focus Groups

For qualitative data (i.e. focus groups and/or interviews), data collectors need topic guides that outline which topics to cover during a session. Ideally, qualitative data are recorded with a voice recorder and later transcribed. This is, however, time and resource intensive: one hour of recorded data takes approximately 3 hours to transcribe into text. Texts then need to be analysed to determine which themes recur. When resources are limited, it is more efficient not to record the session but to have it conducted by two assessors: one to guide the discussion while the others takes notes. Notes from each session can then be compiled to inform the needs assessment.

Sampling Strategies

Different sampling strategies are used for quantitative (survey, questionnaire) and qualitative (interview, focus group) data collection.

Quantitative (Survey) Sampling

Quantitative data are used to describe populations quantitatively by accurately quantifying the percentage of individuals within a population who share a specific characteristic. Surveys can offer important information on attitudes, beliefs, perceptions, level of satisfaction, needs, priorities, service access etc, which can be used to inform project theory, design, resourcing, and implementation planning. For example, if 95% of a community say that they use mobile phone data on a daily basis, a project using a mobile-based information dissemination methods is more likely to be successful than it would be in a community where only 45% of individuals have access to mobile data.

Surveys are relatively easy to analyse and provide anonymity to respondents. However, in order to generate usable quantitative information, surveys must:

  • Use closed questions: Individuals need to be asked the same questions and in the same way. For their responses to be meaningfully analysed using statistical methods, the responses also need to fit into discrete categories. Survey questionnaires are therefore comprised of closed questions (i.e. asking participants to select a category, offer a number, rank items or select a value on a specified scale).
  • Use a representative sampling strategy: There are two ways to achieve statistically representative samples from a population of interest: (1) obtain data from everyone in the population or (2) obtain data from a random sample selection (or the statistical equivalent of a random selection).

It is usually too expensive and/or impractical to gather data from every individual in a community or other target population. Surveys therefore usually rely on data from a smaller group of individuals drawn from within the larger population of interest. It is important that this smaller sample is representative of the larger population.  If not, survey data will be misleading: the percentages of individuals reporting a certain need or perspective may then fail to reflect the true percentage in the population and lead to a range of project miscalculations and resourcing errors. In order to be representative, a survey sample needs to be drawn at random (or the statistical equivalent) from the larger population.

Both of these survey features can be challenging to implement in practice. This leads to short cuts that actually produce invalid (and probably misleading) results. There are many examples of surveys that do not fulfil these basic criteria for validity in the social science literature on the Astronomy community (e.g. of gender composition and gender differences; and of Astronomy capacity and research output). In fact, we request that anyone who finds a properly conducted random sample  survey of astronomers please let us know!

Qualitative Sampling

Sampling is more straight-forward for qualitative data collection. For qualitative sampling, those conducting the needs assessment deliberately seek out key individuals or groups based on characteristics of interest. An example of such sampling for a community would include community leaders of both genders and a small number of community members drawn from each identifiable subgroup within the community. Effort is typically made to seek out different perspectives on multiple dimensions (e.g. interviewing community members with varying economic status and of both genders from within each ethnic group).

Making Use of Secondary Datasets

Secondary datasets comprise of, for example, World Bank or OECD development data. Some links to get started exploring these sources can be found here.

 

 BACK     NEXT > 

Last modified: Thursday, 5 July 2018, 7:35 PM