5 Automated species identification from images and sounds

Normally, when a photo is uploaded to iSpot or one of the other citizen science biodiversity platforms, a person looks at the photo and tries to give an identification. This might happen very quickly, in just a few minutes, or it might take days or even years if the image is of an obscure organism and a suitable expert visits the site only rarely.

Recently automated species-identification services have become available across a range of taxa, whereby a computer compares an image or a sound recording with information in its database to arrive at an identification. For this to work successfully, the system needs to be initially ‘trained’ by loading it with a large number of correctly identified images of each species. With sound recordings this is also the case. For example, when using automated recognition to identify bat calls, it is very important to include data from correctly identified bat calls from the same part of the world as the recording of the species being identified (Reason et al. 2016).

Such an automated system analyses the images or sounds and derives a database of information that is unique to that species. It then uses this complex derived information to identify any new images. The system does not try to match new images to any of the single specific images that were used to derive the database; instead, it compares them with the derived information from large numbers of images of that species.

Activity 2 Using Pl@ntnet

Allow about 30 minutes

Pl@ntnet is an online tool whereby citizen scientists can contribute to a project on monitoring plant biodiversity. Visit the website at or download the Pl@ntnet app, which is available on Google Play or the App Store. Familiarise yourself with the system by uploading the image of a pasque flower (Pulsatilla vulgaris) in Figure 10.

Figure 10 Pasque flower (Pulsatilla vulgaris)

One thing to note with Pl@ntnet is that you need to select the correctproject when you first start, e.g. either ‘Useful Plants’, ‘Western Europe’, ‘USA’ or one of the other projects. If you choose the wrong project, your plant image won’t be compared with the relevant database and might not be found.

Try to identify some plants in your local area and answer the following questions.

1. Which species or images work and which have difficulties? Bear in mind that identification is fully automated and performed by a computer program, which might have trouble identifying which part of an image depicts the target sample.
2. What information does the app/site ask you? Why?
3. Does the app/site give just one identification or does it suggest several alternatives?
4. Does the app/site generate any links to other resources that might give more information about the species?

Answer

1. It is likely that images with one clearly defined flower or leaf will work better than those featuring many small objects.
2. The app or website might ask which part of the world the plant is from, in order to narrow down its search for suitable species. Or it might ask which part of the plant is featured in the picture in order to narrow down the search to just the characteristics of leaves or flowers, for example.
3. Depending on which system you use then it may suggest alternative identifications (with pictures of those various species) and may even attach a probability of accuracy to each alternative.
4. There might be a link from the identification to Wikipedia or other resources that might provide more information about the target species.