This overview compares different individual identification (‘matching’) software that could be potentially used to identify Archey’s frogs.
Disclaimers: this review is not intended to be used as a guide to select “the best” identification software. It is a comparison of software that could fit the requirements to identify Archey’s frog. All information comes from the user manual and website of the software linked under the general information paragraph, unless stated otherwise.
Comparison of software available to automatically identify individual animals.
| APHIS | I3S | Wild-ID | AmphIdent | HotSpotter | IBEIS | |||||
| Packages | ITM | SPM | Classic | Pattern | Contour | Spot | ||||
| Year of latest version | 2017 | 2014 | 2014 | 2015 | 2014 | 2011 | 2018 | 2014 | 2020 | |
| Cost | Free | Free | Free | 850€ (basic package) | Free | Free | ||||
| Supported Environments | Windows | Windows | Windows/Apple/Linux | Windows | Windows/Apple/linux | Windows/Apple/linux | ||||
| Feature- or pixel-based | PB | FB | FB | FB | PB | FB | FB | |||
| Automatization | ± | ± | – | ± | – | – | + | – | + | + |
High automatization (less pre-processing), semi-automatization and limited automatization are represented as “+”, “±”, and “-” respectively.
Based on review by Renet et al. (2019) / The Herpetological Journal
The five software programs we reviewed are:
General information
The APHIS software is freely available online and comes with a detailed User Manual. APHIS offers two different matching algorithms, one of which is a feature-based approach (Spots Pattern Matching, SPM) and the other is pixel-based approach (Image template matching, ITM). Whereas ITM has a much faster preprocessing phase (2-3 seconds per image) than SPM, its performance depends greatly on the quality of the dataset. The developers advice to only use ITM when luminosity, definition and image quality can be standardized.
Matching algorithm
The matching software compares an unknown sample against a set of known candidates, scoring them on their similarity to the sample. Based on these scores, the software delivers a series of potential matches (maximum 100) for posterior visual inspection by the user. The results are stored in text files, formatted as comma-separated CSV.
The algorithm behind the ITM approach uses the matchtemplate function of the Open Computer Vision libraries (OpenCV). The matchtemplate function overlaps the two images, repositions them and gives a score on similarity. The SPM approach is based on the I3S Pattern software, which algorithm is explained in section 5.2.
Required preprocessing
Before the matching can take place, the samples need to be preprocessed by providing reference points on each picture. The ITM approach asks for the selection of two reference points within each picture, ensuring the most accurate two dimensional comparison between the sample and the candidate. The SPM preprocessing phase is more labourish, as the three reference spots highlighting a distinctive feature need to be more carefully chosen.
General information
The Hotspotter software can be freely downloaded and comes with an additional User Manual. Hotspotter uses a feature-based approach for matching individual images.
Matching algorithm
The user provides the software with a rectangular region of interest (ROI) and orientation for each individual picture. From this, hotspotter automatically detects elliptical regions centered on points of interest, the so-called hotspots. If two images have enough similarity in hotspots, they are matched by the software. The software ranks all the potential matches, showing for each match a similarity score and highlighting the section of the images it deems the most similar. It is up to the user to select the final match. The type and size of the dataset affects the height of the score, thus the height of the score is only relative to its dataset. After Hotspotter selects an amount of potential matches, it is up to the user to pick the final match.
The mechanism behind Hotspotter consists of two different matching algorithms. The one-vs-one algorithm, matches each image against each database image separately, with a similar mechanism to that of Wild-ID SIFT algorithm. The second algorithm, the one-vs-many algorithm, matches the hotspots from an individual image to all hotspots from the images within the database, using a Local Naive Bayes Nearest Neighbor algorithm. The combined efforts of these algorithms, form the final similarity score (Crall et al, 2016).
Required preprocessing
As earlier stated, the user needs to provide the region of interest (ROI) and orientation for each individual image. The ROI is the selection of a rectangular area, which must include the distinguishing features of the animal’s body. It is recommended to make the ROI too large instead of too small, as not to cut off any important features.
The orientation of the image is by default horizontal. Although specifying the orientation might not be important for all types of pictures, it is crucial to ensure accurate recognition for overhead pictures. Within the ROI the user draws an axis in a way that can be easily repeated for all images. The developers recommend that for frog images, “the repeatable orientation is selected along the spine, from the tip of the mouth to the tip of the tail”, always in that order.
Related software: IBEIS
The hotspotter software is used as the basis for the matching software of the IBEIS program. The IBEIS program is designed for the storage and management of images, able to compute the species of the animal, detect individual animals and know where an animal is. IBEIS employs algorithms such as “random forest species detection and localization, hessian-affine keypoint detection, SIFT keypoint description, Local Naive Bayes Nearest Neighbor identification using approximate nearest neighbors”. The software is part of the WildBook project. The IBEIS documentation can be found here, yet, if you are lost the author provides some additional guidance within this correspondence.
The IBEIS software is written in Python and can be only operated within a Linux environment, thus it is advised when operating on Windows or OSX I to use a Linux virtual machine. Although based on the Hotspotter software, it is unclear to what extent the matching algorithm of IBEIS is different from its predecessor. However, the IBEIS repository is still being updated, whereas the Hotspotter has not in the last six years.
General information
The software can be downloaded for free on the creator’s webpage (bottom right of the page). Within the zip-file of the software a user guide can be found. Wild-ID uses a feature-based approach.
Matching algorithm
For the matching algorithm the Wild-ID software uses the Scale-Invariant Feature Transform (SIFT) operator. The SIFT operator was designed to extract distinctive features within an image invariant to image scale and rotation. Each image is reduced to a set of so-called “SIFT-features” and it is on these SIFT-features two images are compared. These features are compared on their geometric arrangements and attributes, from which a matching score is calculated. The programme then provides 20 potential matches for an individual picture, from which the user chooses if there is a final match or no match at all.
Required preprocessing
The programme requires little preprocessing, yet the most accurate matching is ensured when the images are cropped to only include the area of interest, which is the area with the animal’s distinctive patterns. It is recommended to crop out as much of the background as possible. The developers give the example of the picture of a giraffe, in which they only used the body and lower neck, cropping the head, legs and the rest of the neck.
General information
The AmphIdent software has to be bought, luckily, a test version is available and comes with a free, but limited, User Manual. The AmphIdent is designed for specific species, the Great Crested Newt (Triturus cristatus), Fire-bellied toad (Bombina bombina), Yellow-bellied toad (Bombina variegeta), Fire salamander (Salamandra salamandra) and Marbled salamander (Ambystoma opacum). However, the software should be able to process different species as long as their patterns consist of bright and dark areas. The software uses a pixel-based approach for its matching algorithm.
Matching algorithm
Although the software works differently per species in the preprocessing phase, it’s matching algorithm remains the same. As explained in Matthe et al. (2017), each image is scaled down 25% per dimension and 4 x 4 squares of the original pixels are averaged and become the image’s new pixels. The absolute differences of these pixels between two images give a similarity score. To be robust against different scales and sizes in cropping, one of the images is scaled and translated in various different combinations. Of all these different investigated transformations, the maximum score is the final similarity score.
Required preprocessing
For the preprocessing phase the pattern of the image needs to be cropped out appropriately, yet this differs per species. For example, with images of the fire-bellied toad a frame needs to be selected that covers the toad’s pattern as much as possible. The user manual provides per species a detailed description of the appropriate cropping.
General information
The I3S software and their user manuals are freely available online, as well as an abundance of tutorials are provided on its Youtube webpage. The I3S software uses a feature-based approach and includes various different packages (Classic, Pattern, Contour, Spot).
If the species to be identified has similarly shaped spots, using Classic is recommended. For irregular shaped spots using Spot is recommended. If the main feature is a distinct contour or line (e.g. whale’s tail), Contour is recommended. When the patterns are too complex and difficult to manually annotate, Pattern should be used. Within Pattern the features are automatically, as opposed to manually, annotated. Although this saves a lot of time in the preprocessing phase, it should only be used if no other package is suitable, as the automatic annotation can result in a reduction in accuracy.
Matching algorithm
Depending on which of the packages you use, the inner workings of the algorithm can differ significantly. Yet, for all packages, an unknown individual is matched with a ranked list of known individuals (maximum 20) based on a similarity score. From this ranked list, the user chooses the final match.
The Matching algorithm I3S Classic uses a formula to calculate a similarity measure based on the spatial relationships between the centers of each spot. I3S Spot works similar to I3S Classic but also includes information on the shape and size of the spot to search for potential matches. I3S Contour overlays the contours of the images and the space between them is used as a measure of similarity. I3S Pattern uses the Speeded-Up Robust Features (SURF) algorithm for feature selection and comparison.
Required preprocessing
I3S Pattern requires the least amount of preprocessing, as features do not have to be manually annotated. However, the user does need to provide three fixed reference points, as this provides information on the image’s angle, rotation and scaling. The preprocessing phase for the Classic, Contour and Spot packages of I3S are much more time-consuming, in which their features need to be annotated in detail.
