Evaluation and Performance of Ecological Niche Models in South America: a whip-spider case study

Abstract

In this work Ecological Niche Models were built using the occurrence records of nine species of the endemic South American whip-spider genus Heterophrynus. With this dataset, we tested the use and compared performance and output similarity of three climatic datasets (BioClim, MERRAclim and ENVIREM) and eight algorithms (RF, BRT, SVM, MaxEnt, MaxLike, GLM, GLMNet and MARS) under three M sizes for each of the nine species. Furthermore, we used one of the climatic datasets, BioClim, to build and project models for two end-of-century SSP scenarios and quantify suitable area lying inside Protected Areas (National Parks and Indigenous Land) in each scenario. Our results suggest MERRAclim is the most dissimilar from other climatic datasets, and that the interpolation artifacts in both BioClim and ENVIREM dictate model output in the Amazon Basin. In our analyses, the algorithms RF and MARS overfitted models, while GLM, GLMNet and MaxLike underfitted models given tested settings. We further illustrate how AUC and TSS statistics are uninformative as evaluation methods for presence-background or presence-pseudoabsence models. We found that Indigenous Land or Territories cover as much suitable area as Integral Protection Areas on average. Some species are estimated to lose over two thirds of their current suitable area by the end of the century, while others to have their suitable area more than doubled. From our conclusions, we emphasize that the use of a single climatic dataset, GCM and/or algorithm should be avoided. Furthermore, we suggest that defining M should be based on building a few models a priori with different M sizes and selecting the one with the best performance and best fit for intended model use.