Assessing performances of pattern informatics method: a retrospective analysis for Iran and Italy

Pattern Informatics (PI) algorithm, which was introduced at the beginning of past decade, uses instrumental earthquake catalogs to investigate the time dependent rate of seismicity in the study area and, based on the information from past events, calculates the probabilities for the occurrence of future large earthquakes. The main measure in this method is the number of events above a specified magnitude threshold Mc that is counted over a gridded area. PI has been applied in several regions of the world and different variants of the method have been developed over the past decade. Hence the problem of formally evaluating and comparing the performances of the different PI variants needs to be addressed from an operational perspective, in order to identify the preferred application scheme and as well as to improve the performances of the method. In this study, PI is applied for the first time to the retrospective analysis of the earthquake catalogs of Iran and Italy, so as to check if this method could forecast the past large events in these two regions with different level of data completeness and complex seismotectonic setting. The original PI algorithm and one of its modified variants, as well as the Relative Intensity (RI) model, are used to check the stability and statistical significance of the obtained results. In order to assess and compare the obtained results, the performances of the different PI variants are analyzed considering different evaluation strategies, which turn out to provide significantly different scores even for the same algorithm variant. We show that a critical point in the assessment of the obtained results is related with the definition and quantification of the space uncertainty of the issued forecasts, i.e. with the extent of the territory where large earthquakes are to be expected. Accordingly, we emphasize the need for an appropriate definition of the evaluation strategies, clearly and unambiguously indicating the area where a large earthquake has to be expected. The study shows that, with respect to application in Iran and Italy, the performances of PI algorithm (both original and modified variants) are highly dependent on the selected evaluation strategy and do not provide better information than the simple RI model, which does not account for temporal properties of seismicity evolution. The overall performances can be improved by introducing specific thresholds that discard the less active cells; however, being based on some posterior optimization, a rigorous prospective testing is required to assess the forecasting capability of the method. In this paper we aim to set up the rules for such testing, including advance definition of the evaluation strategy.