The risk of AI's facial recognition in criminal prosecution: the need for a balance between efficiency and the protection of human rights

Abstract

L’articolo analizza l’uso del riconoscimento facciale nei procedimenti penali brasiliani, in rapporto alle sfide legali ed etiche che ne derivano. Lo studio prende in esame la classificazione di questa tecnologia considerata ‘ad alto rischio’ e i meccanismi necessari per garantire trasparenza, responsabilità e certezza del diritto. Casi concreti in Brasile hanno evidenziato l’urgente necessità di una regolamentazione adeguata. Il contributo valuta anche i rischi del potenziale utilizzo di questa tecnologia come strumento di eccessivo controllo sociale. Affinché l’intelligenza artificiale possa essere utilizzata in modo equo ed efficace nei procedimenti penali appare dunque essenziale stabilire normative chiare, meccanismi di controllo efficaci e linee guida per mitigarne gli impatti negativi, garantendo la tutela dei diritti fondamentali dei cittadini.

This paper examines the use of facial recognition in Brazilian criminal prosecution, addressing the legal and ethical challenges involved. From a comparative perspective with European legislation, particularly the AI Act, the study analyzes the classification of this technology as “high risk” and the mechanisms required to ensure transparency, accountability, and legal security. Concrete cases in Brazil, such as wrongful arrests resulting from AI misidentifications, highlight the urgent need for appropriate regulation. The discussion also explores the impact of AI on the phenomenon of “fossilization” and the issue of “self-fulfilling prophecy”, considering the risk of reinforcing discriminatory patterns and the potential use of this technology as a tool for excessive social control. The study concludes that, for AI to be used fairly and effectively in criminal prosecution, it is essential to establish clear regulations, effective oversight mechanisms, and guidelines to mitigate its negative impacts, ensuring the protection of citizens’ fundamental rights.