AI tools should be incorporated and used, as appropriate, throughout the civil justice system. Particular care should be taken to ensure that any form of used in the civil justice system is free from bias, is transparent and intelligible, and subject to appropriate accountability and human oversight. (ELI Principles of Digitalization of Civil Justice Principle 15)
Sounds good, but what does it mean? There is quite a debate going on about AI in the courtroom. I recently found several quite diverse perspectives on AI and judges in the courtroom: two academic articles on judicial perception and explanation. The use of AI as evidence, the core business of the courtroom, is my theme for this blog. I attended a workshop on it recently.
- Will judges lose their capacity for judicial perception when they use AI?
Judges increasingly use risk assessment tools, judicial analytics, and both general-purpose and domain-specific large language models (LLMs) to inform their reasoning and written decisions. How may the use of assistive AI technologies impact and potentially constrain or degrade judicial discretion as an inherent factor characterizing the human element of the judicial process (Van Domselaar & Banks, 2026).
2. What, in the context of fair procedure, is an explanation?
Parties’ right to effective participation of all those involved in a court procedure involves the right tan explanation of an AI result, or how the right to an explanation should be operationalized to enable litigant contestation. Should explanations be general or specific, to what extent do explanations need to be faithful to the system’s internal behavior or merely provide a plausible approximation, and should more interpretable systems be used, even at the cost of accuracy? (Metikos & Van Domselaar, 2025).
3. What would judges do with AI-produced evidence? It was my privilege to attend a workshop organized by Radboud University’s SPLIT project, and SSR, the Dutch judicial academy. With a group of Dutch judges, I discussed case law and four hypothetical cases. We explored what the use of AI implies for fundamental rights, the rule of law and the professional responsibility of legal professionals. The cases involved various kinds of AI-generated evidence, like a risk score rating a bank used to block access to someone’s business account, a risk score for awarding municipal support, a report on parental care, an assessment of damages. In the group on the risk score, we judges decided we would need a hearing to find out more about the AI product’s context, since it could be relevant and maybe even decisive for the outcome of the procedure.
It turned out that the issue in most of those run-of-the-mill cases was not the substance or content of the AI, but the process in which it was developed.
4. The legal framework. We were guided by article 6.1 of the European Convention on Human rights, about fair procedure and by this case law (translated with DeepL.com (free version) as laid down in Dutch guiding jurisprudence:
This complete, timely, and adequate disclosure must make it possible to assess—or have assessed—the choices made and the data and assumptions used, and, if necessary, to challenge them with justification, so that effective legal protection against decisions based on these choices, data, and assumptions is possible, enabling the court to review the legality of these decisions on this basis.” ECLI:NL:RVS:2017:1259
If a decision made by an administrative body is wholly or partly the result of an automated process—such as the model-based valuation carried out under the WOZ Act —and the interested party wishes to verify the accuracy of the choices made in that automated process, as well as the data and assumptions used therein, and, if necessary, challenge them with supporting arguments, the administrative body must ensure that those choices, assumptions, and data are transparent and verifiable. Without such transparency and verifiability, there is a risk that the parties will be placed on an unequal footing in the proceedings. ECLI:NL:HR:2018:1316
5. The cases. (case texts translated with DeepL.com (free version)
A bank uses an AI model to identify potential money laundering risks among its customers. A self-employed business owner receives a risk score of 92 out of 100. After an investigation by a bank employee, the bank terminates the business relationship. The business owner claims that all transactions are legitimate.. Without a bank account, the business owner can hardly continue his business. The bank provides only limited explanation regarding the score.
We decided we needed a hearing to find out whether the risk score is founded in a legally approved policy and used accordingly. The, what data were used on the basis of what model. I would also like to know whether the bank had geven the business owner/account holder the opportunity for defense.
A municipality uses an AI risk score to select applications for special assistance for additional review. After an investigation, a single parent’s application is denied. The decision does not mention the score, even though it was a factor in the selection. A civil servant states that he made the decision independently. (What documents and explanations do you require under the General Administrative Law Act (AWB)?)
We would have a hearing and ask several questions: does the municipality have an approved policy, which data does it use, is there any indication of a breach of privacy (art. 8 ECHR), did the single parent have an opportunity for defense?
In a dispute between parents with joint custody over the division of caregiving and parenting responsibilities, an AI-generated report on the child’s well-being is submitted. It is based on reports and questionnaires, but the methodology and choice of sources are unclear. One parent disputes the conclusions. (How do you prevent a seemingly objective report from determining the balancing of interests?).
In the hearing in this case, we would ask for all data sources, all prompts used for the data search, all reports and questionnaires. Is there any breach of privacy?
In a damages claim proceeding, one party submits an AI-generated analysis to support its calculation of damages. The opposing party disputes the outcome. They are unaware of the input data, model assumptions, and margins of error. The analysis contains persuasive tables and a legal argument. (When is this analysis sufficiently reliable and verifiable?)
Usually, there will be a counter-report from the other party. However, this case clamours for complete disclosure.
