Legal AI is poised to revolutionise international arbitration, offering improved efficiency and cost savings. In its relative infancy, however, it offers imperfect solutions to a wide range of problems. This article will consider the benefits and risks of legal AI tools, and practical steps both parties and arbitral institutions can take to ensure that the primary function of the arbitral process – achieving a just and enforceable award – continues to be met.

Legal AI

Since the introduction of ChatGPT in 2022, AI has experienced a remarkable surge. Generative AI^[1] changed the way people perceive technology virtually overnight. AI has the potential to transform countless industries, from medicine to music. It has already changed the way the legal industry approaches tasks, including^[2]:

1. Document review and management. The data boom means a lot more documents to review. But AI can sort, order, analyse and identify patterns amongst thousands of documents almost instantly, easing the disclosure and inspection process. Tools which can identify privileged documents or assess whether the sentiment expressed in an email is positive or negative are becoming commonplace.

2. Predictive analytics. While such tools are used less than other forms of AI by firms day-to-day, they have enormous potential. AI models can be trained on historical court data – or even the decisions of a specific judge or arbitrator – to predict a case's outcome. The same logic applies to arguments or strategies favoured by particular lawyers or firms, which can now be forecasted and war-gamed before they are even pursued.

3. Summarisation. Consuming an undigestible volume of information and producing a digestible summary was once the unenviable task of human lawyers. AI can now precis an entire file and produce chronologies in seconds.

4. Communication. Lawyers are increasingly using generative tools to draft simple emails and letters to opposing counsel. AI's ability to transcribe meetings has also vastly improved in recent months.

5. Translation. AI firms are increasingly investing in software which can detect and translate foreign language – of particular help to lawyers in cross-border disputes.

For nearly a century, scientists, philosophers and science-fiction authors have warned of the theoretical dangers of clever machines. Yet, when faced with the sudden reality of powerful AI, society has struggled to agree on clear boundaries for its usage. The legal industry is no different. Dispute resolution, by its nature, is a retrospective practice. Lawyers tend to be more conservative and have a greater reverence for history and tradition than other professionals. It is perhaps unsurprising, then, that the industry has yet to really get to grips with AI or set clear guidelines on how and when it should be used.^[3]

That is not to say the industry will not adapt. It has already experienced profound change over the last few decades and has readily embraced technological advances designed to make lawyers' jobs easier (take Technology Assisted Review ("TAR"), for one). Rather, in an increasingly global and decentralised world, the question is from where change will come. International arbitration – consensual, flexible and with global reach – is well-placed to take the lead. That raises the question: should arbitral parties take advantage of the consensual model to set their own rules governing the use of AI in disputes?

Risks

While perhaps not fashionable to admit, legal AI is not yet what it could be. We were told in 2022 that it would immediately turn our lives upside down; it hasn’t. AI is extremely effective at some things and less so at others. While improvements will continue at a historically remarkable pace, for now at least, legal practitioners are advised to take AI output with a healthy pinch of salt. Whether that involves robust human validation procedures or plain common sense, AI should be treated only as a means of getting to an answer more efficiently, rather than of getting to the answer itself. The latter remains the lawyer's remit, for now at least.

Consider the following few examples:

1. Reliability. AI tools can process and synthesise data in volumes once thought insurmountable, and at speeds once thought impossible. They can also be entirely unreliable and lead to embarrassing errors. Few lawyers will need reminding of the dangers of ChatGPT "hallucinations", but if they do, Mata v Avianca^[4] serves as a stark warning. Other tools abhor a vacuum, providing answers where they don't really exist. Humans are prone to being too trusting of machines' findings (e.g. failing to review underlying documents) and sometimes use tools for jobs they simply aren't equipped for. As AI tools become more widely available and affordable, the risk of serious mistakes from overreliance is likely to increase.

2. Equality of arms. Historically, larger, more affluent law firms have always been able to throw more bodies at tasks than their smaller counterparts. The introduction of AI has the potential to create a yet more uneven playing field. Profitable firms can invest capital in cutting-edge technology, thereby arming future clients with efficient AI-powered insights at lower cost. Clients of less well-heeled outfits, let alone litigants in person, could find themselves at a greater disadvantage. While AI tools will presumably get cheaper over time, the biggest firms will always be able to boast the best and most expensive tech, whatever that next generation of software might look like.

3. Decision-making. Given the dramatic pace of development in recent years, it now seems only a matter of time before we start handing over responsibility for legal decision-making to machines. In China, 'Internet Courts' have been doing just that since 2017. Cases are heard online and adjudicated by advanced AI models. The average duration of a case is around 40 days, the average hearing lasts only 37 minutes, and 98% of the rulings have been accepted without appeal. The Internet Court system operates 24 hours a day, seven days a week. Estonia has also recently adopted an AI judge for small claims disputes.

In a few cases, this makes logical sense (why spend thousands on legal proceedings where AI can simply review black box data and CCTV footage of a road traffic collision, and attribute blame objectively and instantly?). The same cannot always be said for sophisticated commercial disputes which often turn on witness testimony and complex legal issues.

Use by arbitrators

The third risk identified above requires careful thought: should arbitrators be free to use AI?

As dispute resolution becomes more and more expensive, there are compelling cost-saving arguments for the use of AI by decision-makers. Arbitrators already rely on clerks, paralegals or arbitral secretaries to help review large volumes of evidence and submissions and craft written decisions. Why shouldn't a chatbot fulfil the same role?

One task AI is extremely good at now is summarisation. AI can consume enormous volumes of data and produce a pithy summary much faster than any human could. This capability is especially useful – and can result in significant costs savings for clients – when parties arrive at the document production stage of proceedings, for example. So long as human lawyers have built up their own understanding of the document universe, independently of their machine counterparts, summarisation tools can slot into a practitioner's toolkit with relatively low risk.

The issue with summarisation, however, is not AI's output, but what AI omits. If arbitrators relied on AI-generated summaries of every submission, witness statement or pleading that came across their desk, the arbitrator’s understanding of those documents could be warped beyond their intended meaning. Nuance, inference, and tone could all fall by the wayside.

Take the following cautionary example. We asked an AI tool to summarise the following paragraph from a (fictive) witness statement in fewer than 20 words:

On or around 23 January 2011 I attended a meeting with some of the board and other people I didn't know. I do not remember specifics, but I remember receiving a frosty welcome. At this meeting it was suggested to me that my shares would be safeguarded if and when the deal went through. When I asked for some form of guarantee that nothing would be done to prejudice my interest, David said "We wouldn't dream of it". As I got up to leave I saw David smiling out of the corner of my eye.

AI's response:

In January 2011, I attended a board meeting where David assured my shares would be protected, then smiled as I left.

Litigants would expect an arbitrator to read the unabridged paragraph and conclude that the witness was unduly optimistic about the merits of his or her case. Leaving aside the fact that the evidence is (deliberately) poorly drafted, the natural inference of the words is that the guarantee the witness received was not as iron-clad as he or she would like to suggest. AI's summary gives the exact opposite impression. It concludes that a meeting with "some of the board" was a quorate board meeting, ignores the witnesses' unsatisfactory powers of recall, and fails to pick up on sarcasm or body language.

Whilst this might seem an extreme example, a 20-word summary of this paragraph would not be unusual in the context of a short, AI-generated chronology of events. At the least, over-reliance on AI here would allow an arbitrator to carry an unduly positive preconception of the witness into cross-examination. At the worst, it could result in an unjust award.

Suggestions

This article does not advocate for outlawing the use of AI in arbitration. That would be a regressive and reactionary step given AI's obvious benefits. But it is clear from the risks outlined above that overreliance should be taken seriously. The direction of travel, in the EU at least, is towards pragmatic regulation. The underlying objective of the recently enacted Artificial Intelligence Act^[5] is to "strike an optimal (and proportional) balance between innovation and the benefits of AI systems on the one hand, and the protection of fundamental values and fundamental rights on the other." Parties to arbitration might welcome minor adjustments to achieve this end.

One practical step which arbitral institutions ought to consider is the use of model clauses governing the use of AI. Parties can choose whether or not to adopt such wording when preparing Procedural Order No. 1, but at the very least, institutions encouraging parties to discuss these issues at an early stage would be a step in the right direction.

Model clauses should promote fair outcomes and equality of arms. A strict clause might prevent the use of sophisticated AI tools by one party if its opponent lacks access to that software. At a minimum, parties ought to know which kinds of AI tools their opponents intend to use. That approach would expand upon the disclosure requirement in the English Business and Property Courts, where parties set out how they intend to utilise technology to assist the document review process.^[6]

Model clauses should be drafted with future technological advances in mind. Specific, concrete rules would become obsolete very quickly given the rate of AI's development. Institutions might require parties to provide the Tribunal with regular updates on which AI tools are being used, and for what purposes. Depending on parties' appetite for regulation, these disclosures could be paired with a power of veto if the Tribunal deems those purposes improper, or sanctions for non-compliance.

Were parties to make voluntary disclosures of this kind, they might consider it reasonable for the requirement to apply equally to their arbitrators. Any such disclosure should be detailed enough to allow parties to evaluate the potential risks. That approach would promote transparency and could give parties leave to make submissions opposing the arbitrator's use of AI in the interests of a fair outcome.

The vast range of AI tools now in use points in favour of Tribunal disclosure. Suppose Party A and the Tribunal are using the same review tool, onto which all contemporaneous documents have been uploaded. Both are likely to get the same or similar response to the same interrogation of the data. Were Party A to make a submission in reliance on that response, the Tribunal might be more receptive to it than the opposing submission of Party B, which is using a different piece of software. The mere choice of AI tool therefore has the potential to create a competitive advantage in favour of one party. Early-stage disclosure of the tool(s) the Tribunal intends to use would mitigate against this risk.

An arbitrator using AI for low-risk administrative tasks would rarely be contentious (indeed, a model clause could specifically carve-out usage of this kind). But parties would benefit from knowing about other, higher risk uses. A non-native arbitrator might, entirely reasonably, wish to use a translation tool to help draft a Final Award, for example, or otherwise use generative AI to finesse language. But precise words or phrases in one language often lack a foreign equivalent. Where even a slight mistranslation in an Award could change the intended outcome, parties might, entirely reasonably, want to verify the quality of the particular software used. Short of being told which tool was used, parties ought to know whether a tool was used, and be allowed to assess potential mistranslations accordingly.

Conclusion

There is huge potential for AI in international arbitration. It may, in due course, change the way we approach commercial disputes. For now, its risks must be carefully managed. Efficiency should not come at the cost of equitability. Regulatory frameworks like the EU's Artificial Intelligence Act aim to strike a balance between encouraging innovation and safeguarding fundamental rights and fairness. Inserting a reasonable, risk-based AI clause into a procedural order could serve that same purpose. A pragmatic approach would promote transparency, and in turn, allow parties the freedom to make informed decisions on which risks they are willing, or unwilling, to tolerate.

For more on the UK's response to the issues discussed in this article, see our recent article on the AI Opportunities Action Plan.

[1] A form of AI which relies on Large Language Models (“LLMs”), which in turn draw on vast subsets of internet training data to create fresh content – be that text, image, video, audio, code, etc.

[2] An AI tool was used to create this list (albeit its speedy work was reviewed and amended by a human).

[3] The English judiciary issued a high-level guidance note in 2023 (https://www.judiciary.uk/wp-content/uploads/2023/12/AI-Judicial-Guidance.pdf). A much more detailed and consequential rulebook will be needed in due course.

[4] https://casetext.com/case/mata-v-avianca-inc-3

[5] https://artificialintelligenceact.eu/

[6] See template Disclosure Review Document, Section 2, Question 12: https://www.judiciary.uk/wp-content/uploads/2017/11/draft-disclosure-review-document.pdf