There is also the risk of so-called AI ‘hallucination’, nonsensical or grammatically incorrect output created by generative AI models.

For example, when CaseSnappy’s Founder Ayush Sanghavi asked GPT-3.5 to provide him with ‘an overview of the UK Supreme Court’s decision in R v Ayush Sanghavi [2020] UKSC 1’, he was given a rather genuine-sounding case overview on the subject of ‘the proper approach to be taken when determining whether a defendant is unfit to plead and stand trial’, despite the case being entirely made up.

But it’s not just students who can fall victim to AI hallucinations. A well-known example is that of New York lawyer Steven Schwartz, who mistakenly cited fictitious cases generated by ChatGPT in an aviation injury claim. The admittedly 'embarrassed' lawyer was later fined $5,000.

Of course, a model producing nonsensical – or even grammatically incorrect – content would not be conducive to any sort of quality legal research.

However, when the root causes of such hallucination are considered, it is quite easily avoided. Hallucination typically occurs in generative AI output due to a lack of training data, dirty (i.e. inaccurate, incomplete or inconsistent) or noisy (i.e. corrupted) data, a lack of context (i.e. in the prompt) or a lack of constraints. Therefore, an extensive and prudent data selection process and careful prompt engineering can eliminate the risk of hallucination. AI models can also be ‘fine-tuned’ to a particular subject by restricting their training data to data solely on that subject. This can again fight against risks of AI hallucination.