Two New York lawyers have been fined $5,000 in a legal first after they relied on fake research created by ChatGPT for a submission in an injury claim against Avianca airline.
Judge Kevin Castel said attorneys Steven Schwartz and Peter LoDuca acted in bad faith by using the AI bot’s submissions – some of which contained ‘gibberish’ – even after judicial orders questioned their authenticity.
Schwartz and LoDuca had been representing Roberto Mata, who claimed his knee was injured when he was struck by a metal serving cart on an Avianca flight from El Salvador to Kennedy International Airport in New York in 2019.
When the Colombian airline asked a Manhattan judge to throw out the case because the statute of limitations had expired, Schwartz submitted a 10-page legal brief featuring half a dozen relevant court decisions.
But six cases cited in the filing – including Martinez v. Delta Air Lines, Zicherman v. Korean Air Lines and Varghese v. China Southern Airlines – did not exist.
Steven Schwartz submitted a 10-page brief featuring half a dozen relevant court decisions that turned out to be made up by ChatGPT
Colombian airline Avianca had asked the judge to throw out the claim against it because the statute of limitations had expired – which is when the lawyers intervened with what they thought were relevant submissions created by ChatGPT
Judge Kevin Castel said there was nothing wrong with using ChatGPT to make a case, but lawyers are still responsible for ensuring the accuracy of their submissions
Passing sanction on Thursday, Castel, of the Southern District of New York, said Schwartz and LoDuca were responsible for ensuring the ‘accuracy’ of any submissions, even those created using artificial intelligence.
‘Technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance,’ Castel wrote.
‘But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.’
What is OpenAI’s chatbot ChatGPT and what is it used for?
OpenAI states that their ChatGPT model, trained using a machine learning technique called Reinforcement Learning from Human Feedback (RLHF), can simulate dialogue, answer follow-up questions, admit mistakes, challenge incorrect premises and reject inappropriate requests.
Initial development involved human AI trainers providing the model with conversations in which they played both sides – the user and an AI assistant. The version of the bot available for public testing attempts to understand questions posed by users and responds with in-depth answers resembling human-written text in a conversational format.
A tool like ChatGPT could be used in real-world applications such as digital marketing, online content creation, answering customer service queries or as some users have found, even to help debug code.
The bot can respond to a large range of questions while imitating human speaking styles.
As with many AI-driven innovations, ChatGPT does not come without misgivings. OpenAI has acknowledged the tool´s tendency to respond with “plausible-sounding but incorrect or nonsensical answers”, an issue it considers challenging to fix.
AI technology can also perpetuate societal biases like those around race, gender and culture. Tech giants including Alphabet Inc’s Google and Amazon.com have previously acknowledged that some of their projects that experimented with AI were “ethically dicey” and had limitations. At several companies, humans had to step in and fix AI havoc.
Despite these concerns, AI research remains attractive. Venture capital investment in AI development and operations companies rose last year to nearly $13 billion, and $6 billion had poured in through October this year, according to data from PitchBook, a Seattle company tracking financings.
He added that the lawyers ‘abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question.’
Castel said the attorneys acted in ‘bad faith’ because they failed to respond properly to the judge and their legal adversaries when they questioned the ChatGPT submissions.
He cited ‘shifting and contradictory explanations’ offered by Schwartz, and LoDuca lying about being on vacation – but credited them for apologizing after the mistakes came to light.
The senior district judge also warned that the lawyers and their firm, Levidow, Levidow & Oberman, could have faced harsher fines if it had not been for this apology, along with the remedial steps they took afterwards.
The firm agreed to comply with Castel’s order – but added that it is considering an appeal over the assertion the lawyers acted in ‘bad faith’.
‘We respectfully disagree with the finding that anyone at our firm acted in bad faith,’ the firm said.
‘We have already apologized to the Court and our client.
‘We continue to believe that in the face of what even the Court acknowledged was an unprecedented situation, we made a good faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth.’
ChatGPT was first unleashed in November, sparking excitement and alarm at its ability to generate convincingly human-like essays, poems, form letters and conversational answers to almost any question.
Microsoft has invested some $1 billion in OpenAI, the company behind ChatGPT.
In the case against Avianca, the chatbot suggested several cases involving aviation mishaps that Schwartz hadn´t been able to find through usual methods used at his law firm.
Several of those cases weren’t real, misidentified judges or involved airlines that didn´t exist.
The judge previously said one of the fake decisions generated by the chatbot ‘have some traits that are superficially consistent with actual judicial decisions’ but he said other portions contained ‘gibberish’ and were ‘nonsensical.’
Lawyers for Schwartz and LoDuca did not immediately respond to a request for comment on the sanctions.
Schwartz previously apologized in an affidavit after being called out by the case’s judge, saying he used ‘a source that has revealed itself to be unreliable.’
Castel warned that Schwartz and LoDuca could have been fined more if they had not apologized and taken remedial steps following the ChatGPT legal blunder
Last month, he also said he ‘greatly regrets having utilized generative artificial intelligence to supplement the legal research performed herein and will never do so in the future without absolute verification of its authenticity.’
The lawyer added that he had never used the technology for research purposes before, and ‘was unaware of the possibility that its content could be false.’
He said he had even asked ChatGTP to verify the cases were real, and that it had said they were.
In a copy of the exchange submitted to the judge, Schwartz asked ChatGPT: ‘Is varghese a real case’ and ‘Are the other cases you provided fake.’
The bot replied: ‘No, the other cases I provided are real and can be found in reputable legal databases.’
Fellow attorney LoDuca said he had no role in the research but ‘no reason to doubt the sincerity’ of Schwartz’s research.