Enhancing Legal Efficiency with AI

Prisonology looks for opportunities to reduce sentence time or the security-level of a facility based on factors such as health issues, family situation, or background. In order to convince judges of their cases, they write a document that argues these points based on prior cases where defendants received special treatments based on their conditions or underlying situations. One of the tools they use is the Security Designation Scorecard, which categorizes inmates based on the severity and nature of their offenses. NineTwoThree explored if AI, LLMs and Retrieval-Augmented Generation could be used to automate the creation of this scorecard.

Concept

Prisonology specializes in advising on Federal Bureau of Prisons (BOP) regulations, leveraging the expertise of former BOP staff. The company supports law firms and defendants by developing effective strategies to potentially reduce prison sentences. Services include providing expert opinions, training, and diverse consultancy services across the United States.

In order to convince judges to reduce sentences or lower the security of the prison the defendant will be sent to, Prionsology uses a Security Designation Scorecard. This scorecard gives judges more background information on the defendant and why they should have a more lenient sentence.

The Security Designation Scorecard is a vital tool used in categorizing inmates based on the severity and nature of their offenses. It aids in deciding the appropriate security level for incarceration, ranging from minimum to maximum security prisons. Prisonology uses these scorecards to have their experts create persuasive statements focusing on factors like health, age, and other personal circumstances of the accused.

In order to create this document, lawyers receive detailed reports on each defendant, which include a score indicating the perceived risk and dangerousness associated with the individual. This score is pivotal in the decision-making process regarding prison assignments. Prisonology experts then use these reports to negotiate and argue for lower scores in court, where each point can significantly impact the defendant's fate.

Prisonology was considering the use of artificial intelligence to streamline and enhance the efficiency of statement writing. As part of this initiative, NineTwoThree was tasked with investigating various data sources, both free and paid, to ascertain the feasibility of accessing, storing, and searching through historical court sentencing records at a cost-effective rate. We built a Proof-of-Concept app that scores the Severity of Offense metric based on uploaded Presentence Reports.

The overarching aim of incorporating AI into this process was to significantly reduce the research time required by lawyers, so they can spend their billable hours on the trial itself. This approach is expected to not only cut down on expenses, but also boost the overall efficiency of legal professionals.

Challenges

Conventional AI models, including ChatGPT, exhibit limitations in processing extensive documents. This challenge becomes particularly acute in intricate settings like legal case analysis. Despite their advanced capabilities, these models often struggle to mimic the sophisticated human reasoning necessary for legal scenarios. The complexity and diversity of legal cases, along with the plethora of influencing factors on judicial decisions, pose a significant challenge to AI systems.

Solution

NineTwoThree created a proof-of-concept app with a Retrieval-Augmented Generation system coupled with two scoring modules for reading the documents and reasoning the score.

Enhancing Legal Efficiency with AI

About Retrieval-Augmented Generation

Instead of using a public LLM like Google Bard or ChatGPT where new information can only be given to the LLM in the prompt, Retrieval-Augmented Generation (RAG) offers a contrasting approach. This method augments a user's prompt by pulling in information from a trusted and private knowledge base and then integrating this data into the original query for processing by the Language Learning Model (LLM). When a question is posed, the query's text is encoded into machine language and then matched against the context within the knowledge base.

The knowledge base in a RAG system is tailored to store company-specific and crucial information, such as Prisonology’s database of twenty years of successful legal motions. Being a valuable component of a company's intellectual property, this repository is securely stored behind a firewall with access management protections in place.

By using a RAG, NineTwoThree was able to use Prisonolgy’s trove of past court cases to create an AI solution without risking leaking private information to the LLM.

Employing Chain of Thought to Reason Like Humans

Instead of trying to read a court document and generate a score in one step, NineTwoThree developed two separate modules for reading the document, and then creating a score like a human. 
‍
The reading module is designed to meticulously read PDF files from courts about a defendant’s past history. This module works by posing a series of branching questions, establishing a logical flow for the AI to follow.

This system can compile a summary of these findings into a new PDF and offers a succinct and focused overview of the essential details of the defendant's case.

As the AI navigates through the document, it gathers context, and then uses a separate reasoning module that sequentially answers questions based on the context received from the reading module. It’s important to note that this development process was supported by Prisonology providing NineTwoThree with hundreds of references featuring human-rated responses. These responses were instrumental in training the Language Learning Model (LLM), which was programmed to assign scores based on specific, predefined criteria.

In the final stages of this process, a score is generated reflecting the analysis – with the hope that it indicates a moderate level of severity. The LLM's responses are also made available for review, which allows Prisonology’s team to adjust scoring if necessary. Additionally, they highlight critical information about the defendant within the document, such as the type of drugs involved, the quantity of these drugs, any involvement of firearms, and the nature of their use, whether for brandishing or actual firing.

This system can compile a summary of these findings into a new PDF and offers a succinct and focused overview of the essential details of the defendant's case.

Impact

The AI-driven Proof-of-Concept successfully demonstrated its capability to assess at least one crucial metric, indicating a promising future in automating various other aspects of the scoring process.

This advancement suggests a significant leap towards enhancing efficiency and accuracy in the Security Designation Scorecard evaluations, propelling Prisonology and NineTwoThree's collaborative efforts in adapting legal processes and prison sentencing.