We know how corporate enterprises are responding to their need to digitize their contracts and manage them more effectively and how artificial intelligence (AI) has revolutionized the human experience -- everything from retail shopping to customer service and legal work. But this requires humans interacting with AI to resolve complex and high-level decision making. For example, what happens if data input is incomplete, inaccurate, or misleading? AI alone often cannot detect or correct inaccurate data and our human “logic” might make things worse if either AI or human interpretation is mishandled. Most of us cannot solve this conundrum alone. This article responds to common challenges relating to human intelligence seeking to interact effectively with AI.
In February 2021, World Commerce and Contracting’s survey titled When Technology Meets Humanity – The Future of Contract Management1 revealed that 68% of responding contract professionals identified digitization of the contract management lifecycle as a medium or high priority. Why? Because digitization has shown time and again it is crucially important to achieving the speed, flexibility, and consistency that modern organizations are demanding from their contracting processes today.2
We’re seeing an ongoing debate among contract reviewers about which technology is best for performing contract review -- humans or AI based contract lifecycle management (CLM) plus contract intelligence technology systems. These discussions can be spirited, with each side assertively defending its turf.
Neither side is right or wrong, usually. Both humans and AI technology are essential to creating a modern, state-of-the-art contracting process. It’s more a question of how each resource is used, when and why.
We all understand that contract data is important. Contracts are the source of truth for our obligations and our vendors, clients, and others. We know that without clean contract data, companies can reap the following negative results:
- less productivity and growth - having poor contract data requires additional resources to clean up and neutralize its negative effects.
- rising costs - poor contract data costs money to remediate and, the costs of automatically renewing unwanted contracts or missing a manual renewal can be surprisingly expensive.
- missed opportunities - incorrect contract information can lead to missed opportunities to upsell agreements or renegotiate contracts with vendors.
To avoid these pitfalls, organizations need to evaluate when AI and humans operate at their best to effectively leverage the strengths and weaknesses of each type of resource.
If you start by taking a closer look at the values of speed, flexibility, and consistency you can more accurately determine what ratio of human and AI solutions will combine to achieve your goals. As we now know, AI does not fatigue as people do, but at the same time, AI is not 100% accurate. Plus, humans can achieve almost 100% accuracy with many layers of quality control in place although this requires extra diligence and time.
So, what does human mean and where are we today with AI plus humans in the contract review process? Without people, you have no contracts, no business! People have traditionally led contracting processes in all functions whether legal, sales or procurement. Sometimes the people leading contract review projects are the company’s own in-house lawyers and staff. Or the company retains outside help from alternative legal service providers (ALSPs) or managed services functions. Either way, without AI, contract review is a painstaking process likely to generate errors and slowness.
Crossroads of AI and human element - Historically, technology solutions have been regarded as useful tools that can be implemented or leveraged to help human-led efforts succeed. More traditional workflow models are definitely “people-first” with human resources leading the way and technology assisting them. People-first approaches -- while familiar and time-honored -- have their drawbacks. Looking through the lens of speed, flexibility and consistency exposes some difficulties that often arise from the people-first methods.
Speed - When people get tired or bored, their work productivity typically slows down (the law of diminishing returns). They are apt to make errors that must be cleaned up later. People may work with great accuracy or speed, but it’s difficult to achieve both at the same time. To achieve high accuracy, we often sacrifice speed because investing in too much quality control and extra diligence increases the time to review a contract.
Flexibility – People resist change and bring their own attitudes, intuitions, and personalities to their jobs which can blur their logical decision-making skills and inhibit their ability to adjust to different circumstances. A built-in desire to do things “their way” can negatively impact people’s flexibility.
Consistency - From a consistency standpoint, people struggle with completing similar tasks over and over and maintaining the same focus and enthusiasm with each task. Give people unlimited time to review a small number of documents, and they will likely produce a highly accurate result. However, ask those same people to review thousands of documents 40 hours per week over several weeks’ time and pressure them to make no mistakes and consistency will certainly suffer.
For example, consider what overwork does to you. People reviewing contracts first thing in the morning or at the beginning of a project are more fresh, less fatigued, but their productivity can drop precipitously as the work continues to late in the day or extends for weeks or months. People’s ability to produce quality results can plummet over time. Leaders can try to instill controls to prevent this from happening or red-flag issues when they occur but, the controls often fail for various reasons.
AI contract intelligence technology -- while not a panacea that solves all problems -- can complement people’s strengths and limit their inherent weaknesses to enhance speed, flexibility, and consistency. The key to making AI produce great work is to train involved employees properly.
AI training is much like training children about the world around them – a child is taught what words mean and the differences between them. A school bus and city bus are both buses but differ in some respects, and over time, the child learns to distinguish between the two. Similarly, an AI system must be taught to recognize contract clauses and their many permutations which can occur over time. For example, the following are three examples of Assignment Causes without the use of the word “Assignment”:
- Except as provided hereunder or under the Merger Agreement, such Shareholder shall not, directly, or indirectly, transfer, or consent to or permit any such transfer of, any or all its Subject Shares, or any interest therein.
- Company A shall not delegate its duty of performance or assign its obligations under this Agreement without the prior written consent of Company B.
- Neither this Agreement nor any right or obligation arising hereunder may be transferred, in whole or in part, by contract, by operation of law or otherwise, by a Party without the prior written consent of the other Party.
Data scientists -- usually the people responsible for training AI technology -- use sample data sets to teach AI and extract desired material from contracts. For example, AI data must include multiple examples of a clause -- perhaps across thousands or millions of documents -- and someone such as a data scientist needs to designate which contract clauses should be extracted. AI runs on algorithms which are trained on various documents and these training algorithms can be adjusted along the way to optimize results.
Despite all this complexity, here’s the good news -- once your AI app is sufficiently trained on a set of contract data, its performance within that data set is incredibly accurate and helpful to people. Also, AI does not get tired or bored, and it has no attitudes that skew or deplete its reviewing ability. AI can bolster efficiency of contracts, because people no longer enter data manually, then check and recheck the entries.
Answers at your fingertips
Photo courtesy of Deamstime.com illustrating a Linkedin.com article titled, The Future of Artificial Intelligence in the Intelligence Community)3
And my point is? Trained AI can score high levels of accuracy in the range of 95% on the sample data set where human review accuracy would measure closer to 70%. Corporations may be disappointed because they expect the AI to perform at 100% out-of-the-box, but this is an unrealistic goal. For data residing outside the sample set, AI’s accuracy may decrease, because the system is not trained on that data. With certain data sets, it may require adjustments to the AI and/or human assistance to attain accuracy over 90%.
The best AI solutions can perform live tests on unfamiliar data and score high without a great deal of training, but these outcomes are rare. Some solutions – while claiming they are AI -- involve humans supplementing the technology behind the scenes. Technology should supplement human review, not the other way around. So, as a customer of AI considering a purchase, you are well within your rights to ask AI vendors to perform live tests with their own documents before you buy, so you can see the speed, accuracy, and reliability of the AI solution in real-time action without human assistance in the background.
AI runs into trouble when documents are not easily readable in electronic format. Some companies maintain contract records consisting of old paper or image-only scanned documents dating back several decades where contracts were handwritten, not typed. Other documents have been damaged over time. Legacy contracts can be blurry, stained or torn, with pages missing or words smudged.
Due diligence -- a must for document handling!
First, both human and AI review functions must have optical character recognition (OCR). No AI system can accomplish proper reading of a scanned document without a solid OCR capability.
When selecting AI for contract management, corporate clients may think they are buying a fully automated technology system that does not involve people at all, but that assumption is generally incorrect. So, watch out for AI technology providers that rely more on people than they reveal to their clients. Most AI tech providers hire people to review documents behind the scenes, even if that process surrounds only quality control or review of documents that cannot be processed by the AI or OCR technology.
Be aware of compliance regulations worldwide. Some AI providers employ people in other countries but are not up-front about this to their clients. Down the road, clients using these AI providers may find they have unwittingly violated regulatory compliance laws such as the UK/EU General Data Protection Regulations (GDPR)4 or U.S. state data privacy laws such as the California Consumer Privacy Act (CCPA).5 Therefore, corporate clients must be circumspect and do proper due diligence when selecting AI providers. They are well-advised to ask questions not only about the AI but about the human review aspects of the AI provider’s tech deployment.
To summarize -- because many contract data points need to be 99.5% or 100% accurate, AI data abstraction still needs some human review to make sure data is at the highest level of accuracy. Additionally, there has been some distrust about AI in the contract management space as many have touted the power of AI in their systems without much actual success. Buyers of contract management systems should know that AI today is more powerful and can pull out relevant and distinct metadata automatically. Instead of shying away from these types of systems, prospective contract management software buyers should test any AI contract management system “live” on a demo to see how accurate the AI is. If the AI system doesn’t work on day 0, then the system will not work on day 1 and beyond.
END NOTES
- Survey results report: When Technology Meets Humanity – The Future of Contract Management Deloitte and World Commerce & Contracting
- What is AI-Based Contract Management? – Cobblestone article
- The Future of Artificial Intelligence in the Intelligence Community, LinkedIn article by Professor Jeffrey Rath, Feb 1, 2021, illus: Deamstime.com image.
- Information Commissioner’s Office (ICO): Guide to the UK General Data Protection Regulation (UK GDPR)
- California Consumer Privacy Act (CCPA)
ABOUT THE AUTHOR
Patrick Won’s track record as an experienced Manager reflects his history of working in the legal services industry. Specializing in contract lifecycle management using technology and process optimization, he is skilled in Legal Operations Management, Corporate Law, Management, Contract Management, and Project Management.
ABOUT EVISORT
Evisort is on a mission to change how companies interact with their legal documents. Founded out of Harvard Law and MIT, Evisort has been backed by leading early-stage venture funds. Evisort works by using advanced artificial intelligence models to help companies organize and understand the important business data contained in their legal documents. With Evisort, information locked away in documents becomes searchable, and key terms can be surfaced to the right people at the right time. Evisort can be used across all documents across an organization and helps companies reduce costs and improve their compliance and business operations.