Drafting contracts safely with AI

Richard Mabey: A huge welcome to everyone who is joining us for this Juro webinar on drafting contracts safely with generative AI. We've had several hundred GCs, legal counsel, legal ops professionals sign up for this event. So there's a huge amount of interest in it and it's quite a wide and broad-ranging topic.

We're going to cover quite a lot together in the next hour. So what are we going to cover in this webinar? We're going to start with a general overview of generative AI and the interest that we're all seeing in this topic across in-house legal teams. I'll talk through that. And then I'll hand over to Laura Jeffords Greenberg, who we're really fortunate to have here, who has been not only the interim GC at Wroksome, but also experimenting very heavily with ChatGPT. She's going to give a handful of really concrete examples of what legal teams are doing with this technology.

We have with us our general counsel, Michael Haynes, GC here at Juro, who is going to show a live demonstration of our new feature AI Assistant. So we can give a bit of context to the group on how we at Juro are thinking about this technology and actually what you can do already today with our product. And then we're really fortunate to have our customer Lars Krooshof here, who's going to talk a bit about the value that he's been getting out of AI Assistant.

We have had a lot of live questions submitted in advance. And so there is a live Q & A here. If you go to the questions tab, you can submit questions in real time. We will try to get through as many as we can. But equally, there's so much talent here in this room that if you just put some comments in the chat on anything you're thinking about, any views you have, we'll also be super grateful to leverage the wisdom of the group.

OK, so let's start with why is this a topic? So why are we all here?

Why is AI used for drafting contracts?

So the short answer to this question is that actually this is a topic that is not particularly new in legal or legal tech. Over the last probably seven or eight years, we've had a whole bunch of legal tech companies arising. There's been a huge narrative around artificial intelligence. And really ever since the kind of early pioneers of this like Richard Susskind were talking about this topic, we've seen just a whole bunch of interest in it.

If we go to the next slide, if you look at the first generation of generative AI, generative AI really has been a pretty new thing, but actually AI and legal has been occurring since about 2013.

So in 2013, if you think back as GCs, you probably had a lot of outreach from vendors. And when you headed to their websites, they would usually have one of these awful pictures of a robot holding a gavel. And there was kind of a lot of theoretical promise behind AI.

Now, over that period, say 2013 to 2022, there's been fairly limited adoption of AI across legal. It's not nothing, and I know there's a number of people on the line who have worked with vendors and pioneered this, but actually widespread adoption has not really happened.

So why is this? Let's head to the next slide. So there are three key reasons. Really, the first generation of AI, this is prior to this whole generative AI thing coming along, suffered from accuracy issues.

We lawyers rightly demand a very high level of accuracy and we're right to do so. It's part of our job. But really these models were spitting out accuracy somewhere around 90%. And that meant that there was a huge amount of checking that needed to happen in order to get value. And that really started to grate at the time savings that were promised.

The second thing is, especially in contract use cases, you really had to invest a lot of time in training the models yourself. This might mean taking a data set of contracts, using that to train an AI which could then become useful, but there's still a lot of manual effort involved.

And then the final thing is actually the use cases of AI in 2013 to 22 period were relatively limited. So there's a lot of stuff around extracting unstructured data from PDFs. But actually some of the jobs to be done by legal teams were not really covered at all.

So what has really changed in this world? Let's go to the next slide. So the way I think about generative AI is it's kind of the difference between having a bad trainee and a good trainee. So with first generation AI, you would have to basically get that first part of the work done, but then spend a huge amount of time checking and verifying it.

And in the end, often there wasn't a huge time saving. And I remember when I was a solicitor, I had some very, very good trainees and also some not very good trainees. And the not very good trainees ended up not only not saving you time, but because you had to kind of redo their work every time, it was actually increasing the overall workload.

Whereas when you get that magic trainee who really knows their stuff, really is incredibly diligent, it saves you a huge amount of time when it comes to document review and other things.

Save time on drafting contracts

Richard Mabey: So come 2023, and let's go to the next slide, enter generative AI. And this we think has really changed the game. And for legal teams, we think it's changed the game for three reasons.

Reason number one, it's much more accurate. So it's really a step change more accurate than what we've been able to do in the past. And the reason for that is that the large language models or LLMs that are underpinning these technologies have been trained on much, much larger data sets. So it's the difference between having been trained by 10,000 legal contracts or having been trained on the entire internet.

The second thing is these models are really out of the box. So we can talk a little bit about how they can be fine tuned and adjusted for legal, but for the most part, you can get a lot of value from tools like ChatGPT without having to do all that manual training.

And the third thing is generative itself really means creative. So we are also able to find use cases where the creation of the text is paramount. So this includes, of course, things like drafting, but actually lots of other things that Laura is going to talk about in a moment.

So there's obviously a lot of excitement behind this technology. And one reason for that momentum is a report that came out from Goldman Sachs not so long ago.

And they basically stack ranked the industries that they thought were going to be helped or disrupted by generative AI. And legal came out second. And the claim they make, and who knows how they can be quite so specific, but the claim they make is that 44% of legal tasks lend themselves to automation with AI.

And again, I think this is challengeable. I think there's a debate we can have here. But in general, that's what's throwing a lot of venture capital behind this, but also is generating a lot of interest from GCs, legal counsel, legal ops professionals in the topic.

The other thing if we go to the next slide, I think is interesting is we've kind of moved beyond a really small, small core of very forward-looking tech lawyers experimenting with this technology to becoming more mainstream.

And this is an example I really like from the UK. So an English judge called Lord Justice Burr recently used ChatGPT to summarize some key facts in a case. And he described it in a delightfully British way as jolly useful. So even the judiciary are starting to embrace this technology.

So let's head to the next slide. We are also not only beneficiaries of the technology, but we also play special roles as lawyers, as risk guardians. And so the other huge topic which we'll address in this webinar are the risks that go alongside the use of generative AI.

And really they fall in three primary buckets. The first is of course, privacy. So if you're using tools like ChatGPT and throwing in personal data, what is happening to that data? Is it being used to train models? Where is it going? Is it going to the EA? Is it going to the US? All kinds of questions that I'm sure we're all starting to get familiar with.

The second is confidentiality. So there are lots of B2B use cases. And the problem with that is B2B data is often containing sensitive information, may contain company secrets, for example. And again, if you put that into a model in a contract, whatever it is, what is actually happening to that information?

And the third is really accuracy, which is, I think we're all familiar with the news stories of the less rosy kind coming out where people have been caught out in using generative AI. We all know about the hallucinations that these tools can give. So how can we get comfortable as a legal industry that we're going to get the accuracy that we need?

So we're going to answer those questions in a bit more detail when we come onto the demo. We're here from Michael and others on this in a bit. But before we get there,

I'm really happy to have Laura here, who's going to talk through, actually, in the weeds, what are GCs doing with generative AI.

Drafting contracts safely - how to get started with AI

Laura Jeffords Greenberg: Great, well, thank you for having me. I'm excited to talk about how I'm using ChatGPT in my everyday work as legal counsel.

So I'm going to take you on a little bit of a different journey. I'm going to give you an idea of what's possible. In some of the other talks I've done, I do step by step. But this is more of a tasting menu or greatest hits of what you can do. So in some instances, I will show you the prompts that I use and the output. In other ones, I'll just show you the output.

So we can move to the next slide. Before I start with a problem, I like to ask, can ChatGPT help with that? Because ChatGPT is amazing, but it doesn't solve all our problems. So normally, I'll ask myself three questions. What problem am I looking to solve? Is ChatGPT the right tool? It may not be. And can I verify the output? If I can't verify the output, then it's definitely not the right tool to use, because we're not there yet.

It needs verification for the output. When you're approaching prompting ChatGPT, there are two ways to do it. One, you can give it very detailed instructions, you know, step by step, for example, if you were giving it to a trainee or an intern or a first year or summer associate, or you can ask it to do something and ChatGPT will tell you the step by step instructions of how to do that.

So in these 10 minutes, I'm going to go pretty quickly over four main areas that I'm using. One is proofreading. The second is brainstorming. The third is visualizing data. And the fourth is analysis.

And then the super secret one that's not on there that I'm experimenting with right now is actually having ChatGPT write code for me and becoming a legal engineer, if you will, a lawyer who has some engineering on the side.

So first, the first problem I have that ChatGPT can help with is I need help proofreading.

Laura Jeffords Greenberg: So I drafted a mutual nondisclosure agreement that we really wanted to give to the sales team that they could run with and then we would never see again. And you can see here that I could have been a little bit more detail oriented. So I gave it, I uploaded a mutual nondisclosure agreement and said, can you proofread it for me?

And you can see that it's actually come back, identified the sections, and then told me where I can make changes for improvement. Now you can do proofreading for grammar. You can also do it for style, consistency. You can translate it into plain English, which I really like. You can also do tone of voice or voice. Sometimes I've asked it to write in the voice of Ruth Bader Ginsburg. So you can test out things like that that are kind of fun.

You can also target it for a specific audience. Say, I'm writing this for the CEO. I'm writing this for the sales team. And it will be able to translate that language for you. It's really great. If we move to the next slide, I can show you some other ways that I'm using it.

Another one is brainstorming. So I need help generating ideas. So for this talk, I actually just went in and said, hey, I'm giving this talk. Here's the title. Here's my section. Can you come up with some ideas and give me some use cases and then practical examples that I can share with these attorneys? And then I asked it to put it in a table format.

So I really like telling ChatGPT what format it can be in so that I can use and consume that information easier. So you can see the different suggestions that it came up with for brainstorming or generating ideas. We go to the next one. Next slide. So this was a little bit more complicated. The problem is I need to figure out how to present contract data visually.

Contract data visualisation

So imagine that I'm going to have to give a presentation to the CFO and the CEO to provide an overview of where we are with all of our contracts based on key metrics. How do I take that information and then get it to do a quick visual so that I can present it to the leadership team.

Well, instead of taking my data that we have, I actually use ChatGPT to create dummy data for a presentation. So I went in and I prompted it and I said, hey, this is what I want to do. Can you create me an Excel sheet that has contract information in it that I could use to then generate visuals?

So if you go to the next slide, you can see the response that we had there.

Laura Jeffords Greenberg: So I just said, hey, I want contract data. And it pulled out contract ID, party name, type, start end date, contract value, and status, which is great. And then it said, you want me to do more? And I'm a lawyer. So I said, yes, because I can't just go with the first draft. I have to add things to it. So I added a couple other variables and then told it how many rows of data I wanted. And it exported an Excel sheet for me, which you can see in the next slide.

In the next slide with the, yeah, there we go. So I got this Excel sheet and you can see it just made up all this random information. So I have that. So now I actually go to ChatGPT-4. So I'm using plus with where you can actually upload documents.

So then I went to it and I uploaded this Excel and said, hey, help me visualize this data. And I think in the next slide, you can see the prompt that I actually used, which again was just I'm presenting to the CFO and CEO and there's five variables that I need to present to them on. How do I put all these five variables together? And then can you visualize something out of that?

So if we move to the next one, maybe, yes, perfect. And then we can skip over that one. So then you can actually see. So what it did is now I'm giving it a big task and it breaks it down into different instructions. So it's telling me this is how I'm going to break up the data based on the information that you've given me.

So you can see how it's actually manipulating the data, which is great for verifying output. And if you go to the following slide, then you can actually see the visual output that ChatGPT created. So based on the five variables within ChatGPT, I got out these different graphs. So I'm looking at limitation of liability, distribution of contract values, et cetera.

And then you can download them individually, put them in a PowerPoint, do whatever you want. It's pretty cool. So the next problem, because I'm trying to move quickly here, is that I need to answer a question about a company policy. So I upload a company policy document to it. And I say, hey, this employee has asked me whether or not they've been asked to be on a nonprofit board to make a mobile game. I pretended that I was a company that made mobile games.

Laura Jeffords Greenberg: And can this employee serve on the board of a nonprofit that makes mobile games according to our company policy?

So if you jump to the next slide, you can see it actually has a two-step analysis. So the first is that here are the key points. Who does this policy apply to? What is the actual policy? And then the second step, which is in the next slide, is looking at the definition of competitors and it provides an analysis of competitors and then gives me my advice at the end.

And this is exactly the same kind of legal analysis that we would provide as lawyers, right, going through breaking down the policy. And because I actually had ChatGPT create the policy for me, I know I can verify the output and it was accurate and spot on. Then we've moved to another problem regarding analysis. I've actually used ChatGPT in contracts. So this is actually from our work from contracts, which are available online. So I'm not worried about putting them in there.

I took, actually, if you just go to the next slide, you can skip it. This is the example. The next slide actually has the prompt that I used. So what I said is I'm an in-house lawyer. Here's our summary of our representation of warranties. Here's the clause. The other side wants to strike this clause from the agreement. Here's their comment that accompanies their need to strike the clause.

I, on the other side, as the vendor, want to keep it in the agreement. Can you give me as many arguments as possible to keep this clause in the agreement based on my position as the vendor? And if you move to the next slide, then you can see the output and then the touches that I put on it. So it gave me about nine arguments. I thought half of them were not persuasive. Not very good.

And then I went through and then made, you know, compared them all and pulled them together, put them into ChatGPT and said, hey, I now need to put this in Microsoft Word. Please summarize this and make it, you know, short and succinct so that I can put it into a Microsoft Word document. I did that and it was accepted. So that was a great use of it.

This is an excerpt from the full transcript. To watch the webinar in full, click the preview at the top of this page.