Adding the Stop Sequence

[00:00] Okay, so if we refer back to our slide, we can see that each completion should end with a stop sequence to inform the model when the completion ends. And when we were preparing our data, we actually selected this option to add a suffix ending of a new line character,

[00:16] which is backslash n. So as we add completions to our conversation, we need to include this new line character at the end as a stop sequence. Now, we're also using this arrow as a separator, and when experimenting with this, I sometimes found arrows popping up in completions, and some cases where the chatbot seemed to just

[00:36] get into a conversation with itself, asking itself questions and giving itself answers. So we're actually going to add the arrow as a stop sequence as well, because we never want the model to generate an arrow. Okay, so here is your challenge, and it comes in two parts. Firstly, I want you to add the new line character, which is backslash n,

[00:56] and the separator as a stop sequence. And actually, you don't need to have a space in the stop sequence. Secondly, I've said here, I want you to add something to the line below where we update conversation string. So that is this line of code right here.

[01:12] I'm going to leave you to think about that. What would it be good to add to that line? Okay, pause now, get that challenge sorted, and I'll see you back here in just a moment. Okay, so hopefully you managed to do that just fine. So let's come in here, and we'll add our stop, and we know that that will be an array, and it will be an array of strings. So the first

[01:39] string is going to be the new line character, and the second string will be the arrow separator. And of course, it doesn't matter what order they go in. Now, the second challenge, add something to the line below where we update conversation string. Well, if we refer back to

[01:55] this slide, we're going to add a suffix ending of backslash n to all completions. So this is the completion that we're getting back. Let's just add a new line character to the end of that. And what that is going to do is make it really clear to the model when it reads conversation

[02:13] string that it has reached the end of a completion, because that is what the new line character will tell it. And likewise, the arrow separator that we're adding to the stop sequence here, which we're already adding to conversation string right here in the event listeners anonymous

[02:29] function, that will tell the model that it's reached the end of a prompt. Okay, let's hit save and see if it's worked. So I'll start again just by saying hey. All right, and I'm still getting a pretty strange response here. Now, I'm going to ask a question from our data. And if we have a quick look at the data, we can see that the customer support

[02:48] team has a phone number. So I'm going to ask it, what is your phone number? Okay, and look, we're getting a much better quality response. It knows the phone number. It is still hallucinating. We're only here from nine to six 30 weekdays. That is not actually true. But the answer is much shorter, and it's much more manageable.

[03:11] Now I'm going to ask it a question it can't know the answer to, because it simply isn't in the data. I'll ask it, who is your CEO? Okay, interesting. It completely makes up a person, Richard Quing. So for some reason, the large language model has decided that Richard Quing is the most probable name for a CEO of our

[03:32] company. And it's also made up an email address for him, but using at wewingit.com. So again, it's a hallucination, but it's an interesting one. So I think where we are at the moment is that this chatbot works. It can see its data. It's giving us short, manageable answers,

[03:48] which makes sense, but we're still getting too many hallucinations. It is not accurate. Now we are going to get hallucinations, because as I said at the beginning, we're not really using enough data. We've got enough data to show the principle, but not enough data for a full production ready app. But that said, there is one last really

[04:08] important aspect of fine tuning that I want to tell you about. And it is an epochs. What does that mean? Well, when we were building the model, you might have noticed that as the model was fine

[04:21] tuning, it completed various epochs. And in this case, it completed four epochs. Well, maybe four epochs just wasn't enough. So in the next scrim, let's talk about what epochs are,

[04:36] how we can get more of them, and how that is going to help us. So let's check that out next.