The Separator

[00:00] So, we have the fine-tuned chatbot working, or sort of, because actually it's giving us nonsensical gibberish. So, let's just check the settings we have. The presence penalty and frequency penalty aren't doing anything very much, they're almost at their default, so I'm not too worried about them.

[00:18] Now with OpenAI, when the completions are a bit weird, the first thing we might think of is temperature. Remember temperature controls how daring the model will be, how creative, how inventive. Now, we don't want creative, we want factual, so let's put the temperature down to zero.

[00:36] And let's see if that has had any effect. Okay, that completion is like something from a horror movie. Now the chatbot is claiming to be a human, it's begging us not to kill it, and it's just saying, please, please, please, and then creating a word so long it actually breaks our CSS.

[00:56] So, I'm just going to refresh to get rid of that, because it's just a bit scary. Okay, so the temperature hasn't magically solved the problem, but I think a low temperature will be useful, so I'm going to leave it there. Now before we do anything else, let's go back to the criteria for the format of our data.

[01:14] The first one that we had says each prompt ends with a separator to inform the model when the prompt ends and the completion begins. And actually, when we were prepping the data, it told us that it would use this arrow as the separator. So what we need to do is add the separator to the end of our prompt.

[01:33] And I've got the JSONL file right here, and we can see that each prompt does indeed end with a space and an arrow separator. Now we're not adding those separators to our conversation string, and I think that might be causing part of the problem.

[01:49] So in fact, I'm just going to log out conversation string right here. And let's be brave and just see if the chatbot is going to scare us again. And it's done the same creepy thing again.

[02:08] Now let's just have a quick look down in the console. And what we can see there, if I just cut and paste it into the editor, is that we're just getting one continuous line of text. So hey, and I'm not a robot, look like they're being said by the same speaker.

[02:23] But in fact, that was our prompt, and this is the beginning of the completion. So we have got problems there, and that brings us to a challenge. And your challenge is this. I want you to add the arrow separator to the end of our prompt as it is added to conversation string.

[02:42] And I've put here a space before the arrow separator because that is how it's formatted in our JSON-LData. Okay, I'm going to leave the console.log that I just added. So when you're done with this, you can just type something into the chatbot and give it a test.

[02:57] Pause now, get that sorted, and I'll see you back here in just a minute. Okay, hopefully you managed to do that just fine. So I want to come down here and put this in backticks.

[03:14] And now we'll use the dollar sign and the curly braces so we can access the user input dot value. I'm going to start with a space, and I'm going to end with a space and an arrow separator. Okay, let's give that a try.

[03:32] Okay, so we've seen an improvement there. This is by no means perfect, but at least it's working a little bit. Now it feels like we're talking to a representative of the WeWingIt company. It's got the email address. It's still giving us a load of hallucination and weirdness.

[03:51] And if we look down in the console, we have at least got our separator between hey, which was our prompt, and oh dear, which was the beginning of the completion. So we've made some small progress here. But what else could be going wrong? Let's just go back to our data formatting criteria.

[04:09] So we've sorted the first one. Secondly, it says each completion should start with a single white space. Well, if we have a look down in the console, that appears to be true, or is it? I think the next thing that we should do is come down here to fetch reply, and where we're

[04:27] updating conversation string with the completion, we should explicitly add a white space at the beginning. So again, I'm going to put this in backticks, use the dollar sign and curly braces to access the completion from the response, and now I'll add my white space at the beginning.

[04:47] Okay, let's hit save and try that one more time. And that's not really made any improvement. I think that's just as crazy as it was before. But we have now got that white space there for sure. We know it's there.

[05:03] We've ruled that out as a potential cause of problems. So let's go back to the data criteria and look at the third one. Each completion should end with a stop sequence to inform the model when the completion ends. Well, we're back with the mysterious stop sequence.

[05:20] We keep coming across it, and we've never really looked at it in detail. So why don't we deal with that next?