What is Prompt Engineering when it comes to AI?

There are a few different ways that you can go about training an AI model. Providing custom data is imperative to getting great outputs. There are multiple ways to do this successfully and ultimately, the model is going to produce content based on this data so the more specific you make it for your own needs, the better. This is why we always advocate for custom models for users who are looking to get an edge over anything else on the market simply because this will be the best way to get the AI producing content with your voice and tone. You are less reliant on a more generic model. In this post, I want to go over prompt engineering. What it is, what it can be used for and the basic caveats to help you better understand what goes into training AI models.

What is Prompt Engineering?

Prompt engineering involves providing custom data into the promps of the AI model. You'll find that most technologies will give you a way to provide some instructions to the AI on what you want it to create. Some of these models will work if you simply write 'Write me a product description' but the AI would then be unsure of exactly how you want that product description to look like. Prompt engineering is all about teaching the AI the pattern and process of what you expect it to output so instead of just writing the instruction for a Product Description, you may then give it a few examples.

The examples are super important as they enable the AI to see a pattern and understand it. The importance of both the input fields and the output fields are not to be underestimated. The input fields teach the AI that we are looking to receive these inputs in a specific number of fields, for product descriptions this is usually two. The output is then provided below these inputs so the AI can see this pattern and follow it in the future. 'I have received two inputs and I have written an output for a product description'. Follow this again and you reinforce that idea for the AI. Follow it again and the seed you plant in the AI just gets stronger and stronger. This is the power of prompt engineering.

Why Use Prompt Engineering?

Now you know what prompt engineering is, you should know when and when not to use it. In this section we'll go over the advantages of prompt engineering. Prompt engineering is extremely powerful and works best when you are dealing with short, sharp inputs and outputs. The less text you have to put into your input fields and the output will mean that you can reinforce the pattern a lot easier due to the ability to put in more and more examples. The longer the text or output you'd expect to create is, the more difficult it is to get a good output from prompt engineering when using AI.

That isn't to say it is impossible to get a decent output with only giving one or two examples through prompt engineering, it just won't be as finetuned as if you could put that data into a custom model and really give it a lot of examples to work with. If you are just looking to create something fairly quickly and prototype out ideas then prompt engineering is the way to go. If you are looking for a more specific and finely-tuned model experience then creating a custom model with hundreds or thousands of examples is going to be the more optimum path to follow.

The Disadvantages of Prompt Engineering

There are two main disadvantages of prompt engineering that you need to keep in mind. The first is length of the prompt. Most of the AI providers will not allow you to make a prompt over a certain number of characters long. This can be extremely limited, especially if you are dealing with longer texts. This is why summarization and rephrasing models tend to be highly limited because those tools require a decent amount of prompt engineering and also whatever you add as an input adds to that prompt so it can quite quickly max out the total prompt limit. If you feel that you are going to be maxing out a prompt limit then it might be worth to go towards the fine-tuning and building out a custom model / dataset.

The second disadvantage of prompt engineering is cost. Most AI providers charge for both the prompt characters and the output characters. The longer you have the prompt, the more you are charged as the prompt is likely to be larger than the actual output. Maxing out a prompt with examples will get you better AI outputs but it will cost you more. Providing a prompt with no examples may get you some usable outputs and be a lot cheaper but the results compared to a delicately tailored prompt will not be the same. If you have a large prompt and also set the output limit to be high, then you are going to have a model which is fairly expensive to run.

This is all a delicate balancing act. You want to provide quality models and outputs that your users can use but you also understand the need to save money. There is a lot of ways to play this further but as a basic introduction, I hope this has given you some food for thought about how AI models are created and how they can look like with regards to prompt engineering.

Fine-tuning (Custom Models)

The difference between prompt engineering and fine-tuning custom models is that with custom models, you are collecting and curating a wider dataset. Usually, these datasets contain more than 100 examples. This is compared to prompt engineering where you are looking at possibly 5, 10 or 20 max examples on shorter length models. Considering that the dataset is on a magnitude a ton more powerful than prompt engineering, these fine-tuned models are going to create better results and potentially cost less.

The main issue with custom models is to create the datasets. This is a slow and tedious process. Once you have a dataset though, you can upload it into the AI provider you are using and then call on it to run. Due to all the data already being now in the back, you do not need to provide any information in a prompt at all. This makes it cheaper to run and gives users better outputs. It is a no brainer where the future of AI will go and it also shows the importance of quality datasets and how they are the real item of value in this whole process.

Of course, there are a few other things to consider with models such as the Temperature, Top P, Frequency Penalty and other toggles but generally without a strong data backend to a model, your outputs will not be as good as they can be. Content Villain has of course been working with users on custom models since our inception and we love helping out real businesses with their content problems. If you'd like to discuss a custom model with our team, reach out to us today!