Prompt Engineering is a very recent concept, widely used to optimize interactions with large language models (LLMs)
When ChatGPT was first introduced, my initial approach was to ask questions in a straightforward manner. I didn’t think about providing examples, setting the tone, or even to incorporate some characteristics in the request (like acting as a customer support agent specializing in claim handling).
I always found weird the usage of the word engineering on an activity that is hard to measure and quantify, many times its counter intuitive, and other times simply not consistent.
Having said that, it still seems to bring a lot of value when we are struggling with getting the proper response from the LLM.
Prompt engineering is the continuous and iterative task of designing and optimizing prompts to guide language models toward generating accurate and desired responses.
Let’s have a look at some of the most common techniques
Zero-shot learning
I believe this is what everyone does by default, which is simply asking the question without including any examples within the prompt. This approach relies on the existing data within the trained model to be enough to provide the desired response without further efforts.
K-shot learning
Prompt Construction with Demonstrations: Instead of simply asking the model to perform a task, you include example inputs and outputs (demonstrations) within the prompt. These examples clarify what you want the model to do.
For instance, if you want the model to provide information about books, you could prompt it like this: “What was the first book of the Foundation Trilogy? Please include the release year and the rating.” This helps the model understand the specific task it is supposed to complete, guiding it to return relevant details such as the book’s title, its release year, and the associated rating. These examples help the model understand the specific task it is supposed to complete.
The term “k-shot” refers to the number of examples (or “shots”) provided in the prompt. So, if you use 3 examples, you are performing 3-shot prompting; if you provide 1 example, it’s 1-shot prompting, and so on. The more examples you include (i.e., the higher the “k”), the more context the model has to understand the task. However, too many examples can make the prompt too long and might lead to diminishing returns in terms of performance improvements.
This shows very clearly how a carefully thought prompt impacts the output. you may not get the desired response on the first attempt, but iteratively refining it has a high probability of reaching your desired outcome.
Below an example of a 3-shot prompt.
As mentioned before, If we dont include examples in the prompt is called zero shot prompting. The general consensus around this is that few shot prompting tends to yield better results.
Chain of thought prompting
The idea behind chain of thought is that when tackling a complex task we guide the model to break down the solution into smaller, manageable steps. This approach mirrors how we would solve the problem ourselfs, breaking it into clear pieces to make it easier to handle.
Keep in mind that an LLM does not actually perform mathematical calculations, it still uses probabilities as to which is the next best token. In this case the above answer is correct, but in a situation where it is not, we can help the LLM with this chain of thought technique, in that case it would guide the LLM to a good output.
To be honest, I haven’t used this approach at all—maybe the type of answers I usually look for don’t require this method. Yet, this method appears in most, if not all, places where prompt engineering is being discussed. The interesting thing here is its flexibility: some scenarios require complex, structured prompts, while others benefit from simple guidance.
The key is knowing when to employ these techniques for maximum impact, as not every problem calls for the same level of sophistication in the prompt design.
And most importantly, a prompt designed for one specific model won’t necessarily work the same way on a different model.
Role-Based Prompting
This is a technique i see often in the projects I am involved with. It’s simple, but it profoundly impacts the responses.
Role-Based Prompting is a technique where the model is assigned a specific role or persona to guide its responses. By defining the model’s “role,” you provide context that influences how it processes and generates information. This can be useful for tailoring the model’s answers to specific needs, such as expert advice, a particular tone, or a target audience.
See how such a simple role based prompt can alter the entire tone of the response.
describe a pastel de nata
As a Michelin Star food critique, describe a pastel de nata
As a 5 year old, describe a pastel de nata
Use LLM to create JSONs
Another useful way of role based prompting is when we use LLM’s to feed data to a particular application (very generic statement I know). Let me show you an example.
We need to make an API call to a backend system, and the input for this call will be derived from a voice conversation. First, we can use OCI Speech services to transcribe the audio into text. Then, we can leverage the OCI Generative AI service (our LLM) to extract the necessary information from the text and map it to a JSON payload, which will be used to trigger the specific API.
Act as an Integration developer. Please map the values from the text into a FHIR JSON Payload for the API that updates the Vitals on Cerner, and provide only the JSON in your response.
The above prompt is very simplified. In reality we would need to provide the JSON example to make sure we get exactly the structure we need.
The output JSON is quite long, but you would get something like.
This would bridge the gap between unstructured data and structured data in the integration realm, making Integration simpler. This is just an example of the possibilities in this realm.
Wrap Up
There are many more prompt engineering techniques but the above ones are some of the most relevant. In my personal experience the k-shot (or few shot) prompting and the role based prompting are the two I see the most, and are flexible enough for multiples uses.
As mentioned before, each model responds differently to the same prompt. With the wide variety of LLM models available, and frequent updates or model retirements from even the same provider, it’s essential to continuously maintain these prompts to ensure consistent response quality.
Leave a Reply