The Art and Practice of Prompt Engineering

A Step-by-Step Guide to Crafting a Good Prompt and Setting the Right Parameters

Are you tired of getting subpar results from language models? Do you want to know how to get the most out of your LLMs? This blog post will walk you through crafting a good prompt and setting the correct parameters for your LLMs. We will also provide some tips on how to troubleshoot your results.

By Indira Kumar

Crafting good prompts for Large Language Models is an iterative process.

Ask anyone these days, and they would say that ChatGPT is a model that can churn out answers, solve problems, write poetry, generate a text, summarize it, and much more. It has become a valuable tool for many to accomplish tasks that would otherwise consume an incredible amount of time.

To a layman, the ChatGPT is a unique, impressive, outstanding, mind-blowing model that can do impossible tasks in a jiffy. Still, the truth is that it is an interface that takes the response from the chatbot and sends it to a large language model (LLM) in the background. The LLM works on probabilities of words and the predictions that follow, which are then sent back as an output to the chat window.

So, what if you didn't have an interface that understood your need and still had to get the required result? To answer this question, I created this write-up that focuses on prompt engineering and the parameters you can adjust to get the desired output.

What is prompt engineering?

Prompt engineering is the process of crafting a prompt that will elicit the desired output from an LLM. A good prompt will be clear, concise, and informative. It will also give the LLM enough context to generate a relevant and coherent response.

Crafting a good Prompt

Crafting a good prompt is a process of getting to a point where the prompt you give the model provides a satisfactory result. Let us look at some sample prompts and their outputs.

1. Getting started: Unless a model is fine-tuned for the task, merely posing a question or specifying an instruction doesn't necessarily give the desired result. It results in sentence completion and nothing more.

2. Use cues: Adding a cue fetches a structured response.

3. Provide examples: To get a higher-quality response, provide an example of the kind of response that you want.

4. Use descriptive language: Include descriptive details for more guidance and better results. The first example in the picture becomes a reference for the model while it generates the output.

Use descriptive language when writing prompts for LLMs.

How to set the parameters

Now that you have crafted your prompt, you can set the parameters to refine it even further. The parameters control the length, style, and content of the output. Here are some of the most critical parameters to consider.

1. When you find that the response is too long or short, you can adjust the length of the LLM output by setting the minimum or maximum tokens. A token is a collection of characters with semantic meaning for a model. It can be equivalent to a word or part of a word; hence increasing the number of tokens results in significantly more generated text and vice versa.

2. You can set the decoding parameters depending on whether you want a creative or a predictable output.

Greedy decoding: This setting tells the LLM to select the most likely word at each step. This can result in a more predictable output but can also lead to repetitiveness.
Sampling decoding: This setting tells the LLM to select a word from a range of probabilities. This can result in a more creative output but can also lead to less coherent output.
- Temperature: This parameter which can be set between 0.00 and 2.00, controls the randomness of the output. Increasing the temperature results in low-probability words being picked; hence more variability is seen in the outcome.
- Top P (nucleus sampling): Refers to selecting the smallest set of words whose cumulative probability exceeds p. It is based on nucleus sampling that shortlists the top tokens whose sum of likelihoods does not exceed a specific value. This parameter can be set between 0.00 and 1.00.
- Top K: Refers to selecting k words with the highest probabilities at each step. Higher values lead to more variability. The tokens with the highest probabilities get picked, thus creating probable yet meaningful output. Setting top p & top k ensures that we iron out the less probable and, at times, less relevant tokens from getting added to the resultant text. This parameter can also be set between 0.00 and 1.00.

3. Increasing the temperature can sometimes resolve the problem of a text being repeated. However, when text is still repetitive even after setting a higher temperature, you can add a repetition penalty that can be set between 1.00 and 2.00. The higher the penalty, the less likely the results will include repeated text.

4. In situations where the model returns multiple answers, the output can be restricted by using stopping criteria like adding two carriages. This step restricts the output generation to one by telling the LLM to stop generating more tokens.

5. Random seed is yet another parameter in LLMs which can be set to ensure that the output you have generated for specific criteria is reproducible. If the same seed is used, the model returns the same output.

To summarize, if you want to generate a short, predictable output—limit tokens, enable greedy decoding, and decrease temperature. Vice versa, if you want a long, creative output—increase tokens, set a higher temperature, enable sample decoding, and use top p, top k for relevant outcomes.

Troubleshooting Your Results

If you are not getting the desired results, try troubleshooting your prompts and parameters. Here are some tips:

Try different prompts: If you are not getting the desired results with one prompt, try a different one.
Adjust the parameters: If you are not getting the desired results with the default parameters, try adjusting them.
Try a different model: Different models give different outputs. Try a different model if you are still not getting the desired results.
Rethink your use case: After following all the above steps, you need to rethink the use case if you still don't get a satisfactory result.

Conclusion

Lastly, there is more than one-size-fits-all approach to prompt engineering. Hence, It is an iterative process of understanding your need's complexity, the model you are trying, and its parameters.

You may need to experiment with different parameters to get the desired output. However, by following the tips in this blog post, you'll be well on your way to crafting the perfect prompt for your LLM.

Click here to read in detail about prompt engineering.