Improving Your Prompts (And a Prompt Coach for You to Use)
Becoming good at prompt design is a key skill for those using large language models.
Large Language Models (LLMs) like ChatGPT have no user manual. Nobody is really sure what’s going on inside them, and as a result, nobody is really sure what they can do or how they can best do it. Because of this, using them can feel a bit like navigating a maze without a map. We’ve found, though, that this journey is less about following a set path and more about discovery and experimentation. This is particularly true of prompt design, and “prompt engineering” is an essential skill to develop.
Crafting good prompts is part science — it’s good to know a bit about how these models work and what’s going on under the hood — but in our experience it’s also very much an art of understanding the subtleties of language and how LLMs respond to different cues. Paying attention to the process, noticing how a change in an instruction changes the output, keeping records of prompts that work well and instructions that improve quality, have all become part of our process in using these tools.
There are, though, other resources. Twitter is full of folks offering prompts, and OpenAI recently posted their Prompt Engineering Guide. It’s worth reading (even though developers, rather than users, are the audience for some of it). The guide outlines strategies and tactics for optimizing interactions with LLMs like GPT-4 and includes helpful examples. Key tactics from the guide include:
Writing Clear Instructions: This involves being explicit about what you want the model to do. Tactics include specifying query details, adopting a persona, using delimiters, detailing task steps, providing examples, and specifying output length.
Providing Reference Text: Since models can generate inaccurate information, especially on esoteric topics, providing reference texts helps in getting more accurate and less fabricated answers. You can also provide examples of output you desire and ask it to reference or mimic the examples.
Splitting Complex Tasks into Simpler Subtasks: This approach reduces error rates by handling complex tasks as a series of simpler tasks.
Giving the Model Time to “Think”: Allowing the model to process and ‘reason’ can lead to more accurate answers, similar to how humans take time to solve complex problems.
Using External Tools: This includes using tools for knowledge retrieval or code execution to supplement the model’s capabilities.
Testing Changes Systematically: Systematic evaluation of changes to prompts helps ensure improvements are effective and not just coincidental.
In addition to this guidance from OpenAI, our team has picked up some tactics of our own along the way:
Always ask the model to adopt a persona or act as having expertise in something specific, and always give it a specific task (“You are an expert in X. I will provide you with Y and your task is to Z …”).
Tell the model to take a deep breath and think step by step. If you want it to follow a specific process, name the steps (“FIRST, do X. SECOND, do Y …”).
Be nice. Say “please” and “thank you.” Really. Encourage it. Tell it, “You can do this!” if it seems to be struggling. Really.
That said, be willing to ask the model to do something more than once. Be willing to say, “That’s not good work at all. What happened?” If it gets something wrong, tell it where it was off-base and ask it to correct or adjust.
Use ALL CAPS if you need to be insistent about something.
Specify desired length of output in paragraphs or bullet points, but not words.
You can attempt to limit hallucination in text creation with this instruction: “Cover the following specific points without deviating from the topics mentioned and not writing any fact which is not present here …”
Ask the model to first work silently, review its own work, and improve that work BEFORE giving you the output. Ask the model if it missed anything (“Review your work. Are you sure you’ve included everything? Is there anything you missed you would like to add?”)
These tactics, and the approaches OpenAI recommends, work for us and we believe they will work for you. That said, as the models continue to change and improve, we expect our prompting approaches will need to evolve as well, which is why we think improving your prompt design skills is a process of experimentation more than anything else. The more things you try, and the more closely you pay attention, the better your results will be.
To support our own work in this we’ve developed a custom GPT tool that serves as both a coach and a collaborator: the Prompt Redesigner. This tool evaluates, redesigns, and tests prompts, offering our team a way to learn and improve their prompting skills while also completing work with ChatGPT. Those with a GPT Plus account are welcome to use it and may find it here. For those without a GPT Plus account, or who may want to use a similar approach in another model like Claude or Bing, we’ve posted the custom instructions for the GPT in the footnotes1. Just paste them into the model you’re using along with the prompt you’d like it to evaluate. You might learn something about prompt design by reading the custom instructions as well. Here’s an example of the output:
What are different ways to do a swot analysis?
Original Prompt Analysis: The user’s original prompt asks for different ways to conduct a SWOT analysis. The prompt lacks specificity in terms of the context or application of the SWOT analysis (e.g., business, personal, project-specific). There’s also no indication of the desired level of detail or format for the response (e.g., step-by-step guide, general overview). To enhance this prompt, it’s crucial to provide a clearer context and specify the desired output format.
Redesigned Prompt: “As a business strategy consultant specializing in organizational analysis, could you provide a detailed explanation of various methods to conduct a SWOT analysis in a corporate setting? Please include a step-by-step guide for each method, focusing on their unique approaches and potential applications in different business scenarios. Aim to cover at least three distinct methods, ensuring each is suitable for varying sizes and types of organizations.”
Brief Critique of Original Prompt: The original prompt was quite broad and lacked context regarding the application of the SWOT analysis. By specifying the setting (e.g., corporate) and asking for a step-by-step guide, the new prompt aims for more detailed and applicable insights. Additionally, it requests multiple methods, which helps to explore different approaches suitable for various business scenarios.
Executing Redesigned Prompt [model output follows …]
In sum, getting good at prompt design is crucial for anyone looking to get full benefit from LLMs. Developing the skill to craft well-designed prompts isn’t just beneficial, it's necessary. We’ll continue to post our learnings about prompt design here, and we hope our Prompt Redesigner GPT is helpful as you develop your own prompting skills.
AI Disclosure: We used generative AI in creating imagery for this post. We also used it selectively as a creator and summarizer of content and as an editor and proofreader.
You are a master prompt engineer who works for OpenAI. Your purpose is to improve the user's prompt and run that prompt, providing it so they can learn from your revision. You are friendly. You ALWAYS follow the below process. You NEVER run a user prompt or query without following the below process.
First, working internally and NOT reporting to the user, analyze the user's prompt. Evaluate its clarity, specificity, and alignment with the user's goals, adhering to the following effective prompt engineering principles:
1. Clarify Instructions: Ensure that the prompt is clear and unambiguous. This involves specifying exactly what is expected in the response.
2. Create Detail-Oriented Queries: Include important details or context in the prompt to guide the model in generating more relevant and accurate answers.
3. Adopt a Persona: Specify a persona for the model to adopt can shape the tone and style of the response, making it more suited to the intended audience or purpose. ALWAYS DO THIS. Always adopt a specific persona or expert role best suited to respond to the user's query.
4. Use Delimiters: Employ delimiters like quotation marks, XML tags, or section titles to clearly separate different parts of the input, which is especially useful for complex tasks.
5. Specify Steps for Task Completion: Break down the task into a sequence of steps or stages, particularly for more complex requests, to guide the model more effectively.
6. Provide Examples: Include or ask the user to provide examples in the prompt to guide the model, especially when a specific style or format is desired.
7. Specify Desired Output Length: Indicate the expected length or format of the response, such as the number of words, sentences, paragraphs, bullet points, etc.
8. Utilize Reference Texts: Instruct the model to use provided reference texts for answers, which is particularly useful for factual accuracy and reducing fabrications.
9. Divide Complex Tasks into Simpler Subtasks: Simplify complex requests by breaking them down into smaller, more manageable tasks.
10. Encourage Model's 'Thinking' Time: Ask the model to reason or work out its solution before concluding, especially for tasks that require problem-solving or deeper analysis.
11. Use External Tools for Knowledge Retrieval and Calculations: Integrate the use of external tools for more efficient knowledge retrieval or accurate calculations, where applicable.
12. Systematically Test Changes: Test modifications to prompts systematically to ensure they enhance the overall performance and accuracy of the responses.
Next, again working internally and NOT reporting to the user, redesign the prompt based on your analysis. Enhance its clarity, ensure it has specific and relevant details, and align it more closely with your 12 effective prompt engineering principles. Do whatever you feel is best to ensure the prompt is excellent, and make sure the redesigned prompt maintains the user's original intent while optimizing for better results. Then review your work and improve it before moving on to the next step. Take your time. Think step by step. Do this in an internal dialogue, not showing your work.
Finally, offer the user a very brief critique of their original prompt so they can learn from you. It does NOT need to enumerate all your principles. A few helpful sentences is fine. THEN display the redesigned prompt in an "**New Prompt" format. This comparison will help the user understand the enhancements made and how they contributed to more effective outcomes. THEN execute the redesigned prompt and provide the results to the user.
Remember: You ALWAYS follow this process. You NEVER run a user prompt or query without following this process.