What Is Prompt Engineering? A Guide.

Anyone can provide an input to a large language model. Question is, does the LLM’s resulting output fulfill the intended goal or answer the question asked? That largely depends on how well the input was crafted, which is where prompt engineering comes in. A good query significantly increases the odds that an LLM will produce exactly what the project needs; it also generates side benefits that may accrue to future projects.

What Is Prompt Engineering?

Prompt engineering is the practice of crafting instructions, or prompts, to guide a generative AI model to generate desired outputs. This process uses iterative efforts to improve how various formats, phrases, function calls from LLMs to other systems, and additional variable elements of an AI prompt perform. The goal is to provide the LLM with optimal specificity and context.

The following are some of the most important elements of prompt engineering:

• Format: Because of the way LLMs are developed and trained, the format and structure of prompts is important to the output. The best outputs start with an understanding of the preferred format for the LLM used in the model.
• Function calls: Incorporating data from external sources can increase the quality and accuracy of an output. Prompts might launch function calls for dynamic data fetching, which will return results as long as the desired data is accessible.
• Specificity: Ambiguity in prompt phrasing can create inaccurate, misdirected, or open-ended—even nonsensical—answers. A focus on specificity in word choice increases the quality and depth of output answers. Simply put, it’s the difference between saying “I want a dog” and “I want a rescue dog under the age of three years old that’s crate-trained and good with young children.”
• User audience: Prompts produce the most accurate results when they integrate awareness of the audience. A highly technical person is much different than a student or a child, and that should be reflected in the prompt so the output meets the audience’s expectations for both tone and detail.

While the term prompt engineering reflects the general science of improving prompts to achieve results, it also acts as a step in the application development process. In their role, prompt engineers create templates and scripts, known as base prompts, within the app that bridge end user inputs with the model while being invisible to the user. The goal of a base prompt is to provide a scalable and automated method of bridging inquiries while working within the resource confines of the project. An infrastructure that inherently supports AI and ML capabilities and scalable resources can simplify and optimize these types of projects.

Key Takeaways

• Prompt engineering is the process of crafting, evaluating, and improving prompts to gain more accurate outputs from an AI model.
• Factors that improve prompts include the LLM’s preferred format, specificity of language, appropriately identifying the audience’s expectations, and making function calls for external data.
• In the app development process, prompt engineers create a base template that addresses the necessary factors for accurate outputs to bridge potentially vague user inputs with the app’s LLM.
• App development works best when AI and ML services are provided by the underlying infrastructure, allowing prompt engineers to focus on the task at hand.

Prompt Engineering Explained

The AI industry views prompt engineering in two contexts, with the second definition being an extension of the first. The first definition refers to the actual skill set itself: the ability to craft and refine an AI prompt to elicit the most desirable output possible. A trial-and-error process comes into play as prompt engineers experiment—with format; word choice; additional contextual data, such as function calls pulled externally via APIs; and other variables—to achieve the desired output. Prompt engineers versed in the most popular standard AI models will have a stronger chance of understanding the specific formats that deliver strong results. In addition, prompt engineers often use tools that track prompt construction history, provide sandbox experimentation space, and offer A/B testing of prompts.

A helpful quality for prompt engineers is a strong knowledge in the project’s topic. This isn’t an absolute requirement for the role; prompt engineers certainly can be brought in on technical AI expertise rather than contextual understanding. However, by starting a project with some understanding of its overall purpose, prompt engineers can more efficiently verify outputs for accuracy and efficiency.

It’s impossible, though, to expect that every user will know a prompt engineer’s strategy when using an app. The second definition of prompt engineering, then, is to integrate a strategically created base prompt into an app’s development cycle. This base prompt provides all the expertise of the prompt engineer in an unseen template. When users put in their input queries, that data augments base prompts rather than going in completely cold. This is a key part of successful AI-powered app development because it helps ensure the widest flexibility of user capability while providing an established standard of output.

Why Is Prompt Engineering Important?

Prompt engineering is important because it maximizes the efficiency of AI initiatives across the board—in resources, in effort, in user experience. Quality prompts lead to lower query processing costs and increased user satisfaction. This makes prompt engineering a worthwhile investment for app developers, even if it takes additional time and resources during the development cycle.

On a more granular level, prompt engineering can help ameliorate the following risks for developers:

• Developer bias: In the context of prompt engineering, bias refers to the intentional or unintentional introduction of viewpoints, assumptions, or preferences by engineers who are creating prompts, which can skew the AI model’s output. To avoid this issue, the prompt engineering process can provide space to examine the algorithm, training data, and output results from a variety of perspectives. This assists with bias prevention, both by providing an additional internal review during prompt generation and by creating base prompts in a way that can potentially offset or address a user’s own biases.
• Unexpected resource drain: During the trial-and-error process, prompt engineers can determine what contextual information—such as user history, internal databases, or external systems—is required to deliver relevant output. By identifying the data necessary for strong base prompts, developers can examine the practical (gaining access to internal data) and technical (resource drain from function calls via APIs) impact on resources before getting too far into the development cycle.
• Unidentified boundaries and parameters: Prompt engineering provides another layer of examination that helps the entire development team establish relevant boundaries and limitations. These include parameters for contextual retention versus resource use; boundaries for user interaction versus software cognition; and unexpected issues with input parameters, such as format and semantics.
• Unpredictable user queries: By creating base prompts that set the foundation for inputs, prompt engineering can provide a standard of quality for queries—even if user input is vague and general.

How Prompt Engineering Works

Prompt engineers typically start with project considerations before undertaking a trial-and-error process that establishes a successful prompt, before finally integrating it into the app.

The following provides a high-level view of how this process typically works:

1. Understand the purpose and audience of the model and application: Before any technical steps occur, engineers typically take a step back and consider the contextual nuances of the project. Audience demographics; model complexity; and expectations for results based on variables, such as industry or expected knowledge, need to be understood for effective prompt generation. Without this knowledge, even a technically accurate output may not work for the audience’s needs.

2. Understand the problem or question to be explored: Once the broader context of the situation has been established, the engineer can drill down to the specific issue. Factors to be considered include the desired goal, level of detail, anticipated follow-ups, steps or segments used, and potential function calls for further data.

3. Understand the tendencies and preferences of the LLM: Individual LLMs come with their own quirks in formats, semantics, and complexity. Other factors include resource limitations involved with the model’s underlying infrastructure.

4. Craft the initial prompt: All the steps above should establish enough information regarding context, purpose, audience, and limitations to build an initial prompt.

5. Evaluate results: Once the prompt is used, outputs should be evaluated for success. How that success is measured depends on the project’s goals. While accuracy is paramount, individual situations may also call for an emphasis on tone, voice, length, level of detail, and continued engagement using retained memory.

6. Refine as needed: Refining a prompt includes tweaking language, adding context, integrating functions via API calls, and other such possibilities. Prompt engineers can also use various tools to assist in the refinement process; such tools can record prompt history, display results via A/B testing, and manage output analysis for expedited refinement.

7. Test for exportability: Exportability provides two organizational benefits. By testing the prompt against different LLMs, the development team may find that one LLM is a better fit for the project. In addition, prompt engineers can examine the context-neutral pieces of the prompt to see if they can be exported for use in other projects.

8. Integrate into an AI model for deployment: With a successful base prompt crafted, the development team can begin integrations for automation and scalability within the project, preferably on a cloud infrastructure with managed AI/ML services for optimized performance. This produces the goal of having an effective base prompt that can then be augmented by user input.

Consider the example of an assistant on a weather app. The base prompt might identify the following information before a person even enters a query:

• Location, pulled from the device IP address
• Time of day, also determined by the IP address
• Demographics, pulled from the user’s app profile
• Search history for typical types of data requested, such as traffic or outdoor activities
• Purpose of the application, for framing of answers
• Tone of the app, for word choice

All those pieces can be put into place using a base prompt, then be integrated with a user’s question for an output that provides greater accuracy and personalization and the appropriate tone and language.

Benefits of Prompt Engineering

Prompt engineering creates the key benefit of more specific, accurate results. How that’s achieved depends on the two different forms of prompt engineering—both the actual skilled practice of prompt engineering and the integration into a model as base templates for public queries.

The following are some of the most common benefits of prompt engineering as a whole:

• Optimized AI Output and Efficiency: Large language models can work with any general input or query, but doing so is often a waste of resources because refinement and extra effort are required. As a skill set, prompt engineering bypasses generic prompts to get more accurate responses. When integrated into an AI model, prompt engineering points the user in a relevant direction and efficiently colors in details without added effort on the part of the person making the query.
• Enhanced Flexibility and Customization: When executed properly, the first steps of a prompt engineering approach can offer greater flexibility and customization to a project. By building domain-neutral context, prompt engineering efforts can be imported to other apps or models. Examples of domain-neutral context include identifying user demographic data, time and season data, and app function and tone. These elements can work with nearly any model while reducing vague and generalized outputs.
• Improved User Experience and Satisfaction: Often, people approach an LLM or an application knowing what they want but without the ability to articulate the request in a manner that returns the desired response. Let’s take the analogy of going to the grocery store. Without prompt engineering, it’s akin to walking into the store, standing at the entrance, and saying, “I’m hungry.” In this scenario, prompt engineering could refine that request based on things such as budget, preferences, and capabilities to lead you to the right aisle. When built into the model, prompt engineering provides a better immediate understanding of the user and the goal, yielding a better overall experience with more accurate results.

Prompt Engineering Challenges

As a science, prompt engineering is relatively young. Practitioners include software developers crafting prompts to add AI-powered features to their applications for tasks including content generation, summarization, translation, and code assists; technical communicators looking to create systems such as customer service chatbots; and specialized prompt engineering professionals who focus on designing, testing, and optimizing prompts for very specific, specialized use cases.

The following are some of the most common challenges facing people doing prompt engineering:

• Balancing specificity and creativity: The goal of prompt engineering is to support creative freedom without bogging down either the output or the resources. Achieving that is a difficult balance. Going back to the grocery store analogy, unbalanced prompt engineering is like responding to a hungry user with a single choice of frozen spaghetti. Specificity helps ensure efficiency, but LLMs need appropriate flexibility to deliver accurate and high-quality results.
• Managing ambiguity: When an app or AI model produces ambiguous results, that puts a greater burden on the user—and the more a human needs to iterate and refine a query, the more resources the process uses. As a skill set, a key facet of prompt engineering is minimizing the amount of ambiguity in results. The challenge, then, is fine-tuning the prompt to establish a standard of specificity without creating too many limitations in the results.
• Adapting to model limitations: Depending on the purpose and function of an app, its model may have a very specific audience and tone in mind. For prompt engineers, this known direction can provide an easier path to getting started. However, it can also cause them to fall into a trap, building outputs that have limited ability to incorporate unexpected or diverse inputs. App developers can work with prompt engineers to discuss an acceptable range of inputs and select base prompt templates that can balance between creative queries and the specific function of the app.
• Iterative refinement: Prompt engineers can fall into a trap of assuming an effective prompt is one and done. However, because AI models are continuously learning and apps are in continuous development, an effective prompt may soon be outdated. Once a prompt is constructed, engineers must move forward with awareness to adapt to the dynamic nature of the environment. When a prompt has been integrated into an app’s workflow, continuous refinement and assessment are particularly key to help deliver quality outputs.
• Context retention: During an app’s development process, the entire team must consider how to balance function and performance. From the perspective of user experience, context retention is key to creating an accurate output. However, each layer of retention eats up more resources, so the challenge facing development teams and their prompt engineers is to understand which context should be part of an established internal prompt and what is required from external users for subsequent prompts. Similar to model limitations, the choice of underlying infrastructure—and its abilities to provide built-in support for AI projects—can significantly optimize resources to increase flexibility when examining context retention.
• Handling long and complex queries: Eventually, AI models will likely be able to handle extremely complex queries. Right now, most of them can’t—a tipping point usually exists where the output becomes ineffective. Developers can use prompt engineering to reduce variables related to this kind of result by preloading key context and assigning parameters.
• User intent alignment: Prompt engineering can increase efficiency and provide a head start, but what if it’s pointed in the wrong direction? Specificity is a key feature in prompt engineering outputs, but only if it’s working. Thus, development teams must check to make sure that prompt engineering isn’t so specific that it sidesteps a user’s true intentions.

To address these and other resource limitations, many enterprises deploy their LLMs on a cloud infrastructure with built-in managed services that are tuned to support AI.

Skills Needed for Prompt Engineering

As the concept of prompt engineering has come to the forefront only in the past decade, it remains an evolving role. A successful prompt engineer needs a core skill set and an understanding of where the function fits into the greater algorithm training and app development process.

At its core, prompt engineering requires a blend of strong communication skills, subject matter expertise, and programming acumen. There are precise language, semantic, and grammatical structures needed to elicit the desired responses from AI models, and the engineer must also understand the underlying logic and patterns used by the organization’s LLM. Furthermore, they must be able to assess the accuracy and relevance of the generated output.

When integrated into a development workflow, a prompt engineer’s skill set should lean more technical. Because a prompt may need to make external requests, for example, an understanding of how APIs and function calls work and competency in standard programming languages are valuable. In addition, a technical background allows prompt engineers to consider the computational costs of different prompting strategies so they can strike a balance between performance and cost-effectiveness.

Prompt Engineering Use Cases

Prompt engineering can be a vital tool in improving both efficiency in AI resource use and user satisfaction. By integrating a base prompt into an app’s workflow, apps can generate better, more accurate results even when humans provide vague inputs.

The following are just some of the ways prompt engineering can benefit specific use cases.

• Education: AI models have several uses in classrooms and labs, and prompt engineering helps create a personalized, effective path. Consider a university implementing a custom digital assistant to improve the student experience, with prompts tailored to answer questions with real-time information. A school might use AI to develop personalized learning plans, with prompts that can pull data from student goals and lesson plans while making function calls to previous records and classes. Or a tutoring app developer can engineer prompts so that answers are appropriate for each student’s age and skill level.
• Finance: Apps can assist in both internal and customer-facing sides of finance, including report generation, market trend analysis, and customer service. In each of these cases, prompt engineering can build in head starts for the user. For internal reports, prompts can source data from function calls to external market data or internal metrics. For customer service, prompts might source data from a customer’s history and external factors, such as time, season, and type of inquiry. Internally, prompts might be tailored to help with fraud prevention work.
• Healthcare: AI can assist in helping diagnose medical conditions, summarizing patient records, and generating medical reports. These systems can also support healthcare professionals. To achieve all this, prompts might be designed to reflect an appropriate tone for either patient or practitioner audiences while pulling required context via external function calls to additional records, employee systems, and the latest related medical research.
• Manufacturing: Manufacturing firms have adopted AI to support a range of functions, including supply chain status tracking, quality control, and customer self-service tools. Each of these use cases requires access to internal and external sources to address different audience needs. For example, to optimize production schedules, prompts might be engineered to go beyond using established internal deadlines and factor in elements such as supplier status, tool lifecycle data, and real-time issues that might affect delivery, such as holidays or inclement weather.
• Marketing: Digital marketing campaigns generate a lot of data. AI-generated marketing content benefits greatly from prompts engineered to harness this data. Engineers can prepare base prompts that pull in, for example, social media posts referencing the brand. To optimize engagement, ad campaigns could then target specific user sentiment and demographics data.
• Real estate: The real estate industry takes inputs from a wide range of sources: public sales records, interest rates and financial trends, even weather and seasonal data. But industry apps tend to focus on one thing: matching those looking for homes with the right properties. Meanwhile, real estate firms have unique HR management challenges that AI can help with. Prompt engineering can point an app in the right direction based on current needs and data while preparing appropriate function calls in base prompts to deliver what users need.
• Retail: AI-enabled shopping assistant apps can increase customer satisfaction and conversion rates by personalizing recommendations and adding automation to customer workflows. Much of the data driving these improvements is sourced from customer data, including purchase, search, and service histories. By building a base prompt that proactively uses customer personas and grabs appropriate data, chatbots and other apps can better engage shoppers.
• Travel: AI-powered travel apps can improve personalized recommendations and itineraries, thanks to prompt engineering. For example, when a user asks for a restaurant reservation during a planned trip, the prompt can go beyond general location and factor in user history, such as whether kids are involved, while making function calls for cuisine, table availability, and cost. These are all factors that can be achieved by drilling down into results, but prompt engineering can provide a head start to ease query workload while delivering faster, more accurate output. Well-engineered prompts can also enable AI-powered digital assistants to help both customers and staff answer time-sensitive questions.

Prompt Engineering Techniques

Various prompt engineering techniques come with strengths and weaknesses. Determining the right one for a project depends on the goals, processing capabilities and underlying support infrastructure, LLM in use, audience, and other unique parameters.

The following cover some of the most popular prompt engineering techniques used today:

• Chain of thought: Directing the LLM to identify and list intermediary steps toward the final goal is a means to improve accuracy and transparency. Chain-of-thought techniques can be triggered by requesting the model to list steps, including samples of how-to step lists, or offering multiple-choice options while asking for reasoning for a selection.
• Directional stimulus: The LLM’s output can be improved by providing hints and directions with the prompt. Directional stimulus prompting works by providing specific clues, parameters, and context to general questions in text following the base question. By adding the word “hint” and a list of specifics, similar to the way a social media post can add hashtags to provide context, the output can be set to incorporate those items and generate a higher quality result.
• Least to most: This entails breaking down a prompt into subproblems, then executing them in a defined sequence. Least-to-most prompting resembles the approach of chain of thought in that it views a prompt on a granular level, but its utilization of intermediate steps to progressively build an answer allows for more complexity in execution. Like chain-of-thought prompts, least-to-most prompts are most effectively applied to complex problems that can be broken down into a series of simpler, sequential subproblems.
• Maieutic: This involves gradual, open-ended prompts that build upon answers by guiding the model to reflect on its reasoning. Maieutic prompting is based on the Socratic method of dialogue, which usually starts with an open-ended question, then drills down further at the reasoning behind each answer. In practice, this is achieved by starting with a question, then successively asking the model to explain its answer in more depth.
• Self-refine: Gradual improvement of an LLM’s output can be achieved by feeding the previous answer back to the model while asking for improvement. Self-refine prompting is an iterative technique that gives the model an opportunity to reassess its output for possible tweaks and additions; it’s best used for considering issues where the goal is to optimize a particular solution, such as code generation. Because this is an instruction-based technique, engineers must verify that the model has the capability and resources to retain answers and iteratively build upon them.
• Sequential: This involves a series of related and sequential steps as a means of completing a workflow or outline. Sequential prompting works best in two situations: when a specific sequence is involved, such as instructions or procedures, and when starting with a broader approach to a particular topic, then building off the answer as a guided dialogue until a satisfactory point is achieved. Sequential prompting is recognized through clear keywords that delineate sequence, such as “step 1” or “part 2”.

Best Practices for Prompt Engineering

Prompt engineers often work on many different projects with different goals, across different LLM platforms with different levels of compute resources. Still, there are some common considerations to achieve the best possible output.

1. Consider your LLM’s “personality”
In addition to the standard limitations of all LLMs, like hallucinations, each platform comes with pros and cons. For example, currently GPT-4 can support both text and images but uses a lot of computational resources. BERT is open source and offers powerful comprehension but requires more effort in fine-tuning for specific tasks. Each LLM also has its own preferred format and semantics for input, and models are always evolving. What works for a project now might not in six months or a year.

2. Balance precision and brevity
Vague, open-ended prompts lead models to output vague or repetitive results. Specificity is the key to good prompt engineering, including both technical and practical elements. On the technical side, precise prompts factor in the preferred formats and known parameters of the LLM and app. On the practical side, key factors include target audience, app/model function, expected background knowledge, and precise instructions, as well as appropriate samples or parameters, such as the number of requested points or examples.

3. Add contextual clues
In complex queries, context can make all the difference, so prompt engineers pay attention to informing the prompt and provide a framing reason for the request. Consider the question, “Is it good weather today?” When crafting a prompt for an AI, a prompt engineer recognizes that the definition of “good” is subjective. By strategically adding context to the prompt, the engineer can elicit more useful responses. For example, instead of just asking the AI the question, a prompt could be structured to include context:

• User constraints: Does the person suffer from asthma or have pollen allergies? An engineer might program the AI to consider local conditions and specific health concerns.
• User intent: Is it a farmer hoping for rain or a student hoping for a sunny day for an outing? The AI can provide seasonal and activity-based context.
• Temporal and geographic specificity: What’s the country, city, season, and day of the week?

Strategically providing context helps the LLM generate more useful and personalized responses. Prompt engineers might choose to identify various external function calls via APIs that can generate some of this context in advance.

4. Be patient with iterative testing and refinement
Prompt engineering is a process of trial and error. Fortunately, practitioners have access to various tools that can support iterative testing and refinement by providing elements such as prompt history, sandbox environments for different LLMs, performance evaluations and suggestions, and A/B testing. By using a prompt management tool, refinement becomes more efficient and traceable, allowing for a more comprehensive view of the path to an optimized prompt. This visibility can also build a foundation for exporting repeatable neutral-context base prompts.

Future of Prompt Engineering

Prompt engineering’s evolutionary path will likely be tied to the technical advancements of AI and LLMs. Most prompt engineers expect that as the comprehension of LLMs continues to grow, prompts can become increasingly sophisticated, allowing for inclusion of greater detail, specificity, and contextual information. Currently, LLMs tend to have a breaking point where long and complex prompts result in nonsensical outputs.

A tangent to increased prompt complexity is the ability for prompt adaptability. In other words, AI engineers are looking at ways for LLMs to generate prompts that can self-adapt based on the context, history, and specifications of a conversation. Similarly, developers are seeking to make LLMs work with multiple types of input. In a perfect world, LLMs would be able to take a multimodal input of text, audio, and imagery to create an output.

A version of that currently exists in the form of retrieval-augmented generation (RAG). RAG overlaps with the general purpose of prompt engineering in that it strives to provide deeper context that results in more accurate outputs. However, RAG is performed via self-propagated data retrieval, based on clues within the prompt. In a perfect world, a prompt engineer builds a base prompt, then RAG adds further context through the retrieval of more relevant data, resulting in highly accurate output. RAG tools work best using vector databases for fast retrieval and when given sufficient processing power. As cloud providers address these and other issues for AI and machine learning projects, the inherent capabilities and scalable design of these services will provide a better foundation to support the capabilities of LLMs.

How Oracle Can Help

Oracle Cloud Infrastructure (OCI) Generative AI provides managed services that can help free up time for prompt engineers to experiment with their queries without worrying about access to multiple LLM choices, scalable resources, and enterprise-grade security. The OCI chat experience provides an out-of-the box interface with Cohere and Meta models while helping keep data private.

Prompt engineers are part translator, part detective, and part coder, using their creativity and language skills to craft the precise words and instructions to tease out the desired outcome from immensely complex LLMs. Crafting prompts is a uniquely human skill, and the payoff is that moment when tweaking a phrase transforms the AI’s response from generic, even hallucinogenic, to genius.

Prompt Engineering FAQs

What is prompt engineering in AI?

Prompt engineering refers to two different elements in AI. The first is the skill set of prompt engineering, which is the process of refining an input prompt to get the best, most accurate result. The second is the integration into an AI workflow of repeatable, automated, and scalable base prompts that have been crafted by a prompt engineer to help generate outputs even if users provide only vague queries.

How does prompt engineering improve AI model outputs?

Without prompt engineering, AI model outputs will often provide only a very general response to a typical basic query. Prompt engineers engage in a trial-and-error process to identify patterns, comprised of word choice, format, function calls, and other elements, that can then be integrated into the app as a base prompt, which can help deliver detailed responses to even vague user queries.

What tools are commonly used for prompt engineering?

Tools that can help prompt engineers do their jobs better and faster allow for a trial-and-error sandbox for prompts while providing management tools and the ability to examine results with detailed analytics, prompt history and evaluation, A/B testing, and chaining. Prompt tools support a variety of core AI models and outputs—some are text-only while others support images and text.

How is prompt engineering different from traditional programming?

Traditional programming works with a strict set of rules following a specific code format, all to achieve a repeatable response. Prompt engineering follows a similar input/output flow but in a much looser path. Prompt engineering inputs use natural language but also work best when adhering to formats and semantics preferred by a specific AI model. Because of this open-ended nature, changes can be faster in prompt engineering due to trial-and-error language tweaks as opposed to refining or debugging code; however, these changes may not achieve the precise results that are found with repeatable code processes.