Optimize Prompts Cookbook #1939
Conversation
examples/Optimize Prompts.ipynb
Outdated
| "source": [ | ||
| "# Optimize Prompts\n", | ||
| "\n", | ||
| "This cookbook provides a look into an early version of OpenAI's prompt optimization system. Crafting effective prompts is a critical skill when working with AI models. Even experienced users can inadvertently introduce contradictions, ambiguities, or inconsistencies that lead to suboptimal results. The system demonstrated here helps identify and fix common issues, resulting in more reliable and effective prompts.\n", |
There was a problem hiding this comment.
I tink we can edit the first sentence here - like This cookbook demonstrates how we used the Agents SDK together with Evals to build the first version of prompt optimization - aimed at addressing common issues
Obviously not quite like that but like starting off with what we're going to teach you
There was a problem hiding this comment.
oh I see we dont talk about evals in here right now - I feel like we probably should since it's a cookbook and usually those are talking about development workflows
There was a problem hiding this comment.
I think putting this in Objective after providing some context makes sense here!
There was a problem hiding this comment.
Happy to get more feedback on if this is a good structure
| "\n", | ||
| "4. **Dev-Rewriter**: After issues are identified, this agent rewrites the prompt to resolve contradictions and clarify format specifications while preserving the original intent.\n", | ||
| "\n", | ||
| "5. **Few-Shot-Rewriter**: Updates inconsistent example responses to align with the rules in the prompt, ensuring all examples properly comply with the new developer prompt.\n", |
There was a problem hiding this comment.
nit: developer or system prompt - this is in a few other places as well
There was a problem hiding this comment.
I think I'm fairly consistent with developer prompt instead of system prompt here! I think that's the best wording to use
There was a problem hiding this comment.
ah I was just saying that depending on the model it may be called developer or system prompt, but I'm ok with keeping it simple since all new models use developer prompt
| " developer_message: str,\n", | ||
| " messages: List[\"ChatMessage\"],\n", | ||
| ") -> Dict[str, Any]:\n", | ||
| " \"\"\"\n", |
There was a problem hiding this comment.
Can you group these all in a overarching trace, like this example:
from agents import Agent, Runner, trace
async def main():
agent = Agent(name="Joke generator", instructions="Tell funny jokes.")
with trace("Joke workflow"):
first_result = await Runner.run(agent, "Tell me a joke")
second_result = await Runner.run(agent, f"Rate this joke: {first_result.final_output}")
print(f"Joke: {first_result.final_output}")
print(f"Rating: {second_result.final_output}")
Should follow the BP a bit better for grouping these.
There was a problem hiding this comment.
Probably just the whole function would work for this!
Summary
Added Cookbook that outlines how Optimize Prompts Works and provides example of Agents-sdk usage, prompting best practices, and Pydantic models.