Home/HighLevel/Features/AI Employee/HighLevel Prompt Optimizer: test and improve AI agent prompts

AI

HighLevel Prompt Optimizer: test and improve AI agent prompts

In short

Prompt Optimizer is a Labs feature that lets you validate a Conversations AI agent against realistic chat scenarios without altering the live prompt. It runs live test chats, scores outcomes, suggests prompt revisions, and lets you apply an approved variation to production.

AreaAI Employee · Conversation AI
Used forDental practice, Real estate lead generation agency, Online language tutoring platform
Setup steps16 steps in 3 tasks
Last updatedNovember 17, 2026

01Who this is for

  • Founders of service based businesses benefit because the tool catches prompt gaps before customers see them.
  • Small agencies that build AI chatbots gain confidence that configured actions such as bookings or handoffs work as intended.
  • Product teams that maintain multi language AI agents use the feature to ensure translations stay aligned with prompt logic.

02Prompt Optimizer for Conversations AI, explained

Prompt Optimizer creates or reuses customer scenarios based on an agent's current prompt, knowledge base, appointment setup, and configured actions. It then clones the agent and runs live chat executions against those scenarios. Each chat is recorded, actions are tracked, and an AI model scores the result as pass or fail, explaining the reasoning.

The feature lives within the Conversations AI agents area as a Labs option. It shows usage estimates before a run, creates temporary contacts for the test chats, and cleans them up afterward. Results include full transcripts, action evidence, overall accuracy, and a diff view of any generated prompt variations.

Prompt variations remain in a history list. You choose a variation, review the Prompt Diff and AI reasoning, and click Use Prompt to replace the production prompt. Until you click Use Prompt, the live agent continues to use the original prompt.

03How businesses use Prompt Optimizer for Conversations AI

  • Dental practice. The practice generates scenarios that include appointment booking requests. It runs the test chats, checks whether the booking action fires correctly, reviews any failures, and lets Prompt Optimizer suggest prompt tweaks before applying them to the live bot.
  • Real estate lead generation agency. The agency reuses scenarios from a prior run to compare how a new knowledge base article affects the bot’s answers. Prompt Optimizer runs multiple executions per scenario, scores the outcomes, and helps refine the prompt for more accurate property queries.
  • Online language tutoring platform. The platform enables multi language testing by letting Prompt Optimizer generate scenarios in the bot’s configured language. It reviews the AI evaluation of each chat and applies an improved prompt that better handles multilingual student inquiries.

04How to set up and use Prompt Optimizer for Conversations AI in HighLevel

Configure a test run

  1. Generate new test scenarios that reflect the current prompt, knowledge base, appointment setup, actions, and business context.
  2. Select previously saved scenarios if you want to compare performance after changes.
  3. Set the number of chat executions that should run for each scenario.
  4. Review the pre run usage summary and confirm the temporary test contacts that will be created; note that temporary contacts do not stop configured actions from executing.
  5. Ensure any calendars, workflows, or routing destinations used by actions are test versions to avoid affecting production.

Test the cloned agent with live chats

  1. Start live chat executions so the cloned agent receives customer style messages and replies in real time.
  2. Track any configured actions such as bookings, handovers, workflow triggers, or field updates during each execution.
  3. Open each execution to review the full transcript of customer messages and agent replies.
  4. View the AI evaluation that marks the chat as pass or fail and explains the reasoning behind the score.
  5. Remember that real actions may fire during testing; use dedicated test resources whenever possible.

Improvise prompt improvements

  1. Examine failed chats and the AI generated reasoning.
  2. Click the Improvise button to let the system create a revised prompt variation.
  3. Run the new variation against the selected scenarios.
  4. Check the updated accuracy and compare the original and new prompts with Prompt Diff.
  5. Select the best performing variation and click Use Prompt to apply it to the production agent.
  6. Note that AI evaluations are probabilistic, so repeated runs can yield different pass/fail outcomes.

05What it looks like in HighLevel

06Settings that matter

  • Auto Optimize. Runs multiple optimization attempts automatically until the target accuracy or variation limit is reached.
  • Use Prompt. Applies the chosen optimized prompt variation to the production Conversations AI agent.

07Things to know before you rely on it

  • Prompt Optimizer is a Labs feature and may change as feedback is incorporated.
  • Testing runs a cloned agent, but configured actions still execute during the test.
  • AI evaluations are probabilistic; repeated tests can yield different pass/fail outcomes.
  • Auto Optimize respects configured variation limits and may stop before reaching the target accuracy.

08If something goes wrong

  • A failed optimization is caused by an underlying calendar, workflow, handover destination, Knowledge Base source, or field mapping issue.. Correct the specific configuration, such as the calendar, workflow, or Knowledge Base, before re running Prompt Optimizer.

09How it connects to the rest of HighLevel

Michael's take

Michael Reimer
Michael ReimerFounder of CRM Pros, HighLevel Commander and Certified Admin

Be aware that Prompt Optimizer runs live chats against a cloned agent, but configured actions can still fire. That means you could unintentionally trigger appointments or workflows if you don’t point the test at isolated resources.

When you need confidence that a new prompt or knowledge update won’t surprise real customers, Prompt Optimizer gives you a repeatable way to spot weaknesses and see concrete improvements before you push anything live.

Read the full details on HighLevel →Published November 3, 2026 · Updated November 17, 2026

Common questions

Common questions about Prompt Optimizer for Conversations AI.

Does Prompt Optimizer modify my live Conversations AI agent automatically?

No. All testing and optimization happen on a cloned agent. The production prompt stays unchanged until you review a variation and click Use Prompt.

Can I test the same scenario more than once?

Yes. Running multiple chat executions for a scenario helps you see if the agent behaves consistently or if failures appear intermittently.

Can I reuse scenarios from a previous run?

Yes. Reusing earlier scenarios lets you compare performance after changes to the prompt, knowledge base, actions, or appointment settings.

What should I do when a chat fails?

Open the transcript, review the action evidence and AI reasoning, then determine whether the issue stems from the prompt, Knowledge Base content, action configuration, or evaluation criteria before adjusting anything.

Why can repeated tests produce different results?

The underlying language model is probabilistic, so customer messages, agent replies, and evaluations can vary across executions.

Does Prompt Optimizer support multiple languages?

Yes. Scenarios, customer messages, evaluations, and any optimized prompts stay aligned to the language set for the Conversations AI agent.

Next step

Pick the features that matter. Then get the plan that unlocks them.

Explore the features you want, start with the five at the top, and upgrade through my link when you are ready. The bonuses and the team come with it.

The features directory explains every HighLevel feature in plain English. Browse it to see what the platform can do.

Back to the FeaturesSee the full details
⌃

Site preview

Explore the next page

This destination is reserved in the page inventory and is not built yet.