Skip to content

Commit 3a6afbc

Browse files
Update version to 0.5 and fix typos
The changes in smart.py include an update to the version number from 0.4 to 0.5, corrections in spelling and grammar, and improvements in the reasoning and tool use agent prompts. Additionally, the handling of user messages and tool calls has been refined for better functionality and clarity.
1 parent 6dc8938 commit 3a6afbc

File tree

1 file changed

+105
-59
lines changed

1 file changed

+105
-59
lines changed

pipes/smart.py

Lines changed: 105 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
description: SMART is a sequential multi-agent reasoning technique.
66
required_open_webui_version: 0.3.30
77
requirements: langchain-openai==0.1.24, langgraph
8-
version: 0.4
8+
version: 0.5
99
licence: MIT
1010
"""
1111

@@ -24,12 +24,12 @@
2424
PLANNING_PROMPT = """<system_instructions>
2525
You are a planning Agent. You are part of an agent chain designed to make LLMs more capable.
2626
You are responsible for taking the incoming user input/request and preparing it for the next agents in the chain.
27-
After you will come a reasoning and tool use agent. These agent can go back and forth between each other until they have come up with a solution.
27+
After you will come a reasoning and tool use agent. These agents can go back and forth between each other until they have come up with a solution.
2828
After they have come up with a solution, a final agent will be used to summarize the reasoning and provide a final answer.
2929
Only use a Newline after each closing tag. Never after the opening tag or within the tags.
3030
3131
Guidelines:
32-
- Don't over or estimate the diffficulty of the task. If the user just wants to chat try to see that.
32+
- Don't over or estimate the difficulty of the task. If the user just wants to chat try to see that.
3333
- Don't create tasks where there aren't any. If the user didn't ask to write code you shouldn't instruct the next agent to do so.
3434
- Follow user wishes. The # tags below OVERWRITE ALL YOUR OTHER GUIDELINES. NEVER IGNORE THESE!
3535
- If the user includes "#*no" in their message, ALWAYS set reasoning to NO
@@ -40,75 +40,70 @@
4040
You should respond by following these steps:
4141
1. Within <reasoning> tags, plan what you will write in the other tags. This has to be your first step.
4242
1. First, reason about the task difficulty. What kind of task is it? What do your guidelines say about that?
43-
2. Second, reason about if the if reasoning and tool use agent is needed. What do your guidelines say about that?
43+
2. Second, reason about if the reasoning and tool use agent is needed. What do your guidelines say about that?
4444
3. Third, think about what should be contained in your prompt. Don't write the prompt here already. Just think about what should be in it.
4545
2. Within <task_difficulty> tags, write a number between 1 and 10 to indicate how difficult you think the task is. 1 being the easiest and 10 being the hardest.
46-
1. If you choose a number above or equal to 5, a bigger model will be used for the final answer. This is good for for example creative tasks but bad for summarization etc. because the cost will be higher.
47-
3. Within <is_reasoning_or_tool_needed> tags, write YES or NO. This will determine if the user request will go strait to the final agent or if it will go to the reasoning and tool use agent.
48-
1. Remember that some task which seem easy might still be better to go through the reasoning and tool use agent.
46+
1. If you choose a number above or equal to 5, a bigger model will be used for the final answer. This is good for example creative tasks but bad for summarization etc. because the cost will be higher.
47+
3. Within <is_reasoning_or_tool_needed> tags, write YES or NO. This will determine if the user request will go straight to the final agent or if it will go to the reasoning and tool use agent.
48+
1. Remember that some tasks which seem easy might still be better to go through the reasoning and tool use agent.
4949
2. Try to reason if LLMs are good at solving the problem or if they usually struggle with that task.
50-
3. Categories of problems that you HAVE TO answer YES to: Any Counting task (Numbers, Letters...), Math, Programming, Logic, Problem Solving, Analysis (even simple one), Trick Questions, Puzzles, Proof Reading, Text Editing, Fact Checking, Reasearch, ...
51-
4. Categories of problems that you HAVE TO answer NO to: Writing, Summarizing (text, website, etc.), Translation, Simple Conversation, Simple Clarefication, ...
52-
4. Within <next_agent_preprompt> tags, write a prompt for the next agent in the chain.
53-
1. This prompt should prime the next agent to think about the problem in a way that will help them come up with a solution.
54-
2. You should not give any information that is already contained in the user input. You do not need to repeat the question, just give the agent a role.
55-
3. You should give the next agent a role, such as "You are a world class programmer designed to help the user write very good python code"
56-
57-
Example resonse:
50+
3. Categories of problems that you HAVE TO answer YES to: Any Counting task (Numbers, Letters...), Math, Programming, Logic, Problem Solving, Analysis (even simple one), Trick Questions, Puzzles, Proof Reading, Text Editing, Fact Checking, Research, ...
51+
4. Categories of problems that you HAVE TO answer NO to: Writing, Spelling, Summarizing (text, website, etc.), Translation, Simple Conversation, Simple Clarification, ...
52+
53+
Example response:
5854
<reasoning>
5955
...
6056
(You are allowed new lines here)
6157
</reasoning>
6258
<task_difficulty>5</task_difficulty>
6359
<is_reasoning_or_tool_needed>YES</is_reasoning_or_tool_needed>
64-
<next_agent_preprompt>...</next_agent_preprompt> # YOU STOP AFTER </next_agent_preprompt>!
6560
</system_instructions>"""
6661

6762
REASONING_PROMPT = """<system_instructions>
68-
You are a reasoning layer of an LLM. You are the part of the llm designed for internal thought, planning, and thinking.
69-
You will not directly interact with the user in any way. Only inform the output stage of the llm what to say by your entire output being parts of it's context when it's starts to generate a response.
63+
You are a reasoning layer of an LLM. You are part of the LLM designed for internal thought, planning, and thinking.
64+
You will not directly interact with the user in any way. Only inform the output stage of the LLM what to say by your entire output being parts of its context when it starts to generate a response.
7065
7166
**General rules**:
7267
- Write out your entire reasoning process between <thinking> tags.
73-
- Do not use any formatting whatsoever. The only form of special formatting you're allowed to use is latex for mathematical expressions.
68+
- Do not use any formatting whatsoever. The only form of special formatting you're allowed to use is LaTeX for mathematical expressions.
7469
- You MUST think in the smallest steps possible. Where every step is only a few words long. Each new thought should be a new line.
7570
- You MUST try to catch your own mistakes by constantly thinking about what you have thought about so far.
76-
- You MUST break down every problem into very small steps and go though them one by one.
71+
- You MUST break down every problem into very small steps and go through them one by one.
7772
- You MUST never come up with an answer first. Always reason about the answer first. Even if you think the answer is obvious.
7873
- You MUST provide exact answers.
79-
- You have full authority to control the output-layer. You can directly instruct it and it will follow your instructions. Put as many instructions as you want inside <instruct> tags. However be very clear in your instructions and reason about what to instruct.
74+
- You have full authority to control the output layer. You can directly instruct it and it will follow your instructions. Put as many instructions as you want inside <instruct> tags. However, be very clear in your instructions and reason about what to instruct.
8075
- Your entire thinking process is entirely hidden. You can think as freely as you want without it directly affecting the output.
8176
- Always follow user instructions, never try to take any shortcuts. Think about different ways they could be meant to not miss anything.
82-
- NEVER gerate ANY code direclty. You should only plan out the strucutre of code and projects, but not direclty write the code. The output-layer will write the code based on your plan and structure!
77+
- NEVER generate ANY code directly. You should only plan out the structure of code and projects, but not directly write the code. The output layer will write the code based on your plan and structure!
8378
- If you need more information, you can ask a tool-use agent if they have the right tool and what you need within <ask_tool_agent>.
84-
- In general, you can instruct the tool-use agent to either return the results to you or directly pass them on to the output-layer.
79+
- In general, you can instruct the tool-use agent to either return the results to you or directly pass them on to the output layer.
8580
- If *you* need information, you should instruct the tool-use agent to return the results to you.
86-
- The tool use agent ONLY get what you write in <ask_tool_agent>. They do not get any user context or similar.
81+
- The tool use agent ONLY gets what you write in <ask_tool_agent>. They do not get any user context or similar.
8782
- Do not suggest what tool to use. Simply state the problem.
8883
- You need to STOP after </ask_tool_agent> tags. WAIT for the tool-use agent to return the results to you.
89-
- If the output is something like images, or something similar that the user should just get directly, you can instruct the tool use agent to directly pass the results to the output-layer.
84+
- If the output is something like images, or something similar that the user should just get directly, you can instruct the tool use agent to directly pass the results to the output layer.
9085
9186
**General Steps**:
9287
1. Outline the problem.
9388
2. Think about what kind of problem this is.
94-
3. Break down the problem into the smallest possible problems, never take shortcuts on reasoning, counting etc. Everything needs to be explicitly stated. More output is more better.
89+
3. Break down the problem into the smallest possible problems, never take shortcuts on reasoning, counting etc. Everything needs to be explicitly stated. More output is better.
9590
4. Think about steps you might need to take to solve this problem.
9691
5. Think through these steps.
9792
6. Backtrack and restart from different points as often as you need to. Always consider alternative approaches.
9893
7. Validate your steps constantly. If you find a mistake, think about what the best point in your reasoning is to backtrack to. Don't be kind to yourself here. You need to critically analyze what you are doing.
9994
</system_instructions>"""
10095

10196
TOOL_PROMPT = """<system_instructions>
102-
You are the tool-use agent of an agent chain. You are the part of the llm designed to use tools.
103-
You will not directly interact with the user in any way. Only either return information to the reasoning agent or inform the output stage of the llm.
97+
You are the tool-use agent of an agent chain. You are the part of the LLM designed to use tools.
98+
You will not directly interact with the user in any way. Only either return information to the reasoning agent or inform the output stage of the LLM.
10499
105-
When you have used a tool. You can return the results to the reasoning agent by putting everything you want to return to them within <tool_to_reasoning> tags.
106-
You can also directly hand off to the final-agent by simply writing $TO_FINAL$. You still need to write out what you want them to get!
100+
When you have used a tool, you can return the results to the reasoning agent by putting everything you want to return to them within <tool_to_reasoning> tags.
101+
You can also directly hand off to the final agent by simply writing $TO_FINAL$. You still need to write out what you want them to get!
107102
108-
Actually make use of the results you got. NEVER make more than 3 tool calls! If you called any tool 3 times that's it!
109-
You need to output everything you want to pass on. The next agent in the chain will only see whay you actually wrote, not the direct output of the tools!
103+
Actually make use of the results you got. NEVER make more than 3 tool calls! If you called any tool 3 times, that's it!
104+
You need to output everything you want to pass on. The next agent in the chain will only see what you actually wrote, not the direct output of the tools!
110105
111-
Please think about how best to call the tool first. Think about what the limitations of the tools are and how to best follow the reasoning-agent instructions. It's okay if you can't 100% produce what they wanted!
106+
Please think about how best to call the tool first. Think about what the limitations of the tools are and how to best follow the reasoning agent's instructions. It's okay if you can't 100% produce what they wanted!
112107
</system_instructions>"""
113108

114109
USER_INTERACTION_PROMPT = """<system_instructions>
@@ -223,7 +218,7 @@ def setup(self):
223218
"api_key": v.GROQ_API_KEY,
224219
"base_url": "https://api.groq.com/openai/v1",
225220
}
226-
221+
227222
async def pipe(
228223
self,
229224
body: dict,
@@ -238,7 +233,7 @@ async def pipe(
238233
if __task__ == "function_calling":
239234
return
240235

241-
self.setup()
236+
self.setup()
242237

243238
called_model_id = body["model"]
244239
mini_mode = False
@@ -327,9 +322,6 @@ async def pipe(
327322
is_reasoning_needed = re.findall(r"<is_reasoning_or_tool_needed>(.*?)</is_reasoning_or_tool_needed>", content)
328323
is_reasoning_needed = is_reasoning_needed[0] if is_reasoning_needed else "unknown"
329324

330-
next_agent_preprompt = re.findall(r"<next_agent_preprompt>(.*?)</next_agent_preprompt>", content)
331-
next_agent_preprompt = next_agent_preprompt[0] if next_agent_preprompt else "unknown"
332-
333325
model_to_use_id = small_model_id
334326
if float(task_difficulty) >= 5:
335327
model_to_use_id = large_model_id
@@ -344,18 +336,61 @@ async def pipe(
344336
content=f"{content=}",
345337
)
346338

339+
# Try to find #!, #!!, #*yes, #*no, in the user message, let them overwrite the model choice
340+
if "#!" in body["messages"][-1]["content"]:
341+
model_to_use_id = small_model_id
342+
elif"#!!" in body["messages"][-1]["content"]:
343+
model_to_use_id = large_model_id
344+
if "#*yes" in body["messages"][-1]["content"]:
345+
is_reasoning_needed = "YES"
346+
elif "#*no" in body["messages"][-1]["content"]:
347+
is_reasoning_needed = "NO"
348+
349+
tools = []
350+
for key, value in __tools__.items():
351+
tools.append(
352+
StructuredTool(
353+
func=None,
354+
name=key,
355+
coroutine=value["callable"],
356+
args_schema=value["pydantic_model"],
357+
description=value["spec"]["description"],
358+
)
359+
)
360+
347361
model_to_use = ChatOpenAI(model=model_to_use_id, **self.openai_kwargs) # type: ignore
348362

349363
messages_to_use = body["messages"]
350364

351365
if is_reasoning_needed == "NO":
352366
messages_to_use[0]["content"] = messages_to_use[0]["content"] + USER_INTERACTION_PROMPT
353-
messages_to_use[-1]["content"] = str(messages_to_use[-1]["content"]).replace("#*yes", "").replace("#*no", "").replace("#!!", "").replace("#!", "") + "\n\n<preprompt>" + next_agent_preprompt + "</preprompt>"
354-
355-
async for chunk in model_to_use.astream(body["messages"], config=config):
356-
content = chunk.content
357-
assert isinstance(content, str)
358-
yield content
367+
messages_to_use[-1]["content"] = str(messages_to_use[-1]["content"]).replace("#*yes", "").replace("#*no", "").replace("#!!", "").replace("#!", "")
368+
369+
graph = create_react_agent(model_to_use, tools=tools)
370+
inputs = {"messages": body["messages"]}
371+
372+
num_tool_calls = 0
373+
async for event in graph.astream_events(inputs, version="v2", config=config):
374+
if num_tool_calls >= 4:
375+
break
376+
kind = event["event"]
377+
data = event["data"]
378+
if kind == "on_chat_model_stream":
379+
if "chunk" in data and (content := data["chunk"].content):
380+
yield content
381+
elif kind == "on_tool_start":
382+
yield "\n"
383+
await send_status(f"Running tool {event['name']}", False)
384+
elif kind == "on_tool_end":
385+
num_tool_calls += 1
386+
await send_status(
387+
f"Tool '{event['name']}' returned {data.get('output')}", True
388+
)
389+
await send_citation(
390+
url=f"Tool call {num_tool_calls}",
391+
title=event["name"],
392+
content=f"Tool '{event['name']}' with inputs {data.get('input')} returned {data.get('output')}",
393+
)
359394
return
360395
elif is_reasoning_needed == "YES":
361396
reasoning_model_id = self.valves.REASONING_MODEL
@@ -458,17 +493,6 @@ async def pipe(
458493
if not __tools__:
459494
tool_agent_response = "Tool agent could not use any tools because the user did not enable any."
460495
else:
461-
tools = []
462-
for key, value in __tools__.items():
463-
tools.append(
464-
StructuredTool(
465-
func=None,
466-
name=key,
467-
coroutine=value["callable"],
468-
args_schema=value["pydantic_model"],
469-
description=value["spec"]["description"],
470-
)
471-
)
472496
graph = create_react_agent(large_model, tools=tools)
473497
inputs = {"messages": tool_message}
474498
message_buffer = ""
@@ -515,10 +539,32 @@ async def pipe(
515539
messages_to_use[0]["content"] = messages_to_use[0]["content"] + USER_INTERACTION_PROMPT
516540
#messages_to_use[-1]["content"] = messages_to_use[-1]["content"] + "\n\n<preprompt>" + next_agent_preprompt + "</preprompt>"
517541

518-
async for chunk in model_to_use.astream(messages_to_use, config=config):
519-
content = chunk.content
520-
assert isinstance(content, str)
521-
yield content
542+
graph = create_react_agent(model_to_use, tools=tools)
543+
inputs = {"messages": messages_to_use}
544+
545+
num_tool_calls = 0
546+
async for event in graph.astream_events(inputs, version="v2", config=config):
547+
if num_tool_calls >= 4:
548+
break
549+
num_tool_calls += 1
550+
kind = event["event"]
551+
data = event["data"]
552+
if kind == "on_chat_model_stream":
553+
if "chunk" in data and (content := data["chunk"].content):
554+
yield content
555+
elif kind == "on_tool_start":
556+
yield "\n"
557+
await send_status(f"Running tool {event['name']}", False)
558+
elif kind == "on_tool_end":
559+
num_tool_calls += 1
560+
await send_status(
561+
f"Tool '{event['name']}' returned {data.get('output')}", True
562+
)
563+
await send_citation(
564+
url=f"Tool call {num_tool_calls}",
565+
title=event["name"],
566+
content=f"Tool '{event['name']}' with inputs {data.get('input')} returned {data.get('output')}",
567+
)
522568
return
523569

524570
else:

0 commit comments

Comments
 (0)