[MODEL I/O - Langauge Models] LLM How to - 1

- 출처 : https://python.langchain.com/docs/modules/model_io/

- 이 블로그 글은 LangChain API document의 글을 기반으로 번역되었으며 이 과정에서 약간의 내용이 추가되었습니다.

- MODEL I/O는 Prompts, Language Model, Output Parser로 이루어져 있습니다.

- How-to에서는 Langchain에서 LLM을 다루는 여러 가지 방법에 대해 설명합니다.

- 본 글에서는 Async API, Custom LLM, Fake LLM, Human input LLM을 다룹니다.

 

 

 

1. Async API for LLMs

 

asyncio는 Python에서 사용할 수 있는 대표적인 비동기 지원 라이브러리입니다. Langchain은 asyncio 라이브러리를 활용하여 LLM에 대한 비동기 지원을 제공합니다.

 

비동기 지원은 여러 개의 LLM을 동시에 호출할 때 유용합니다. 이런 호출은 네크워크에 의존하기 때문입니다. 현재 OpenAI, PromptLayerOpenAI, ChatOpenAI, Anthropic이 지원되며, 다른 LLM에 대한 비동기 지원은 로드맵에 있습니다.

 

이번 세션에서는 agenerate 메서드를 사용하여 OpenAI LLM을 비동기적으로 호출하는 것을 다뤄보겠습니다.

import time
import asyncio
from langchain.llms import OpenAI

def generate_serially():
    llm = OpenAI(temperature = 0.9)
    for _ in range(10):
        resp = llm.generate(['hello, how are you?'])
        print(resp.generations[0][0].text)

async def async_generate(llm):
    resp = await llm.agenerate(['hello, how are you?'])
    print(resp.generations[0][0].text)

async def generate_concurrently():
    llm = OpenAI(temperature=0.9)
    tasks = [async_generate(llm) for _ in range(10)]
    await asyncio.gather(*tasks)

s = time.perf_counter()
generate_serially()
elapsed = time.perf_counter() - s
print('\033[1m' + f"Serial  executed in {elapsed:0.2f} seconds." + '\033[0m')

s = time.perf_counter()
await generate_concurrently()
elapsed = time.perf_counter() - s
print('\033[1m' + f"Concurrent executed in {elapsed:0.2f} seconds." + '\033[0m')

I'm doing great, thanks for asking. How about you?

...

I'm doing great, thank you for asking. How about you?
Serial  executed in 14.38 seconds.


I'm doing well, thank you. How about yourself?

...

I'm doing well. How about you?
Concurrent executed in 2.91 seconds.

 

 

 

2. Custom LLM Wrapper

LangChain에서 지원되는 LLM 대신 Custom LLM이나 다른 Wrapper를 사용하는 경우 Custom LLM Wrapper를 만드는 방법을 안내합니다. 

 

Custom LLM이 구현해야 할 유일한 필수 요소는 다음과 같습니다.

 - 문자열과 중단 단어(Optional)를 입력으로 받아 문자열을 반환하는 _call 메서드

 

추가로 구현할 수 있는 요소는 다음과 같습니다.

 - 클래스의 출력을 돕기 위해 사용되는 _identifying_params. 사전(dictionary)를 반환해야 합니다.

 

이번 세션에서는 입력의 첫 N개 문자만 반환하는 매우 간단한 사용자 정의 LLM을 구현해 보겠습니다.

from typing import Any, List, Mapping, Optional
from langchain.callbacks.manager import CallbackManagerForLLMRun
from langchain.llms.base import LLM

class CustomLLM(LLM):
    n : int
    
    @property
    def _llm_type(self) -> str:
        return 'custom'
    
    def _call(self, prompt: str, stop :Optional[List[str]] = None, run_manager : Optional[CallbackManagerForLLMRun] = None) -> str:
        if stop is not None:
            raise ValueError('stop kwargs are not permitted.')
        return prompt[:self.n]
    
    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        return {"n": self.n}
    
llm = CustomLLM(n = 10)

print(llm("This is a foobar thing"))
print('\n')
print(llm)

This is a 


CustomLLM
Params: {'n': 10}

 

 

 

3. Fake LLM

이번에는 테스트에 사용할 수 있는 Fake LLM 클래스를 소개합니다. 이 클래스는 LLM에 대한 호출을 가로채고, LLM이 특정한 방식으로 응답하는 경우 어떻게 작용하는지 시뮬레이션할 수 있습니다.

 

from langchain.llms.fake import FakeListLLM
from langchain.agents import load_tools, initialize_agent, AgentType

tools = load_tools(['python_repl'])
responses = [
    "Action: Python REPL\nAction Input: print(2 + 2)",
    'Final Answer: 4'
]
llm = FakeListLLM(responses=responses)
agent = initialize_agent(tools, llm, agent = AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose = True)
agent.run('what is 2 + 2')

> Entering new  chain...
Action: Python REPL
Action Input: print(2 + 2)
Observation: Python REPL is not a valid tool, try another one.
Thought:Final Answer: 4

> Finished chain.
'4'

 

 

 

4. Human Input LLM

 

HumanInputLLM 클래스는 Fake LLM과 유사하게 테스트, 디버깅 혹은 교육 목적으로 사용할 수 있는 pseudo LLM 클래스입니다.  이 클래스를 통해 LLM에 대한 호출을 가로채고, 프롬프트를 받았을 때 인간이 어떻게 응답하는지 시뮬레이션할 수 있습니다.

 

이 세션은 출력에 오류가 있어 원본 페이지의 내용으로 대체합니다.

from langchain.llms.human import HumanInputLLM
from langchain.agents import load_tools, initialize_agent, AgentType

tools = load_tools(['wikipedia'])
llm = HumanInputLLM(prompt_func=lambda prompt: print(f"\n===PROMPT====\n{prompt}\n=====END OF PROMPT======"))
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose = True)
agent.run("What is 'Bocchi the Rock!'?")

    
    
    > Entering new AgentExecutor chain...
    
    ===PROMPT====
    Answer the following questions as best you can. You have access to the following tools:
    
    Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.
    
    Use the following format:
    
    Question: the input question you must answer
    Thought: you should always think about what to do
    Action: the action to take, should be one of [Wikipedia]
    Action Input: the input to the action
    Observation: the result of the action
    ... (this Thought/Action/Action Input/Observation can repeat N times)
    Thought: I now know the final answer
    Final Answer: the final answer to the original input question
    
    Begin!
    
    Question: What is 'Bocchi the Rock!'?
    Thought:
    =====END OF PROMPT======
    I need to use a tool.
    Action: Wikipedia
    Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.
    Observation: Page: Bocchi the Rock!
    Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
    
    Page: Manga Time Kirara
    Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.
    
    Page: Manga Time Kirara Max
    Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.
    Thought:
    ===PROMPT====
    Answer the following questions as best you can. You have access to the following tools:
    
    Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.
    
    Use the following format:
    
    Question: the input question you must answer
    Thought: you should always think about what to do
    Action: the action to take, should be one of [Wikipedia]
    Action Input: the input to the action
    Observation: the result of the action
    ... (this Thought/Action/Action Input/Observation can repeat N times)
    Thought: I now know the final answer
    Final Answer: the final answer to the original input question
    
    Begin!
    
    Question: What is 'Bocchi the Rock!'?
    Thought:I need to use a tool.
    Action: Wikipedia
    Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.
    Observation: Page: Bocchi the Rock!
    Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
    
    Page: Manga Time Kirara
    Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.
    
    Page: Manga Time Kirara Max
    Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.
    Thought:
    =====END OF PROMPT======
    These are not relevant articles.
    Action: Wikipedia
    Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.
    Observation: Page: Bocchi the Rock!
    Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
    Thought:
    ===PROMPT====
    Answer the following questions as best you can. You have access to the following tools:
    
    Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.
    
    Use the following format:
    
    Question: the input question you must answer
    Thought: you should always think about what to do
    Action: the action to take, should be one of [Wikipedia]
    Action Input: the input to the action
    Observation: the result of the action
    ... (this Thought/Action/Action Input/Observation can repeat N times)
    Thought: I now know the final answer
    Final Answer: the final answer to the original input question
    
    Begin!
    
    Question: What is 'Bocchi the Rock!'?
    Thought:I need to use a tool.
    Action: Wikipedia
    Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.
    Observation: Page: Bocchi the Rock!
    Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
    
    Page: Manga Time Kirara
    Summary: Manga Time Kirara (まんがタイムきらら, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.
    
    Page: Manga Time Kirara Max
    Summary: Manga Time Kirara Max (まんがタイムきららMAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.
    Thought:These are not relevant articles.
    Action: Wikipedia
    Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.
    Observation: Page: Bocchi the Rock!
    Summary: Bocchi the Rock! (ぼっち・ざ・ろっく!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankōbon volumes as of November 2022.
    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
    Thought:
    =====END OF PROMPT======
    It worked.
    Final Answer: Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.
    
    > Finished chain.





    "Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim."