2023-11-24
Llama Packs - a community-driven pre-packaged module center that allows you to quickly launch your Large Language Model (LLM) applications. Importing these modules can be used for a wide range of use cases, from building a Streamlit application to setting up an advanced retrieval system based on Weaviate, to resume parsers that can perform structured data extraction. Equally important, you can customize these modules according to your preferences.
Overview
Llama Packs can be described in two ways:
- On one hand, they are pre-packaged modules that can be initialized with parameters and can be used out of the box to achieve a given use case (whether it's a complete RAG (Reusable, Additive, Composable) workflow, application templates, etc.). You can also import sub-modules (such as LLMs, query engines) for direct use.
- On the other hand, Llama Packs are also templates that you can view, modify, and use.
They can be downloaded with a single line of code using the llama_index Python library or the Command Line Interface (CLI):
Command Line Interface (CLI):
llamaindex-cli download-llamapack <pack_name> --download-dir <pack_directory>dir <pack_directory>
Python
from llama_index.llama_pack import download_llama_pack
# download and install dependencies
VoyageQueryEnginePack = download_llama_pack(
"<pack_name>", "<pack_directory>"
)
Llama Packs can span different levels of abstraction - some are complete pre-packaged templates (complete Streamlit / Gradio applications), while others combine smaller modules (such as our SubQuestionQueryEngine with Weaviate).
Example Demo
The best way to showcase the features of Llama Packs is to demonstrate an example. We will walk through a simple Llama Pack that provides users with a RAG pipeline setup with Voyage AI embedding.
First, we download and initialize the Pack with a set of documents:
from llama_index.llama_pack import download_llama_pack
# download pack
VoyageQueryEnginePack = download_llama_pack("VoyageQueryEnginePack", "./voyage_pack")
# initialize pack (assume documents is defined)
voyage_pack = VoyageQueryEnginePack(documents)
Each Llama Pack implements a get_modules() function that allows you to inspect/use the modules.
modules = voyage_pack.get_modules()
display(modules)
# get LLM, vector index
llm = modules["llm"]
vector_index = modules["index"]
Llama Packs can be run out of the box. By calling run, we execute the RAG pipeline and get a response. In this setup, you don't need to worry about the internals.
# this will run the full pack
response = voyage_pack.run("What did the author do growing up?", similarity_top_k=2)
print(str(response))
The author spent his time outside of school mainly writing and programming. He wrote short stories and attempted to write programs on an IBM 1401. Later, he started programming on a TRS-80, creating simple games and a word processor. He also painted still lives while studying at the Accademia.
The second important thing is that you have full access to the code of Llama Packs. This allows you to customize Llama Packs, remove code, or use it as a reference to build your own applications. Let's take a look at the downloaded pack in voyage_pack/base.py and replace OpenAI LLM with Anthropic:
from llama_index.llms import Anthropic
...
class VoyageQueryEnginePack(BaseLlamaPack):
def __init__(self, documents: List[Document]) -> None:
llm = Anthropic()
embed_model = VoyageEmbedding(
model_name="voyage-01", voyage_api_key=os.environ["VOYAGE_API_KEY"]
)
service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model)
self.llm = llm
self.index = VectorStoreIndex.from_documents(
documents, service_context=service_context
)
def get_modules(self) -> Dict[str, Any]:
"""Get modules."""
return {"llm": self.llm, "index": self.index}
def run(self, query_str: str, **kwargs: Any) -> Any:
"""Run the pipeline."""
query_engine = self.index.as_query_engine(**kwargs)
return query_engine.query(query_str)
You can directly import the module and run it again:
from voyage_pack.base import VoyageQueryEnginePack
voyage_pack = VoyageQueryEnginePack(documents)
response = voyage_pack.run("What did the author do during his time in RISD?")
print(str(response))
RECENT AI NEWS
RECENT AI TOOLS
Generate tattoo designs from text prompts.
Build and deploy full-stack web applications.
Translate text into multiple languages
Generate images and videos from text.
Create interactive lessons with AI.
Here is a summary of the main use case of the product in six words:“Generate marketing assets with AI assistance”.
Video to 3D models for animations
Create viral short videos from text.
Automatically create faceless short videos