Memory-augmented LLMs: Retrieving Information using a Gated Encoder LLM (RIGEL)
MLabs, an AI and Blockchain consultancy, seeks funding through RFP3 - Memory-augmented LLMs. They propose addressing the shortcomings of large language models (LLMs), such as ChatGPT, concerning the veracity of generated responses. MLabs aims to develop a new module for LLMs, enabling them to reference the source material, providing explanation and evidence to support responses, thereby reducing the risk of factually incorrect outputs or hallucinations. Their solution involves compressing context vectors and storing them externally, using hierarchical self-attention for efficient data compression and retrieval.
The project includes three parts: large-scale contextual compression experiments, training and evaluating the compression and retrieval module, and integration with the existing LLM. The total funding request is USD 140,000 for a 36-week project. Risks include the novelty of the concept, vector retrieval system performance, machine learning training, software development, and project management, which they aim to mitigate with their experienced team and contingency plans.
MLabs has a proven track record in ML/AI deployments and aims to leverage their expertise to enhance LLMs.