Introducing Semantic Caching and a Dedicated MongoDB LangChain Package for Gen AI Apps

Prakul Agarwal, Erick Friis, and Jacob Lee
March 20, 2024 | Updated: March 25, 2024
#genAI

We are in an unprecedented time in history where developers can build transformative AI applications quickly, without being AI experts themselves. This ability is enabling new classes of applications that can better serve customers with conversational AI for assistance and automation, advanced reasoning and analysis using AI-powered retrieval, and recommendation systems.

Behind this revolution are large language models (LLMs) that can be prompted to solve for a wide range of use cases. However, LLMs have various limitations, like knowledge cutoff and a tendency to hallucinate. To overcome these limitations, they must be integrated with proprietary enterprise data sources to build reliable, relevant, and high-quality generative AI applications. That’s where MongoDB plays a critical role in the modern generative AI stack.

Developers use MongoDB Atlas Vector Search as a vital part of the generative AI technique known as retrieval-augmented generation (RAG). RAG is the process of feeding LLMs the supplementary data necessary to ground their responses, ensuring they're dependable and precise. LangChain has been a critical part of this journey since the public launch of Atlas Vector Search, enabling developers to build better retriever systems powered by vector search and store conversation history in the operational database.

Today, we are excited to announce support for two enhancements:

  1. Semantic cache powered by Atlas vector search, which improves the performance of your apps

  2. A dedicated LangChain-MongoDB package for Python and JS/TS developers, enabling them to build advanced applications even more efficiently

The MongoDB Atlas integration with LangChain can now power all the database requirements for building modern generative AI applications: vector search, semantic caching (currently only available in Python), and conversation history.

Earlier, we announced the launch of MongoDB LangChain Templates, which enable the developers to quickly deploy RAG applications, and provided a reference implementation of a basic RAG template using MongoDB Atlas Vector Search and OpenAI and a more advanced Parent-document Retrieval RAG template using MongoDB Atlas Vector Search. We are excited about our partnership with LangChain and will continue innovating.

Improve LLM application performance with semantic cache

Semantic cache improves the performance of LLM applications by caching responses based on the semantic meaning or context within the queries themselves. This is different from a traditional cache that works based on exact keyword matching. In the era of LLM the value of semantic cache is increasing tremendously, enabling sophisticated user experiences that closely mimic human interactions. For example, if two different users enter two different prompts, “give me suggestions for a comedy movie” and “recommend a comedy movie”, the semantic cache can understand that the intent behind the queries are same and return a similar response, even though different keywords are used, whereas a traditional cache will fail.

Diagram of using semantic cache using MongoDB Atlas Vector Search. Before using cache, each use query sends to LLM, which takes up time and money. With the cache, the system checks if the query has already been processed by the LLM and only new queries are sent to the LLM. Saving time and money
Figure 1: Semantic cache using MongoDB Atlas Vector Search

Check out this video walkthrough for the semantic cache:

Accelerate development with a dedicated package

With a dedicated LangChain-MongoDB package, MongoDB is even more deeply integrated with LangChain. The Python and Javascript packages contain the following LangChain Integrations: MongoDBAtlasVectorSearch (Vector stores) and MongoDBChatMessageHistory (Chat Messages Memory). In addition, the Python package includes the MongoDBAtlasSemanticCache (LLM Caching).

The new package langchain-mongodb contains all the MongoDB-specific implementations and needs to be installed separately from langchain, which includes all the core abstractions. Earlier, everything was in the same package, making it challenging to correctly version and communicate what version should be used and whether any breaking changes were made.

Find out more about the langchain-mongodb package:

Get started today

  • Check out this accompanying tutorial and notebook on building advanced RAG with MongoDB and LangChain, which contains a walkthrough and use cases for using semantic cache, vector search, and chat message history.

  • Check out the “PDFtoChat” app to see langchain-mongodb JS in action. It allows you to have a conversation with your proprietary PDFs using AI and is built with MongoDB Atlas, LangChain.js, and TogetherAI. It’s an end-to-end SaaS-in-a-box app and includes user authentication, saving PDFs, and saving chats per PDF.

  • Read the excellent overview of semantic caching using LangChain and MongoDB.

← Previous

New Data Modeling Learning Path and Certification

This post is also available in: Deutsch , Français , Español , Português , Italiano , 한국어 , 简体中文 . Data modeling is a crucial part of software development. It's what gives structure to the data so that it can be analyzed, used to make decisions, and built into useful applications. It can be complex and challenging, especially when you're juggling performance and maintainability. But here's the good news: a well-designed data model can boost your app's performance. That's why we've put together a new learning path and certification to help you level up your data modeling skills. Whether you're a seasoned developer looking to deepen your understanding or a newcomer eager to learn, this content is designed to give you the tools you need to create efficient and effective data models. Let's dive in and start building better apps together. Deep dive into data modeling with MongoDB The new free, online MongoDB Data Modeling Path will help you build your knowledge from the ground up and prepare you for the certification exam. The curated sequence of videos, hands-on labs, and quizzes provides you with a guided journey to learning data modeling at your own pace. The learning path will show you a step-by-step method for creating effective data models. You'll learn how to identify entities and workloads, map relationships, and when to embed or reference them. In addition, you'll learn different schema design patterns, how to recognize and address MongoDB anti-patterns, and the basics of schema lifecycle management. And, here's a little something extra: Once you complete the learning path, you'll automatically score a 50% discount on the new certification ! Learn fast, certify faster With an extensive array of new educational resources on data modeling for MongoDB, we are thrilled to introduce our new credential, the MongoDB Associate Data Modeler certification , which will allow you to validate and showcase your expertise in the field. Our certifications are officially recognized by professional institutions, validating and acknowledging your MongoDB expertise. They're a valuable asset for advancing in your role and enhancing your marketability for future positions. Certified individuals earn bragging rights, inclusion in the Credly Talent Directory , and a distinguished Credly badge, making it easy to share your achievements. The MongoDB Associate Data Modeler certification is designed for experienced users. Candidates should have some familiarity with JSON, and the MongoDB Query API , including aggregations and data modeling on MongoDB. You should also understand the tradeoffs of simplicity versus performance of data modeling techniques. This certification attests to your competency in designing, building, and evolving effective data models in MongoDB, and incorporating data governance into your work. Explore MongoDB’s new learning path and certification designed to elevate and validate your data modeling skills.

March 20, 2024
Next →

Building AI With MongoDB: Integrating Vector Search And Cohere to Build Frontier Enterprise Apps

Cohere is the leading enterprise AI platform, building large language models (LLMs) which help businesses unlock the potential of their data. Operating at the frontier of AI, Cohere’s models provide a more intuitive way for users to retrieve, summarize, and generate complex information. Cohere offers both text generation and embedding models to its customers. Enterprises running mission-critical AI workloads select Cohere because its models offer the best performance-cost tradeoff and can be deployed in production at scale. Cohere’s platform is cloud-agnostic. Their models are accessible through their own API as well as popular cloud managed services, and can be deployed on a virtual private cloud (VPC) or even on-prem to meet companies where their data is, offering the highest levels of flexibility and control. Cohere’s leading Embed 3 and Rerank 3 models can be used with MongoDB Atlas Vector Search to convert MongoDB data to vectors and build a state-of-the-art semantic search system. Search results also can be passed to Cohere’s Command R family of models for retrieval augmented generation (RAG) with citations. Check out our AI resource page to learn more about building AI-powered apps with MongoDB. A new approach to vector embeddings It is in the realm of embedding where Cohere has made a host of recent advances. Described as “AI for language understanding,” Embed is Cohere’s leading text representation language model. Cohere offers both English and multilingual embedding models, and gives users the ability to specify the type of data they are computing an embedding for (e.g., search document, search query). The result is embeddings that improve the accuracy of search results for traditional enterprise search or retrieval-augmented generation. One challenge developers faced using Embed was that documents had to be passed one by one to the model endpoint, limiting throughput when dealing with larger data sets. To address that challenge and improve developer experience, Cohere has recently announced its new Embed Jobs endpoint . Now entire data sets can be passed in one operation to the model, and embedded outputs can be more easily ingested back into your storage systems. Additionally, with only a few lines of code, Rerank 3 can be added at the final stage of search systems to improve accuracy. It also works across 100+ languages and offers uniquely high accuracy on complex data such as JSON, code, and tabular structure. This is particularly useful for developers who rely on legacy dense retrieval systems. Demonstrating how developers can exploit this new endpoint, we have published the How to use Cohere embeddings and rerank modules with MongoDB Atlas tutorial . Readers will learn how to store, index, and search the embeddings from Cohere. They will also learn how to use the Cohere Rerank model to provide a powerful semantic boost to the quality of keyword and vector search results. Figure 1: Illustrating the embedding generation and search workflow shown in the tutorial Why MongoDB Atlas and Cohere? MongoDB Atlas provides a proven OLTP database handling high read and write throughput backed by transactional guarantees. Pairing these capabilities with Cohere’s batch embeddings is massively valuable to developers building sophisticated gen AI apps. Developers can be confident that Atlas Vector Search will handle high scale vector ingestion, making embeddings immediately available for accurate and reliable semantic search and RAG. Increasing the speed of experimentation, developers and data scientists can configure separate vector search indexes side by side to compare the performance of different parameters used in the creation of vector embeddings. In addition to batch embeddings, Atlas Triggers can also be used to embed new or updated source content in real time, as illustrated in the Cohere workflow shown in Figure 2. Figure 2: MongoDB Atlas Vector Search supports Cohere’s batch and real time workflows. (Image courtesy of Cohere) Supporting both batch and real-time embeddings from Cohere makes MongoDB Atlas well suited to highly dynamic gen AI-powered apps that need to be grounded in live, operational data. Developers can use MongoDB’s expressive query API to pre-filter query predicates against metadata, making it much faster to access and retrieve the more relevant vector embeddings. The unification and synchronization of source application data, metadata, and vector embeddings in a single platform, accessed by a single API, makes building gen AI apps faster, with lower cost and complexity. Those apps can be layered on top of the secure, resilient, and mature MongoDB Atlas developer data platform that is used today by over 45,000 customers spanning startups to enterprises and governments handling mission-critical workloads. What's next? To start your journey into gen AI and Atlas Vector Search, review our 10-minute Learning Byte . In the video, you’ll learn about use cases, benefits, and how to get started using Atlas Vector Search.

April 25, 2024