Building a SaaS-ready, RAG-powered chatbot with the GPT-4o (Omni) model

Create your retrieval-augmented generation (RAG) app with user authentication using prompts in Databutton’s agentic framework

If you're looking to build a business on an AI-driven product, Retrieval-Augmented Generation (RAG) chatbots are a very popular topic. Just Google "Chat PDF" and you'll find dozens of companies building a SaaS business on that very topic. This blog post will provide the technical details to help you build one from scratch in Databutton. The information will also make it easier for you to write appropriate prompts and share context with Databutton's agentic framework, making the whole process straightforward and simple.

My earlier posts mostly cover the core concepts and functional aspects of building such a chatbot. However, while these foundational elements are important, the question remains: is this product ready to be an MVP for customer demos or early testing? 

AI Chatbot with your Knowledge base

Build AI-powered chat on your business data using OpenAI and retrieval augmented generation (RAG) — All Through Prompts

The Databutton BlogAvra

So, to make our app market-ready … the first thing I would prioritise is adding a user-authentication feature. It secures the app, manages user access, and ensures that only authorised users can use it. We'll use Firebase for user authentication, the preferred service for most application developers.

Next… how about improving and fine-tuning our RAG framework?

We'll make a couple of key improvements to enhance the speed and quality of results. After all, the value of a chatbot like this is based not only on the accuracy of its answers but also on how quickly it produces them. 

Reuse the stored embeddings – We can use an open-source vector database, like Chroma DB. This database stores indexed content (e.g. from uploaded PDFs) - allowing users to ask questions based on that content without needing to re-index again. This approach saves time and reduces costs.

Augmented Generation – Update our RAG framework the latest AI model. For better speed in the augmentation process, using OpenAI’s latest model, GPT-4 Omni, is the best choice. It provides with fast speed and affordable pricing. Since gpt-4o provides a 128,000 token size as context window, we can easily pass additional context from the similarity search we perform from the vector database. This aids the augmented response with a better quality of response. Furthermore, leveraging this expansive context window ensures that the chatbot can handle more complex queries with greater efficiency and accuracy.

As before, we use the Langchain python library as the orchestration tool to handle parsing, chunking, and embedding the document into the vector database. Langchain also performs similarity searches and generates responses based on the relevant context.

But the biggest advantage here is building this whole functional app just via prompting with Databutton’s Agentic Framework.

Databutton is an online platform where you can build full-stack apps conversationally from scratch! Prompting is key while building in Databutton. 

• The functional part (backend) is generated in Python and available as API endpoints, which can be called from the frontend.
• The UI (frontend) is generated in React JS.

Read more here — 

AI Agent Frameworks for Full Stack App Development & Software Engineering

How AI Agent Frameworks like Databutton and Devin pushing the LLMs limits

The Databutton BlogAvratanu Biswas

Building Next-Gen Apps with AI Agents

For businesses that want to become efficient and win with the use of custom internal tools and automation.

The Databutton BlogAvratanu Biswas

1. Adding Authentication to the app 

Every Databutton app comes with a default authentication wrapper template as a UI component.

If we prompt the Databutton agent, it can quickly add authentication to our UI. Here’s a prompt to use:

I would like to add the component #AuthWrapper and protect my page

Let's look into the code where the Agent added the AuthWrapper component successfully in the main app.

....

return (
    <AuthWrapper>
      <Flex w="full" h="full" direction="column">
        <Box
          p={2}
          borderBottom="1px"
          textAlign="center"
          borderColor="lightgray"
          bg="black"
        >
          <Heading size="2xl" fontWeight="bold" color="white" mt={4}>
            PDF Knowledge Bot
          </Heading>
....

</AuthWrapper>
  );
}

However, there is some configurations necessary when working with Firebase Authentication (key steps jotted down below) : 

1. Setting up Firebase Authentication
2. Adding Firebase config to the template manually or via prompting
3. Configuring Sign-in methods
4. Adding Authorized domains

For detailed instructions on getting started with firebase, check out this comprehensive documentation from Databutton.

2. Building the initial UI in React

Best approach while building the UI —  is to think through the page in terms of UI components (reusable building blocks for the front-end) .

Here’s the prompt used — 

I would like to build an UI component. — A text input box for PDF URL ( light gray placeholder color and borders in black ) — A text input box for the unique name — A black confirm button. All these within a sidebar.

Next, for building a simple chat component — 

A chat component in the middle of the main app. 
— Prompt input box and a send button. 
— The output of chat : Chat should have an AI and UI message thread like a conversation.

We now have a blueprint of our app! This is a good start for our UI.

We can integrate the backend later and make further improvements to the frontend as needed.

3. Building the Backend (API endpoints) in Python

While building the backend it’s very crucial that we prompt our backend’s input and output model well. 

Here’s a detailed write up on Databutton building Python backend.

AI agents writing Python code (FastAPI Routers) for Full-Stack App Development

Databutton — a Full-Stack AI App builder, leverages an Agentic framework to build Python(FastAPI routers) backend and a React powered…

The Databutton BlogAvratanu Biswas

For this RAG framework, we need at least 4 API endpoints to make it work! Much like with the UI, it's important to consider how many distinct core functionalities or "input/outputs" your application has.

• Upload the PDF — The idea is that our end-users will upload a PDF URL from the front end. The backend will then load the URL, split the content, and prepare it for embedding.

But how did I prompt it successfully? Here’s my prompt:

Note — Pasting docs and additional context always help Databutton Agents to perform better. 

• Embed the Texts — Another key step in RAG is embedding the texts. We dedicated an entire endpoint for this — handling both embedding and storing the index on Databutton.

Databutton has its own storage, and the Databutton agents are very familiar with their internal SDKs. While prompting, make sure to include the documentation related to embedding from Langchain and ask the agent to use its own SDK to upload the indexed data.

It’s worthwhile to note the usage of Databutton’s SDK here. 
This functionality allows us to upload the indexed vector database to Databutton’s storage with a unique name provided by the end-user.

db.experimental.folder.upload(request.chroma_name, chroma_dir)

• Performing Similarity Search — Our next step is to perform a similarity search based on the user query and retrieve the results from the stored index. The retrieved results will be further passed to the LLM along with custom Prompt to create enhanced response from the LLM.

This allows our end-users to query their stored indexed data on Databutton’s storage. As mentioned earlier, by reusing the stored data and bypassing the embedding step, we enhance reusability and ensure cost-effectiveness.

• Augment Generation with streaming — Using the latest GPT-4 Omni model, the AI will generate the final response efficiently.
  Here’s the prompt used to create such backends in Databutton —

I would like to create a simple chatbot style backend. Where users give an input and Open AI LLM respond to it. The response must be streaming to the front-end.

Databutton under-the-hood takes care of the streaming implementation.

4. Integrating the backend, Iterating Over the UI and Deploying the app 

While integrating the backend (API routers) with the UI, providing the agent with a clear and concise prompt is crucial. Here’s a detailed doc on practical tips and tricks while working with Databutton —  https://docs.databutton.com/databutton-tips-and-tricks/practical-tips-and-strategies

Using the hashtag for Integration the backend with the UI — By using the “#” in the prompt input box, all the UI components and API endpoints pop up. This helps to naturally guide which endpoint must be integrated where.

UI Component Level → Main UI Page — In Databutton, it’s a good practice to work on the UI component level and gradually stitch the components together into a full app (page). This approach allows the agent to handle small amounts of code at a time.

Improving the UI to make it market-ready and attractive is one of the most critical and time-consuming steps when building complex apps with numerous functionalities. Note — this isn’t magical work in a single day where everything just falls into place. It’s an iterative and conversational process between the Databutton Agent and the user. 

App Deployment — Deployment is super simple in Databutton! With just one click, the app is accessible with an unique URL ( of course we can further customize the url or add our own domain )

The app can be accessed via this link — https://avra.databutton.app/chat-with-pdfurls

Databutton is making it possible to build amazing apps through conversational prompting. Start building with Databutton today!