No Code App Builder

Building a SaaS-ready, RAG-powered chatbot with the GPT-4o (Omni) model

Avratanu Biswas

May 21, 2024 — 9 min read

Create your retrieval-augmented generation (RAG) app with user authentication using prompts in Databutton’s agentic framework

If you're looking to build a business on an AI-driven product, Retrieval-Augmented Generation (RAG) chatbots are a very popular topic. Just Google "Chat PDF" and you'll find dozens of companies building a SaaS business on that very topic. This blog post will provide the technical details to help you build one from scratch in Databutton. The information will also make it easier for you to write appropriate prompts and share context with Databutton's agentic framework, making the whole process straightforward and simple.

My earlier posts mostly cover the core concepts and functional aspects of building such a chatbot. However, while these foundational elements are important, the question remains: is this product ready to be an MVP for customer demos or early testing?

So, to make our app market-ready … the first thing I would prioritise is adding a user-authentication feature. It secures the app, manages user access, and ensures that only authorised users can use it. We'll use Firebase for user authentication, the preferred service for most application developers.

User-authentication screen connected with Firebase.

Next… how about improving and fine-tuning our RAG framework?

We'll make a couple of key improvements to enhance the speed and quality of results. After all, the value of a chatbot like this is based not only on the accuracy of its answers but also on how quickly it produces them.

Reuse the stored embeddings – We can use an open-source vector database, like Chroma DB. This database stores indexed content (e.g. from uploaded PDFs) - allowing users to ask questions based on that content without needing to re-index again. This approach saves time and reduces costs.

Note: Here’s a quick overview of why we store embeddings. The main purpose is **reusability** — so we don’t have to embed the content of the PDF file every time. This approach is both cost-effective and a time saver.

Augmented Generation – Update our RAG framework the latest AI model. For better speed in the augmentation process, using OpenAI’s latest model, GPT-4 Omni, is the best choice. It provides with fast speed and affordable pricing. Since gpt-4o provides a 128,000 token size as context window, we can easily pass additional context from the similarity search we perform from the vector database. This aids the augmented response with a better quality of response. Furthermore, leveraging this expansive context window ensures that the chatbot can handle more complex queries with greater efficiency and accuracy.

As before, we use the Langchain python library as the orchestration tool to handle parsing, chunking, and embedding the document into the vector database. Langchain also performs similarity searches and generates responses based on the relevant context.

But the biggest advantage here is building this whole functional app just via prompting with Databutton’s Agentic Framework.

Databutton is an online platform where you can build full-stack apps conversationally from scratch! Prompting is key while building in Databutton.

The functional part (backend) is generated in Python and available as API endpoints, which can be called from the frontend.
The UI (frontend) is generated in React JS.

1. Adding Authentication to the app

Every Databutton app comes with a default authentication wrapper template as a UI component.

The AuthWrapper component in Databutton’s App.

If we prompt the Databutton agent, it can quickly add authentication to our UI. Here’s a prompt to use:

I would like to add the component #AuthWrapper and protect my page

Let's look into the code where the Agent added the AuthWrapper component successfully in the main app.

....

return (
    <AuthWrapper>
      <Flex w="full" h="full" direction="column">
        <Box
          p={2}
          borderBottom="1px"
          textAlign="center"
          borderColor="lightgray"
          bg="black"
        >
          <Heading size="2xl" fontWeight="bold" color="white" mt={4}>
            PDF Knowledge Bot
          </Heading>
....

</AuthWrapper>
  );
}

However, there is some configurations necessary when working with Firebase Authentication (key steps jotted down below) :

Setting up Firebase Authentication
Adding Firebase config to the template manually or via prompting
Configuring Sign-in methods
Adding Authorized domains

For detailed instructions on getting started with firebase, check out this comprehensive documentation from Databutton.

2. Building the initial UI in React

Best approach while building the UI — is to think through the page in terms of UI components (reusable building blocks for the front-end) .

We created the sidebar UI component and integrated to the main page.

Here’s the prompt used —

I would like to build an UI component. — A text input box for PDF URL ( light gray placeholder color and borders in black ) — A text input box for the unique name — A black confirm button. All these within a sidebar.

Next, for building a simple chat component —

A chat component in the middle of the main app.
— Prompt input box and a send button.
— The output of chat : Chat should have an AI and UI message thread like a conversation.

Created the chat UI component and further integrated it to the main page.

We now have a blueprint of our app! This is a good start for our UI.

We can integrate the backend later and make further improvements to the frontend as needed.

3. Building the Backend (API endpoints) in Python

While building the backend it’s very crucial that we prompt our backend’s input and output model well.

Example of an initial prompt to the Backend Agent.

Here’s a detailed write up on Databutton building Python backend.

For this RAG framework, we need at least 4 API endpoints to make it work! Much like with the UI, it's important to consider how many distinct core functionalities or "input/outputs" your application has.

The backend has at least four endpoints (represented in blue). The schematic shows how the backend and frontend are integrated in brief. For example, when the "Confirm" button is pressed at the UI, it triggers two API endpoints: Upload PDF and Text to Embeddings. When the "Send" button is pressed, it triggers the Similarity Search endpoint, which takes in the user query to search the stored embeddings. The gathered contexts are then passed to the Augmented Generation endpoint to generate a response.

Upload the PDF — The idea is that our end-users will upload a PDF URL from the front end. The backend will then load the URL, split the content, and prepare it for embedding.

But how did I prompt it successfully? Here’s my prompt:

Left image — Prompt to create the upload PDF functionality. Right Image — Code generated by Databutton Agent

Note — Pasting docs and additional context always help Databutton Agents to perform better.

Embed the Texts — Another key step in RAG is embedding the texts. We dedicated an entire endpoint for this — handling both embedding and storing the index on Databutton.

Databutton has its own storage, and the Databutton agents are very familiar with their internal SDKs. While prompting, make sure to include the documentation related to embedding from Langchain and ask the agent to use its own SDK to upload the indexed data.

Generated endpoint for storing embeddings over Databutton storage.

It’s worthwhile to note the usage of Databutton’s SDK here.
This functionality allows us to upload the indexed vector database to Databutton’s storage with a unique name provided by the end-user.

db.experimental.folder.upload(request.chroma_name, chroma_dir)

Performing Similarity Search — Our next step is to perform a similarity search based on the user query and retrieve the results from the stored index. The retrieved results will be further passed to the LLM along with custom Prompt to create enhanced response from the LLM.

This allows our end-users to query their stored indexed data on Databutton’s storage. As mentioned earlier, by reusing the stored data and bypassing the embedding step, we enhance reusability and ensure cost-effectiveness.

Left Image — Code generated for performing the similarity search. Right Image — Databutton Agent collaboratively test the code and ensures that it works well within the chat option.

Augment Generation with streaming — Using the latest GPT-4 Omni model, the AI will generate the final response efficiently.
Here’s the prompt used to create such backends in Databutton —

I would like to create a simple chatbot style backend. Where users give an input and Open AI LLM respond to it. The response must be streaming to the front-end.

Databutton under-the-hood takes care of the streaming implementation.

Left Image — Generating the backend. Right Image — By iterating over the endpoint, we can easily tweak the functionality to our use case. For example, we can pass relevant contexts or update the model to `gpt-4o-2024-05-13`. Note: Databutton’s code editor offers the flexibility to edit and rewrite the code as needed.

4. Integrating the backend, Iterating Over the UI and Deploying the app

While integrating the backend (API routers) with the UI, providing the agent with a clear and concise prompt is crucial. Here’s a detailed doc on practical tips and tricks while working with Databutton — https://docs.databutton.com/databutton-tips-and-tricks/practical-tips-and-strategies

Using the hashtag for Integration the backend with the UI — By using the “#” in the prompt input box, all the UI components and API endpoints pop up. This helps to naturally guide which endpoint must be integrated where.

Left Image — Using the hastag over the prompt input box. Right Image — Prompting to integrate the API endpoints to the UI.

UI Component Level → Main UI Page — In Databutton, it’s a good practice to work on the UI component level and gradually stitch the components together into a full app (page). This approach allows the agent to handle small amounts of code at a time.

UI components are building blocks of the UI ( pages )

Improving the UI to make it market-ready and attractive is one of the most critical and time-consuming steps when building complex apps with numerous functionalities. Note — this isn’t magical work in a single day where everything just falls into place. It’s an iterative and conversational process between the Databutton Agent and the user.

App Deployment — Deployment is super simple in Databutton! With just one click, the app is accessible with an unique URL ( of course we can further customize the url or add our own domain )