This article was authored by a member of the HubSpot developer community, Roman Kozak.
As generative AI appears more stable and safe, its integration across various products and broad usage in digital marketing is becoming a clear advantage. AI-powered chatbots or assistants will find their place in different areas. So, the competition will be shifted to communicational interface designs to serve user needs instantly.
Building a chatbot like ChatGPT might look like an easy job today, but it's not. So, why is it difficult? Well, more is needed than just to hook up generative AI API to a chat interface. It still requires a few things: managing prompt and context windows, saving conversations to a database to preserve a chat history, and enriching the chatbot with business-related specific information. The last task itself is complex and requires a knowledge database with embeddings and semantic search by the user’s query. The bottom line is that good AI chatbots are possible to create, but there have been a lot of headaches with them until recently.
Lately, OpenAI presented a game-changing tool – Assistants API, making it much easier to build self-hosted chatbots based on GPT. It handles all the mentioned hurdles and provides a convenient API. To demonstrate an example with Assistants API, I will show how to build an AI assistant in HubSpot that will perform lead qualification screening by following the BANT framework. Here is the page with the working demo. You can play with it by pretending to be interested in marketing agency services, Alpha.
Prerequisites
Before we dive deep into Assistants API, a few things are expected. You have an OpenAI dev account and are familiar with the OpenAI dashboard. You know some basics of LLM prompt engineering and have basic knowledge of JavaScript. Also, this tutorial is supplemented by demo code and bot programming instructions, which you can find in my Github repo.
Assistants API
To better understand Assistants API, perceive it as a collection of pre-built tools in a single package.
At its core lies the well-known AI text generation service. It simply takes a given text and returns the best guess of how it might evolve if a human continues writing it. The provided initial text is commonly known as a prompt. Writing prompts might be tedious and require some practice. By using Assistants API, you will get a setup with a default prompt to ensure that the AI generates text in a dialogue form. So, it is to be sure you get a chatbot behavior. But you can still add instructions for how you want the model to generate responses in a dialogue. Typically, in these instructions, you might want to steer the model to respond as a virtual assistant – hence the name Assistants API.
Beyond combining instructions for the bot and dialogues into one prompt, the tool also ensures that with the progression of the conversation, the prompt remains relatively small while retaining the core information about the entire conversation. OpenAI, at a certain point, starts to summarize the conversation to preserve the context and maintain the prompt in the boundaries. Regardless of automatic conversation distortion for a prompt, all conversation is stored intact as a thread with a unique ID on OpenAI servers that can be retrieved, previewed, or continued as chat at any time.
An Assistant can also leverage additional tools to get extra info or perform math operations to support a conversation that requires more data. The Assistants API currently supports three types of tools: Code Interpreter, Knowledge Retrieval, and Custom Function calling. Here is the video where OpenAI folks show everything in action during the presentation day.
You can explore the Assistants API through the Assistants playground or by visiting OpenAI's Assistants API overview docs.
Setting Up a Custom Assistant
After getting general knowledge about Assistants API, let’s delve into its practical aspects by creating a copy of the demo assistant. To begin, navigate to the OpenAI's Assistants page. Here, you'll find a straightforward user interface. Click the “Create” button to start the process. Then, you will see a sidebar on the right. Then follow these next steps:
- Enter the name field: “Alpha's Sales Rep”.
- Copy instructions from this file and paste them in the Instruction box.
- Choose the latest GPT4 model in the model drop-down.
- We don’t need extra tools. Keep it as is.
- Save it.
If you’re curious how the assistant performs, click the “Test” button in the upper-right corner. It allows you to interact with your assistant in a sandbox environment and ensure the assistant works well.
Here's a short video walking through these steps:
Creating Custom Bot
Let’s integrate our demo assistant into HubSpot. We'll start by creating a chatflow.
Let’s create a bot by performing these steps:
- Open your HubSpot account and navigate to Conversations > Chatflows.
- Click "Create chatflow" in the upper right corner.
- Select "Website".
- Select the bot template "Start from scratch".
- Click Next.
- Choose the "inbox and language" settings.
- Click "Create".
You will be redirected to the bot actions editor page. Here, to match the demo bot, you can change the Welcome message to “Got any questions about services?” and the next message to “Hi! I am an AI Sales Rep – Adam. What can I do to help you today?”.
Next, click the plus sign at the bottom and choose the “Run a code snippet” action in the left sidebar. Here, give it a name and description as you see fit. Clear the code editor box from the default code. Go to the index.js, copy the whole file, and paste it into the action’s code editor. Save it.
A few words about the code that was used in the previous step. This code is generated by the ncc npm package from a simple JavaScript file written by me and OpenAI JavaScript SDK. Ncc combined those two entities together so we could get a single text file to paste into the custom code action. I’ve supplemented the initial JavaScript file with comments on almost every line so it will help you understand it better.
The last step before testing is to add the Assistant ID and OpenAI API key. Open the custom code action in the actions editor again and click the “Open in full page editor” link there. In the right sidebar, click the Variables tab. Here, add two variables: OPENAI_API_KEY and ASSISTANT_ID. You can get an OpenAI API key here and an Assistant ID here. Save it, and you will be ready to test.
Here's a short video walking through creating a custom bot:
Customizing the Bot for Other Use Cases
In case you need to modify the lead qualification bot or even create a different bot. Open the Assistants playground and start modifying the instructions text.
It might be a bit overwhelming to work with the demo instructions. You can remove them and write plain text instructions. Most of the time, people start with instructions as plain text and test the assistant. Then, change instructions until they are happy with the assistant's answers.
Instead of plain text, I've designed instructions in an unconventional format for a prompt — JSON. By trial and error, I figured out that JSON makes the model more disciplined and works more stable with each iteration. There are no special rules on how to design these JSON instructions. In this tutorial, I made up the field names and values in the instructions myself. So, if you find JSON instructions useful for you, it's totally up to you how to organize the JSON.
Also, in more complex use cases, you might consider using assistant tools. A retrieval tool and custom functions, especially, might give extra knowledge and functionality to your bot.
Summary
In this tutorial, I’ve demonstrated the Assistants API technology, which allows us to build an AI-powered chatbot. I encourage you to experiment with it and see how it might serve your business needs. If you find it useful, I will be happy to discuss use cases and technology implementation with you. I am available on LinkedIn.