Welcome! In this tutorial, we will walk you through the different files involved in building your offline AI companion. The following files are used:
Before we dive into how everything works, let's look at how the files are organized. It's important to set up the project correctly so that all parts work together smoothly.
For this tutorial, we will store everything on the E: drive. Here's how the folder structure should look:
E:\
│
├── ai.py # Python script that runs the backend AI server
├── personality.txt # Text file that stores the AI's personality (optional)
├── memory\ # Folder to store the AI's memory (SQLite database)
│ └── memory_agent.db # SQLite3 database that stores conversation history
│
├── run_ai.bat # Batch file to start the FastAPI server
│
└── static\ # Folder containing the static files (HTML, CSS, JS, and images)
├── index.html # The main webpage where the user interacts with the AI
├── style.css # The CSS file that defines the look and feel of the page
├── chat.js # The JavaScript file that handles user input and AI response
├── male.bmp # User avatar image
├── girl.bmp # AI assistant avatar image
Here's what each file and folder does:
memory_agent.db) that stores all past conversations. This helps the AI remember what you've talked about before.personality.txt?The personality.txt file is where you define what kind of personality you want your AI to have. You might want your AI to act like a friendly assistant, or maybe you want it to be more serious and professional. This file gives you full control over how the AI talks and behaves.
Each person can create their own personality.txt, and that's why we keep it separate. You can change the file any time you want the AI to act differently. Here's an example of what you might put in personality.txt:
You are a friendly and caring assistant. You are always supportive and like to help people with their problems.
This description is simple, but it tells the AI to be friendly and supportive. You can make your AI funny, serious, curious, or anything else you want!
Tip: If you don’t have a personality.txt file, the AI will use a default personality, which might be more neutral. Customize this file to give your AI the personality you want!
Description: This Python script is the core of your project. It handles communication with the AI model, retrieves and stores conversations, and makes the AI smarter by using previous conversations to influence future responses.
import time
import ollama
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from fastapi.responses import PlainTextResponse, FileResponse
from pydantic import BaseModel
import uvicorn
import os
from annoy import AnnoyIndex
import numpy as np
# Declare constants for the model names
CHAT_MODEL = 'llama3' # For generating text responses
EMBEDDING_MODEL = 'nomic-embed-text' # For generating 384-dimensional embeddings
PERSONALITY_FILE = "personality.txt" # The file containing the system prompt or personality
ANNOY_INDEX_FILE = "memory.ann"
VECTOR_DIM = 384 # Annoy index dimensionality
# Initialize FastAPI app
app = FastAPI()
# CORS middleware setup
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # Allow all origins, or specify a domain.
allow_credentials=True,
allow_methods=["*"], # Allow all HTTP methods.
allow_headers=["*"], # Allow all headers.
)
# Serve static files from the "static" directory
app.mount("/static", StaticFiles(directory="static"), name="static")
# Initialize Annoy index
annoy_index = AnnoyIndex(VECTOR_DIM, 'angular')
# Check if Annoy index exists and load it if present
if os.path.exists(ANNOY_INDEX_FILE):
annoy_index.load(ANNOY_INDEX_FILE)
# Define the PromptModel
class PromptModel(BaseModel):
prompt: str # Ensure the prompt field is a string
# Load the system prompt/personality from the file
def load_personality():
try:
with open(PERSONALITY_FILE, 'r') as file:
return file.read().strip()
except FileNotFoundError:
return "Default system instructions or personality prompt."
# Dummy function to simulate the chat model interaction (using llama3)
def chat_with_model(system_prompt, user_prompt):
# This is where you'd integrate with the actual AI model, such as llama3.
# For now, it returns a dummy response.
return f"System says: {system_prompt}. You said: {user_prompt}"
# Store conversation and its embedding into Annoy index
def store_conversation(prompt, response):
# Simulate generating an embedding from the prompt/response
embedding = np.random.rand(VECTOR_DIM).tolist() # Dummy embedding generation
# Add the new item to the in-memory Annoy index (rebuilding from scratch)
global annoy_index
annoy_index = AnnoyIndex(VECTOR_DIM, 'angular') # Rebuild a fresh index
annoy_index.add_item(annoy_index.get_n_items(), embedding)
# Save the updated index
annoy_index.build(10) # Build the index with 10 trees
annoy_index.save(ANNOY_INDEX_FILE)
# Retrieve similar conversations (dummy implementation)
def get_similar_conversations(prompt):
# Dummy similar conversations retrieval
return []
# FastAPI route to handle prompt input
@app.post("/send_prompt/")
def send_prompt(data: PromptModel):
prompt = data.prompt
# Load personality/system prompt from file
system_prompt = load_personality()
# Prepare the conversation with the system prompt and the user's input
convo = [
{'role': 'system', 'content': system_prompt}, # System instructions, only for AI's context
{'role': 'user', 'content': prompt} # User input
]
# Get the AI response (using a proper model integration)
response = ollama.chat(CHAT_MODEL, convo)['message']['content']
# Store the new conversation (user prompt and response)
store_conversation(prompt, response)
# Return only the response, not the system prompt
return PlainTextResponse(response)
# Serve the index.html file at the root URL
@app.get("/")
def read_root():
return FileResponse('static/index.html')
# Start the FastAPI app
if __name__ == "__main__":
uvicorn.run(app, host="127.0.0.1", port=8000)
Description: This HTML file defines the webpage layout where you interact with the AI. It includes a text area for user input and a chat container for displaying messages.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Virtual Brigid Chat</title>
<link rel="stylesheet" href="/static/style.css">
</head>
<body>
<!-- Chat Container -->
<div id="chat-container" class="chat-container">
<!-- Messages will dynamically appear here -->
</div>
<!-- Typing Input Section -->
<div class="input-container">
<textarea id="user-input" placeholder="Enter your message..." required></textarea>
<button id="send-btn" onclick="sendPrompt()">Send</button>
</div>
<!-- Link to your JavaScript file -->
<script src="/static/chat.js"></script>
</body>
</html>
Description: This file defines the look and feel of the webpage, controlling the layout, colors, fonts, and other visual elements. Here's an example snippet:
/* General Styles */
* {
margin: 0;
padding: 0;
box-sizing: border-box;
font-family: 'Poppins', sans-serif;
}
body {
background-color: #000; /* Background color of the chat interface */
color: #fff;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
height: 100vh;
}
/* Chat Container */
.chat-container {
display: flex;
flex-direction: column;
gap: 10px;
padding: 20px;
width: 100%; /* Ensure it takes full width of the parent */
max-width: 900px; /* Allow a comfortable maximum width */
margin: auto;
height: 80vh;
overflow-y: auto;
background-color: #1E3E62;
border-radius: 10px;
}
/* Message Bubbles */
.message-bubble {
display: flex;
align-items: center;
max-width: 80%; /* Limit each message to 80% of the chat container */
padding: 10px;
border-radius: 10px;
word-wrap: break-word;
margin-bottom: 10px;
font-size: 100%; /* Reduce text size to 75% of original */
}
/* Outgoing Messages (Your Prompt) */
.message-bubble.outgoing {
background-color: #0B192C;
align-self: flex-end; /* Align the bubble to the right */
display: flex;
flex-direction: row-reverse; /* Avatar on the right */
max-width: 80%; /* Set max width to 80% of container */
margin-left: auto; /* Push the bubble to the right */
color: #fff;
}
/* Ensure the avatar is on the right side for outgoing messages */
.message-bubble.outgoing .chat-details img {
margin-left: 10px; /* Space between message and avatar */
margin-right: 0; /* Remove right margin */
width: 35px;
height: 35px;
border-radius: 50%;
object-fit: cover;
}
/* Incoming Messages (Her Reply) */
.message-bubble.incoming {
background-color: #FF6500;
align-self: flex-start; /* Align the bubble to the left */
display: flex;
flex-direction: row; /* Default row direction */
max-width: 80%; /* Set max width to 80% of container */
color: #000;
border: 2px solid #0B192C;
}
/* Ensure the avatar is on the left side for incoming messages */
.message-bubble.incoming .chat-details img {
margin-right: 10px; /* Space between avatar and message */
margin-left: 0; /* Remove left margin */
width: 35px;
height: 35px;
border-radius: 50%;
object-fit: cover;
}
/* Ensure that the message text stays aligned properly */
.message-content p {
margin: 0;
padding: 5px 10px;
font-size: inherit;
}
/* Typing Input Container */
.input-container {
display: flex;
padding: 10px;
background-color: #343541; /* Match the theme color */
width: 100%;
max-width: 900px; /* Matches the chat container width */
margin-top: 10px;
border-radius: 10px;
}
/* Input Textarea */
#user-input {
flex-grow: 1;
padding: 10px;
border: none;
border-radius: 5px;
font-size: 1rem;
outline: none;
background-color: #555;
color: #fff;
}
/* Send Button */
#send-btn {
margin-left: 10px;
padding: 10px 15px;
background-color: #acecbe;
border: none;
border-radius: 5px;
cursor: pointer;
color: #000;
}
/* Typing Animation (Dots) */
.typing-animation {
display: inline-flex;
align-items: center;
gap: 3px;
}
.typing-dot {
width: 8px;
height: 8px;
background-color: #acecbe;
border-radius: 50%;
animation: animateDots 1.2s linear infinite;
}
@keyframes animateDots {
0%, 100% {
transform: translateY(0);
}
50% {
transform: translateY(-8px);
}
}
/* Scrollbar Styling */
:where(.chat-container, textarea)::-webkit-scrollbar {
width: 6px;
}
:where(.chat-container, textarea)::-webkit-scrollbar-track {
background: #444654;
border-radius: 25px;
}
:where(.chat-container, textarea)::-webkit-scrollbar-thumb {
background: #acecbe;
border-radius: 25px;
}
Description: This JavaScript file handles user input, sends the prompt to the backend AI, and appends the AI's response to the chat window. It also auto-resizes the input field as you type.
// Attach the input event listener to the textarea for auto-resizing
document.getElementById('user-input').addEventListener('input', autoResizeTextarea);
async function sendPrompt() {
const textarea = document.getElementById('user-input');
const userInput = textarea.value;
if (userInput.trim() === "") return;
textarea.value = '';
textarea.style.height = '55px';
textarea.focus();
appendUserMessage(userInput);
// Add assistant typing animation before making the fetch request
console.log("Adding assistant typing animation");
let assistantDiv = appendAssistantMessage(); // This function adds the typing dots
console.log("Assistant typing animation added");
// Force a reflow/repaint to ensure the typing animation is visible
await new Promise(resolve => requestAnimationFrame(resolve));
try {
// Start the fetch request
console.log("Sending fetch request...");
const response = await fetch('http://127.0.0.1:8000/send_prompt/', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt: userInput })
});
console.log("Fetch request sent, awaiting response...");
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
// Read the streaming response and append it to the assistant message
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
// Remove typing animation (dots) and display the response text
const typingAnimation = assistantDiv.querySelector('.typing-animation');
if (typingAnimation) {
typingAnimation.remove(); // Remove the dots when response starts coming in
} else {
console.error("Typing animation element not found in assistantDiv");
}
console.log("Response chunk received:", chunk);
assistantDiv.querySelector('p').textContent += chunk;
const chatWindow = document.getElementById('chat-container');
chatWindow.scrollTop = chatWindow.scrollHeight; // Auto-scroll to bottom
}
} catch (error) {
console.error("Error fetching response:", error);
assistantDiv.querySelector('p').textContent = "Error getting response. Please try again.";
}
}
// Function to append user's outgoing message to the chat window
function appendUserMessage(message) {
const chatWindow = document.getElementById('chat-container');
const outgoingChat = document.createElement('div');
outgoingChat.classList.add('message-bubble', 'outgoing');
outgoingChat.innerHTML = `
<div class="chat-details">
<img src="/static/male.bmp" alt="Your Avatar"> <!-- Ensure avatar is added here -->
<div class="message-content">
<p>${message}</p>
</div>
</div>
`;
chatWindow.appendChild(outgoingChat);
chatWindow.scrollTop = chatWindow.scrollHeight;
}
// Function to add typing animation for the assistant
function appendAssistantMessage() {
const chatWindow = document.getElementById('chat-container');
const incomingChat = document.createElement('div');
incomingChat.classList.add('message-bubble', 'incoming'); // Ensure incoming class is added
incomingChat.innerHTML = `
<div class="chat-details">
<img src="/static/girl.bmp" alt="Her Avatar"> <!-- Ensure avatar is added here -->
<div class="message-content">
<p><span class="typing-animation">
<span class="typing-dot"></span>
<span class="typing-dot"></span>
<span class="typing-dot"></span>
</span></p>
</div>
</div>
`;
chatWindow.appendChild(incomingChat);
chatWindow.scrollTop = chatWindow.scrollHeight;
return incomingChat;
}
// Function to automatically resize the textarea based on content
function autoResizeTextarea() {
const textarea = document.getElementById('user-input');
textarea.style.height = 'auto'; // Reset the height first to get the scroll height correctly
textarea.style.height = textarea.scrollHeight + 'px'; // Adjust height based on content
}
// Attach the input event to trigger resizing when typing
document.getElementById('user-input').addEventListener('input', autoResizeTextarea);
// Handle pressing "Enter" to send the message
function handleKeyPress(event) {
if (event.key === 'Enter') {
sendPrompt();
}
}
// Attach the keypress event listener for Enter key
document.getElementById('user-input').addEventListener('keypress', handleKeyPress);
Description: This batch file is a simple script that helps you start the Python FastAPI server, which runs the AI backend. It allows you to start the project with a single click.
@echo off
echo Starting the AI Server...
uvicorn ai:app --reload --port 8000
pause