Files Needed for Your Offline AI Companion

To get your offline AI companion up and running, there are a few files and programs you'll need. This section will guide you through everything, explaining why each part is important and how it fits into the bigger picture. Don’t worry—I'll break it down step by step so even if you’re not super technical, you’ll understand how it all works.

1. Ollama.exe

Download Ollama.exe here

Ollama is the program that allows you to run your AI offline. Think of it like the engine that powers the entire experience. Without this, your AI won’t have the "brain" to think, respond, or hold conversations. This is where the Llama3 model (the actual AI) will live and run on your computer, so you can interact with it like you would with something like ChatGPT, but totally offline.

Steps to Get Ollama Running:

• Download and install Ollama.exe from the link above.
• Once installed, you'll need to make sure the Llama3 model is set up correctly. Don’t worry, we’ll cover that in the next sections.

2. Portable Python (WinPython)

Download WinPython 3.11.9 here

We use WinPython to keep Python portable and self-contained. This version of Python doesn't require installation, making it easier to run the offline AI on different systems. It's crucial for ensuring all dependencies and scripts can run smoothly without needing a full Python installation.

Steps to Get WinPython Set Up:

• Download the WinPython executable from the link above.
• Once downloaded, extract and set up WinPython anywhere on your machine. No installation is required!
• Open the WinPython Command Prompt and you’re ready to go.

3. Windows Tools C++ Build Tools

Download Windows C++ Build Tools here

Some Python libraries, especially those needed for AI, require a C++ compiler. Windows Tools C++ is required to compile some of the dependencies you'll need for the AI. Without this, you might encounter errors when setting up the environment.

Steps to Set Up C++ Build Tools:

• Download the C++ Build Tools from the link above.
• Run the installer and make sure to install the required components for C++ development.
• After installation, you’ll be ready to compile and use the AI.

4. requirements.txt

Download the requirements.txt file here

The requirements.txt file is a simple text file that lists all the important tools or "dependencies" your offline AI needs to work. When you run this file, it automatically downloads all the programs your AI needs. Think of it like a shopping list that your computer follows to grab the right ingredients.

Here’s what you’ll see in the file and why each one is needed:

• annoy==1.17.3: Annoy is a tool for finding similar items quickly. It helps the AI store and retrieve past conversations and memories efficiently, allowing for better context recall over time.
• fastapi==0.115.0: FastAPI is like a communication bridge. It helps the AI take your input and respond in a way that makes sense. It's fast and efficient, making sure you don’t have to wait long for responses.
• uvicorn==0.31.0: Uvicorn is the engine that helps FastAPI work properly. It keeps everything running smoothly, making sure the AI is always ready to talk.
• pydantic==2.9.2: This tool helps with organizing and validating data. It makes sure everything going in and out of the AI makes sense and works as it should.
• ollama==0.2.1: This is the version of Ollama we’re using. It’s the program that runs your AI model, making everything possible.
• numpy==1.23.5: Numpy is a fundamental package for scientific computing. It’s used to handle mathematical operations, which can be useful for managing large amounts of data efficiently.
• requests==2.32.3: Requests is used to handle HTTP requests easily. It helps the AI connect to web services or APIs, ensuring that data can be retrieved and sent smoothly.
• beautifulsoup4==4.10.0: BeautifulSoup is used for parsing and extracting information from HTML and XML files. It’s helpful for web scraping or cleaning up data for the AI to process more effectively.

How to Use the Requirements.txt File:

1. Download the requirements.txt file from the link above.
2. Once downloaded, open a terminal or command prompt and navigate to where your requirements.txt file is saved.
3. Type the following command:

pip install -r requirements.txt

This will automatically download and set up everything your offline AI needs to work.

With these files and a little patience, you’ll be well on your way to having your very own offline AI companion. Next up, we’ll go over how to set up Llama3 so it starts responding to you!