Changes Made Today

• Switched from ChromaDB to Annoy: The persistent memory system was updated from using ChromaDB to Annoy for handling memory recall and embedding storage. Annoy now helps with vector-based search and persistent memory, storing embeddings locally on disk.
• Integration of Annoy: Updated functions to use the Annoy library, specifically to add embeddings, store, and retrieve conversational history across sessions.
• Persistent Memory: The new setup ensures persistent memory remains after shutdown by saving and reloading the Annoy index.
• Index Building: Ensured that the index is built and saved after conversations are processed. Introduced checks for the index being loaded or built properly to avoid errors.
• Error Fixes: Resolved errors such as "You can't add an item to a loaded index" by managing index creation and loading more efficiently.

TODO

• Teach AI to format text: Implement logic to format text based on context such as:
  • Code formatting (JavaScript, Python, HTML, etc.)
  • Letter writing (formal/informal structure)
  • Bullet points and lists for step-by-step instructions
• Text Preprocessing: Add text preprocessing steps for formatting based on the type of response required (e.g., code snippets, documentation, casual conversations).
• Improve Memory Efficiency: Optimize Annoy's memory use and retrieval to handle larger conversation histories more effectively.
• Advanced Conversation Context: Work on retrieving more contextually relevant conversations based on embeddings and similarity scores.
• Testing and Debugging: Continue testing for edge cases where persistent memory could fail and ensure robustness.