Instinct.cpp is your comprehensive toolkit for effortlessly building AI agent applications. Compatible with the OpenAI Assistant API, it offers a range of ready-to-use applications and utilities tailored for LLM-powered services. Whether you're developing chatbots or code interpreters, instinct.cpp provides powerful frameworks and built-in integrations for a streamlined development experience.
instinct.cpp is an innovative toolkit designed for developing AI applications powered by large language models (LLMs). Often referred to as langchain.cpp, this library provides robust alternatives to the OpenAI Assistant API and essential utilities for building advanced AI Agent applications, including chatbots and code interpreters.
Key Features
Ready-to-Use Applications
-
Assistant API Server: A fully compatible agent service with OpenAI's Assistant API features:
- mini-assistant-api: A single binary deployment that includes a vector database and all necessary dependencies for straightforward setup.
- mighty-assistant-api: (Work In Progress) A scalable, cloud-native implementation designed for distributed components and multi-tenant support.
-
chat-agent: A command-line application that builds a knowledge index from various document formats (PDF, TXT, MD, etc.) and launches an HTTP server that integrates seamlessly with OpenAI’s
ChatCompletionAPI.
Versatile Frameworks
- Integration for Privacy-Centric LLM Providers: Out-of-the-box support for a variety of LLM services, such as Ollama, vllm, llama.cpp server, and nitro.
- Building Blocks for Application Patterns: Create chatbots and other AI applications easily with functional chaining components for composable LLM pipelines, and utilize agent patterns like ReACT, OpenAI-based tool agents, and LLMCompilers.
Documentation and User Guides
Explore comprehensive guides and documentation for all built-in applications and the core library:
Roadmap
Be a part of our journey! The complete project plan and milestones can be found on our Project Kanban. Upcoming releases include major features such as long-short memory support, performance tuning, and further enhancements to the assistant APIs. We invite contributions and collaboration from the community to help shape the future of instinct.cpp.
Join Us
Your feedback and contributions are invaluable! Engage with us through our Discord server or reach out via email at robinqu@gmail.com. Together, we can advance AI technology and explore the potentials of language models more effectively.