> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/volcengine/OpenViking/llms.txt
> Use this file to discover all available pages before exploring further.
# Architecture Overview
> Understanding OpenViking's layered architecture and component design
OpenViking is a context database designed for AI Agents, unifying all context types (Memory, Resource, Skill) into a directory structure with semantic retrieval and progressive content loading.
## System Architecture
OpenViking follows a layered architecture pattern separating concerns across Client, Service, Core Modules, and Storage layers:
```
┌────────────────────────────────────────────────────────────────────────────┐
│ OpenViking System Architecture │
├────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ │
│ │ Client │ │
│ │ (OpenViking)│ │
│ └──────┬──────┘ │
│ │ delegates │
│ ┌──────▼──────┐ │
│ │ Service │ │
│ │ Layer │ │
│ └──────┬──────┘ │
│ │ │
│ ┌─────────────────────────┼─────────────────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Retrieve │ │ Session │ │ Parse │ │
│ │ (Context │ │ (Session │ │ (Context │ │
│ │ Retrieval) │ │ Management)│ │ Extraction)│ │
│ │ search/find │ │ add/used │ │ Doc parsing │ │
│ │ Intent │ │ commit │ │ L0/L1/L2 │ │
│ │ Rerank │ │ archive │ │ Tree build │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ │ │ Memory extraction │ │
│ │ ▼ │ │
│ │ ┌─────────────┐ │ │
│ │ │ Compressor │ │ │
│ │ │ Compress/ │ │ │
│ │ │ Deduplicate │ │ │
│ │ └──────┬──────┘ │ │
│ │ │ │ │
│ └────────────────────────┼────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Storage Layer │ │
│ │ AGFS (File Content) + Vector Index │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────────┘
```
## Core Modules
Provides unified entry point for all operations, delegating to the Service layer.
* Exposes all public APIs (find, search, add\_resource, session, etc.)
* Handles HTTP/SDK client implementations
* Supports both embedded and remote modes
Implements business logic and decouples from transport layer, enabling reuse across HTTP Server and CLI.
| Service | Responsibility | Key Methods |
| ------------------- | ---------------------- | ------------------------------------------------------------------- |
| **FSService** | File system operations | ls, mkdir, rm, mv, tree, stat, read, abstract, overview, grep, glob |
| **SearchService** | Semantic search | search, find |
| **SessionService** | Session management | session, sessions, commit, delete |
| **ResourceService** | Resource import | add\_resource, add\_skill, wait\_processed |
| **RelationService** | Relation management | relations, link, unlink |
| **PackService** | Import/export | export\_ovpack, import\_ovpack |
| **DebugService** | Debug service | observer (ObserverService) |
Handles context retrieval with intent analysis and hierarchical search.
* **IntentAnalyzer**: Analyzes query intent, generates 0-5 typed queries
* **HierarchicalRetriever**: Directory-level recursive search using priority queue
* **Rerank**: Scalar filtering + model reranking for refined results
Manages conversation lifecycle and memory extraction.
* Message recording and usage tracking
* Session compression and archiving
* 6-category memory extraction (profile, preferences, entities, events, cases, patterns)
* LLM-based deduplication decisions
Handles document parsing and context extraction.
* Document parsing (PDF/MD/HTML/Code)
* Tree building and directory creation
* Async semantic generation (L0/L1)
* AST-based code skeleton extraction
Compresses and extracts memories from sessions.
* 6-category memory extraction
* LLM deduplication decisions
* Conflict resolution (merge/delete/skip)
Dual-layer storage separating content from index.
* **VikingFS**: Virtual filesystem with URI abstraction
* **AGFS**: Content storage (L0/L1/L2, multimedia files, relations)
* **Vector Index**: Semantic search (URIs, vectors, metadata)
## Data Flow
### Adding Context
```mermaid theme={null}
graph LR
A[Input File] --> B[Parser]
B --> C[TreeBuilder]
C --> D[AGFS]
C --> E[SemanticQueue]
E --> F[Vector Index]
B -.->|No LLM| B2[Parse & Convert]
C -.->|Move Files| C2[Queue Semantic]
E -.->|Async| E2[L0/L1 Generation]
```
Parse documents, create file and directory structure (no LLM calls)
Move temp directory to AGFS, enqueue for semantic processing
Async bottom-up L0/L1 generation using VLM
Build index for semantic search
### Retrieving Context
```mermaid theme={null}
graph LR
A[Query] --> B[Intent Analysis]
B --> C[Hierarchical Retrieval]
C --> D[Rerank]
D --> E[Results]
B -.-> B2[0-5 TypedQueries]
C -.-> C2[Directory Recursion]
D -.-> D2[Refined Scoring]
```
Analyze query intent, generate 0-5 typed queries by context type
Directory-level recursive search using priority queue
Scalar filtering + model reranking for refined scoring
Return contexts sorted by relevance
### Session Commit
```mermaid theme={null}
graph LR
A[Messages] --> B[Compress]
B --> C[Archive]
C --> D[Memory Extraction]
D --> E[Storage]
B -.-> B2[Keep Recent N Rounds]
C -.-> C2[Generate L0/L1]
D -.-> D2[6 Categories]
E -.-> E2[AGFS + Vector Index]
```
Accumulate conversation messages and usage records
Keep recent N rounds, archive older messages
Generate L0/L1 for history segments
Extract 6-category memories from messages
Write to AGFS + vector index
## Deployment Modes
For local development and single-process applications
```python theme={null}
client = OpenViking(path="./data")
```
* Auto-starts AGFS subprocess
* Uses local vector index
* Singleton pattern
For team sharing, production deployment, and cross-language integration
```python theme={null}
# Python SDK
client = SyncHTTPClient(
url="http://localhost:1933",
api_key="xxx"
)
```
* Server runs as standalone process
* Clients connect via HTTP API
* Supports any language
## Design Principles
OpenViking follows these core design principles to ensure scalability and maintainability:
| Principle | Description |
| --------------------------- | ------------------------------------------------------------------------------------------ |
| **Pure Storage Layer** | Storage only handles AGFS operations and basic vector search; Rerank is in retrieval layer |
| **Three-Layer Information** | L0/L1/L2 enables progressive detail loading, saving token consumption |
| **Two-Stage Retrieval** | Vector search recalls candidates + Rerank improves accuracy |
| **Single Data Source** | All content read from AGFS; vector index only stores references |
## Code Example: Client Initialization
```python theme={null}
from openviking import OpenViking
from openviking.http_client import SyncHTTPClient
# Embedded mode (local development)
client = OpenViking(path="./data")
# HTTP mode (production)
client = SyncHTTPClient(
url="http://localhost:1933",
api_key="your-api-key"
)
# Add resource
await client.add_resource(
"https://docs.example.com/api.pdf",
reason="API documentation"
)
# Search across all contexts
results = await client.find("authentication methods")
for ctx in results.resources:
print(f"Resource: {ctx.uri}")
print(f"Abstract: {ctx.abstract}")
```
## Related Concepts
Learn about Resource, Memory, and Skill types
Understand L0/L1/L2 progressive loading
Unified resource identifier system
Dual-layer storage architecture