Architecture Overview

Understanding OpenViking’s layered architecture and component design

OpenViking is a context database designed for AI Agents, unifying all context types (Memory, Resource, Skill) into a directory structure with semantic retrieval and progressive content loading.

System Architecture

OpenViking follows a layered architecture pattern separating concerns across Client, Service, Core Modules, and Storage layers:

┌────────────────────────────────────────────────────────────────────────────┐
│                        OpenViking System Architecture                       │
├────────────────────────────────────────────────────────────────────────────┤
│                                                                            │
│                              ┌─────────────┐                               │
│                              │   Client    │                               │
│                              │ (OpenViking)│                               │
│                              └──────┬──────┘                               │
│                                     │ delegates                            │
│                              ┌──────▼──────┐                               │
│                              │   Service   │                               │
│                              │    Layer    │                               │
│                              └──────┬──────┘                               │
│                                     │                                      │
│           ┌─────────────────────────┼─────────────────────────┐            │
│           │                         │                         │            │
│           ▼                         ▼                         ▼            │
│    ┌─────────────┐          ┌─────────────┐          ┌─────────────┐      │
│    │  Retrieve   │          │   Session   │          │    Parse    │      │
│    │  (Context   │          │  (Session   │          │  (Context   │      │
│    │  Retrieval) │          │  Management)│          │  Extraction)│      │
│    │ search/find │          │ add/used    │          │ Doc parsing │      │
│    │ Intent      │          │ commit      │          │ L0/L1/L2    │      │
│    │ Rerank      │          │ archive     │          │ Tree build  │      │
│    └──────┬──────┘          └──────┬──────┘          └──────┬──────┘      │
│           │                        │                        │             │
│           │                        │ Memory extraction      │             │
│           │                        ▼                        │             │
│           │                 ┌─────────────┐                 │             │
│           │                 │ Compressor  │                 │             │
│           │                 │ Compress/   │                 │             │
│           │                 │ Deduplicate │                 │             │
│           │                 └──────┬──────┘                 │             │
│           │                        │                        │             │
│           └────────────────────────┼────────────────────────┘             │
│                                    ▼                                      │
│    ┌─────────────────────────────────────────────────────────────────┐    │
│    │                         Storage Layer                            │    │
│    │               AGFS (File Content)  +  Vector Index               │    │
│    └─────────────────────────────────────────────────────────────────┘    │
│                                                                            │
└────────────────────────────────────────────────────────────────────────────┘

Core Modules

Client Layer

Provides unified entry point for all operations, delegating to the Service layer.

• Exposes all public APIs (find, search, add_resource, session, etc.)
• Handles HTTP/SDK client implementations
• Supports both embedded and remote modes

Service Layer

Implements business logic and decouples from transport layer, enabling reuse across HTTP Server and CLI.

Service	Responsibility	Key Methods
FSService	File system operations	ls, mkdir, rm, mv, tree, stat, read, abstract, overview, grep, glob
SearchService	Semantic search	search, find
SessionService	Session management	session, sessions, commit, delete
ResourceService	Resource import	add_resource, add_skill, wait_processed
RelationService	Relation management	relations, link, unlink
PackService	Import/export	export_ovpack, import_ovpack
DebugService	Debug service	observer (ObserverService)

Retrieve Module

Handles context retrieval with intent analysis and hierarchical search.

• IntentAnalyzer: Analyzes query intent, generates 0-5 typed queries
• HierarchicalRetriever: Directory-level recursive search using priority queue
• Rerank: Scalar filtering + model reranking for refined results

Session Module

Manages conversation lifecycle and memory extraction.

• Message recording and usage tracking
• Session compression and archiving
• 6-category memory extraction (profile, preferences, entities, events, cases, patterns)
• LLM-based deduplication decisions

Parse Module

Handles document parsing and context extraction.

• Document parsing (PDF/MD/HTML/Code)
• Tree building and directory creation
• Async semantic generation (L0/L1)
• AST-based code skeleton extraction

Compressor Module

Compresses and extracts memories from sessions.

• 6-category memory extraction
• LLM deduplication decisions
• Conflict resolution (merge/delete/skip)

Storage Layer

Dual-layer storage separating content from index.

• VikingFS: Virtual filesystem with URI abstraction
• AGFS: Content storage (L0/L1/L2, multimedia files, relations)
• Vector Index: Semantic search (URIs, vectors, metadata)

Data Flow

Adding Context

1. Parser

   Parse documents, create file and directory structure (no LLM calls)

2. TreeBuilder

   Move temp directory to AGFS, enqueue for semantic processing

3. SemanticQueue

   Async bottom-up L0/L1 generation using VLM

4. Vector Index

   Build index for semantic search

Retrieving Context

1. Intent Analysis

   Analyze query intent, generate 0-5 typed queries by context type

2. Hierarchical Retrieval

   Directory-level recursive search using priority queue

3. Rerank

   Scalar filtering + model reranking for refined scoring

4. Results

   Return contexts sorted by relevance

Session Commit

1. Messages

   Accumulate conversation messages and usage records

2. Compress

   Keep recent N rounds, archive older messages

3. Archive

   Generate L0/L1 for history segments

4. Memory Extraction

   Extract 6-category memories from messages

5. Storage

   Write to AGFS + vector index

Deployment Modes

Embedded Mode

For local development and single-process applications

client = OpenViking(path="./data")

• Auto-starts AGFS subprocess
• Uses local vector index
• Singleton pattern

HTTP Mode

For team sharing, production deployment, and cross-language integration

# Python SDK
client = SyncHTTPClient(
    url="http://localhost:1933",
    api_key="xxx"
)

• Server runs as standalone process
• Clients connect via HTTP API
• Supports any language

Design Principles

OpenViking follows these core design principles to ensure scalability and maintainability:

Principle	Description
Pure Storage Layer	Storage only handles AGFS operations and basic vector search; Rerank is in retrieval layer
Three-Layer Information	L0/L1/L2 enables progressive detail loading, saving token consumption
Two-Stage Retrieval	Vector search recalls candidates + Rerank improves accuracy
Single Data Source	All content read from AGFS; vector index only stores references

Code Example: Client Initialization

from openviking import OpenViking
from openviking.http_client import SyncHTTPClient

# Embedded mode (local development)
client = OpenViking(path="./data")

# HTTP mode (production)
client = SyncHTTPClient(
    url="http://localhost:1933",
    api_key="your-api-key"
)

# Add resource
await client.add_resource(
    "https://docs.example.com/api.pdf",
    reason="API documentation"
)

# Search across all contexts
results = await client.find("authentication methods")

for ctx in results.resources:
    print(f"Resource: {ctx.uri}")
    print(f"Abstract: {ctx.abstract}")

Related Concepts

Context Types

Learn about Resource, Memory, and Skill types

Context Layers

Understand L0/L1/L2 progressive loading

Viking URI

Unified resource identifier system

Storage

Dual-layer storage architecture