wait_processed

Wait for asynchronous resource processing to complete

Overview

Wait for all queued semantic processing and vectorization tasks to complete. This ensures that resources and skills are fully processed and indexed before you query them.

After adding resources or skills, OpenViking processes them asynchronously. Use this method to wait for completion before searching or retrieving content.

Method Signature

Python SDK

client.wait_processed(
    timeout: Optional[float] = None
) -> Dict[str, Any]

Parameters

timeout · float

Maximum time to wait in seconds. If not provided, waits indefinitely until all processing completes.

Response

semantic_queue · object

Status of the semantic processing queue (L0/L1 generation)

Show semantic_queue properties

processed · integer

Number of items successfully processed

error_count · integer

Number of items that failed processing

errors · array

List of error objects with message field

vector_queue · object

Status of the vectorization queue

Show vector_queue properties

processed · integer

Number of items successfully vectorized

error_count · integer

Number of items that failed vectorization

errors · array

List of error objects with message field

Examples

Basic Usage

Python SDK

import openviking as ov

client = ov.OpenViking(path="./data")
client.initialize()

# Add resource without waiting
result = client.add_resource("./docs/guide.md")

# Wait for processing to complete
status = client.wait_processed()
print(f"Processed: {status['semantic_queue']['processed']} items")

Response

{
  "status": "ok",
  "result": {
    "semantic_queue": {
      "processed": 12,
      "error_count": 0,
      "errors": []
    },
    "vector_queue": {
      "processed": 12,
      "error_count": 0,
      "errors": []
    }
  },
  "time": 2.4
}

With Timeout

Python SDK

try:
    status = client.wait_processed(timeout=30.0)
    print("Processing complete!")
except Exception as e:
    print(f"Timeout or error: {e}")

Batch Processing Pattern

Python SDK

# Add multiple resources without waiting
files = [
    "./docs/intro.md",
    "./docs/api.md",
    "./docs/examples.md"
]

for file in files:
    client.add_resource(file, wait=False)
    print(f"Queued: {file}")

# Wait for all to complete
print("Waiting for processing...")
status = client.wait_processed()

print(f"✓ Processed {status['semantic_queue']['processed']} items")
if status['semantic_queue']['error_count'] > 0:
    print(f"⚠ Errors: {status['semantic_queue']['errors']}")

Integration with Search

Python SDK

# Add resource and wait
client.add_resource(
    "https://raw.githubusercontent.com/volcengine/OpenViking/refs/heads/main/README.md",
    wait=False
)

# Wait for processing
client.wait_processed()

# Now safe to search
results = client.find("what is openviking")
for r in results.resources:
    print(f"Found: {r.uri} (score: {r.score:.4f})")

Processing Queues

OpenViking uses two asynchronous queues for resource processing:

1. Semantic Queue

Processes resources to generate:

• L0 (Abstract): High-level summary of the resource
• L1 (Overview): Detailed overview with key points

2. Vector Queue

Indexes resources for semantic search:

• Generates embeddings for content chunks
• Stores vectors in the vector database
• Enables semantic similarity search

Both queues must complete before resources are fully searchable.

Best Practices

When to use wait_processed()

Use wait_processed() when:

• You need to search resources immediately after adding them
• You’re adding resources in a script or initialization phase
• You need to verify processing completed successfully
• You’re running tests that depend on resource availability

Batch processing

For better performance when adding multiple resources:

# Add all resources first (fast)
for resource in resources:
    client.add_resource(resource, wait=False)

# Wait once for all to complete
client.wait_processed()

This is more efficient than waiting after each individual resource.

Handling timeouts

Set appropriate timeouts based on resource size:

• Small files (< 1MB): 10-30 seconds
• Medium files (1-100MB): 1-5 minutes
• Large files or directories: 5-30 minutes

try:
    client.wait_processed(timeout=300.0)  # 5 minutes
except TimeoutError:
    # Handle timeout - processing continues in background
    print("Still processing, check back later")

Error handling

Check for processing errors:

status = client.wait_processed()

if status['semantic_queue']['error_count'] > 0:
    for error in status['semantic_queue']['errors']:
        print(f"Error: {error['message']}")

if status['vector_queue']['error_count'] > 0:
    for error in status['vector_queue']['errors']:
        print(f"Vectorization error: {error['message']}")

Response Examples

All Complete

{
  "status": "ok",
  "result": {
    "semantic_queue": {
      "processed": 15,
      "error_count": 0,
      "errors": []
    },
    "vector_queue": {
      "processed": 15,
      "error_count": 0,
      "errors": []
    }
  },
  "time": 3.2
}

Related Methods

• add_resource - Add resources (supports wait parameter)
• add_skill - Add skills (supports wait parameter)
• find - Search resources after processing
• abstract - Get L0 abstract (requires processing)
• overview - Get L1 overview (requires processing)