Pagination & Filtering

Efficiently retrieve large datasets and filter results using pagination and filtering.

Pagination

Why Pagination?

Performance: Returning all results in one response is slow and wasteful.

Example:

10,000 vaults × 5 KB each = 50 MB response (slow, expensive)
100 vaults × 5 KB each = 500 KB response (fast, efficient)

All list endpoints support pagination:

GET /v1/vaults
GET /v1/documents
GET /v1/recipients
GET /v1/policies
etc.

Page-Based Pagination

How It Works

Request Specific Page:

GET /v1/vaults?page=1&limit=25

Response:

{
  "vaults": [
    { "id": "vault_1", "name": "Vault 1" },
    { "id": "vault_2", "name": "Vault 2" },
    ...
  ],
  "pagination": {
    "page": 1,
    "limit": 25,
    "total": 250,
    "total_pages": 10,
    "has_more": true
  }
}

Query Parameters

page (integer, default: 1):

Page number (1-indexed)
Minimum: 1
Maximum: No hard limit, but performance degrades for very high page numbers

limit (integer, default: 25):

Number of results per page
Minimum: 1
Maximum: 100
Recommended: 25-50 for best performance

Example:

GET /v1/vaults?page=2&limit=50

Returns vaults 51-100 (page 2, 50 per page).

Response Fields

pagination object:

{
  "page": 2,            // Current page number
  "limit": 50,          // Results per page
  "total": 250,         // Total number of results across all pages
  "total_pages": 5,     // Total number of pages
  "has_more": true      // Whether there are more pages after this one
}

Iterating Through Pages

Get All Results:

async function getAllVaults() {
  const allVaults = [];
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const response = await fetch(
      `https://api.torvussecurity.com/v1/vaults?page=${page}&limit=100`
    );
    const data = await response.json();

    allVaults.push(...data.vaults);
    hasMore = data.pagination.has_more;
    page++;
  }

  return allVaults;
}

With Error Handling:

async function getAllVaults() {
  const allVaults = [];
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    try {
      const response = await fetch(
        `https://api.torvussecurity.com/v1/vaults?page=${page}&limit=100`,
        {
          headers: {
            'Authorization': `Bearer ${apiKey}`
          }
        }
      );

      if (!response.ok) {
        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
      }

      const data = await response.json();

      allVaults.push(...data.vaults);
      hasMore = data.pagination.has_more;
      page++;

      // Optional: Delay between pages to avoid rate limits
      if (hasMore) {
        await new Promise(resolve => setTimeout(resolve, 100));
      }

    } catch (error) {
      console.error(`Error fetching page ${page}:`, error);
      throw error;
    }
  }

  return allVaults;
}

Advantages

✅ Simple: Easy to understand and implement ✅ Random access: Jump to any page directly (page=5) ✅ Total count: Know total number of results upfront

Disadvantages

❌ Cursor drift: If items are added/deleted during pagination, you might see duplicates or miss items ❌ Performance: Slow for very high page numbers (database must skip previous results)

When to use: Small to medium datasets (under 10,000 results), or when you need random page access.

Cursor-Based Pagination

How It Works

Request First Page:

GET /v1/vaults?limit=25

Response:

{
  "vaults": [
    { "id": "vault_1", "name": "Vault 1" },
    { "id": "vault_2", "name": "Vault 2" },
    ...
  ],
  "pagination": {
    "limit": 25,
    "next_cursor": "eyJpZCI6InZhdWx0XzI1IiwiY3JlYXRlZCI6MTcwMDAwMDAwMH0",
    "has_more": true
  }
}

Request Next Page (using next_cursor):

GET /v1/vaults?limit=25&cursor=eyJpZCI6InZhdWx0XzI1IiwiY3JlYXRlZCI6MTcwMDAwMDAwMH0

Query Parameters

cursor (string, optional):

Opaque cursor string from previous response
Don't construct manually (use value from next_cursor)
Omit for first page

limit (integer, default: 25):

Number of results per page
Minimum: 1
Maximum: 100

Response Fields

pagination object:

{
  "limit": 25,                                    // Results per page
  "next_cursor": "eyJ...",                        // Cursor for next page (null if no more pages)
  "has_more": true                                // Whether there are more results
}

Note: total is NOT included (not possible with cursor pagination).

Iterating Through Pages

Get All Results:

async function getAllVaults() {
  const allVaults = [];
  let cursor = null;

  while (true) {
    const url = cursor
      ? `https://api.torvussecurity.com/v1/vaults?cursor=${cursor}&limit=100`
      : `https://api.torvussecurity.com/v1/vaults?limit=100`;

    const response = await fetch(url, {
      headers: { 'Authorization': `Bearer ${apiKey}` }
    });
    const data = await response.json();

    allVaults.push(...data.vaults);

    if (!data.pagination.has_more) {
      break;
    }

    cursor = data.pagination.next_cursor;
  }

  return allVaults;
}

Advantages

✅ No cursor drift: Consistent results even if data changes during pagination ✅ Performance: Fast for large datasets (no skipping required) ✅ Real-time data: Best for live data that changes frequently

Disadvantages

❌ No random access: Can't jump to specific page (must iterate from start) ❌ No total count: Don't know total number of results upfront

When to use: Large datasets (10,000+ results), real-time data, or when consistency is critical.

Choosing Pagination Method

Feature	Page-Based	Cursor-Based
Simplicity	✅ Simple	⚠️ Slightly complex
Random access	✅ Yes (`page=5`)	❌ No
Total count	✅ Yes	❌ No
Performance (large datasets)	⚠️ Slow	✅ Fast
Consistency	⚠️ Cursor drift possible	✅ Consistent
Use case	Small-medium datasets, UI pagination	Large datasets, bulk exports

Default: If unsure, use page-based (simpler, good for most use cases).

Switch to cursor-based if:

Dataset is very large (10,000+ results)
Data changes frequently during pagination
You're doing bulk exports or background processing

Filtering

Query Parameters

Filter by Field:

GET /v1/vaults?status=active

Returns only vaults with status=active.

Multiple Filters (AND logic):

GET /v1/vaults?status=active&owner_id=user_123

Returns vaults where status=active AND owner_id=user_123.

Common Filters

By Status:

GET /v1/vaults?status=active
GET /v1/vaults?status=archived

By Owner:

GET /v1/vaults?owner_id=user_123

By Date Range:

GET /v1/vaults?created_after=2025-01-01&created_before=2025-12-31

By Type:

GET /v1/documents?type=pdf
GET /v1/documents?type=image

Date Filters

Available Date Filters:

created_after - Results created after date (inclusive)
created_before - Results created before date (inclusive)
updated_after - Results updated after date
updated_before - Results updated before date

Date Format: ISO 8601 (YYYY-MM-DD or YYYY-MM-DDTHH:MM:SSZ)

Examples:

# Vaults created in 2025
GET /v1/vaults?created_after=2025-01-01&created_before=2025-12-31

# Documents updated in last 7 days
GET /v1/documents?updated_after=2025-10-01

# Recipients created this month
GET /v1/recipients?created_after=2025-10-01&created_before=2025-10-31

Field Selection

Select Specific Fields (reduces response size):

GET /v1/vaults?fields=id,name,created_at

Response:

{
  "vaults": [
    {
      "id": "vault_1",
      "name": "Vault 1",
      "created_at": "2025-01-15T10:30:00Z"
      // Other fields omitted
    }
  ]
}

All Fields (default):

GET /v1/vaults

Returns all fields for each vault.

Benefits:

✅ Smaller response size (faster)
✅ Reduced bandwidth usage
✅ Only retrieve data you actually need

Sorting

Query Parameters

Sort by Field:

GET /v1/vaults?sort=created_at

Default sort order: ascending (oldest first).

Sort Descending:

GET /v1/vaults?sort=created_at:desc

Sort Ascending (explicit):

GET /v1/vaults?sort=created_at:asc

Multiple Sort Fields

Sort by Multiple Fields (comma-separated):

GET /v1/vaults?sort=status:asc,created_at:desc

Sorts by status ascending, then created_at descending (for vaults with same status).

Sortable Fields

Common Sortable Fields:

id - Resource ID (alphabetical)
name - Resource name (alphabetical)
created_at - Creation date (chronological)
updated_at - Last update date (chronological)
status - Status (alphabetical)

Example Use Cases:

Newest vaults first:

GET /v1/vaults?sort=created_at:desc

Alphabetical by name:

GET /v1/vaults?sort=name:asc

Most recently updated:

GET /v1/vaults?sort=updated_at:desc

Search

Full-Text Search

Search Across All Fields:

GET /v1/vaults?q=important

Searches vault name, description, and other text fields for "important".

Search Specific Fields:

GET /v1/vaults?q=name:important

Searches only name field.

Search Examples

Search Vaults:

# Find vaults containing "legal"
GET /v1/vaults?q=legal

# Find vaults with "2025" in name
GET /v1/vaults?q=name:2025

# Find vaults with "confidential" in description
GET /v1/vaults?q=description:confidential

Search Documents:

# Find documents containing "contract"
GET /v1/documents?q=contract

# Find PDF documents
GET /v1/documents?q=type:pdf

# Find documents with "invoice" in filename
GET /v1/documents?q=filename:invoice

Advanced Search

Combine Search with Filters:

GET /v1/vaults?q=legal&status=active&sort=created_at:desc

Finds active vaults containing "legal", sorted by newest first.

Search with Pagination:

GET /v1/vaults?q=important&page=1&limit=50

Search results are paginated like regular list results.

Combining Features

Complete Example

Use Case: Find all active vaults created in 2025, containing "legal" in name or description, sorted by most recent, with only ID and name returned.

Request:

GET /v1/vaults?status=active&created_after=2025-01-01&created_before=2025-12-31&q=legal&sort=created_at:desc&fields=id,name&limit=50

Breakdown:

status=active - Filter: only active vaults
created_after=2025-01-01 - Filter: created in 2025 or later
created_before=2025-12-31 - Filter: created in 2025 or earlier
q=legal - Search: contains "legal"
sort=created_at:desc - Sort: newest first
fields=id,name - Select: only ID and name
limit=50 - Pagination: 50 results per page

Response:

{
  "vaults": [
    {
      "id": "vault_123",
      "name": "Legal Documents 2025"
    },
    {
      "id": "vault_456",
      "name": "Legal Contracts"
    },
    ...
  ],
  "pagination": {
    "page": 1,
    "limit": 50,
    "total": 127,
    "total_pages": 3,
    "has_more": true
  }
}

Best Practices

Always Use Pagination

❌ Bad (retrieves all results at once):

const response = await fetch('https://api.torvussecurity.com/v1/vaults');
const { vaults } = await response.json();

If you have 10,000 vaults, this request will:

Take 30+ seconds
Return 50+ MB response
Likely hit timeout or memory limits

✅ Good (use pagination):

const vaults = [];
let page = 1;
let hasMore = true;

while (hasMore) {
  const response = await fetch(
    `https://api.torvussecurity.com/v1/vaults?page=${page}&limit=100`
  );
  const data = await response.json();

  vaults.push(...data.vaults);
  hasMore = data.pagination.has_more;
  page++;
}

Choose Appropriate Page Size

Too Small (limit=1):

Too many API calls (slow)
Hits rate limits faster

Too Large (limit=1000):

Slow responses
High memory usage
May hit timeout

Recommended: limit=25-100

Good balance of speed and efficiency
Default is limit=25

Use Field Selection

Only Request Fields You Need:

❌ Bad (returns all fields):

GET /v1/vaults

✅ Good (returns only needed fields):

GET /v1/vaults?fields=id,name

Benefits:

10-50% smaller responses
Faster JSON parsing
Less bandwidth usage

Cache When Appropriate

Cache Paginated Results (if data doesn't change frequently):

const cache = new Map();
const CACHE_TTL = 5 * 60 * 1000; // 5 minutes

async function getCachedPage(page) {
  const cacheKey = `vaults:page:${page}`;
  const cached = cache.get(cacheKey);

  if (cached && Date.now() - cached.timestamp < CACHE_TTL) {
    return cached.data;
  }

  const response = await fetch(
    `https://api.torvussecurity.com/v1/vaults?page=${page}&limit=50`
  );
  const data = await response.json();

  cache.set(cacheKey, { data, timestamp: Date.now() });
  return data;
}

Invalidate Cache After Mutations:

async function createVault(vaultData) {
  const vault = await fetch(/* create vault */);

  // Invalidate cache (new vault added)
  cache.clear();

  return vault;
}

Handle Edge Cases

Empty Results:

const response = await fetch('https://api.torvussecurity.com/v1/vaults');
const { vaults, pagination } = await response.json();

if (vaults.length === 0) {
  console.log('No vaults found');
}

Single Page of Results:

const { vaults, pagination } = await response.json();

if (!pagination.has_more) {
  console.log('All results on one page');
}

Very Large Result Sets:

// Don't try to fetch ALL results if there are millions
// Instead, process in batches

async function processBatches(batchSize = 100) {
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const { vaults, pagination } = await fetchPage(page, batchSize);

    // Process batch
    await processBatch(vaults);

    hasMore = pagination.has_more;
    page++;

    // Optional: limit total pages processed
    if (page > 100) {
      console.warn('Reached max pages (10,000 results)');
      break;
    }
  }
}

Error Handling

Invalid Parameters

Invalid Page Number:

GET /v1/vaults?page=0

Response:

{
  "error": "Validation failed",
  "errors": [
    {
      "field": "page",
      "message": "Page must be greater than 0"
    }
  ]
}

Invalid Limit:

GET /v1/vaults?limit=1000

Response:

{
  "error": "Validation failed",
  "errors": [
    {
      "field": "limit",
      "message": "Limit must not exceed 100"
    }
  ]
}

Invalid Cursor

Malformed or Expired Cursor:

GET /v1/vaults?cursor=invalid_cursor

Response:

{
  "error": "Invalid cursor",
  "message": "Cursor is malformed or expired. Please start from the first page."
}

Solution: Start over from first page (omit cursor parameter).

Code Examples

Python

import requests

def get_all_vaults(api_key):
    vaults = []
    page = 1
    has_more = True

    while has_more:
        response = requests.get(
            'https://api.torvussecurity.com/v1/vaults',
            headers={'Authorization': f'Bearer {api_key}'},
            params={'page': page, 'limit': 100}
        )
        response.raise_for_status()
        data = response.json()

        vaults.extend(data['vaults'])
        has_more = data['pagination']['has_more']
        page += 1

    return vaults

# Usage
vaults = get_all_vaults(api_key)
print(f'Total vaults: {len(vaults)}')

JavaScript/Node.js

const axios = require('axios');

async function getAllVaults(apiKey) {
  const vaults = [];
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const { data } = await axios.get(
      'https://api.torvussecurity.com/v1/vaults',
      {
        headers: { Authorization: `Bearer ${apiKey}` },
        params: { page, limit: 100 }
      }
    );

    vaults.push(...data.vaults);
    hasMore = data.pagination.has_more;
    page++;
  }

  return vaults;
}

// Usage
const vaults = await getAllVaults(apiKey);
console.log(`Total vaults: ${vaults.length}`);

Cursor-Based (JavaScript)

async function getAllVaultsCursor(apiKey) {
  const vaults = [];
  let cursor = null;

  while (true) {
    const params = { limit: 100 };
    if (cursor) params.cursor = cursor;

    const { data } = await axios.get(
      'https://api.torvussecurity.com/v1/vaults',
      {
        headers: { Authorization: `Bearer ${apiKey}` },
        params
      }
    );

    vaults.push(...data.vaults);

    if (!data.pagination.has_more) break;
    cursor = data.pagination.next_cursor;
  }

  return vaults;
}

FAQ

Can I get all results in one request?

No. All list endpoints are paginated. Maximum limit is 100 results per page.

Why? Performance and efficiency. Returning thousands of results at once is slow and wasteful.

Solution: Use pagination to fetch all results across multiple requests.

What's the maximum number of results I can retrieve?

No limit. You can paginate through millions of results if needed (though it will take time).

Rate limits apply: Fetching 1,000,000 results at 100/page = 10,000 API calls. Plan accordingly.

How do I know the total count without fetching all pages?

Page-based pagination: The total field in pagination object tells you total count.

Cursor-based pagination: Total count is NOT available. You must iterate through all pages.

Can I jump to a specific result (e.g., result #5000)?

Page-based: Yes. Calculate page: page = Math.ceil(5000 / limit).

Cursor-based: No. Must iterate from start.

Do filters affect pagination?

Yes. Pagination applies AFTER filters.

Example:

GET /v1/vaults?status=active&page=1&limit=50

Returns first 50 active vaults (not first 50 of all vaults).

What happens if data changes during pagination?

Page-based: Cursor drift possible (may see duplicates or miss results).

Cursor-based: Consistent results (no duplicates or missed results).

Best practice: Use cursor-based for data that changes frequently.

API Reference: Complete API endpoint documentation
API Best Practices: Performance and optimization
Rate Limiting: Rate limit details
Error Handling: Error codes and responses

Last Updated: October 8, 2025

Pagination​

Why Pagination?​

Page-Based Pagination​

How It Works​

Query Parameters​

Response Fields​

Iterating Through Pages​

Advantages​

Disadvantages​

Cursor-Based Pagination​

How It Works​

Query Parameters​

Response Fields​

Iterating Through Pages​

Advantages​

Disadvantages​

Choosing Pagination Method​

Filtering​

Query Parameters​

Common Filters​

Date Filters​

Field Selection​

Sorting​

Query Parameters​

Multiple Sort Fields​

Sortable Fields​

Search​

Full-Text Search​

Search Examples​

Advanced Search​

Combining Features​

Complete Example​

Best Practices​

Always Use Pagination​

Choose Appropriate Page Size​

Use Field Selection​

Cache When Appropriate​

Handle Edge Cases​

Error Handling​

Invalid Parameters​

Invalid Cursor​

Code Examples​

Python​

JavaScript/Node.js​

Cursor-Based (JavaScript)​

FAQ​

Can I get all results in one request?​

What's the maximum number of results I can retrieve?​

How do I know the total count without fetching all pages?​

Can I jump to a specific result (e.g., result #5000)?​

Do filters affect pagination?​

What happens if data changes during pagination?​

Related Guides​

Pagination

Why Pagination?

Page-Based Pagination

How It Works

Query Parameters

Response Fields

Iterating Through Pages

Advantages

Disadvantages

Cursor-Based Pagination

How It Works

Query Parameters

Response Fields

Iterating Through Pages

Advantages

Disadvantages

Choosing Pagination Method

Filtering

Query Parameters

Common Filters

Date Filters

Field Selection

Sorting

Query Parameters

Multiple Sort Fields

Sortable Fields

Search

Full-Text Search

Search Examples

Advanced Search

Combining Features

Complete Example

Best Practices

Always Use Pagination

Choose Appropriate Page Size

Use Field Selection

Cache When Appropriate

Handle Edge Cases

Error Handling

Invalid Parameters

Invalid Cursor

Code Examples

Python

JavaScript/Node.js

Cursor-Based (JavaScript)

FAQ

Can I get all results in one request?

What's the maximum number of results I can retrieve?

How do I know the total count without fetching all pages?

Can I jump to a specific result (e.g., result #5000)?

Do filters affect pagination?

What happens if data changes during pagination?

Related Guides