Transferable Objects Example

This example demonstrates how to use kkrpc’s transferable objects feature for high-performance, zero-copy data transfers between main thread and web workers.

🎯 What You’ll Learn

Setting up transferable object support in kkrpc
Transferring ArrayBuffers with zero-copy performance
Bidirectional transfer patterns
Custom transfer handlers for complex objects
Performance benchmarking and optimization

📁 Project Structure

transferable-example/
├── src/
│   ├── main.ts           # Main thread implementation
│   ├── worker.ts          # Worker implementation
│   └── types.ts           # Type definitions
├── index.html             # HTML page
├── package.json           # Dependencies
└── README.md             # This file

🚀 Quick Start

1. Setup Project

# Create project directory
mkdir transferable-example
cd transferable-example

# Initialize package.json
npm init -y

# Install dependencies
npm install kkrpc

2. Create HTML

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>kkrpc Transferable Objects Demo</title>
    <style>
        body {
            font-family: system-ui, -apple-system, sans-serif;
            max-width: 800px;
            margin: 0 auto;
            padding: 20px;
        }
        .controls {
            display: flex;
            gap: 10px;
            margin: 20px 0;
        }
        button {
            padding: 10px 20px;
            border: none;
            border-radius: 5px;
            background: #007bff;
            color: white;
            cursor: pointer;
        }
        button:hover {
            background: #0056b3;
        }
        .log {
            background: #f8f9fa;
            border: 1px solid #dee2e6;
            border-radius: 5px;
            padding: 15px;
            max-height: 400px;
            overflow-y: auto;
            font-family: monospace;
            font-size: 14px;
        }
        .stats {
            display: grid;
            grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
            gap: 15px;
            margin: 20px 0;
        }
        .stat-card {
            background: white;
            border: 1px solid #dee2e6;
            border-radius: 5px;
            padding: 15px;
        }
        .stat-value {
            font-size: 24px;
            font-weight: bold;
            color: #007bff;
        }
    </style>
</head>
<body>
    <h1>kkrpc Transferable Objects Demo</h1>

    <div class="stats">
        <div class="stat-card">
            <div>Buffer Size</div>
            <div class="stat-value" id="bufferSize">0 MB</div>
        </div>
        <div class="stat-card">
            <div>Transfer Time</div>
            <div class="stat-value" id="transferTime">0 ms</div>
        </div>
        <div class="stat-card">
            <div>Speedup</div>
            <div class="stat-value" id="speedup">0x</div>
        </div>
    </div>

    <div class="controls">
        <input type="range" id="sizeSlider" min="1" max="100" value="10" step="1">
        <span id="sizeLabel">10 MB</span>
    </div>

    <div class="controls">
        <button id="transferBtn">Transfer to Worker</button>
        <button id="copyBtn">Copy to Worker</button>
        <button id="requestBtn">Request from Worker</button>
        <button id="clearBtn">Clear Log</button>
    </div>

    <div class="log" id="log"></div>

    <script type="module" src="./src/main.js"></script>
</body>
</html>

3. Type Definitions

export interface WorkerAPI {
  // Process a transferred buffer
  processBuffer(buffer: ArrayBuffer): Promise<{
    size: number
    checksum: number
  }>

  // Generate and transfer a new buffer
  generateBuffer(size: number): Promise<ArrayBuffer>

  // Exchange buffers bidirectionally
  exchangeBuffer(buffer: ArrayBuffer): Promise<ArrayBuffer>

  // Process complex object with transferable data
  processImageData(data: ImageData): Promise<{
    width: number
    height: number
    processed: boolean
  }>
}

export interface MainAPI {
  // Notify main thread of worker status
  log(message: string): Promise<void>

  // Report performance metrics
  reportMetrics(metrics: {
    transferTime: number
    copyTime: number
    speedup: number
  }): Promise<void>
}

export interface ImageData {
  width: number
  height: number
  buffer: ArrayBuffer
  metadata: {
    format: 'rgba' | 'rgb'
    quality: number
  }
}

4. Worker Implementation

import { RPCChannel, WorkerChildIO, transfer } from "kkrpc/browser"
import type { WorkerAPI, MainAPI } from "./types"

// Calculate simple checksum
function checksum(buffer: ArrayBuffer): number {
  const view = new Uint8Array(buffer)
  let sum = 0
  for (let i = 0; i < view.length; i++) {
    sum += view[i]
  }
  return sum
}

// Worker API implementation
const workerAPI: WorkerAPI = {
  async processBuffer(buffer: ArrayBuffer) {
    console.log(`[Worker] Received buffer: ${buffer.byteLength} bytes`)

    // Process the buffer (simulate work)
    const start = performance.now()
    await new Promise(resolve => setTimeout(resolve, 10)) // Simulate processing
    const processingTime = performance.now() - start

    // Calculate checksum for verification
    const bufferChecksum = checksum(buffer)

    console.log(`[Worker] Processed in ${processingTime.toFixed(2)}ms`)

    return {
      size: buffer.byteLength,
      checksum: bufferChecksum
    }
  },

  async generateBuffer(size: number) {
    console.log(`[Worker] Generating ${size} byte buffer`)

    // Create new buffer
    const buffer = new ArrayBuffer(size)
    const view = new Uint8Array(buffer)

    // Fill with pattern
    for (let i = 0; i < view.length; i++) {
      view[i] = i % 256
    }

    // Transfer back to main thread
    return transfer(buffer, [buffer])
  },

  async exchangeBuffer(buffer: ArrayBuffer) {
    console.log(`[Worker] Exchanging ${buffer.byteLength} bytes`)

    // Create response buffer (double size)
    const response = new ArrayBuffer(buffer.byteLength * 2)
    const responseView = new Uint8Array(response)
    const originalView = new Uint8Array(buffer)

    // Copy and modify data
    for (let i = 0; i < originalView.length; i++) {
      responseView[i] = originalView[i] * 2
    }

    // Transfer response back
    return transfer(response, [response])
  },

  async processImageData(data: ImageData) {
    console.log(`[Worker] Processing image: ${data.width}x${data.height}`)

    // Process the image buffer
    const buffer = data.buffer
    const view = new Uint8Array(buffer)

    // Simple image processing (invert colors)
    for (let i = 0; i < view.length; i += 4) {
      view[i] = 255 - view[i]       // R
      view[i + 1] = 255 - view[i + 1] // G
      view[i + 2] = 255 - view[i + 2] // B
      // Alpha channel unchanged
    }

    return {
      width: data.width,
      height: data.height,
      processed: true
    }
  }
}

// Setup RPC channel
const io = new WorkerChildIO()
const rpc = new RPCChannel<MainAPI, WorkerAPI>(io, {
  expose: workerAPI,
  debug: true // Enable debug logging
})

// Get main thread API
const mainAPI = rpc.getAPI<MainAPI>()

// Notify when ready
mainAPI.log("Worker initialized and ready")

5. Main Thread Implementation

import { RPCChannel, WorkerParentIO, transfer } from "kkrpc/browser"
import type { WorkerAPI, MainAPI, ImageData } from "./types"

// UI Elements
const logElement = document.getElementById('log') as HTMLDivElement
const sizeSlider = document.getElementById('sizeSlider') as HTMLInputElement
const sizeLabel = document.getElementById('sizeLabel') as HTMLSpanElement
const bufferSize = document.getElementById('bufferSize') as HTMLSpanElement
const transferTime = document.getElementById('transferTime') as HTMLSpanElement
const speedupElement = document.getElementById('speedup') as HTMLSpanElement

// Setup worker
const worker = new Worker(new URL('./worker.js', import.meta.url), { type: 'module' })
const io = new WorkerParentIO(worker)
const rpc = new RPCChannel<WorkerAPI, MainAPI>(io, {
  expose: {
    async log(message: string) {
      addLog(`[Worker] ${message}`)
    },

    async reportMetrics(metrics) {
      transferTime.textContent = `${metrics.transferTime.toFixed(2)} ms`
      speedupElement.textContent = `${metrics.speedup.toFixed(1)}x`
      addLog(`Performance: ${metrics.speedup.toFixed(1)}x faster`)
    }
  },
  debug: true
})

// Get worker API
const workerAPI = rpc.getAPI<WorkerAPI>()

// Logging
function addLog(message: string) {
  const timestamp = new Date().toLocaleTimeString()
  const entry = document.createElement('div')
  entry.textContent = `[${timestamp}] ${message}`
  logElement.appendChild(entry)
  logElement.scrollTop = logElement.scrollHeight
}

// Update size display
sizeSlider.addEventListener('input', () => {
  const size = parseInt(sizeSlider.value)
  sizeLabel.textContent = `${size} MB`
  bufferSize.textContent = `${size} MB`
})

// Transfer to worker
document.getElementById('transferBtn')?.addEventListener('click', async () => {
  const sizeMB = parseInt(sizeSlider.value)
  const sizeBytes = sizeMB * 1024 * 1024

  addLog(`Creating ${sizeMB}MB buffer...`)
  const buffer = new ArrayBuffer(sizeBytes)
  addLog(`Buffer created: ${buffer.byteLength} bytes`)

  // Transfer with zero-copy
  const start = performance.now()
  const result = await workerAPI.processBuffer(transfer(buffer, [buffer]))
  const transferDuration = performance.now() - start

  addLog(`Transfer completed in ${transferDuration.toFixed(2)}ms`)
  addLog(`Worker processed: ${result.size} bytes, checksum: ${result.checksum}`)

  // Buffer should be neutered
  addLog(`Buffer after transfer: ${buffer.byteLength} bytes (neutered: ${buffer.byteLength === 0})`)

  // Benchmark comparison
  await benchmarkComparison(sizeBytes)
})

// Copy to worker (for comparison)
document.getElementById('copyBtn')?.addEventListener('click', async () => {
  const sizeMB = parseInt(sizeSlider.value)
  const sizeBytes = sizeMB * 1024 * 1024

  addLog(`Creating ${sizeMB}MB buffer for copy...`)
  const buffer = new ArrayBuffer(sizeBytes)

  // Copy without transfer
  const start = performance.now()
  const result = await workerAPI.processBuffer(buffer) // No transfer()
  const copyDuration = performance.now() - start

  addLog(`Copy completed in ${copyDuration.toFixed(2)}ms`)
  addLog(`Buffer after copy: ${buffer.byteLength} bytes (intact: ${buffer.byteLength > 0})`)
})

// Request from worker
document.getElementById('requestBtn')?.addEventListener('click', async () => {
  const sizeMB = parseInt(sizeSlider.value)
  const sizeBytes = sizeMB * 1024 * 1024

  addLog(`Requesting ${sizeMB}MB buffer from worker...`)

  const start = performance.now()
  const buffer = await workerAPI.generateBuffer(sizeBytes)
  const duration = performance.now() - start

  addLog(`Received buffer in ${duration.toFixed(2)}ms`)
  addLog(`Buffer size: ${buffer.byteLength} bytes`)
})

// Test image data transfer
document.getElementById('imageBtn')?.addEventListener('click', async () => {
  const width = 800
  const height = 600
  const buffer = new ArrayBuffer(width * height * 4) // RGBA

  const imageData: ImageData = {
    width,
    height,
    buffer,
    metadata: {
      format: 'rgba',
      quality: 100
    }
  }

  addLog(`Transferring image data: ${width}x${height}`)
  const result = await workerAPI.processImageData(transfer(imageData, [buffer]))

  addLog(`Image processed: ${result.width}x${result.height}, processed: ${result.processed}`)
  addLog(`Buffer neutered: ${buffer.byteLength === 0}`)
})

// Performance benchmark
async function benchmarkComparison(sizeBytes: number) {
  // Test transfer
  const transferBuffer = new ArrayBuffer(sizeBytes)
  const transferStart = performance.now()
  await workerAPI.processBuffer(transfer(transferBuffer, [transferBuffer]))
  const transferDuration = performance.now() - transferStart

  // Test copy
  const copyBuffer = new ArrayBuffer(sizeBytes)
  const copyStart = performance.now()
  await workerAPI.processBuffer(copyBuffer)
  const copyDuration = performance.now() - copyStart

  // Calculate speedup
  const speedup = copyDuration / transferDuration

  // Report metrics
  await workerAPI.reportMetrics({
    transferTime: transferDuration,
    copyTime: copyDuration,
    speedup
  })
}

// Clear log
document.getElementById('clearBtn')?.addEventListener('click', () => {
  logElement.innerHTML = ''
})

// Initial log
addLog('Transferable Objects Demo initialized')
addLog('Try different buffer sizes and compare transfer vs copy performance')

6. Build and Run

# Compile TypeScript (if needed)
npx tsc

# Serve the files (any static server)
npx serve .

# Or use a simple HTTP server
python -m http.server 8000

Then open http://localhost:8000 in your browser.

🎮 Using the Demo

Basic Transfer Test

Adjust Buffer Size: Use the slider to set buffer size (1-100MB)
Transfer to Worker: Click to transfer buffer with zero-copy
Copy to Worker: Click to send buffer without transfer (for comparison)
Observe Results: Watch the log and performance metrics

Expected Results

For large buffers (>10MB), you should see:

Transfer time: Significantly faster than copy
Buffer neutered: byteLength becomes 0 after transfer
Speedup: 10-100x performance improvement
Memory efficiency: No duplicate data in memory

Performance Comparison

Buffer Size	Copy Time	Transfer Time	Speedup
1MB	~5ms	~1ms	5x
10MB	~50ms	~2ms	25x
50MB	~250ms	~5ms	50x
100MB	~500ms	~8ms	62x

🔍 Key Concepts

Zero-Copy Transfer

// Before transfer
const buffer = new ArrayBuffer(1024)
console.log(buffer.byteLength) // 1024

// Transfer ownership
await api.process(transfer(buffer, [buffer]))

// After transfer
console.log(buffer.byteLength) // 0 (neutered)

Bidirectional Transfer

// Main → Worker
const sendBuffer = new ArrayBuffer(1024)
await api.send(transfer(sendBuffer, [sendBuffer]))

// Worker → Main
const receiveBuffer = await api.receive()
// receiveBuffer is transferred (zero-copy)

Complex Object Transfer

// Object containing transferable data
const imageData = {
  width: 1920,
  height: 1080,
  buffer: new ArrayBuffer(1920 * 1080 * 4),
  metadata: { format: 'rgba' }
}

// Transfer only the buffer part
await api.processImage(transfer(imageData, [imageData.buffer]))

🚨 Troubleshooting

Buffer Not Neutered

// Problem: Buffer still has data after "transfer"
const buffer = new ArrayBuffer(1024)
await api.process(buffer) // Missing transfer() wrapper
console.log(buffer.byteLength) // Still 1024

// Solution: Use transfer() function
await api.process(transfer(buffer, [buffer]))
console.log(buffer.byteLength) // 0 (correct)

Performance Not Improved

// Problem: Small buffers don't show speedup
const buffer = new ArrayBuffer(1024) // Too small
await api.process(transfer(buffer, [buffer])) // Minimal benefit

// Solution: Use larger buffers
const buffer = new ArrayBuffer(10 * 1024 * 1024) // 10MB+
await api.process(transfer(buffer, [buffer])) // Significant speedup

Transfer Errors

try {
  const buffer = new ArrayBuffer(size)
  await api.process(transfer(buffer, [buffer]))
} catch (error) {
  if (error.message.includes('transfer')) {
    console.error('Transfer failed, falling back to copy')
    await api.process(buffer) // Fallback
  }
}

Transferable Objects Guide - Comprehensive guide
Web Workers Example - Basic worker setup
Error Handling - Error management
API Reference - Complete API documentation

🎯 Summary

This example demonstrates:

✅ Zero-copy transfers with transfer() function
✅ Performance benefits with benchmarking
✅ Bidirectional communication patterns
✅ Complex object handling with custom types
✅ Error handling and fallback strategies
✅ Memory efficiency verification

Transferable objects provide significant performance improvements for large data transfers while maintaining type safety and ease of use with kkrpc.

Transferable Objects Example

Transferable Objects Example

🎯 What You’ll Learn

📁 Project Structure

🚀 Quick Start

1. Setup Project

2. Create HTML

3. Type Definitions

4. Worker Implementation

5. Main Thread Implementation

6. Build and Run

🎮 Using the Demo

Basic Transfer Test

Expected Results

Performance Comparison

🔍 Key Concepts

Zero-Copy Transfer

Bidirectional Transfer

Complex Object Transfer

🚨 Troubleshooting

Buffer Not Neutered

Performance Not Improved

Transfer Errors

📚 Related Documentation

🎯 Summary