Transferable Objects Example

Transferable Objects Example

This example demonstrates how to use kkrpc’s transferable objects feature for high-performance, zero-copy data transfers between main thread and web workers.

🎯 What You’ll Learn

• Setting up transferable object support in kkrpc
• Transferring ArrayBuffers with zero-copy performance
• Bidirectional transfer patterns
• Custom transfer handlers for complex objects
• Performance benchmarking and optimization

📁 Project Structure

transferable-example/
├── src/
│   ├── main.ts           # Main thread implementation
│   ├── worker.ts          # Worker implementation
│   └── types.ts           # Type definitions
├── index.html             # HTML page
├── package.json           # Dependencies
└── README.md             # This file

🚀 Quick Start

1. Setup Project

# Create project directory
mkdir transferable-example
cd transferable-example

# Initialize package.json
npm init -y

# Install dependencies
npm install kkrpc

2. Create HTML

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>kkrpc Transferable Objects Demo</title>
    <style>
      body {
        font-family:
          system-ui,
          -apple-system,
          sans-serif;
        max-width: 800px;
        margin: 0 auto;
        padding: 20px;
      }
      .controls {
        display: flex;
        gap: 10px;
        margin: 20px 0;
      }
      button {
        padding: 10px 20px;
        border: none;
        border-radius: 5px;
        background: #007bff;
        color: white;
        cursor: pointer;
      }
      button:hover {
        background: #0056b3;
      }
      .log {
        background: #f8f9fa;
        border: 1px solid #dee2e6;
        border-radius: 5px;
        padding: 15px;
        max-height: 400px;
        overflow-y: auto;
        font-family: monospace;
        font-size: 14px;
      }
      .stats {
        display: grid;
        grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
        gap: 15px;
        margin: 20px 0;
      }
      .stat-card {
        background: white;
        border: 1px solid #dee2e6;
        border-radius: 5px;
        padding: 15px;
      }
      .stat-value {
        font-size: 24px;
        font-weight: bold;
        color: #007bff;
      }
    </style>
  </head>
  <body>
    <h1>kkrpc Transferable Objects Demo</h1>

    <div class="stats">
      <div class="stat-card">
        <div>Buffer Size</div>
        <div class="stat-value" id="bufferSize">0 MB</div>
      </div>
      <div class="stat-card">
        <div>Transfer Time</div>
        <div class="stat-value" id="transferTime">0 ms</div>
      </div>
      <div class="stat-card">
        <div>Speedup</div>
        <div class="stat-value" id="speedup">0x</div>
      </div>
    </div>

    <div class="controls">
      <input type="range" id="sizeSlider" min="1" max="100" value="10" step="1" />
      <span id="sizeLabel">10 MB</span>
    </div>

    <div class="controls">
      <button id="transferBtn">Transfer to Worker</button>
      <button id="copyBtn">Copy to Worker</button>
      <button id="requestBtn">Request from Worker</button>
      <button id="clearBtn">Clear Log</button>
    </div>

    <div class="log" id="log"></div>

    <script type="module" src="./src/main.js"></script>
  </body>
</html>

3. Type Definitions

src/types.ts

export interface WorkerAPI {
  // Process a transferred buffer
  processBuffer(buffer: ArrayBuffer): Promise<{
    size: number
    checksum: number
  }>

  // Generate and transfer a new buffer
  generateBuffer(size: number): Promise<ArrayBuffer>

  // Exchange buffers bidirectionally
  exchangeBuffer(buffer: ArrayBuffer): Promise<ArrayBuffer>

  // Process complex object with transferable data
  processImageData(data: ImageData): Promise<{
    width: number
    height: number
    processed: boolean
  }>
}

export interface MainAPI {
  // Notify main thread of worker status
  log(message: string): Promise<void>

  // Report performance metrics
  reportMetrics(metrics: { transferTime: number; copyTime: number; speedup: number }): Promise<void>
}

export interface ImageData {
  width: number
  height: number
  buffer: ArrayBuffer
  metadata: {
    format: "rgba" | "rgb"
    quality: number
  }
}

4. Worker Implementation (src/worker.ts)

import { RPCChannel, transfer } from "kkrpc/browser"
import { workerSelfTransport } from "kkrpc/worker"
import type { MainAPI, WorkerAPI } from "./types"

// Calculate simple checksum
function checksum(buffer: ArrayBuffer): number {
  const view = new Uint8Array(buffer)
  let sum = 0
  for (let i = 0; i < view.length; i++) {
    sum += view[i]
  }
  return sum
}

// Worker API implementation
const workerAPI: WorkerAPI = {
  async processBuffer(buffer: ArrayBuffer) {
    console.log(`[Worker] Received buffer: ${buffer.byteLength} bytes`)

    // Process the buffer (simulate work)
    const start = performance.now()
    await new Promise((resolve) => setTimeout(resolve, 10)) // Simulate processing
    const processingTime = performance.now() - start

    // Calculate checksum for verification
    const bufferChecksum = checksum(buffer)

    console.log(`[Worker] Processed in ${processingTime.toFixed(2)}ms`)

    return {
      size: buffer.byteLength,
      checksum: bufferChecksum
    }
  },

  async generateBuffer(size: number) {
    console.log(`[Worker] Generating ${size} byte buffer`)

    // Create new buffer
    const buffer = new ArrayBuffer(size)
    const view = new Uint8Array(buffer)

    // Fill with pattern
    for (let i = 0; i < view.length; i++) {
      view[i] = i % 256
    }

    // Transfer back to main thread
    return transfer(buffer, [buffer])
  },

  async exchangeBuffer(buffer: ArrayBuffer) {
    console.log(`[Worker] Exchanging ${buffer.byteLength} bytes`)

    // Create response buffer (double size)
    const response = new ArrayBuffer(buffer.byteLength * 2)
    const responseView = new Uint8Array(response)
    const originalView = new Uint8Array(buffer)

    // Copy and modify data
    for (let i = 0; i < originalView.length; i++) {
      responseView[i] = originalView[i] * 2
    }

    // Transfer response back
    return transfer(response, [response])
  },

  async processImageData(data: ImageData) {
    console.log(`[Worker] Processing image: ${data.width}x${data.height}`)

    // Process the image buffer
    const buffer = data.buffer
    const view = new Uint8Array(buffer)

    // Simple image processing (invert colors)
    for (let i = 0; i < view.length; i += 4) {
      view[i] = 255 - view[i] // R
      view[i + 1] = 255 - view[i + 1] // G
      view[i + 2] = 255 - view[i + 2] // B
      // Alpha channel unchanged
    }

    return {
      width: data.width,
      height: data.height,
      processed: true
    }
  }
}

// Setup RPC channel
const transport = workerSelfTransport()
const rpc = new RPCChannel<MainAPI, WorkerAPI>(transport, {
  expose: workerAPI,
  debug: true // Enable debug logging
})

// Get main thread API
const mainAPI = rpc.getAPI()

// Notify when ready
mainAPI.log("Worker initialized and ready")

5. Main Thread Implementation (src/main.ts)

import { RPCChannel, transfer } from "kkrpc/browser"
import { workerTransport } from "kkrpc/worker"
import type { ImageData, MainAPI, WorkerAPI } from "./types"

// UI Elements
const logElement = document.getElementById("log") as HTMLDivElement
const sizeSlider = document.getElementById("sizeSlider") as HTMLInputElement
const sizeLabel = document.getElementById("sizeLabel") as HTMLSpanElement
const bufferSize = document.getElementById("bufferSize") as HTMLSpanElement
const transferTime = document.getElementById("transferTime") as HTMLSpanElement
const speedupElement = document.getElementById("speedup") as HTMLSpanElement

// Setup worker
const worker = new Worker(new URL("./worker.js", import.meta.url), { type: "module" })
const transport = workerTransport(worker)
const rpc = new RPCChannel<WorkerAPI, MainAPI>(transport, {
  expose: {
    async log(message: string) {
      addLog(`[Worker] ${message}`)
    },

    async reportMetrics(metrics) {
      transferTime.textContent = `${metrics.transferTime.toFixed(2)} ms`
      speedupElement.textContent = `${metrics.speedup.toFixed(1)}x`
      addLog(`Performance: ${metrics.speedup.toFixed(1)}x faster`)
    }
  },
  debug: true
})

// Get worker API
const workerAPI = rpc.getAPI()

// Logging
function addLog(message: string) {
  const timestamp = new Date().toLocaleTimeString()
  const entry = document.createElement("div")
  entry.textContent = `[${timestamp}] ${message}`
  logElement.appendChild(entry)
  logElement.scrollTop = logElement.scrollHeight
}

// Update size display
sizeSlider.addEventListener("input", () => {
  const size = parseInt(sizeSlider.value)
  sizeLabel.textContent = `${size} MB`
  bufferSize.textContent = `${size} MB`
})

// Transfer to worker
document.getElementById("transferBtn")?.addEventListener("click", async () => {
  const sizeMB = parseInt(sizeSlider.value)
  const sizeBytes = sizeMB * 1024 * 1024

  addLog(`Creating ${sizeMB}MB buffer...`)
  const buffer = new ArrayBuffer(sizeBytes)
  addLog(`Buffer created: ${buffer.byteLength} bytes`)

  // Transfer with zero-copy
  const start = performance.now()
  const result = await workerAPI.processBuffer(transfer(buffer, [buffer]))
  const transferDuration = performance.now() - start

  addLog(`Transfer completed in ${transferDuration.toFixed(2)}ms`)
  addLog(`Worker processed: ${result.size} bytes, checksum: ${result.checksum}`)

  // Buffer should be neutered
  addLog(`Buffer after transfer: ${buffer.byteLength} bytes (neutered: ${buffer.byteLength === 0})`)

  // Benchmark comparison
  await benchmarkComparison(sizeBytes)
})

// Copy to worker (for comparison)
document.getElementById("copyBtn")?.addEventListener("click", async () => {
  const sizeMB = parseInt(sizeSlider.value)
  const sizeBytes = sizeMB * 1024 * 1024

  addLog(`Creating ${sizeMB}MB buffer for copy...`)
  const buffer = new ArrayBuffer(sizeBytes)

  // Copy without transfer
  const start = performance.now()
  const result = await workerAPI.processBuffer(buffer) // No transfer()
  const copyDuration = performance.now() - start

  addLog(`Copy completed in ${copyDuration.toFixed(2)}ms`)
  addLog(`Buffer after copy: ${buffer.byteLength} bytes (intact: ${buffer.byteLength > 0})`)
})

// Request from worker
document.getElementById("requestBtn")?.addEventListener("click", async () => {
  const sizeMB = parseInt(sizeSlider.value)
  const sizeBytes = sizeMB * 1024 * 1024

  addLog(`Requesting ${sizeMB}MB buffer from worker...`)

  const start = performance.now()
  const buffer = await workerAPI.generateBuffer(sizeBytes)
  const duration = performance.now() - start

  addLog(`Received buffer in ${duration.toFixed(2)}ms`)
  addLog(`Buffer size: ${buffer.byteLength} bytes`)
})

// Test image data transfer
document.getElementById("imageBtn")?.addEventListener("click", async () => {
  const width = 800
  const height = 600
  const buffer = new ArrayBuffer(width * height * 4) // RGBA

  const imageData: ImageData = {
    width,
    height,
    buffer,
    metadata: {
      format: "rgba",
      quality: 100
    }
  }

  addLog(`Transferring image data: ${width}x${height}`)
  const result = await workerAPI.processImageData(transfer(imageData, [buffer]))

  addLog(`Image processed: ${result.width}x${result.height}, processed: ${result.processed}`)
  addLog(`Buffer neutered: ${buffer.byteLength === 0}`)
})

// Performance benchmark
async function benchmarkComparison(sizeBytes: number) {
  // Test transfer
  const transferBuffer = new ArrayBuffer(sizeBytes)
  const transferStart = performance.now()
  await workerAPI.processBuffer(transfer(transferBuffer, [transferBuffer]))
  const transferDuration = performance.now() - transferStart

  // Test copy
  const copyBuffer = new ArrayBuffer(sizeBytes)
  const copyStart = performance.now()
  await workerAPI.processBuffer(copyBuffer)
  const copyDuration = performance.now() - copyStart

  // Calculate speedup
  const speedup = copyDuration / transferDuration

  // Report metrics
  await workerAPI.reportMetrics({
    transferTime: transferDuration,
    copyTime: copyDuration,
    speedup
  })
}

// Clear log
document.getElementById("clearBtn")?.addEventListener("click", () => {
  logElement.innerHTML = ""
})

// Initial log
addLog("Transferable Objects Demo initialized")
addLog("Try different buffer sizes and compare transfer vs copy performance")

6. Build and Run

# Compile TypeScript (if needed)
npx tsc

# Serve the files (any static server)
npx serve .

# Or use a simple HTTP server
python -m http.server 8000

Then open http://localhost:8000 in your browser.

🎮 Using the Demo

Basic Transfer Test

1. Adjust Buffer Size: Use the slider to set buffer size (1-100MB)
2. Transfer to Worker: Click to transfer buffer with zero-copy
3. Copy to Worker: Click to send buffer without transfer (for comparison)
4. Observe Results: Watch the log and performance metrics

Expected Results

For large buffers (>10MB), you should see:

• Transfer time: Significantly faster than copy
• Buffer neutered: byteLength becomes 0 after transfer
• Speedup: 10-100x performance improvement
• Memory efficiency: No duplicate data in memory

Performance Comparison

Buffer Size	Copy Time	Transfer Time	Speedup
1MB	~5ms	~1ms	5x
10MB	~50ms	~2ms	25x
50MB	~250ms	~5ms	50x
100MB	~500ms	~8ms	62x

🔍 Key Concepts

Zero-Copy Transfer

// Before transfer
const buffer = new ArrayBuffer(1024)
console.log(buffer.byteLength) // 1024

// Transfer ownership
await api.process(transfer(buffer, [buffer]))

// After transfer
console.log(buffer.byteLength) // 0 (neutered)

Bidirectional Transfer

// Main → Worker
const sendBuffer = new ArrayBuffer(1024)
await api.send(transfer(sendBuffer, [sendBuffer]))

// Worker → Main
const receiveBuffer = await api.receive()
// receiveBuffer is transferred (zero-copy)

Complex Object Transfer

// Object containing transferable data
const imageData = {
  width: 1920,
  height: 1080,
  buffer: new ArrayBuffer(1920 * 1080 * 4),
  metadata: { format: "rgba" }
}

// Transfer only the buffer part
await api.processImage(transfer(imageData, [imageData.buffer]))

🚨 Troubleshooting

Buffer Not Neutered

// Problem: Buffer still has data after "transfer"
const buffer = new ArrayBuffer(1024)
await api.process(buffer) // Missing transfer() wrapper
console.log(buffer.byteLength) // Still 1024

// Solution: Use transfer() function
await api.process(transfer(buffer, [buffer]))
console.log(buffer.byteLength) // 0 (correct)

Performance Not Improved

// Problem: Small buffers don't show speedup
const buffer = new ArrayBuffer(1024) // Too small
await api.process(transfer(buffer, [buffer])) // Minimal benefit

// Solution: Use larger buffers
const buffer = new ArrayBuffer(10 * 1024 * 1024) // 10MB+
await api.process(transfer(buffer, [buffer])) // Significant speedup

Transfer Errors

try {
  const buffer = new ArrayBuffer(size)
  await api.process(transfer(buffer, [buffer]))
} catch (error) {
  if (error.message.includes("transfer")) {
    console.error("Transfer failed, falling back to copy")
    await api.process(buffer) // Fallback
  }
}

📚 Related Documentation

• Transferable Objects Guide - Comprehensive guide
• Web Workers Example - Basic worker setup
• Error Handling - Error management
• API Reference - Complete API documentation

🎯 Summary

This example demonstrates:

✅ Zero-copy transfers with transfer() function
✅ Performance benefits with benchmarking
✅ Bidirectional communication patterns
✅ Complex object handling with custom types
✅ Error handling and fallback strategies
✅ Memory efficiency verification

Transferable objects provide significant performance improvements for large data transfers while maintaining type safety and ease of use with kkrpc.