karu v0.4.0

Stateless byte range transport

Reading without a cursor

Karu treats every read as a complete request containing an object, byte position, length, destination buffer, and caller tag. Because the position travels with the request, reads can run independently without sharing a moving cursor. Before data moves, Karu resolves each window to its real byte position, rejects invalid ranges, and groups reads from the same object so the engine can build an efficient transfer plan.

Combining nearby ranges

Nearby ranges can share one larger transfer when that costs less than separate network trips. Strict limits on gaps, transfer size, pieces, and extra bytes keep this optimization from fetching too much data.

Local and remote reads

Local files use direct positional reads. Remote objects use HTTP requests for exact byte ranges. Karu keeps several transfers active, so one slow response does not stop the rest of the batch. The engine tracks queued work, active transfers, retries, and completed results while reusing network connections and TLS sessions. Credentials are resolved separately, without blocking bytes already in motion.

Checking every result

Karu accepts a result only when its returned range and byte count match the request. It retries temporary failures with bounded delays, then separates merged data into the original buffers and caller tags.

Detailed synchronization between Rumi, Karu, and storage Rumi maps a pixel window to compressed frames and submits tagged byte ranges. Karu resolves credentials, coalesces transfers, fetches from storage, scatters the returned bytes, and completes each original tag so Rumi can decode. RUMI KARU Pixel window x · y · width · height read Frame index select compressed frames plan Tagged ranges offset · length · frame tag submit batch Completions buffer · status · frame tag scatter Transfer plan sort · group · coalesce bytes Transport resolve · sign · parallel GET ranges Storage local · HTTP · cloud each completion wakes Rumi with its original frame tag exact bytes return to Karu Rumi decodes ready frames while other reads finish
Figure 1 Rumi turns a pixel window into tagged frame ranges. Karu groups those ranges, obtains the required credentials, and retrieves only the needed bytes. Each result keeps its frame tag, so Rumi can decode completed frames while the remaining reads continue.