Configuration reference

Karu captures configuration when a client is created. The client keeps that snapshot for its entire lifetime, so later changes to the process environment do not change requests that are already using it.

karu_config_create() reads supported environment variables and records the home and cache directories used for credential discovery. karu_config_create_empty() starts without those implicit sources, which is useful for isolated applications and tests.

Global options replace values captured from the environment. Path options are more specific and win for the longest matching VSI prefix. A path scope includes the named resource and its descendants, but stops at a segment boundary.

Option names ignore letter case and unknown names are rejected. Aliases are converted to their canonical names before priority is evaluated, so an alias cannot bypass a more specific path option.

AWS_DEFAULT_PROFILE, AWS_DEFAULT_REGION, AWS_ENDPOINT_URL_S3, AWS_ENDPOINT_URL, and HUGGING_FACE_HUB_TOKEN use their canonical AWS or Hugging Face names. SOURCE_PROXY_URL maps to SOURCE_ENDPOINT. CURL_CA_BUNDLE and SSL_CERT_FILE map to KARU_HTTP_CA_BUNDLE.

Runtime

Limits apply per client. With multiple worker processes, compare KARU_IO_THREADS=1 and 4 on the full workload, including decoding.

The four coalescing ceilings may be overridden for one batch through karu_client_submit_with(), karu_client_fetch_with(), or the C++ SubmitOptions type. Other runtime options are fixed when the client is created. Local files only merge overlapping or contiguous ranges.

AWS S3

Static profiles, credential_process, and web-identity profiles are handled natively. Chained source-profile AssumeRole and IAM Identity Center should be provided by the application's AWS SDK through the callback API. A credential_process is terminated if it exceeds KARU_REQUEST_TIMEOUT.

AWS_S3_ENDPOINT is a service root. With virtual hosting enabled, an endpoint of https://objects.example.test becomes https://bucket.objects.example.test/key. With virtual hosting disabled it becomes https://objects.example.test/bucket/key.

Source Cooperative

source://account/product/key and /vsisource/account/product/key resolve to the Source Data Proxy. Public reads need no configuration and remain unsigned. Source credentials use their own namespace and callback kind, so Karu never forwards ambient AWS credentials to the proxy.

Installing a Source credential option or a KARU_CREDENTIALS_SOURCE callback enables signed requests automatically. SOURCE_NO_SIGN_REQUEST=YES always wins; SOURCE_NO_SIGN_REQUEST=NO requests credentials explicitly and uses the source-coop profile when no profile is named.

The profile created for the Source CLI uses credential_process:

[profile source-coop]
credential_process = source-coop creds
endpoint_url = https://data.source.coop

Run source-coop login, then set SOURCE_PROFILE=source-coop in Karu. Karu executes the profile's credential process and refreshes its temporary result when needed. The process is terminated if it exceeds KARU_REQUEST_TIMEOUT; Karu does not implement the browser login or read the CLI keyring. The profile's endpoint_url is intentionally ignored because SOURCE_ENDPOINT owns endpoint selection for /vsisource/ paths.

Google Cloud Storage

ADC supports service_account, authorized_user, and OIDC external_account sources backed by a file or a simple URL. Service-account impersonation is supported. Environment-specific external-account suppliers can use the callback API.

With environment discovery enabled, metadata credentials are attempted after ADC files. An explicit GCS_METADATA_ENDPOINT also enables that source for an otherwise empty configuration. GCS_METADATA_DISABLED=YES always disables it.

Azure Storage

Connection-string DefaultEndpointsProtocol, EndpointSuffix, BlobEndpoint, and DfsEndpoint values are honored. AZURE_AUTHORITY_HOST, AZURE_STORAGE_SCOPE, and AZURE_STORAGE_RESOURCE keep identity and storage endpoints explicit for sovereign clouds. AZURE_STORAGE_ENDPOINT is a convenience option; the standard endpoint fields in AZURE_STORAGE_CONNECTION_STRING are also supported.

HTTP and Hugging Face

KARU_HTTP_HEADERS accepts newline-separated Name: value entries. libcurl does not forward authorization to a different redirect origin. HF_ENDPOINT changes the Hugging Face service root.

With environment discovery enabled, Karu follows the Hugging Face CLI's default token location: $HF_TOKEN_PATH, then $HF_HOME/token, then ${XDG_CACHE_HOME:-~/.cache}/huggingface/token. A missing token file means anonymous access. karu_config_create_empty() does not inspect a default cache, but explicitly setting HF_TOKEN_PATH or HF_HOME opts into that one file.

The token file is read while each request attempt is materialized. Logging in or rotating the file therefore affects the next operation without putting a credential or cache inside an object. Karu trims surrounding whitespace and reports an existing but unreadable token file as a credential error.

Range and Host are always owned by Karu. Cloud and Hugging Face requests also reserve authentication and provider-signature headers. Use the dedicated credential options or callback instead of injecting those headers through KARU_HTTP_HEADERS.

Object transfers and their redirects are restricted to HTTP and HTTPS.

On Linux and macOS, Karu discovers a system CA bundle at runtime when no CA option is set. This avoids relying solely on the build machine's certificate path.

With OpenSSL and no CA option set, Karu prefers libcurl's hashed CA directory, as on Debian and Ubuntu. Only certificates in that directory are trusted. Otherwise, libcurl 7.87+ uses the available bundle without its default directory so it can cache the certificates. Set KARU_HTTP_CA_PATH to include a directory; other TLS backends keep their defaults.

The bundle cache lasts up to 24 hours. Bundle updates may not take effect until it expires.

Custom providers

karu_config_set_credentials_provider() installs one callback for AWS, GCS, Azure, or Source Cooperative. The callback receives the canonical VSI path. It returns strings that Karu copies immediately, an optional Unix expiry, and an optional canonical cache_prefix.

Use a prefix only when the credential is valid for that path and every object beneath it. For example, /vsis3/team-bucket/ shares one renewable value across that bucket. Without a prefix Karu calls the provider for every request, which is appropriate when the provider or vendor SDK owns its own cache.

Source uses KARU_CREDENTIALS_SOURCE and the same access-key, secret, and session-token fields as AWS, but its cache namespace is separate. A Source prefix therefore starts with /vsisource/, not /vsis3/.

Karu coalesces simultaneous callback refreshes for the same path. Refresh is lazy: the first request inside the refresh window obtains a new value while later requests for that same path wait for it. There is no background thread. Callbacks for different paths may run concurrently.

The callback must be thread-safe and must not re-enter that same client. Karu cannot interrupt caller code, so the callback must place its own deadline on any I/O.