> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Scaling and storage
> Size the API and Brainstore compute in a self-hosted data plane with Terraform variables and Helm resource settings.
These settings size the API and Brainstore compute for your workload. The defaults are suitable for most deployments. Tune them for high ingestion volume, heavy eval workloads, or cluster capacity constraints.
## Configure AWS API ECS services
On AWS, Terraform module v6.0 introduces ECS as the target runtime for the API, split into three services by workload type. Each service scales automatically through Application Auto Scaling, adjusting its running task count between a minimum and maximum based on multiple metrics, including CPU target tracking and event loop timing. The module exposes optional variables to tune per-task size and task counts. The defaults are suitable for most deployments. See [Upgrade to Terraform module v6.0](/docs/admin/self-hosting/upgrade/v6) for the migration.
1024 CPU units equal 1 vCPU.
**`braintrust-api`** handles general API traffic.
| Variable | Default |
| -------------------------- | ------- |
| `braintrust_api_cpu` | 1024 |
| `braintrust_api_memory` | 8192 |
| `braintrust_api_min_count` | 3 |
| `braintrust_api_max_count` | 50 |
**`braintrust-api-ingest`** handles ingestion paths (`/logs3`, `/otel/v1/traces`).
| Variable | Default |
| --------------------------------- | ------- |
| `braintrust_api_ingest_cpu` | 1024 |
| `braintrust_api_ingest_memory` | 8192 |
| `braintrust_api_ingest_min_count` | 6 |
| `braintrust_api_ingest_max_count` | 200 |
**`braintrust-api-background`** handles background paths (evals, function invocation, proxy).
| Variable | Default |
| ------------------------------------- | ------- |
| `braintrust_api_background_cpu` | 1024 |
| `braintrust_api_background_memory` | 8192 |
| `braintrust_api_background_min_count` | 3 |
| `braintrust_api_background_max_count` | 50 |
For example, to raise the ingestion service's floor for a high-volume deployment:
```hcl theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
braintrust_api_ingest_cpu = 2048
braintrust_api_ingest_min_count = 12
```
Raise `braintrust_api_ingest_min_count` for deployments with spiky ingestion volume or if you expect a launch that will suddenly increase log traffic, and `braintrust_api_background_min_count` for heavy eval workloads.
## Configure Helm API workload isolation
This feature is available to Kubernetes deployments using Helm chart 6.13.0+. AWS ECS deployments use the Terraform-managed routing described above.
Setting `api.workloadIsolation.enabled: true` creates dedicated `braintrust-api-ingest` and `braintrust-api-background` Deployments and Services alongside the default `braintrust-api` pool. The pools share the same image and base configuration while allowing independent replica counts, resources, probes, rollout settings, and disruption budgets, so classified ingestion or background load does not consume the default API pool's capacity.
Your ingress or gateway must keep `braintrust-api` as its default backend and route the following paths to the specialized pools:
| Pool | Paths |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `braintrust-api-ingest` | `POST /logs3`, `POST /otel/v1/traces`, `POST /attachment`, `POST /attachment/status` |
| `braintrust-api-background` | `POST /v1/eval` (exact) and `POST /v1/eval/` (prefix), `POST /function/eval`, `POST /function/sandbox`, `POST /function/use`, `POST /function/invoke-async-batch`, `POST /function/insert-functions`, `POST /automation/logs/trigger`, and all methods for `/v1/proxy/chat/completions` and `/v1/proxy/responses` |
| `braintrust-api` (default) | All requests not matched by an ingest or background route |
If your ingress cannot match requests by HTTP method (for example, GKE Ingress), route the listed paths for all methods instead.
Workload isolation uses fixed capacity and does not enable autoscaling. Configure replica counts with `api.replicas`, `api.workloadIsolation.ingest.replicas`, and `api.workloadIsolation.background.replicas`.
### Roll out workload isolation for an existing deployment
For an existing deployment, stage the rollout so you can verify each pool before routing traffic to it:
1. Create the pools without routing traffic to them or switching Brainstore's internal AI proxy. Set `api.workloadIsolation.brainstoreAiProxyToBackground: false` and apply:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
api:
workloadIsolation:
enabled: true
brainstoreAiProxyToBackground: false
```
2. Verify the ingest and background pools are ready. Then update your ingress or gateway to route the classified paths to the new Services, and set `brainstoreAiProxyToBackground: true` to switch Brainstore's internal AI proxy to the background pool:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
api:
workloadIsolation:
enabled: true
brainstoreAiProxyToBackground: true
```
To roll back, first return the classified paths and `brainstoreAiProxyToBackground` to the default API Service and verify it is serving them, then disable workload isolation in the chart. The chart cannot update an external ingress or gateway on its own, so disabling isolation before rerouting would leave the classified paths pointed at Services that no longer exist.
### Integrate with the Istio VirtualService
When using the chart-managed Istio VirtualService, set `virtualService.workloadIsolation.enabled: true` after the isolated pools are healthy. The chart then renders the ingest and background route contract before your existing `virtualService.http` rules. The classified routes take precedence, so do not use `virtualService.http` to override a classified path.
Enable this only after the ingest and background pools are running and ready.
This option requires both `virtualService.enabled: true` and `api.workloadIsolation.enabled: true`. The chart fails to render if either is missing.
```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
virtualService:
workloadIsolation:
enabled: true
```
### Configuration reference
| Key | Default | Description |
| ------------------------------------------------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `api.workloadIsolation.enabled` | `false` | Create dedicated ingest and background API pools. |
| `api.workloadIsolation.brainstoreAiProxyToBackground` | `true` | Route Brainstore's internal AI proxy to the background pool. Set to `false` while staging pools for an existing deployment. |
| `api.workloadIsolation.ingest.name` | `braintrust-api-ingest` | Name of the ingest Deployment. |
| `api.workloadIsolation.ingest.replicas` | `3` | Number of ingest pod replicas. |
| `api.workloadIsolation.ingest.service.name` | `braintrust-api-ingest` | Name of the ingest Service. |
| `api.workloadIsolation.ingest.podDisruptionBudget.enabled` | `true` | Enable a PodDisruptionBudget for the ingest pool. |
| `api.workloadIsolation.ingest.podDisruptionBudget.minAvailable` | `1` | Minimum available ingest pods during voluntary disruptions. |
| `api.workloadIsolation.ingest.topologySpread.enabled` | `true` | Enable topology spread constraints for the ingest pool. |
| `api.workloadIsolation.background.name` | `braintrust-api-background` | Name of the background Deployment. |
| `api.workloadIsolation.background.replicas` | `3` | Number of background pod replicas. |
| `api.workloadIsolation.background.service.name` | `braintrust-api-background` | Name of the background Service. |
| `api.workloadIsolation.background.podDisruptionBudget.enabled` | `true` | Enable a PodDisruptionBudget for the background pool. |
| `api.workloadIsolation.background.podDisruptionBudget.minAvailable` | `1` | Minimum available background pods during voluntary disruptions. |
| `api.workloadIsolation.background.topologySpread.enabled` | `true` | Enable topology spread constraints for the background pool. |
| `virtualService.workloadIsolation.enabled` | `false` | Render the ingest and background route contract in the chart-managed VirtualService. Requires `virtualService.enabled: true` and `api.workloadIsolation.enabled: true`. |
### Default pool availability settings
Helm chart 6.13.0 or later also exposes availability settings for the default `braintrust-api` pool. The defaults preserve the chart's previous behavior, so no configuration changes are required when upgrading. The ingest and background pools inherit these values as their base configuration.
| Key | Default | Description |
| ------------------------------------------- | ----------------------------- | -------------------------------------------------------------------- |
| `api.strategy.type` | `RollingUpdate` | Rollout strategy for the API Deployment. |
| `api.strategy.rollingUpdate.maxSurge` | `100%` | Maximum number of extra pods created during a rollout. |
| `api.strategy.rollingUpdate.maxUnavailable` | `0` | Maximum number of pods that can be unavailable during a rollout. |
| `api.podDisruptionBudget.enabled` | `false` | Enable a PodDisruptionBudget for the default API pool. |
| `api.podDisruptionBudget.minAvailable` | `1` | Minimum available default-pool pods during voluntary disruptions. |
| `api.topologySpread.enabled` | `false` | Enable topology spread constraints for the default API pool. |
| `api.topologySpread.maxSkew` | `1` | Maximum pod-count skew across topology domains. |
| `api.topologySpread.topologyKey` | `topology.kubernetes.io/zone` | Node label that defines the topology domain. |
| `api.topologySpread.whenUnsatisfiable` | `ScheduleAnyway` | Whether the scheduler places pods when the constraint cannot be met. |
## Configure Brainstore fast readers
Fast readers are isolated Brainstore nodes dedicated to serving predictable UI queries (paginated viewers, span and trace lookups), preventing resource-intensive ad-hoc queries from making the UI unresponsive.
* **GCP and Azure**: Fast readers are enabled by default starting in Helm chart v5.0.0. See the configuration reference below.
* **AWS**: Fast readers are enabled by default (2 nodes) starting in Terraform module v5.5.0. On earlier module versions they are disabled by default. Set `brainstore_fast_reader_instance_count` in your Terraform configuration to control the node count, or set it to `0` to opt out (recommended for sandbox or non-production deployments).
Upgrading to Helm chart v5.0.0 from an earlier version automatically creates fast reader nodes. By default, 2 fast reader nodes are created with the same resource profile as standard reader nodes (CPU: 16, memory: 32Gi). Verify that your cluster has capacity for these additional nodes before upgrading.
If you have custom `brainstore.readinessProbe` overrides pointing to `/status`, remove them before upgrading to Helm chart v5.0.0+. The `/status` readiness endpoint has a bug where it never recovers after a failure, which can permanently mark Brainstore nodes as not ready. Remove any `brainstore.readinessProbe` or `brainstore.fastreader.readinessProbe` customizations and rely on the chart defaults.
### Configuration reference
Fast readers are configured under the `brainstore.fastreader` key in your `values.yaml`. If you have customized `brainstore.reader` settings, mirror those customizations to `brainstore.fastreader`.
| Key | Default | Description |
| --------------------------------------------------- | ----------------------- | ---------------------------------------------- |
| `brainstore.fastreader.name` | `brainstore-fastreader` | Name of the Deployment, Service, and ConfigMap |
| `brainstore.fastreader.replicas` | `2` | Number of fast reader pod replicas |
| `brainstore.fastreader.service.port` | `4000` | Service port |
| `brainstore.fastreader.service.type` | `ClusterIP` | Kubernetes service type |
| `brainstore.fastreader.resources.requests.cpu` | `16` | CPU request |
| `brainstore.fastreader.resources.requests.memory` | `32Gi` | Memory request |
| `brainstore.fastreader.resources.limits.cpu` | `16` | CPU limit |
| `brainstore.fastreader.resources.limits.memory` | `32Gi` | Memory limit |
| `brainstore.fastreader.objectStoreCacheMemoryLimit` | `1Gi` | Object store memory cache limit |
| `brainstore.fastreader.objectStoreCacheFileSize` | `1000Gi` | Object store file cache size |
| `brainstore.fastreader.cacheDir` | `/mnt/tmp/brainstore` | Local cache mount path |
| `brainstore.fastreader.volume.size` | `""` | Ephemeral storage size; required for Azure ACS |
| `brainstore.fastreader.extraEnvVars` | `[]` | Additional environment variables |
| `brainstore.fastreader.nodeSelector` | `{}` | Node selector for scheduling |
| `brainstore.fastreader.tolerations` | `[]` | Pod tolerations |
| `brainstore.fastreader.affinity` | `{}` | Pod affinity rules |
Azure users must explicitly set `brainstore.fastreader.volume.size` when using Azure Container Storage (`enableAzureContainerStorageDriver: true`):
```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
brainstore:
fastreader:
volume:
size: "100Gi"
```
On GKE Autopilot, set `brainstore.fastreader.volume.size` to configure ephemeral storage requests. Match the resource profile of your standard reader nodes:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
brainstore:
fastreader:
resources:
requests:
cpu: "16"
memory: "32Gi"
limits:
cpu: "16"
memory: "32Gi"
objectStoreCacheFileSize: "900Gi"
volume:
size: "1000Gi"
```
## Brainstore resource configuration
This section applies to GCP and Azure deployments using the Helm chart (v5.0.1+). AWS deployments manage Brainstore resources automatically.
Starting in Helm chart v5.0.1, the `resources` block for each Brainstore component (`brainstore.reader`, `brainstore.writer`, `brainstore.fastreader`) is passed through as-is to the Kubernetes pod spec. You can omit `limits` entirely, set them to `{}`, or supply any valid Kubernetes resource spec.
Omitting `limits` sets the pod QoS class to Burstable, which prevents CPU throttling and allows pods to use available node capacity. This can improve query performance on nodes with spare capacity, but increases the risk of resource contention if multiple pods compete for the same node.
```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# With limits (Guaranteed QoS — default)
brainstore:
reader:
resources:
requests:
cpu: "16"
memory: "32Gi"
limits:
cpu: "16"
memory: "32Gi"
# Without limits (Burstable QoS — allows burst CPU/memory)
brainstore:
reader:
resources:
requests:
cpu: "16"
memory: "32Gi"
writer:
resources:
requests:
cpu: "4"
memory: "16Gi"
fastreader:
resources:
requests:
cpu: "16"
memory: "32Gi"
```
### Auto-derived Brainstore environment variables
As of Helm chart v5.1.0, `BRAINSTORE_RESPONSE_CACHE_URI` and `BRAINSTORE_CODE_BUNDLE_URI` are automatically populated from your `objectStorage` configuration and do not need to be set manually. The chart derives these values as follows:
| Cloud | Source key | `BRAINSTORE_RESPONSE_CACHE_URI` | `BRAINSTORE_CODE_BUNDLE_URI` |
| ----- | ----------------------------------------------------------------------------------- | ------------------------------------------- | ---------------------------- |
| AWS | `objectStorage.aws.responseBucket` / `objectStorage.aws.codeBundleBucket` | `s3:///brainstore-cache` | `s3://` |
| Azure | `objectStorage.azure.responseContainer` / `objectStorage.azure.codeBundleContainer` | `az:///brainstore-cache` | `az://` |
| GCP | `objectStorage.google.apiBucket` | `gs:///brainstore-cache` | `gs://` |
If you previously configured these via `extraEnvVars`, remove those overrides after upgrading to v5.1.0 to avoid conflicts.