Storage Buckets
Storage Buckets are project-scoped persistence containers for workflow outputs.
They are designed for three main storage patterns:
- media artifacts (images/video)
- metadata key/value state
- time-indexed measurements and events
Buckets make workflow output reproducible, auditable, and available for downstream analytics.
Why Buckets Matter
Without persistent storage, operational workflows lose traceability.
Buckets enable:
- post-incident replay and evidence retention
- stateful automation across executions
- historical trend analysis and reporting
Storage Modes
Media Storage
Use for frame captures, event snapshots, clips, and generated artifacts.
Metadata Storage
Use for structured key/value state, run context, configuration snapshots.
Timeseries Storage
Use for metrics, counters, confidence scores, and trend signals over time.
Bucket Lifecycle
- Create bucket per domain or workflow family
- Define naming and collection strategy
- Write from workflow nodes
- Read from workflows, dashboards, or APIs
- Apply retention and cleanup process
Design Guidance
- Keep naming explicit (
line3-defect-media,site-a-telemetry) - Separate high-volume media from analytical timeseries buckets
- Treat metadata as controlled state, not unbounded logs
- Use collections/folders for predictable retrieval patterns
Operational Best Practices
- Store critical outputs at key decision nodes
- Normalize metadata keys for consistency
- Limit write amplification in hot workflows
- Define retention policy by data type and compliance needs
- Periodically verify read-path performance for analytics workloads
Security and Access
- Buckets inherit project scope and access boundaries
- Apply least-privilege access patterns to operations tooling
- Avoid storing secrets in plain metadata
- Audit write/read/delete paths for sensitive workflows
Bucket Reference Pages
- Bucket Media Storage
- Bucket Metadata Storage
- Bucket Timeseries Storage
- Bucket Operations and Retention