Baseten 推出自动扩缩容计划，提前扩容降低冷启动延迟

TL;DR

Autoscaling schedules reduce cold-start delays by raising the minimum number of replicas before predictable traffic arrives. You can set a schedule to recur (e.g., weekday mornings) or to run once (e.g., ahead of a product launch). Configure up to 10 schedules per environment through the dashboard, CLI, or Management API.

Autoscaling adjusts the number of replicas to match incoming traffic. When demand increases, new replicas need time before they can handle requests. This is their cold-start time, which includes allocating compute resources, starting the serving software, and loading the model. Requests that wait during a cold start have longer end-to-end response times.⁠

Autoscaling schedules help avoid cold-start delays by starting additional replicas before traffic arrives. You can set a schedule to recur (e.g., every weekday morning) or to run once (e.g., ahead of a launch).

Schedule autoscaling around demand

Many workloads have predictable traffic. Autoscaling schedules let you change your autoscaling settings at set times, so you don’t need to maintain an external cron job to do it.

Recurring peaks: Raise the minimum number of replicas during work hours and lower it overnight. For example, a meeting-transcription app can scale up before meetings begin and back down after the peak.
Planned events: Schedule a temporary capacity increase for product launches, load tests, or migrations to a new deployment.

Illustrative example: starting additional replicas before an expected traffic surge can help keep end-to-end response times consistent as request volume increases.

How autoscaling schedules work

Each schedule sets a time window and the minimum and maximum number of replicas for that window. The minimum sets the lowest number of replicas the autoscaler keeps running during the window. The maximum limits the total number of replicas the autoscaler can run. When no schedule is active, the environment uses its default settings.

To configure a schedule:

Create up to 10 schedules per environment, all using one shared timezone.
Choose a daily, hourly, or one-time window. Windows can’t overlap and must last at least five minutes.
Use the dashboard, CLI, or Management API.

Two autoscaling schedules for a production environment

Get started

Start replicas before your expected traffic arrives to reduce cold-start delays. Follow the setup guide to configure schedules through the dashboard, CLI, or Management API, or talk with our engineers about your workload.