Review Azure Machine Learning capacity floors
On this page
Unneeded minimum capacity can keep costs running, so ask owners to justify deployment and training-cluster minimums during low demand. Flag unexplained floors for review, not automatic removal. Low CPU alone is not enough reason to reduce capacity.
What you need first
- Tool
- Use the Azure portal, Azure's browser-based management tool. Start at your subscription's Cost Analysis page for endpoint costs. Use the read-only checklist below with configuration and metric results from an authorized Azure Machine Learning colleague.
- Access
- For endpoint costs, have at least Billing Reader access on the endpoint's subscription. Ask an authorized Azure Machine Learning colleague to provide read-only configuration and metric results for the selected resources.
- If you do not use that tool
- Send your Azure Machine Learning owner the subscription, workspace, endpoint, deployment and cluster names, plus the review period. Request minimums, autoscale rules, metrics and job activity. Ask a subscription billing colleague for endpoint costs filtered by workspace and both endpoint tags.
Why this is worth a look
Autoscaling cannot release capacity below its configured minimum. Managed online deployments scale using metric and schedule rules. AmlCompute, Azure Machine Learning's managed training clusters, releases nodes as jobs finish but keeps its minimum running even without jobs. A cluster minimum of zero allows unused nodes to be deallocated to avoid charges.
Run this check
CHECKLISTRun these steps manually in Azure portal Cost Analysis and against owner-provided configuration and metric results. Review one named workspace and a recorded period; do not save configuration changes.
[ ] Record the subscription, workspace, managed online endpoint and deployment names, and AmlCompute training-cluster names in scope.
[ ] Choose and record a review period that includes normal peaks and scheduled work. Ask the owner to confirm that it is representative.
[ ] Use at least Billing Reader access on the subscription for endpoint costs. Obtain configuration, metric and job-activity results from an authorized Azure Machine Learning colleague.
Managed online deployments
[ ] Record whether autoscale is enabled. For enabled profiles, record minimum, maximum and default capacity in VM instances, plus applicable metric and schedule rules.
[ ] Review CpuUtilizationPercentage (percent) and RequestLatency (milliseconds) where available over the recorded period. Ask the owner which other available metrics matter for this deployment.
[ ] Compare retained instance counts with the applicable minimum and the owner's workload requirements. Mark unexplained minimum capacity for review, not deletion.
AmlCompute training clusters
[ ] Record minimum and maximum node counts, idle time before scale-down in seconds, and job-driven scaling during the review period.
[ ] For a nonzero minimum, ask whether work requires nodes to remain running after jobs finish. Record the reason or mark the floor for owner review.
Endpoint cost evidence
[ ] In the Azure portal, select Cost Analysis for the subscription. Select Add filter, choose Resource, then select the Azure Machine Learning workspace.
[ ] Add Tag filters azuremlendpoint: <endpoint name> and azuremldeployment: <deployment name>.
[ ] Record the cost period and displayed cost with its currency. Keep these endpoint costs separate from the training-cluster review.
Decision
[ ] Record each resource's owner, minimum, reason for retaining it, and whether a setting review is needed. Make no resource changes.How to confirm it
- 01
Choose a representative period
Select one workspace and name the deployments and training clusters to review. Ask owners to include normal peaks and scheduled work in the period so a quiet interval does not become the basis for a capacity cut.
- 02
Separate required capacity from unexplained floors
Compare deployment minimums with metric and schedule rules. For training clusters, check whether a nonzero minimum remains necessary after jobs finish. Record an owner explanation for each retained floor.
- 03
Prioritize with endpoint costs
Use the checklist's Cost Analysis filters to isolate each managed online deployment's costs. Record the cost period alongside the workload period before using the result to prioritize reviews.
- 04
Request a setting review, not a deletion
Send unexplained floors to the resource owner for approval before any change. For a training cluster with no need to retain unused nodes, ask the owner to consider a minimum of zero. Do not apply that recommendation to online deployments.
Before making changes
Treat a quiet period as representative only after the owner confirms it covers normal peaks and scheduled work. Low CPU or latency alone does not prove capacity is unnecessary. This review assumes enabled autoscale profiles for deployment-floor comparisons; disabled autoscale needs a separate configuration review. Endpoint cost tags do not establish training-cluster costs, and displayed endpoint cost is not a savings estimate.