Google CloudData & storage3 min read

Separate job trouble from Dataflow cost

Sources checked September 10, 202630 minutes
On this page

This check is due for a source refresh. Confirm the current documentation before you rely on provider-specific details.

THE SHORT ANSWER

Compare Dataflow estimates with Cloud Billing actuals before choosing cost-reduction work, or you may rank the wrong jobs because estimates can differ from billed cost. Use errors, delays, and scaling patterns for engineering follow-up, not as proof of spend.

What you need first

Tool
Use the Google Cloud console, Google's web interface. Start at Dataflow > Jobs, select a job, then open Cost. For Spark, use Cloud Monitoring, Google's metrics service: in Metrics Explorer, filter by "dataproc" and select a "Cloud Managed Service for Apache Spark" resource.
Access
Ask authorized service and billing colleagues for read-only results for the selected project, jobs, Spark resources, and billing account. Have each colleague cover their own service or billing scope.
If you do not use that tool
Send a Dataflow or Spark owner and a billing colleague the project, region, resource identifiers, billing account, and review period. Request Dataflow Cost and Autoscaling results, actual Dataflow costs from Cloud Billing reports, and relevant Spark metrics or autoscaler logs.

Why this is worth a look

You can send engineering effort to the wrong workload if you treat estimates or performance problems as proof of spend. Dataflow estimates multiply resource-use metrics by regional resource prices, while contractual discounts and temporary billing adjustments can change actual cost. Spark metrics and autoscaler logs diagnose performance, not billed-cost ranking.

Run this check

CHECKLIST

Follow these steps in the Google Cloud console, or ask authorized colleagues to return the results. Record cost and performance separately for the same resources and review period. Make no changes.

Read-only cost and performance review
READ-ONLY COST AND PERFORMANCE REVIEW

Execution surfaces:
- Google Cloud console: Dataflow job views and Cloud Billing reports.
- Cloud Monitoring: metrics service, using Metrics Explorer for Spark.
- Cloud Logging: log service, for Spark autoscaler logs.
Access: ask authorized service and billing colleagues for read-only results for their respective scopes if you cannot view them.
Scope: record project, Dataflow region and job IDs, relevant Spark cluster, job, batch, or session IDs, and billing account.
Time window: record start, end, and time zone. Mark results with different scopes or periods as unmatched.
Units: keep displayed cost currency and resource units. Worker values are counts. CPU utilization is CPU used divided by CPU available. Streaming Engine backlog time is in seconds.

[ ] In Dataflow > Jobs, select a job and open Cost. Select the review period. Record the estimated total and contributions from vCPUs, memory, Dataflow Shuffle or Streaming Engine data processed, and SSD or HDD usage.
[ ] Obtain actual Dataflow costs from Cloud Billing reports for the billing account. Record the available scope and period beside the estimate. Leave the cost ranking unresolved if they cannot be matched.
[ ] Review job errors, stage progress, and delays. For streaming jobs, also review data freshness and the Autoscaling tab. Record worker counts and mean CPU utilization over the review period. For Streaming Engine jobs, record backlog seconds and autoscaling rationale.
[ ] For Managed Service for Apache Spark, open Metrics Explorer. Type "dataproc" in "Filter by resource or metric name" and select the relevant "Cloud Managed Service for Apache Spark" resource. Record metric names, units, period, and gaps.
[ ] For an autoscaled Spark cluster, ask its owner for autoscaler logs from Cloud Logging covering the same period. Request scaling recommendations, metrics used in decisions, original cluster size, and new target size.
[ ] Record two separate outcomes: a cost question for the billing colleague and a performance question for the workload owner. Attach the relevant results. Do not change jobs, clusters, policies, or alerts.

How to confirm it

  1. 01

    Choose a comparable scope

    Choose the jobs or Spark resources and one review period. Give service and billing colleagues the same identifiers and time zone. Do not compare a single-job estimate with a broader billing total.

  2. 02

    Confirm the cost question

    Compare Dataflow Cost results with Cloud Billing reports before ranking jobs. Use resource contributions to identify what drives the estimate, but leave actual-cost conclusions open when billing scope or time differs.

  3. 03

    Assign performance follow-up

    Send errors, delays, and scaling patterns to the workload owner. For Dataflow streaming jobs, compare worker changes with CPU utilization and backlog where available. For Spark, use metrics and autoscaler logs to frame the next diagnostic question.

  4. 04

    Review trade-offs before changes

    Ask the workload owner to assess both latency and cost before changing Dataflow autoscaling. A lower worker utilization hint can improve scale-up speed but increase total cost if more workers are provisioned.

Before making changes

Assume the views cover comparable resources and periods. Treat missing metrics as gaps, not zero usage. Dataflow Cost does not support Prime jobs or include additional GPU accelerator cost. It approximates TPU accelerator cost with vCPU and memory costs even though TPU workers do not incur those charges. An empty backlog chart without Streaming Engine does not mean no backlog. Spark custom metrics depend on enabled sources and overrides, and retention limits historical review.

Skip this review if you have neither Dataflow nor Managed Service for Apache Spark workloads. Also skip it if you need only Spark billed-cost attribution, because the Spark steps diagnose performance rather than billed cost.

Primary sources