Real Production Scenarios
Real-world architecture, system migration, and design challenges.
How does GCP VPC networking work and what are Shared VPC and VPC peering?
Google Cloud VPC (Virtual Private Cloud) is a global, private network that provides connectivity for GCP resources. Unlike AWS VPCs which are regional, GCP VPCs are global by default with subnets in specific regions.
GCP VPC Key Characteristics
Global VPC: A single VPC spans all GCP regions. Resources in the same VPC can communicate across regions using internal IPs without extra configuration.
Subnets: Regional resources with a defined CIDR range. Two modes exist: auto mode auto-creates subnets in each region, custom mode gives full control over all subnets.
Firewall Rules: Applied at the VPC level using tags or service accounts to target instances. Rules are stateful. Unlike AWS, there are no network ACLs – all filtering is done through firewall rules.
Shared VPC
Shared VPC allows a host project to share its VPC network with service projects. Multiple projects share the same networking while keeping workloads isolated per project. The host project owns and manages the VPC, subnets, and firewall rules while service projects deploy resources into the shared subnets.
Use Shared VPC for centralized network administration, consistent firewall policy enforcement, and simplifying inter-project connectivity within an organization.
VPC Peering
VPC Peering connects two VPCs so resources can communicate using internal IPs without routing through the public internet. Peering works across projects and organizations. Peering is non-transitive: if VPC A peers with B and B peers with C, A cannot reach C through B.
Use VPC Peering for connecting VPCs in different projects or organizations, sharing services privately, and achieving lower latency compared to external routing.
What is BigQuery and how does it differ from traditional relational databases?
BigQuery is Google Cloud’s fully managed, serverless data warehouse designed for large-scale analytics. It can query petabytes of data in seconds using SQL without requiring infrastructure management.
Key Architecture Differences
BigQuery uses columnar storage (Capacitor format) which makes analytical queries fast by reading only relevant columns rather than entire rows. Traditional RDBMS use row-based storage optimized for transactional workloads with frequent single-record reads and writes.
BigQuery separates compute and storage, allowing each to scale independently. Traditional databases tightly couple compute and storage on the same server.
BigQuery uses a distributed query engine (Dremel) that automatically parallelizes queries across thousands of nodes. Traditional databases are typically single-node or manually sharded.
BigQuery vs Traditional Databases
BigQuery excels at OLAP workloads: aggregations, joins across billions of rows, analytics dashboards. Traditional RDBMS (PostgreSQL, MySQL) excel at OLTP: fast single-row inserts/updates with ACID transactional guarantees.
BigQuery requires no indexes, vacuuming, or schema optimization. Pricing is per-query (bytes scanned) or flat-rate. Traditional databases require DBA management, index tuning, and ongoing optimization.
Key BigQuery Features
Partitioning by date, range, or ingestion time reduces scan costs. Clustering on filtered columns improves query performance. BigQuery ML runs machine learning models using SQL. Streaming inserts via Storage Write API. Native integration with Dataflow, Pub/Sub, Looker, and Data Studio.
What are the different storage classes in Google Cloud Storage and when should you use each?
Google Cloud Storage (GCS) is a unified object storage service for unstructured data. It offers four storage classes optimized for different access patterns and cost requirements.
Standard Storage: Best for frequently accessed or hot data. No minimum storage duration. Highest storage cost, lowest access cost. Use for active website content, mobile apps, gaming assets, and data analytics requiring low latency.
Nearline Storage: Best for data accessed less than once per month. 30-day minimum storage duration. Lower storage cost than Standard with a small retrieval fee. Use for backups, long-tail multimedia content, and monthly-accessed archives.
Coldline Storage: Best for data accessed less than once per quarter. 90-day minimum storage duration. Very low storage cost with higher retrieval fee. Use for disaster recovery, compliance archives, and infrequently accessed backups.
Archive Storage: Best for data accessed less than once per year. 365-day minimum storage duration. Lowest storage cost with highest retrieval fee. Use for long-term preservation, regulatory compliance data, and cold archival storage.
Key GCS Features
All storage classes share the same API. Object versioning supports recovery from accidental deletions. Lifecycle policies automate transitions between storage classes. Strong consistency for all read/write operations. Object Lock and retention policies for compliance. Signed URLs provide time-limited access without authentication.
What is Google Cloud Pub/Sub and how does it differ from traditional message queues?
Google Cloud Pub/Sub is a fully managed, real-time messaging service that enables asynchronous communication between independent applications at scale. It follows the publish-subscribe pattern where publishers send messages to topics and subscribers receive messages from subscriptions.
Core Concepts
Topic: A named resource to which publishers send messages. Subscription: A named resource representing the stream of messages from a single, specific topic. Publisher: Application that creates and sends messages to a topic. Subscriber: Application that receives messages from a subscription.
Delivery Models
Pull delivery: Subscriber explicitly calls an API to retrieve messages. Suitable for batch processing and when the subscriber controls the rate. Push delivery: Pub/Sub sends messages to a webhook endpoint. Suitable for real-time processing and serverless architectures.
How Pub/Sub Differs from Traditional Message Queues
Traditional queues like RabbitMQ or SQS use a point-to-point model where each message is consumed by one consumer. Pub/Sub supports fan-out natively – one message can be delivered to multiple subscriptions simultaneously.
Pub/Sub is fully serverless and scales automatically to millions of messages per second. It integrates natively with Dataflow, BigQuery, Cloud Storage, and Cloud Functions for streaming pipelines.
Key Use Cases
Event-driven microservices decoupling. Stream analytics with Dataflow. Log aggregation and metric collection. IoT data ingestion from millions of devices. Triggering Cloud Functions or Cloud Run services on events.
How does GCP IAM work and what is the difference between service accounts and user accounts?
GCP IAM (Identity and Access Management) controls who can do what on which GCP resources. It uses a policy-based model with three main components: principals (who), roles (what permissions), and resources (which resources).
Key IAM Concepts
Principals: Google accounts, service accounts, Google groups, Google Workspace domains, or Cloud Identity domains.
Roles: Collections of permissions. Three types exist — Basic roles (Owner, Editor, Viewer), Predefined roles (fine-grained, service-specific), and Custom roles (user-defined).
Policies: Bindings that attach roles to principals on a resource.
Service Accounts vs User Accounts
User Accounts represent a human user (developer, admin). They authenticate with passwords and OAuth2, are managed in Google/Cloud Identity, and are used for interactive access like gcloud CLI or Console.
Service Accounts represent an application or workload (non-human). They authenticate using cryptographic keys or workload identity, are managed in GCP per project, and are used by VMs, Cloud Functions, GKE pods, etc.
Key Differences
Service accounts have no password — they use RSA key pairs or metadata server tokens. Service accounts can be impersonated by other principals (act as). GKE Workload Identity links Kubernetes service accounts to GCP service accounts, eliminating key files. Service account keys should be rotated regularly and avoided when possible — prefer Workload Identity or ADC (Application Default Credentials).
Best Practices
Apply principle of least privilege — grant minimum required permissions. Use predefined roles over basic roles. Avoid using Editor/Owner roles in production. Use Workload Identity for GKE workloads instead of key files. Audit IAM policies with Cloud Asset Inventory and IAM Recommender.
What is Google Cloud Run and when should you use it instead of GKE?
Google Cloud Run is a fully managed serverless container platform that automatically scales containerized workloads, including to zero when not in use.
What is Cloud Run?
Cloud Run runs any stateless container that listens on HTTP. You bring a Docker image, and Cloud Run handles all infrastructure – load balancing, scaling, SSL, and billing.
Key Characteristics
- Serverless: No infrastructure management, scales to zero
- Pay-per-use: Billed per request + CPU/memory during request processing
- Knative-based: Built on open Knative standards
- Any language/framework: Works with any Docker container
Cloud Run vs GKE
| Aspect | Cloud Run | GKE |
|---|---|---|
| Infrastructure | Fully managed | Partially managed |
| Scaling | Automatic (0 to N) | Manual/HPA |
| Cost model | Per request | Per node hour |
| Startup time | Cold starts ~1-2s | N/A (pods warm) |
| Stateful workloads | No | Yes |
| Custom networking | Limited | Full control |
| Persistent storage | No (use GCS/CloudSQL) | Yes (PV/PVC) |
| Long-running jobs | Limited (timeout) | Yes |
When to Use Cloud Run
- API backends and microservices: HTTP APIs that can be stateless
- Event-driven workloads: Triggered by Pub/Sub, Cloud Scheduler, Eventarc
- Batch processing: Short-lived tasks from message queues
- Variable or spiky traffic: Scales to zero saves costs for low-traffic services
- Prototyping and MVPs: Fast deployment without cluster setup
When to Use GKE
- Stateful applications: Databases, message brokers with persistent storage
- Long-running background jobs: No timeout constraints
- Complex networking: Service mesh, custom ingress controllers
- GPU/specialized hardware: Machine learning training workloads
- Multiple containers per pod: Sidecar patterns (Envoy, log agents)
- Fine-grained scaling control: Custom HPA metrics
Cloud Run Example
# Deploy a container to Cloud Run
gcloud run deploy my-service \
--image gcr.io/PROJECT/my-app:latest \
--platform managed \
--region us-central1 \
--allow-unauthenticated \
--min-instances 1 \
--max-instances 100 \
--memory 512MiWhat is Google Kubernetes Engine (GKE) and how does it differ from self-managed Kubernetes?
Google Kubernetes Engine (GKE) is a fully managed Kubernetes service on Google Cloud that handles the complexity of managing Kubernetes clusters, letting teams focus on running applications.
GKE vs Self-Managed Kubernetes
Control Plane Management
- GKE: Google manages the control plane (API server, etcd, scheduler, controller manager). You don’t pay for control plane compute in Standard mode; Autopilot mode is fully managed.
- Self-managed: You provision, configure, secure, upgrade, and monitor all control plane components.
Node Management
- GKE Standard: You manage node pools; Google handles OS patching, automatic repairs, and upgrades with your configured policies.
- GKE Autopilot: Google manages nodes entirely – you only pay per Pod, not per node.
- Self-managed: Full responsibility for node provisioning, OS updates, and scaling.
GKE Key Features
Release Channels
- Rapid: Latest Kubernetes versions for early testing
- Regular: Balanced stability (default)
- Stable: Maximum stability for production
Auto Upgrade and Auto Repair
GKE automatically upgrades node pools to match the cluster version and repairs unhealthy nodes.
Workload Identity
Secure way for pods to access GCP services without service account keys:
gcloud container clusters create my-cluster \
--workload-pool=PROJECT_ID.svc.id.googNode Pools
Groups of nodes with the same configuration (machine type, labels, taints). You can have multiple node pools for different workload types (CPU-optimized, GPU, spot).
GKE Autopilot
Fully managed Kubernetes:
- Per-Pod billing (no unused node capacity costs)
- Automatically optimizes resource requests
- Built-in security baselines enforced
- Google manages all node infrastructure
GKE Modes Comparison
| Feature | GKE Standard | GKE Autopilot | Self-managed |
|---|---|---|---|
| Node management | Partial | Full | Full |
| Control plane | Managed | Managed | Self-managed |
| Cost model | Per node | Per pod | Infrastructure cost |
| Flexibility | High | Medium | Full |
| Operational overhead | Low | Minimal | High |
Cloud-Native Integrations
- Cloud Load Balancing: Automatic L7/L4 load balancer provisioning
- Cloud Storage: Persistent Disk and Filestore integration
- Cloud Monitoring/Logging: Built-in observability with Cloud Operations
- Binary Authorization: Policy enforcement for container images
- Anthos: Multi-cloud and on-premises cluster management