Home Documentation Stay organized with collections Save and categorize content based on your preferences. Data analytics and pipelines overview Load, transform, and analyze data to provide business intelligence insights. Explore data analytics and pipelines in Google Cloud Read documentation and Cloud Architecture Center articles about data analytics and pipeline products, capabilities, and procedures. description Introduction to loading data into BigQuery Learn how to batch load or stream data into BigQuery. account_tree Data and analytics resources Plan your approach with Architecture Center resources across a variety of data and analytics topics. (Goes to Architecture Center.) account_tree Data analytics design patterns Plan design patterns for real-world data analytics solutions. (Goes to Architecture Center.) account_tree Jump start solution: analytics lakehouse Plan how to design an analytics lakehouse to store, process, and activate data. (Goes to Architecture Center.) Training, blog articles, and more Go to training courses, blog articles, and other related resources. school Data analyst learning path Study Google Cloud technologies essential to the data analyst role. (Goes to Skills Boost Training.) school BI and analytics with Looker learning path Study how to explore data in Looker and set up self-serve analytics for your users. (Goes to Skills Boost Training.) cloud Decision tree for data & analytics workloads View a decision tree to choose services for running data and analytics workloads. (Goes to Google Cloud blog.) Data analytics and pipelines products by use case Expand sections or use the filter to find products and guides for typical use cases. Data analysis and pipelines platforms and product suites Analyze your local and external data with machine learning and AI tools, then securely share and visualize insights - with zero infrastructure management. BigQuery Understand your data using a fully managed, highly scalable data warehouse with built-in ML. BigLake in BigQuery Query structured data in external data stores with access delegation. BigQuery Storage Use storage optimized for running analytic queries over large datasets and high-throughput streaming ingestion and high-throughput reads. Introduction to migrating to BigQuery Migrate your data warehouse to BigQuery using free-to-use tools that help you with each phase of migration, including assessment and planning, SQL translation for more than 10 dialects, data transfer, and data validation. Data analysis Query massive datasets with SQL, geospatial analytics, and BI tools that support ad hoc and programmatic analysis as well as data sharing. Analytics Hub A data exchange platform that enables you to share data and insights at scale across organizational boundaries with a robust security and privacy framework. BigQuery Analytics Maximize your data analysis investments when running analytic queries on large datasets. BigQuery ML Create and run machine learning (ML) models by using GoogleSQL queries, plus access LLMs and Cloud AI APIs to perform artificial intelligence (AI) tasks like text generation or machine translation. Dataproc Perform batch processing, querying, and streaming using a managed Apache Spark and Hadoop service. Earth Engine Google Earth Engine is a geospatial processing service. With Earth Engine, you can perform geospatial processing at scale, powered by Google Cloud Platform. Looker Access, analyze, and act on an up-to-date, trusted version of your data Looker Studio Tell great data stories to support better business decisions. Data governance Control and manage quality throughout the lifecycle of your data as you share it across and outside your organization. Data Catalog Discover and understand your data using a fully managed and scalable data discovery and metadata management service. Dataplex Organize your data into lakes and zones, and automate data management and governance across that data to power analytics at scale. Sensitive Data Protection Discover and redact sensitive data. Introduction to data governance in BigQuery Implement and enforce BigQuery data governance policies. Data ingestion Migrate, stream, and batch-load data into a serverless, high-throughput storage architecture. BigQuery Data Transfer Service Automates data movement into BigQuery on a scheduled, managed basis. Your analytics team can lay the foundation for a BigQuery data warehouse without writing a single line of code. Cloud Data Fusion Quickly build and manage data pipelines using fully managed, code-free data integration with a graphical interface. Dataflow Develop real-time batch and stream data processing pipelines. Dataproc Perform batch processing, querying, and streaming using a managed Apache Spark and Hadoop service. Dataproc Metastore Use a fully managed Apache Hive metastore (HMS) that runs on Google Cloud to manage your data lake and metadata, plus provide interoperability between the various data processing engines and tools you're using. Dataproc Serverless Use Dataproc Serverless to run Spark batch workloads without provisioning and managing your own cluster. Datastream A serverless and easy-to-use change data capture (CDC) and replication service. Pub/Sub Ingest event streams from anywhere, at any scale. Storage Transfer Service Transfer data between Cloud Storage services such as AWS S3 and Cloud Storage. Transfer Appliance Ship large volumes of data to Google Cloud using rackable storage. Data orchestration Organize and optimize your workload management chain with seamless connections across data sources and processes. Cloud Composer Create, schedule, monitor, and manage workflows using a fully managed orchestration service built on Apache Airflow. Dataform Dataform offers an end-to-end experience that helps data teams build, version control, and orchestrate SQL workflows in BigQuery. Data processing Transform your data with powerful ETL, SQL workflow, and DML tools. Dataflow Develop real-time batch and stream data processing pipelines. Dataproc Perform batch processing, querying, and streaming using a managed Apache Spark and Hadoop service. Related products, guides, and sites What is Vertex AI? Learn about the Vertex AI machine learning (ML) platform that lets you train and deploy ML models and AI applications, and customize large language models (LLMs) for use in your AI-powered applications. Blockchain Analytics Get indexed blockchain data made available through BigQuery for easy analysis through SQL. Dataprep by Trifacta Explore, clean, and prepare data for analysis. (Goes to external website.)