Cloud Storage and Data Acquisition

Managed Ingestion Pipelines

Managed ingestion pipelines such as AWS Glue, Google Cloud Dataflow, and Azure Data Factory expose REST APIs for orchestrating extract-transform-load and extract-load-transform jobs that land source data in cloud storage.

GitHub

Documentation

📖
Documentation
https://docs.aws.amazon.com/glue/latest/webapi/Welcome.html
📖
Documentation
https://cloud.google.com/dataflow/docs/reference/rest
📖
Documentation
https://learn.microsoft.com/en-us/rest/api/datafactory/

API entry from apis.yml

apis.yml Raw ↑ 
aid: cloud-storage-and-data-acquisition:managed-ingestion
name: Managed Ingestion Pipelines
tags:
- Data Pipelines
- ETL
- Managed
image: https://kinlane-productions2.s3.amazonaws.com/apis-json/apis-json-logo.jpg
humanURL: https://aws.amazon.com/glue/
properties:
- url: https://docs.aws.amazon.com/glue/latest/webapi/Welcome.html
  type: Documentation
- url: https://cloud.google.com/dataflow/docs/reference/rest
  type: Documentation
- url: https://learn.microsoft.com/en-us/rest/api/datafactory/
  type: Documentation
description: Managed ingestion pipelines such as AWS Glue, Google Cloud Dataflow, and Azure Data Factory
  expose REST APIs for orchestrating extract-transform-load and extract-load-transform jobs that land
  source data in cloud storage.