Airbyte

Description

Airbyte is open source data movement infrastructure for building extract and load (EL) data pipelines. It is designed for versatility, scalability, and ease-of-use.

Uses and Functionnalities

Teams and organizations need efficient and timely data access to an ever-growing list of data sources. In-house data pipelines are brittle and costly to build and maintain. Airbyte's unique open source approach enables your data stack to adapt as your data needs evolve.

• Wide connector availability: Airbyte’s connector catalog comes “out-of-the-box” with over 600 pre-built connectors. These connectors can be used to start replicating data from a source to a destination in just a few minutes.

• Long-tail connector coverage: You can easily extend Airbyte’s capability to support your custom use cases through Airbyte's No-Code Connector Builder.

• Robust platform provides horizontal scaling required for large-scale data movement operations, available as Cloud-managed or Self-managed.

• Accessible User Interfaces through the UI, PyAirbyte (Python library), API, and Terraform Provider to integrate with your preferred tooling and approach to infrastructure management.

Airbyte is suitable for a wide range of data integration use cases, including AI data infrastructure and EL(T) workloads. Airbyte is also embeddable within your own app or platform to power your product.

CICD integration method

To enable airbyte, set the following variables in your main configuration file.

airbyte = {
      enabled      = true
      version      = "1.7.0"
      namespace    = "airbyte"
      component    = "dp-components"
      release_name = "airbyte"
      ingress      = true
      url_prefix   = "airbyte"
}

Releases

Official documentation

Airbyte documentation