APACHE AIRFLOW SAP BW CONNECTOR by Nextlytics

SAP BW & Apache Airflow:
More flexibility through bilateral system integration

The Apache Airflow workflow management platform orchestrates a large number of systems in the corporate context. Many interfaces can be conveniently controlled via operators in the workflow, only the connection to SAP systems cannot be realized without further ado.

SAP Business Warehouse (BW), on the other hand, lacks the ability to use open-source libraries and to freely choose the programming language for data manipulation. Through a two-way integration, the obstacles are overcome and the strengths of both systems can be combined in operational use cases. Thus the organization of the data processes gains flexibility regarding development language, linking possibilities and usable libraries.

Linking process chains in SAP BW and workflows in Apache Airflow

The Nextlytics Apache Airflow - BW connector enables plug-and-play integration between the Apache Airflow workflow management platform and SAP BW.

  • Endpoint in SAP BW to start process chains externally
  • User-defined process chain step that enables starting a workflow in Apache Airflow
  • Plug-and-play operator for synchronous and asychronous execution of process chains from Airflow

Functional range of the process chain step in SAP BW

System ID transfer

Simple system ID transfer

The trigger process optionally sends the system ID during the API call. In Airflow, the system ID is used to recognize whether the process chain came from the development or the production system.

Transfer

Transfer of any parameters

Additional transfer parameters can be passed on specifically for an individual workflow run. These can be, for example, file locations, target system abbreviations or temporary table names.

Synchron

Synchronous and asynchronous execution

After a workflow is started in Apache Airflow, the process chain can either proceed with the execution of subsequent process steps (asynchronous) or the process chain waits for the workflow to complete (synchronous).

Testing

Connection testing

The integrated connection test improves error handling. Connection errors due to incorrect access data are thus determined before the first execution.

Processing Data

Success-dependent processing of data

In the case of synchronous execution, the process chain can react differently to the incorrect execution of a workflow in Airflow than in the case of success.

SAPBW_Konnektor_1a

Functionality of the operator in Apache Airflow

Security

Secure authentification via Airflow backend

The connection data for SAP BW is encrypted in the Airflow backend during integration and is therefore not visible in the program code.

Synchronous and asynchronous execution

The workflow in Apache Airflow can also wait for the process chain to complete (synchronous mode) or proceed directly with subsequent steps (asynchronous mode). In synchronous mode, the status check is executed within the trigger task.
Synchrone Workflow
Combination

Can be easily combined with other operators

The connector can be added to the workflow in a modular way as a user-defined operator. Accordingly, it can be placed at the desired position in any workflow.

Automatic connection management

The management of the connection into the BW is controlled within the operator. During workflow creation, no additional code is required to manage X-CSRF tokens, cookies and the settings of proxies. Only the connection information needs to be stored in the backend.
Automatization
SAPBW_Konnektor_2a

Advantages of the connector

Installation

The NextLytics Airflow-BW connector is provided by BW as a transport and by Airflow as a custom operator. A detailed installation guide is available for the setup, which will help you to prepare both systems for the two-way integration in 10 minutes.

Do you have questions or are you interested in the connector?
Please do not hesitate to contact us.

NextLytics is always at your side as an experienced project partner. We help you to effectively solve your data problems from data integration to the use of machine learning models. Use the form below to ask your question and we will get back to you as soon as possible. 

We look forward to hearing from you!

Markus
Markus Suhr, Consultant Data Science & Engineering
Profilbild_Sebastian_Uhlig
Sebastian Uhlig, Solution Architect & Co-CEO

Contact us