SAP Datasphere and Databricks

Earlier this year we announced SAP Datasphere, a comprehensive data service built on SAP Business Technology Platform (SAP BTP) that enables every data professional to deliver seamless and scalable access to mission-critical business data. And along with SAP Datasphere, we announced an open data ecosystem of leading technology partners that are closely integrating their data and AI platforms with SAP Datasphere. Jointly, we are on a mission to help customers radically simplify their data landscapes with a business data fabric.

The Business Data Fabric With Databricks

One of these key partnerships is with Databricks, which delivers an open and unified Data Intelligence Platform for data, analytics and AI use cases. This partnership enables organizations to integrate SAP data, with its complete business context, with the Databricks data lakehouse architecture unlike ever before.

3 Ways to Unlock the Value of Business Data

The enhanced interoperability between SAP Datasphere and Databricks ensures that your data engineers and data scientists no longer spend countless hours extracting SAP data and rebuilding its critical semantics. Instead, they can use SAP Datasphere as the tailored integration point to connect, unify, and manage SAP and non-SAP data. And they can do so without data duplication by using SAP Datasphere to federate business data to the Databricks data lakehouse, enabling extended planning and analysis and gaining more accurate predictions from AI.

Reference architecture of SAP Datasphere and Databricks integration. Picture credits: Databricks.

Customers can unlock the value of business data in three ways using SAP Datasphere and Databricks.

1. Simplify access to SAP and blended data for AI and advanced analytics

Many SAP customers use Databricks with SAP data and other sources for advanced analytics and artificial intelligence (AI) initiatives. With SAP Datasphere self-service capabilities, data users discover and model business data within a governed environment. They can quickly identify the appropriate SAP data with the associated business context and prepare it for their data projects, cutting data preparation time and ensuring more accurate outcomes. Data scientists and engineers can then use the FedML library with SAP Datasphere to federate SAP Datasphere data directly into the Databricks data lakehouse, accelerating advanced analytics and AI projects. They can also write insights back into SAP Datasphere to enable extended planning and analysis use cases.

Take the example of a global retailer who wanted to anticipate customer needs and deliver a better omnichannel experience. With rapidly changing customer preferences and supply chain disruption, the company wanted to rely on real-time predictions to promptly adjust its sourcing and restocking strategy. The retailer is now federating harmonized business data from multiple SAP systems to deliver up-to-date sales and inventory data to Databricks and blending it with customer sentiment data in Databricks to accelerate data science use cases. Data preparation has been highly reduced, and inferences from machine learning models are written back to SAP Datasphere to enable further analytics and planning in SAP Analytics Cloud.

2. Harmonize SAP and Databricks lakehouse data without duplication

Our customers are accelerating business 360 analytics and planning use cases by harmonizing SAP and non-SAP Data in SAP Datasphere — without running ETL processes or having to make sense of SAP data stripped of its business semantics.

Take the example of a fashion retail company that needed to react faster to changes in buying behaviors to meet customer demand. The company uses data federation capabilities in SAP Datasphere to bring data like market indicators from Databricks Data Intelligence Platform to SAP Datasphere and harmonize it with SAP data without duplication. By relying on rich data without the burden of data movements, the company can be informed in real time, allowing for more agile and trusted decisions with planning and analysis.

3. Build, train and deploy machine learning models for business insights

Customers get faster time-to-value and more accurate predictions from AI when you can rely on fresh data from across your business. You can turbocharge your AI and ML in Databricks with semantically rich business data to ensure models are accurate and trained more quickly. Our customers use the FedML library with SAP Datasphere to federate SAP data to the Databricks data lakehouse and blend it with other enterprise data for advanced analytics and data science use cases.

As of November 2023 (release 2023.24), customers can now use SAP Datasphere Replication Flow to replicate SAP data to the cloud object store accessible to Databricks.

For example, you could build a demand forecasting model in Databricks that uses point of sales (POS) data and other data from SAP — such as on-hand inventory, sales, open orders, etc. — to give your model business context and improve your predictions. You could then write back inferences to SAP Datasphere and run reports in SAP Analytics Cloud to compare forecasts and actuals.

Learn more

To learn more about how SAP Datasphere and Databricks work together, check out the following:

• SAP TechEd 2023: Accelerate Databricks Projects With Semantically Rich SAP Data
• Free Trial: SAP Datasphere Guided Experience
• Technical blog: Federating queries to Databricks from SAP Datasphere for real-time analytics in SAP Analytics Cloud
• Technical blog: Using FedML library with SAP Datasphere and Databricks