Introduction:

Hey Folks! I’m Navya and here I am with my first blogpost!

In the era of data-driven decision-making, seamless integration and real-time access to data from diverse sources are paramount for organizations seeking competitive advantage.

Qualtrics is a cloud-based platform that specializes in experience management, including survey creation, feedback collection, data analysis, and actionable insights. It enables organizations to design and distribute surveys to gather feedback and insights from customers, employees, and other stakeholders.

Key features of Qualtrics include:

• Creating surveys with various question types
• Collect responses in real-time
• Set up automated actions based on survey responses
• Integrate with other platforms and systems, enabling seamless data exchange and collaboration

You can read and understand more about Qualtrics in the following link: About Qualtrics

Qualtrics, allows organizations to collect feedback and insights through surveys. However, for comprehensive analysis and decision-making, these survey responses need to be integrated into a central data repository with business data. This is where SAP Datasphere, with its data integration and management capabilities, comes into play.

For analysing data, SAP Datasphere is a flexible, scalable, and cost-effective solution; with integrated tools and data governance, and Self-service analytics, you can easily cleanse, enrich, and visualize your data. It provides a large set of default connections to access data from a wide range of sources, which might be in the cloud as well as on-premises, or from SAP as well as from non-SAP sources or partner tools.

To extend connectivity beyond SAP Datasphere standard remote connectivity and cover additional data sources we can create a connection to a partner tool and generate an Open SQL schema in SAP Datasphere.

In this blog post, I will share my journey of exploring a specific scenario – the integration of Qualtrics survey data into SAP Datasphere. There are different options for bringing Qualtrics information into SAP Datasphere, however in this blog post, we are looking at how SAP CAP extension application can be used to bring data from Qualtrics into SAP BTP.

In the above architecture, the SAP CAP application retrieves Qualtrics survey data and its responses via Rest service. This data is persisted in SAP HANA Cloud which also acts as backing service for the extension application.

In my experiment, I’ve focused on bringing Qualtrics data to SAP Datasphere. However, you can bring data from various sources such as SAP S/4HANA, Customer Experience Systems, other Business Systems.Furthermore, you can harness data from hyperscaler platforms like Google BigQuery, Databricks as well.

Let’s get started!

cds init <your-project-name>

cf api <API-Endpoint-of-your-landscape>
cf login

cds add hana --for production

cds add mta

mbt build
cf deploy <generated-mtar-file-path>

Create an event listener in Qualtrics for completed response

To create an event listener in Qualtrics, enabling the initiation of a webhook callback upon receiving a survey response notification,

Upon receiving a survey response from the user, Qualtrics system initiates a webhook callback to the earlier created CAP application which acts as a event listener. To create an event listener, you can use the eventssubscriptions api.

For example, in my case, we are much interested in the event which is surveyengine.completedResponse.{SurveyID} as that signals that a survey has been completed.

Run the following curl command to create an event subscription with your survey’s parameters:

curl -X POST -H 'X-API-TOKEN: {your-api-token}' -H 'Content-Type: application/json' -d '{
"topics": "surveyengine.completedResponse.{your-survey-id}",
"publicationUrl": "{your-cap-application-url}",
"encrypt": false
}' 'https://{your-datacenter-id}.qualtrics.com/API/v3/eventsubscriptions/'

Generate Test Responses

After finishing the survey event subscription, we can utilize one of Qualtrics’ features to create test responses. You can access it by choosing the option Generate test responses in Tools in the edit view of your survey.

Enter the number of test responses you would like to generate and Click Generate to generate the responses.

After waiting for your test to be completed, you will be shown a list of the test iterations, along with a Survey Test Summary. This summary includes a number count of test iterations, saved responses and more.

Check Survey Response in the CAP Application

Open your CAP application, using the URL generated earlier on SAP BTP and by clicking on Surveys, you can visualize the test survey response.

Create a Key for a HDI Container to access it in SAP Datasphere

You can create a key for the HDI container either using Cloud Foundry CLI or using SAP BTP Cockpit. Using Cockpit, you can find and open the HDI container instance based on the name specified in mta.yaml file.

Once you’re inside your HDI container, access the “service keys” section. From there, select the option to “create” a new service key for your HDI container which will contain the connection details like JDBC URL, host, user, password and more.

Once you’ve successfully generated the service key, proceed to download the associated JSON file. which will be used in establishing the connection to the database within SAP Datasphere.

Use this key to create a connection in SAP Datasphere

Prior to establishing your connection, ensure that you have generated a space within your SAP Datasphere account. (If not, follow the below steps to create one)

To create a Space, click on the Space management icon on the bottom left, and click on the Create button on the top right.

Enter a name for your Space. The Space ID will auto-populate. In this example, let’s call our Space DEV SPACE. Click on Create, and you’ve successfully created your Space in SAP Datasphere.

Create a connection with your CAP Application’s HANA Database

In the “Connections” tab, open the relevant space of your choice.

In the local connections section, click on create, and search for SAP HANA in connection types.

Connection properties is configured by populating it with necessary details like Host, Port, User Name and Password from the service key JSON file of the HDI container.

Input your desired business name and description(optional) for the connection. Finally, click on “Create Connection” to finalize the process.

You will be able to see the Connection created under the local connections section.

Use connection to connect with remote table and Model Survey Data in Data Builder

In Data Builder, we can create the New Graphical View to view the data from Qualtrics.

The connection created earlier can be found in “Connection Folder” of the “Sources” section.

Navigate inside your table where the data is present and drag and drop your table from the panel on the left over to the canvas. After dragging, a pop-up for Import Table appears, click on Import and Deploy to add it to the canvas.

To access the data within the table, locate the “Data Viewer” icon. Upon clicking this icon, you will encounter a “Delayed Data Viewing” message at the bottom section. Scroll downwards and select “View Data” to proceed.

Now you should be able to view your data and connect with Remote tables in SAP Datashpere.

Summary

SAP Datasphere provides us the different approaches to connect with the source system. One of the ways we have used here is to create a database user to allow external tools to connect to the space and write data to Open SQL schemas associated with the space.

In this blog post, we’ve explored the process of bringing Qualtrics data into SAP Datasphere via Database Users/Open SQL Schemas approach. We achieved this by deploying a CAP Application on SAP BTP which uses SAP HANA Cloud to store Qualtrics Survey data. This integration empowers us to leverage Data Builder within SAP Datashpere for the modeling and enriching of business operational data with experience data.

Before this blog post comes to an end, I would like to thank my team at SAP for their continuous support and express special gratitude to Lalit Mohan Sharma ,Uma Anbazhagan and PVN PavanKumar, who supported me in validating and publishing this post.

What’s next?

Well, Thank you for your engagement and patience throughout my blog post. Your interest is truly valued. We’re eager to hear about your own encounters and insights, not to mention your innovative concepts for additional SAP Datasphere applications, whether involving SAP or non-SAP solutions.

Stay enthusiastic and maintain your spirit of curiosity – there’s much more to explore on the horizon!

Reference & Further Reading

• About Qualtrics
• Integrating Data and Managing Spaces in SAP Datasphere
• Integrating Data via Database Users/Open SQL Schemas
• Integrating Data via Database Users/Open SQL Schemas
• How to create SAP Datasphere service instance in the SAP BTP Platform

For more information and related blog posts on the topic page for SAP Datasphere.

If you have questions about SAP Datasphere you can submit them in the Q&A area for SAP Datasphere in the SAP Community.