Understanding the Differences

Understanding the differences between both service offerings simplifies the migration. While Application Logging uses a shared stack to serve multiple customers, each Cloud Logging Service instance is a separate stack. Due to this performance isolation, Cloud Logging Service can provide additional features, like custom dashboards and alerting, unrestricted ingestion without quota limitations, configurable data retention periods, and higher storage volume. You can find an extensive feature comparison on our feature page.

While authorization for Application Logging replicates exactly the same scope level defined by CF UAA for logs, Cloud Logging Service requires configuring authorization.

Cloud Logging Service does not provide a free-of-charge service plan like the lite plan of Application Logging yet. When migrating from a lite plan, you must change the instance to a paid service plan (see Service Plans and Pricing).

Planning Cloud Logging Service Instances for your CF Application Log Load

This section discusses how many Cloud Logging Service instances you need, and how many apps to bind to one instance. We recommend revisiting which apps are bound to which logging service plan instance and not having a one-to-one mapping of service plans. Compared to Application Logging, Cloud Logging Service instances can handle significantly more log volume. For the majority of use cases, peak ingestion performance should not be a problem for either of the services.

What previously has been sent to multiple service instances can now be sent to one. This solution allows for additional analyses and can be the more cost-efficient option.

For Application Logging, log ingestion is limited by the service plan quota limit, which defines the theoretical maximum of log storage. Cloud Logging instances have auto-scaling capabilities and a pay-per-resource usage model in which it checks usage hourly.

To determine the storage demand, check the statistics dashboard in Application Logging on how many logs (and bytes) you are shipping and how many have been dropped. When calculating the storage demand, consider the data retention. While data retention is 7 days for Application Logging, it is configurable in Cloud Logging Service. A change in data retention multiplies your storage demand. For example, increasing retention from 7 to 30 days increases storage demand by the factor of 4.28.

If you have multiple applications forming a single solution and the combined log volume fits a single Cloud Logging instance, we recommend binding them all to that instance. This allows you to analyze your application data together and investigate cross-application problems. However, if your applications do not serve a common solution, you can get a better separation of concerns by binding them to different Cloud Logging Service instances. It will give you more overall capacity. Additionally, each service instance can use its own user management (IDP). This can be helpful to manage access to the logs for operators.

Find more detailed information on sizing, performance, and costs for Cloud Logging Service plans at Service Plans and Pricing.

Logging Service Migration

For the migration, we propose you run Application Logging and Cloud Logging service instances, and bind your applications to both in parallel for the duration of the Application Logging retention of 7 days. This allows a smooth transition where you can check if everything works and you do not lose any logs.

Configure Identity Provider

In Cloud Logging, you have to configure your own user management (IDP). To have the same Single Sign On (SSO) experience, you must bring your own IAS account.

The recommended integration with Cloud Access Manager (CAM) requires you execute two steps.

• First, create a tenant SAML application in SAP BTP’S Identity Authentication Service (IAS)
• Second, integrate the tenant with CAM

Note: SAML authentication cannot be enabled on an existing Cloud Logging Service instance with a service update. Parameters have to be passed on service creation.

Instantiate Cloud Logging Instances and Bind Your Applications

Create instances of Cloud Logging Service and bind your applications based on our documentation.

• Enter your SAML configuration as a configuration parameter derived from the previous step. You cannot enable SAML authentication on an existing Cloud Logging Service instance with a service update.
• You have more freedom to configure, compared to Application Logging. For example, you can configure up to 90 days of retention. See Configuration Parameters to have a suitable parametrization.

Adjust Custom Fields Configuration When Sending with Logging Libraries

This section applies only if you are sending custom fields with one of our logging libraries (Java/NodeJS). Application Logging requires configuring custom fields in the logging library, while Cloud Logging Service supports them without additional configuration.

The migration requires a configuration change to support both services simultaneously. You must send custom fields in #cf nested object and at top-level.

NODEJS

The CF NodeJS logging library automatically detects which services you are bound to (Application Logging and/or Cloud Logging Service) and adjusts sending custom fields since version v6.5.0. Make sure you updated the logging library to this version or higher.

JAVA

You must configure the CF Java logging library, dependent on the logging backend you use.

For Log4j Logging Backend

Set retainOriginal to true in your log4j XML configuration file. That means replacing <customField mdcKeyName="custom-field" retainOriginal="false" /> with <customField mdcKeyName="custom-field" retainOriginal="true" /> for all custom fields.

For Logback Logging Backend

For each custom-field configuration line, another one has to be appended to retain that custom field at top level. For example, the <customFieldMdcKeyName>custom-field</customFieldMdcKeyName> configuration line should be followed by this one: <retainFieldMdcKeyName>custom-field</retainFieldMdcKeyName>.

After deprovisioning the Application Logging instance, you can remove all configuration regarding custom fields.

Configure your Cloud Logging Instance

You can use Cloud Logging service in different contexts, therefore we cannot provide a single configuration that fits all. To have the “Application Logging” experience:

• Change the OpenSearch Dashboards landing page
• Set pin filters by default in the same menu as the landing page (we plan to have selected this in the future automatically)

[Optional] Configure Metric Sending

Without additional efforts, Cloud Logging does not receive resource usage metrics or custom metrics injected.

Cloud Logging service does not provide the “Metrics” dashboard showing resource utilization. Note: Resource metrics will be added in the future, track progress in PERFX-6063.

You can send metrics using Telegraf, Spring Boot Actuator or explore the new OpenTelemetry-based ingestion. Note that you have to create your own dashboard for metrics.

Check Cloud Logging Instance Works as Expected

Check if logs arrive and are parsed correctly. Remember, you have your own OpenSearch Dashboards, and your own URL which can be read from the cf service-key and that contains all endpoints and credentials.

• To learn what you can do, check the using your Cloud Logging Service instance documentation and see how to ingest, alert, manage access rights, and so on.
• The Cloud Logging Service lacks certain dashboards that are present in Application Logging:
  • The “Statistics” dashboard about dropped logs is not provided for Cloud Logging Service.
  • The “Metrics” dashboard showing resource utilization is not provided for Cloud Logging Service.
  • The “Help” dashboard is replaced by the “Using your Cloud Logging instance” dashboard.

Deprovision Application Logging Instances

If Cloud Logging Service instances hold all the logs which are in Application Logging, you can unbind the applications from Application Logging instances and delete the Application Logging service instance. Existing bindings must be deleted before the instance can be deprovisioned.

• Run cf services and cf apps
• Run cf unbind-service <App Name> <Instance Name>
• Run cf delete-service <Instance Name>