This is documentation for Apprenda 7 and 8.
Documentation for newer version is available at https://new.docs.apprenda.com.

Logging from Your Application

Logging is a traditional and essential tool that assists in monitoring certain activities conducted by your application's Users as well as debugging problems that arise during ordinary usage. Apprenda provides a centralized logging service that can be used by your application. Features of Apprenda's logging service include:

  • Searching capability
  • Scrubbing of sensitive information from log messages, such as password info
  • E-mail notification for certain log levels
  • Automatic capture of contextual information, such as the User that conducted the action resulting in a log message

Before taking advantage of these features, it is necessary to understand why the Platform provides its own logging service as well as how to use it in your application.

Viewing Your Logs

Global Logging levels (Debug, Information, Warning, Error and Fatal) are set by your Platform Operator.  This means that events that are logged for your applications are limited to messages where the severity level is at or more severe than the Global Logging Level (e.g., if the Global Logging Level is set to Error, you will receive Error and Fatal messages only). In order to receive event logs at a less severe level for your application, you must set a Log Override.

Log Overrides can be set in the Developer Portal or in an application's Deployment Manifest. Event logs can be viewed in the Developer Portal.

It should be noted that Platform Operators can also view log messages and Log Overrides through the System Operations Center.

Why a Centralized Logging Service?

Most Platform instances run on multiple servers. A typical application may have several components (user interfaces, .NET services, etc.), each of which may be deployed multiple times on different severs. Each component typically would log to disk on a local machine, and it would become unwieldy to review and search for log messages on several machines. Additionally, there are general security and efficiency concerns that pertain to accessing log messages, especially in a production environment. 

To address these general concerns, the Platform provides its own logging service and makes it available to all Platform applications automatically.

Logging via the Apprenda .NET Guest App API

Apprenda, via its Guest App API, logs messages for .NET applications. The log level is configurable and log messages not meeting the appropriate level are discarded. It should be noted that the Platform will ignore any local logging configuration and automatically supply an implementation that wires your log calls to the Platform’s centralized logging services.

In your Visual Studio project, reference the following DLL in your application:

  • SaaSGrid.API

In code, an application can create a logger instance and use that instance as follows:

C# Example – Logging via the .NET API

    [ServiceContract]
    public class MyService
    {
        private static readonly ILogger log = 
        LogManager.Instance().GetLogger(typeof(MyService));

        [OperationContract]
        public void MyMethod()
        {
            if (log.IsDebugEnabled)
            {
                log.Debug("At start of my method!");
            }
            try
            {
                DoWork();
            }
            catch (Exception e)
            {
                if (log.IsErrorEnabled)
                {
                log.Error("Unknown error", e);
                }
            }
            if (log.IsDebugEnabled)
            {
                log.Debug("At end of my method!");
            }
        }
    }

Logging for Java Applications 

The information below pertains to logging for applications that contain Java Web Application workloads deployed on Tomcat or JBoss containers. Please note that logging for Linux Services and Docker workloads function as indicated in their dedicated documentation pages.

The Platform handles logging for Java applications by leveraging log4j (version 1.2). Log4j loggers are mapped to Platform Log Overrides. Since Log Overrides can be modified after an application has been deployed, this mapping allows developers the freedom to modify logging behavior at any point in an application's lifecycle.

The Platform ships with an out-of-the-box Apprenda log4j Appender bootstrap policy that determines the logging configuration of Java guest applications. Your Platform Operator has considerable control over how logging is configured for Java applications (including configuration for Tomcat and JBoss containers), and may choose to update or disable this bootstrap policy. Your Platform Operator may also choose to replace the Apprenda log4j Appender bootstrap policy with a custom implementation and provide you with instructions on how it can be leveraged.

The documentation below explains the basics of how Java logging works on the Platform and provides details on how to leverage this functionality in two common use cases: 1) when the Apprenda log4j Appender bootstrap policy is invoked and 2) when no bootstrap policy is invoked. It has been broken up into the following categories:

It should be noted that any necessary log4j libraries should be included in an application's WAR file

How log4j loggers work with Platform log overrides

On the Platform, log messages for Java Application that leverage log4j flow as follows:

  • Developers generate logging messages from the application code at specific logging levels using log4j’s logging APIs.
  • Messages from the application are passed to loggers defined in the application's log4j configuration file.
  • Normal log4j log level filtering is applied to generated log messages: messages with a log level that is “higher” than or equal to the level defined on the logger are passed on to appenders, messages that with a level are “lower” than the logger’s level are filtered out.
  • Appenders defined for each logger direct where the forwarded messages go. Appenders may, for instance, direct messages to a particular file. In order for messages to appear in the Platform's logging system, they must be forwarded to the Apprenda Platform appender. 
  • Messages that are not associated with a specific logger are filtered and forwarded by the root logger.
  • Messages received by the Apprenda Platform appender are filtered a second time according to any Log Overrides configured for the application (if no overrides apply, they are then filtered according to the Global Logging level). 
  • Messages successfully forwarded by the Apprenda Platform appender will be sent to the appropriate Platform logging database and will be accessible to the Development Team that owns the application through the Developer Portal. Platform Operators will be able to view these messages in the System Operations Center.

The Platform allows for four types of Log Overrides, which will override the Global Logging level according the following restrictions:

  • None/Version: the override applies to all log messages for the application version
  • Tenant (available for multi-tenant applications in Sandbox or Published only): the override applies only to log messages for the specified Tenant
  • User (available for multi-tenant applications in Sandbox or Published only): the override applies only to log messages for the specified User
  • Source/Tag: the override applies if the value specified matches the logger name / class name from which the logs originate. It should be noted that wildcard suffix matching (e.g., "com.apprenda*" will apply the override to any log message with a string that begins with "com.apprenda") is supported for this option.

Log Overrides may be defined for an application through a Deployment Manifest included with its Application Archive. They may also be viewed and managed through the Monitor tab in the Developer Portal.

How log4j config files are translated into Log Overrides at upload time

When an Application Archive that contains a Java Web Application component is uploaded to the Platform, the following occurs:

  • Log4j configuration files are parsed. 
  • For each logger found in the log4j config file, the Platform will create a corresponding "Source" log override using the logger name as the Source restriction. If the logger defined in log4j is a “class logger” (the last element of the “foo.bar.Baz” logger name starts with an upper case), then the Platform creates a matching Log Override (e.g. the override will be “foo.bar.Baz”). If the logger is a “package logger” (a logger for all classes in a specific package, based on current convention, the last element starts with a lower case , e.g. “foo.bar”), the Platform creates a “wildcard override” (e.g. “foo.bar*”)
  • A special Source override will be created for the root logger. A valid log4j configuration always requires a root logger to be specified. In order to match that, we create a log override with a special name of “root” that is used to match the “<root />” element from the log4j configuration. The level of the “root” logger controls the log level at which all classes in the application that do not have an explicitly specified logger will log . In addition, the root logger would control the level of messages that are passed to ALL appenders – both the Apprenda Platform appender, as well as any other appenders that the Platform Operator may have provided (e.g if they have file appenders, messages that pass through based on the root logger specification will go to both the Apprenda-specific and any custom appenders that the Platform Operator might have created). Creating this override allows the developer to control the root logger log level (and change it after the application has been uploaded). 
  • A Version override that corresponds to the log level of the root logger will be created. The root logger defines logging behavior for messages that are not associated with a specifically defined logger and therefore determines which messages pass from the application to the Apprenda Platform appender. However, the class names associated with these messages will not match the Source restriction ("root") for the override created for the "root" logger, which means these messages would be filtered out by the Apprenda Platform appender. The creation of the "blank" Version override is therefore intended to allow the messages forwarded by the root logger to pass through the Apprenda Platform appender to the SOC/Developer Portal.

The screenshot above shows the overrides created when a Java application with the following log4j configuration was uploaded to the Platform:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"  threshold="debug" debug="true">  

<logger name="com.springapp.mvc">
  <level value="Info" />
</logger>
<root>
  <level value="Error" />
</root>

</log4j:configuration>
  • The  com.springapp.mvc logger with a log level set to "Info" corresonds to the first override in the screenshot, the com.springapp.mvc* Source/Tag override. 
  • The root logger defined in the log4j config file resulted in a second override in the screenshot, the root Source/Tag override with the level set to the root logger level. This override can be updated by by developers to manipulate the root logger level through log overrides.
  • A Version override is also created with the level set to the root loggers' level (the third override in the screenshot).

What happens on the Java side when a Log Override is created

Although all Log Overrides create a filtering rule in the Apprenda Platform appender, Source overrides are unique in that they additionally create a log4j logger in the runtime logging configuration of the application (if one does not already exist). 

Precedence of Source/Tag overrides over the other types of Log Overrides

Because loggers are tied to Source overrides, Source overrides will take precedence over other types of Log Overrides. When a message is passed from the application to a logger (including the root logger), the log level will correspond to the Source override and will determine whether or not a log message is passed by the log4j logger from the application to the Apprenda Platform appender. This means that all messages will be initially filtered at the logger according to the log level of the Source override before they reach the Apprenda Platform appender (where all other types of overrides are applied). Therefore, whenever a Source override is involved in the filtering of log messages, the Source override dominates the filtering process:

  • Messages are sent to the SOC if the their level is greater than or equal to the source override level, regardless of any additional (and otherwise applicable) overrides.
  • Messages are dropped if their level is less than the Source override level, regardless of any additional (and otherwise applicable) overrides.

As an example, let's consider a setup with the following overrides:

  • com.apprenda.foo Tag/Source override at ERROR
  • Tenant override for tenantId=1 at INFO
  • User override for userId=2 at DEBUG
  • Tenant override for tenantId=2 at FATAL
Log message from app com.apprenda.foo
Logger at ERROR
Apprenda
Platform
appender
SOC/ Dev Portal Logs
WARN message
(com.apprenda.foo)
(tenantId=1)
WARN < ERROR

Message is filtered out
before it gets to the Apprenda Platform appender

  Does not appear in logs
ERROR message
(com.apprenda.foo)
(tenantId=2)
ERROR>= ERROR

Message goes through

Most permissive = ERROR

Message goes through

Appears in logs

Log Overrides for System.out and System.err

By default the contents of the System.out and System.err streams are not directed to to the Platform's logs. In order to direct these streams to the Platform logs (so that content is visible through the Developer Portal or SOC), create the following Source/Tag overrides:

  • For sysout, add a Source override with value System.out at Info
  • For syserr, add a Source override with value System.err at Error

Unlike log overrides for other “Sources,”  log override for “System.out" and "System.err" will not somehow change what the application outputs to the console. Rather, it will simply create the “filter rule” in the Apprenda appender that allows the logs coming out of the corresponding streams of the application to appear in the SOC/dev portal. 

The Apprenda log4j Appender Bootstrap Policy

The Apprenda log4j Appender bootstrap policy is installed out of the box with the Platform and is enabled by default on fresh Platform installs. Your Platform Operator may choose to configure the Platform to invoke this bootstrap policy for all Java deployments, or may allow you to choose whether or not to invoke it through a Custom Property setting. Regardless of how the bootstrap policy is invoked, the information below describes what happens when the associated bootstrapper runs for a Java Web Application workload deployment.

In a nutshell, the bootstrapper modifies the application's log4j configuration file (log4j.xml or log4j.properties). If the application does not include a log4j config file, the bootstrapper creates a default configuration where all of the application’s Log4j logging output is sent to the Platform

The details of what the bootstrapper does is as follows:

Discovery of Log4j configuration files bundled with the application

The bootstrapper looks for log4j configuration files in the following order:

  • log4j.xml in the Apprenda archive's merge/WEB-INF/classes/ directory
  • log4j.properties in the Apprenda archive's merge/WEB-INF/classes/ directory
  • log4j.xml in the war's WEB-INF/classes/ directory
  • log4.properties in the war's WEB-INF/classes/ directory

Modification of log4j configuration file, if found

If the bootstrapper finds a log4j configuration file, it will modify its contents in the following way:

  • Remove all appenders from the configuration file
  • Remove all appender-refs from all loggers, including the root logger
  • Add an Apprenda logger at DEBUG level if it does not exist (i.e. logger for com.apprenda)
  • Add an Apprenda appender that will forward messages to the Apprenda Platform appender
  • Add an appender-ref that references the Apprenda appender to the root logger if the root logger is defined. If the root logger has not been configured by the developer, the root logger is configured and set to the level specified in the Default Application Log4j Root Logger Level Custom Property (or set to Info if the Custom Property is not set).

Creation of log4j configuration file if the application does not provide one

If a log4j configuration file is not found, the bootstrapper creates a minimal log4j.xml config file to log messages coming from Apprenda-specific classes:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"  threshold="debug" debug="true">  
<appender class="com.apprenda.guestapp.logging.ApprendaPlatformAppender" name="apprendaPlatform" />
 
<logger name="com.apprenda" additivity="false">
      <level value="debug" />
      <appender-ref ref="apprendaPlatform"/>
</logger>
 
<root>
  <level value="{DefaultApplicationLog4jRootLoggerLevel custom property}"/><!-- level defaults to Info if the custom property is not set -->
  <appender-ref ref="apprendaPlatform"/>
</root>

</log4j:configuration>

It should be noted that the bootstrapper will run only when a Java Web Application workload is deployed. Any log overrides that are created at runtime will not be affected by the bootstrapper.

Java logging behavior when no logging bootstrap policy is invoked

The scenarios below describe how logging can be configured for Java applications when logging behavior is NOT affected by the Apprenda log4j Appender bootstrapper (or any other bootstrapper implemented by your Platform Operator to configure Java logging). Some of the scenarios below are dependent upon the the cooperation of your Platform Operator.

The snippets below do not define the appenders that are referenced in the loggers. However, should you wish to view messages through the Apprenda Platform, you must send logs to the Apprenda Platform appender by referencing the following class (your may configure the name however you choose):

<appender class="com.apprenda.guestapp.logging.ApprendaPlatformAppender" name="apprendaAppender" />

It should be noted that although the snippets reflect sections of a log4j config file, any log4j config file must be valid and complete in order to be recognized by the Platform (see the default config file generated by the Platform for an example).

Scenario 1. Developer wants all logging to be governed by Apprenda 

The developer can reference the Apprenda appender in the root logger, and set the root log level to Debug. By doing this, all log events are sent to the Platform, and the Apprenda Platform appender will govern logs through Log Overrides.

<root>
  <level value="DEBUG" />
  <appender-ref ref="apprendaAppender" />
</root>

Scenario 2. Developer wants all logging to go to a file at Info

The developer can reference a file appender at the root logger, and set the root log level to Info.

<root>
  <level value="INFO"/>
  <appender-ref ref="fileAppender" />
</root>

Scenario 3. Developer wants certain packages to go to a file at Info and doesn't care about anything else

The developer can create loggers for the desired packages, and add a file appender to the loggers.

<logger name="foo.bar">
  <level value="Info" />
  <appender-ref ref="someFileAppender" />
</logger>
<logger name="foo.baz">
  <level value="Info" />
  <appender-ref ref="someFileAppender" />
</logger>

Scenario 4. Developer wants certain packages to go to a file at Info and certain others to go to Apprenda at Debug

The developer can create loggers for the desired packages, and reference the appropriate appenders.

<logger name="foo.bar">
  <level value="Info" />
  <appender-ref ref="someFileAppender" />
</logger>
<logger name="foo.baz">
  <level value="Debug" />
  <appender-ref ref="apprendaAppender" />
</logger>

Scenario 5a. Developer wants certain packages to go to a file at Info and everything else to go to Apprenda at Debug

In order to get the "certain packages" to go to a file, the developer can create loggers for the desired packages, reference a file appender, and set the additivity to False.

In order to get "everything else" to go to Apprenda at Debug, the root logger can be set to Debug with an Apprenda appender.

<logger name="foo.bar" additivity="false"> <!-- aditivity=false to avoid sending to root logger -->
  <level value="Info" />
  <appender-ref ref="someFileAppender" />
</logger>
<root>
  <level value="Debug" />
  <appender-ref ref="apprendaAppender" />
</root>

Scenario 5b. Developer wants certain packages to go to a file at Debug and everything else to go to Apprenda at Info

In order to get the "certain packages" to go to a file, the developer can create loggers for the desired packages, reference a file appender, and set the additivity to False.

In order to get "everything else" to go to Apprenda at Info, the root logger can be set to Info with an Apprenda appender.

<logger name="foo.bar" additivity="false"> <!-- aditivity=false to avoid sending to root logger, otherwise the override we extracted would result in the message being logged -->
   <level value="Debug" />
   <appender-ref ref="someFileAppender" />
</logger>
<root>
   <level value="Info" />
   <appender-ref ref="apprendaAppender" />
</root> 

Scenario 6a. Developer wants certain packages to go to a file at Info, and everything (including the "certain packages") to Apprenda at Debug

This scenario is a bit more complicated. The file appender needs to be set to drop Debug messages, because the logger for the desired packages must be set to Debug in order for the messages to be sent to Apprenda.

<appender name="infoFileAppender" class="org.apache.log4j.FileAppender">
  <filter class="org.apache.log4j.varia.LevelRangeFilter">
    <param name="LevelMax" value="FATAL"/>
    <param name="LevelMin" value="INFO"/>
    <param name="AcceptOnMatch" value="true"/>
  </filter>
</appender>
 
<logger name="foo.bar" additivity="false">
  <level value="Debug" />
  <appender-ref ref="infoFileAppender" />
  <appender-ref ref="apprendaAppender" />
</logger>
 
<root>
  <level value="Debug" />
  <appender-ref ref="apprendaAppender" />
</root>

Scenario 6b. Developer wants certain packages to go to a file at Debug, and everything (including the "certain packages") to Apprenda at Info

This scenario is also a bit more complicated. The Apprenda appender needs to be set to drop Debug messages, because the logger for the desired packages must be set to Debug in order for the messages to be sent to the file.

<appender name="infoApprendaAppender" class="[apprendaAppenderClassName]">
  <filter class="org.apache.log4j.varia.LevelRangeFilter">
  <param name="LevelMax" value="FATAL"/>
  <param name="LevelMin" value="INFO"/>
  <param name="AcceptOnMatch" value="true"/>
  </filter>
</appender>
<logger name="foo.bar" additivity="false">
  <level value="Debug" />
  <appender-ref ref="someFileAppender" />
  <appender-ref ref="infoApprendaAppender" /> <!-- Need to filter so the message never makes it to the Apprenda appender or the extracted override will apply -->
</logger>
<root>
  <level value="Info" />
  <appender-ref ref="apprendaAppender" />
</root>

Scenario 7. Developer wants everything to go to a file at Info, and certain packages to go to Apprenda at Debug

This is similar to scenario #5, but we flip it around. The developer can create a logger for the "certain packages" at Debug level, and set the Apprenda appender as the appender for the logger.

<appender name="infoFileAppender" class="org.apache.log4j.FileAppender">
  <filter class="org.apache.log4j.varia.LevelRangeFilter">
    <param name="LevelMax" value="FATAL"/>
    <param name="LevelMin" value="INFO"/>
    <param name="AcceptOnMatch" value="true"/>
  </filter>
</appender>
 
<logger name="com.foo">
  <level value="Debug" />
  <appender-ref="apprendaAppender" />
  <appender-ref="infoFileAppender" />
</logger>
 
<root>
  <level value="Info" />
  <appender-ref="someFileAppender" />
</root>

Scenario 8. Developer wants certain packages to log to a file at Info and anything else for a particular tenant to log to Apprenda at Debug

The developer can create loggers for the desired packages and use a file appender to send them to a file. In terms of getting logs for a specific tenant at Debug, the root logger must be configured at Debug level, so that we can apply the tenant override on the appender side.

<logger name="com.foo">
  <level value="Info" />
  <appender-ref ref="fileAppender" />
</logger>
 
<root>
  <level value="Debug" />
  <appender-ref ref="apprendaAppender" />
</root>

Configuration for the Application Server/Container

Initial logging configuration for the Tomcat or JBoss container is configured by the Platform Operator. The Log Override/logger relationship is not, however, restricted to the application, as developer-created Log Overrides will apply to the application container runtime configuration as well. One exception to this is the root log override; this will apply to the application logging configuration only and will not affect the application server/container.