Dazl is not just another Go logging framework. We're not here to reinvent Go logging for the nth time. Dazl is an abstraction layer that provides a unified interface and configuration format on top of existing logging frameworks, using a proven approach adapted from popular frameworks in other languages like slf4j.
Dazl is a pluggable logging abstraction with support for multiple existing backend frameworks:
Dazl loggers add numerous features on top of existing frameworks:
- Decouples Go libraries from specific logging implementations
- Makes logging configurable via YAML configuration files
- Structured logging with support for JSON or console encoding and user-defined fields
- Hierarchical loggers, inheritance, sampling and other advanced features
- Supports runtime configuration changes for easy debugging
- Use cases
- Getting started
- Go API
- Configuration files
- Runtime configuration changes
- Custom logging frameworks
There are numerous benefits to using a logging abstraction, but dazl is designed to serve a couple of important use cases for two types of applications in particular.
Go libraries designed to be imported and used by other Go modules can use dazl to avoid adding a dependency on a specific logging framework, tying their users to the same framework. Additionally, dazl enables your users to configure loggers and log formats indepenedently, with no added work for you. The users of your library ought to be able to select their own logging framework and configure the format and severity of log outputs. Simply add a dependency on the dazl logger, and leave it up to your users to import whichever logging backend they desire.
One of the most common use cases for Go applications is in cloud applications deployed in containers and on platforms like Kubernetes. Most Go logging frameworks provide programmatic APIs for configuring loggers, levels, formats, and other logging options. Using dazl in Go-based services enables your users to configure logging independent of the code, eliminating the need to recompile code to modify the verbosity or format of application logs.
To start using dazl, first add the framework to your go.mod
:
go get -u github.com/atomix/dazl
Additionally, add the dependency for one of the logging backendsL
go get -u github.com/atomix/dazl/zap
Import the logging backend in your module main
:
main.go
package main
import (
"github.com/atomix/dazl"
_ "github.com/atomix/dazl/zap"
)
var log = dazl.GetLogger()
func main() {
log.Info("Hello world!")
}
To avoid adding unnecessary dependencies on logging frameworks, logging backends should only be imported within
your module main
.
Create a logging.yaml
file to configure the logging framework:
logging.yaml
encoders:
json:
fields:
- message
- level:
format: uppercase
- caller:
format: short
- timestamp:
format: iso8601
writers:
stdout:
encoder: json
rootLogger:
level: info
outputs:
- stdout
To print logs to an output, the logging.yaml
configuration must configure one or more writers
and configure the
rootLogger
or child loggers
with outputs
to output messages from those loggers to the configured writer(s).
Finally, when you run the application main
from the same directory as your logging.yaml
file, you should see
the logs printed to stdout:
go run ./main.go
For more information check out the example module.
The dazl logger does not log to any output unless a logging framework is imported. To maintain independence from any
particular logging backend, applications should only import a specific logging framework from within a main
file.
Libraries designed to be imported by other projects should never import a logging backend themselves. Instead, leave
the specific logging framework implementation up to your users.
The logging backend is configured by importing the framework into your application's main
package.
To configure dazl to use the zap logging backend, add the zap
framework
to your module's go.mod
:
go get -u github.com/atomix/dazl/zap
Then import the github.com/atomix/dazl/zap
framework implementation in your main
package:
package main
import _ "github.com/atomix/dazl/zap"
func main() {
...
}
To configure dazl to use the zerolog logging backend, add the zerolog
framework
to your module's go.mod
:
go get -u github.com/atomix/dazl/zerolog
Then import the github.com/atomix/dazl/zerolog
framework implementation in your main
package:
package main
import _ "github.com/atomix/dazl/zerolog"
func main() {
...
}
The typical usage of the framework is to create a Logger
once at the top of each Go package:
var log = dazl.GetLogger()
Package loggers are assigned the name of the package calling GetLogger()
or GetPackageLogger()
. So, if you
call dazl.GetLogger()
from the github.com/atomix/atomix/runtime
package, the logger will be assigned the
name github.com/atomix/atomix/runtime
. The naming strategy becomes important for
logger configuration and, in particular, inheritance.
const author = "kuujo"
var log = dazl.GetLogger()
func main() {
log.Infof("The author of dazl is %s", author)
}
2023-03-31T01:13:30.607Z INFO main.go:12 The author of dazl is kuujo
Alternatively, custom loggers can be retrieved via GetLogger
:
var log = dazl.GetLogger("foo/bar")
Logger names must be formatted in path format, with each element separated by a /
. This format is used
to establish a hierarchy for inheritence of logger configurations.
All loggers are descendants of the root logger:
var log = dazl.GetRootLogger()
Dazl supports a fairly standard set of log levels for loggers:
debug
info
warn
error
fatal
panic
The levels for each logger can be configured individually via their configuration:
loggers:
github.com/atomix/atomix/runtime:
level: warn
The Logger
interface exposes methods for simple logging, formatted logging, and structured logging with typed
fields for each log level:
Debug(args ...any)
Debugf(msg string, args ...any)
Debugw(msg string, fields ...Field)
Info(args ...any)
Infof(msg string, args ...any)
Infow(msg string, fields ...Field)
Warn(args ...any)
Warnf(msg string, args ...any)
Warnw(msg string, fields ...Field)
- ...
Messages will only be written to log outputs if the configured level of the logger is higher than the message level.
Structured logging is supported for the JSON encoding, and JSON fields are configurable via
the Logger
API.
The simplest way to add fields to your structured logs is to call one of the *w
methods on the Logger
interface.
These methods accept an arbitrary number of ...Field
s to write to the logs. Fields are typed and named values
that can be constructed via functions in the dazl
package:
log.Warnw("Something went wrong!",
dazl.String("user", user.Name),
dazl.Uint64("user-id", user.ID))
Alternatively, you can create a structured logger with a fixed set of fields using the WithFields
method:
var log = dazl.GetLogger().WithFields(
dazl.String("user", user.Name),
dazl.Uint64("id", user.ID))
log.Warn("Something went wrong!")
When the logger is output to a JSON encoded writer, the above code will log the fields as part of the JSON object:
{"timestamp":"2023-04-07T19:24:09-07:00","logger":"2/4","message":"Something went wrong!","user":"Jordan Halterman","id":5678}
Loggers can be configured via a YAML configuration file. The configuration files may be in one of many locations on the file system:
logging.yaml
~/logging.yaml
/etc/dazl/logging.yaml
The configuration file contains a set of loggers
which specifies the level and outputs of each logger,
writers
which specify where to write log messages, and encoders
defining how to encode log messages.
The encoders
section of the configuration defines how dazl encodes log messages. Dazl supports two
encodings: json
and console
. Each encoder defines the set of fields
to output and optionally the
format of the fields.
encoders:
json:
fields:
...
console:
fields:
...
Encoders are referenced by writers
and used to encode messages.
Note that support for some encoding options such as renaming keys or formatting fields depends on whether those features are supported the underlying logging framework. If some requested features are not supported by the imported logging framework, dazl may panic at startup.
The json
encoder configuration defines the set of fields to include in all JSON formatted messages. Each
JSON field also supports an additional key
to override the default JSON key for that field:
encoders:
json:
# A list of fields to include
fields:
# The log message
- message:
# The JSON key for the field
key: msg
# The logger name
- name:
# The JSON key for the field
key: logger
# The log level
- level:
# The JSON key for the field
key: level
# The level format: 'uppercase' or 'lowercase'
format: uppercase
# The time at which the message was logged
- timestamp:
# The JSON key for the field
key: time
# The time format: 'iso8601' or 'unix'
format: iso8601
# The line of code at which the message was logged
- caller:
# The JSON key for the field
key: caller
# The caller format: 'short' or 'long'
format: short
# The log stacktrace
- stacktrace:
# The JSON key for the field
key: trace
The console
encoder configuration defines the colums to encode with each log message:
encoders:
console:
# A list of fields to include
fields:
# The log message
- message
# The logger name
- name
# The log level
- level:
# The level format: 'uppercase' or 'lowercase'
format: uppercase
# The time at which the message was logged
- timestamp:
# The time format: 'iso8601' or 'unix'
format: iso8601
# The line of code at which the message was logged
- caller:
# The caller format: 'short' or 'long'
format: short
# The log stacktrace
- stacktrace
Both the json
and console
encoders support the following set of fields:
message
name
level
timestamp
caller
stacktrace
If any field is excluded from the encoder's field list, dazl will attempt to exclude that field from the log output, but some logging backends may not support this ability and therefore may include those fields in their output.
The format of the log level can be configured for each encoder via the format
key:
encoders:
json:
level:
format: uppercase
Defined level formats include:
uppercase
- upper case level namelowercase
- lower case level name
Note that support for level formats depends on support from the imported logging backend. Dazl may panic at startup if the underlying logging framework does not support the configured level format.
The format of the timestamp can be configured for each encoder via the format
key:
encoders:
json:
timestamp:
format: ISO8601
Defined timestamp formats include:
ISO8601
unix
Note that support for timestamp formats depends on support from the imported logging backend. Dazl may panic at startup if the underlying logging framework does not support the configured timestamp format.
The format of the caller can be configured for each encoder via the format
key:
encoders:
json:
caller:
format: uppercase
Defined caller formats include:
short
full
Note that support for caller formats depends on support from the imported logging backend. Dazl may panic at startup if the underlying logging framework does not support the configured level format.
# A set of named writers for loggers to write to.
# All writers must specify an 'encoder' to use
writers:
# The stdout writer
stdout:
# The name of the encoder to use for the writer
encoder: console
# The stderr writer
stderr:
# The name of the encoder to use for the writer
encoder: console
# Remaining writers are files
file:
# The path to the file
path: ./example.log
# The name of the encoder to use for the writer
encoder: json
The default logging configuration is configured via the rootLogger
key:
rootLogger:
level: info
All loggers inherit their default configuration from the root logger. The root logger configuration should at
least specify a minimum log level
for all loggers, and at least one outputs
to a writer.
rootLogger:
level: info
outputs:
- stdout
Once the rootLogger
has been defined, all other loggers that are descendants of the root logger may be defined and
configured in the loggers
section of the configuration file:
loggers:
github.com/atomix/atomix/runtime:
level: info
github.com/atomix/atomix/sidecar:
level: warn
Once you've defined the set of writers to which to write your application logs, named loggers can be directed to
those writers via their configured outputs
:
rootLogger:
level: info
outputs:
- stdout
Each output must specify a writer
to write to. As with log levels, loggers inherit the outputs of their ancestors, so
with an output to the stdout
writer in the root logger configuration, all loggers will have an output to stdout
.
In some cases, you may want to restrict the verbosity of logs to one output without restricting the verbosity of all
messages for a logger. For example, you may want to write info
and higher messages to the console for
human readability, and debug
and higher messages to a file for later debugging. Each output supports a level
that can be used to filter the logs to that output:
loggers:
github.com/atomix/atomix:
# Pass through all messages over 'debug' level to the outputs
level: debug
outputs:
# Limit the messages from this logger to stdout to 'info' level
- stdout:
level: info
# Pass through all messages to the 'file' writer
- file
writers:
stdout:
encoder: console
file:
path: ./app.log
encoder: json
Descendants may override their ancestor loggers' output configurations. This can be done by simply specifying the
same output name. For example, to override the log level for the rootLogger
's stdout
output:
# Configure the stdout writer to use console encoding
writers:
stdout:
encoder: console
rootLogger:
level: info
outputs:
# Output the root logger to stdout
- stdout
loggers:
github.com/atomix/atomix/runtime:
outputs:
# Limit outputs from this logger to stdout to minimum of 'warn' level
stdout:
level: warn
Samplers can be added to either loggers to reduce the number of messages logged:
loggers:
github.com/atomix/atomix/runtime:
sample:
basic:
interval: 10
Samplers can also be added to individual outputs to limit only the messages to that particular output:
loggers:
github.com/atomix/atomix/runtime:
outputs:
stdout:
sample:
basic:
interval: 10
The basic
sampler logs every nth message below maxLevel
according to the configured interval
:
loggers:
github.com/atomix/atomix/runtime:
sample:
basic:
interval: 10
maxLevel: debug
The random
sampler randomly logs messages below maxLevel
by choosing a random integer between 0
and interval
:
loggers:
github.com/atomix/atomix/runtime:
sample:
random:
interval: 10
maxLevel: debug
The path-like format used for logger names is used to establish a hierarchy of loggers. The dazl configuration enables developers and their users to configure individual loggers at runtime. Log levels are inherited by descendants of a logger. This enables users to easily enable logging for specific Go packages, their subpackages, or entire Go modules with a single configuration change:
# Enable debug logging for all Atomix code
loggers:
github.com/atomix:
level: debug
You can set the dazl.Level
for a logger at startup time via configuration files or at runtime
via the Logger
API to control the granularity of a logger's output:
dazl.GetLogger("github.com/atomix").SetLevel(dazl.DebugLevel)
The root logger is the ancestor of all other loggers and can be configured via GetRootLogger
:
dazl.GetRootLogger().SetLevel(dazl.InfoLevel)
If the level for a logger is not explicitly set, it will inherit its level from its nearest ancestor in
the logger hierarchy. For example, setting the github.com/atomix/atomix/runtime
logger to the debug
level will change the loggers for all loggers in the github.com/atomix/atomix/runtime/...
packages
to the debug
level.
A reference configuration file detailing and documenting all the available configuration
options in logging.yaml
is available in the examples directory of this repo.
You can set the dazl.Level
for a logger at startup time via configuration files or at runtime
via the Logger
API to control the granularity of a logger's output:
dazl.GetLogger("github.com/atomix").SetLevel(dazl.DebugLevel)
The root logger is the ancestor of all other loggers and can be configured via GetRootLogger
:
dazl.GetRootLogger().SetLevel(dazl.InfoLevel)
Dazl provides several existing implementations of logging frameworks:
Logging frameworks are implemented by implementing the Framework
interface:
const name = "example"
type Framework struct{}
func (f Framework) Name() string {
return name
}
Frameworks should register when imported:
func init() {
dazl.Register(&Framework{})
}
Framework
implementations should implement one or more of the *EncodingFramework
interfaces to indicate support
for encoding formats. Frameworks implement these interfaces to provide Encoder
s which are used by dazl to create new
Writer
s:
type JSONEncoder struct{}
func (e *JSONEncoder) NewWriter(writer io.Writer) (dazl.Writer, error) {
...
}
To implement support for JSON encoding, implement the JSONEncodingFramework
interface:
func (f Framework) JSONEncoder() Encoder {
return &JSONEncoder{}
}
To implement support for JSON encoding, implement the ConsoleEncodingFramework
interface:
func (f Framework) ConsoleEncoder() Encoder {
return &ConsoleEncoder{}
}
Encoder
implementations may support configuration options by implementing optional interfaces with the
following methods:
WithMessageKey(key string) (dazl.Encoder, error)
WithNameEnabled() (dazl.Encoder, error)
WithNameKey(key string) (dazl.Encoder, error)
WithLevelEnabled() (dazl.Encoder, error)
WithLevelKey(key string) (dazl.Encoder, error)
WithLevelFormat(format dazl.LevelFormat) (dazl.Encoder, error)
WithTimestampEnabled() (dazl.Encoder, error)
WithTimestampKey(key string) (dazl.Encoder, error)
WithTimestampFormat(format dazl.TimestampFormat) (dazl.Encoder, error)
WithCallerEnabled() (dazl.Encoder, error)
WithCallerKey(key string) (dazl.Encoder, error)
WithCallerFormat(format dazl.CallerFormat) (dazl.Encoder, error)
WithStacktraceEnabled() (dazl.Encoder, error)
WithStacktraceKey(key string) (dazl.Encoder, error)
Encoder
s create Writer
s for use by the dazl Logger
. All Writer
s must implement the following methods:
WithName(name string) dazl.Writer
WithSkipCalls(calls int) dazl.Writer
Debug(msg string)
Info(msg string)
Warn(msg string)
Error(msg string)
Panic(msg string)
Fatal(msg string)
Writer
implementations may optionally support additional features by implementing optional interfaces
by adding the following methods:
WithStringField(name string, value string) dazl.Writer
WithIntField(name string, value int) dazl.Writer
WithInt32Field(name string, value int32) dazl.Writer
WithInt64Field(name string, value int64) dazl.Writer
WithUintField(name string, value uint) dazl.Writer
WithUint32Field(name string, value uint32) dazl.Writer
WithUint64Field(name string, value uint64) dazl.Writer
WithFloat32Field(name string, value float32) dazl.Writer
WithFloat64Field(name string, value float64) dazl.Writer
WithBoolField(name string, value bool) dazl.Writer
WithStringSliceField(name string, values []string) dazl.Writer
WithIntSliceField(name string, values []int) dazl.Writer
WithInt32SliceField(name string, values []int32) dazl.Writer
WithInt64SliceField(name string, values []int64) dazl.Writer
WithUintSliceField(name string, values []uint) dazl.Writer
WithUint32SliceField(name string, values []uint32) dazl.Writer
WithUint64SliceField(name string, values []uint64) dazl.Writer
WithFloat32SliceField(name string, values []float32) dazl.Writer
WithFloat64SliceField(name string, values []float64) dazl.Writer
WithBoolSliceField(name string, values []bool) dazl.Writer
WithErrorField(name string, err error) dazl.Writer