Docker Logs

Where are Docker logs?

When it comes to Docker logs, you either want to inspect your container logs or the logs for the Docker daemon.

Docker Container logs

On Docker, container logs can either be inspected by using the “logs” command or they can be stored on an external system (like Logstash or syslog) in order to be analyzed later on.

When they are sent to an external system, you will need to have a logging driver installed for Docker to send its container logs.

Some particularities may exist depending on the image used.

One good example is the NGINX image actually redirecting access and error logs to some specific devices on your Linux system (/dev/stdout and /dev/stderr).

Similarly, the Apache Web Server will redirect the logs directly to specific file descriptors (/proc/self/fd/1) on your system.

However, in most of the cases, using the “logs” command is enough.

Docker Daemon Logs

On the other hand, the Docker daemon is also producing some logs for you to inspect.

Depending on the system you are running Docker on, you will find the Docker daemon logs in different places :

• /var/log/daemon.log : for Debian distributions;
• /var/log/messages : for RHEL and Oracle Linux;
• journalctl -u docker.service : for Ubuntu 16.04+ and CentOS 7/8
• /var/log/upstart/docker.log : for Ubuntu distributions still using upstart;
• AppData/Local : for Windows operating systems;

As a small example, if you installed Docker on Ubuntu or Debian, you would run this command to inspect your Docker daemon logs.

$ tail -f -n 1000 /var/log/daemon.log | grep --color docker

Alternatively, if you are using systemd to manage your services, you can inspect Docker daemon logs with the journalctl command.

$ sudo journalctl -u docker.service | less

View Docker Logs using the logs option

In order to view and inspect logs on Docker, you have to use the “docker logs” command with custom options.

$ docker logs <container_id>

$ docker logs <container_name>

For example, in order to see the Docker logs from a Grafana server started from a Docker image, you would run

$ docker logs grafana

View Docker logs since a specific date

When you are inspecting your Docker logs, you often want to limit the output to a given number of lines, not to be flooded with information.

In order to view Docker logs since a specific date or duration, use the “–since” option with a date or a duration.

For example, to see Docker container logs since 10 minutes, you would write

$ docker logs --since 10m <container_id>

You can also write a date format as long as it is provided in ISO format

$ docker logs --since 2019-10-20T10:00:00 <container_id>

Note that the logs will be shown given the current date of your system and not the date of the Docker logs.

View Docker logs until a specific date

Similarly to the since option, you can choose to inspect your container logs until a specific date.

You can also specify absolute dates (like an ISO date for example) or relative like 10 minutes, 30 minutes or 50 minutes.

To view logs until a specific date, use the “–until” option with a date or a duration.

$ docker logs --until 10m <container_id>

You can also provide a date format like you did before for the since option.

$ docker logs --until 2019-10-20T10:00:00 <container_id>

Tail Docker Logs using the tail option

In some cases, you want to restrict the number of lines printed on your screen from the Docker logs.

In order to achieve this result, you will have to use the “–tail” option in the following way.

$ docker logs --tail <number> <container_id|name>

For example, in order to show 100 lines from your Grafana for Docker logs, you will have to write

$ docker logs --tail 100 grafana

$ docker logs --tail 100 4921d714d338

Exporting Docker Logs using logging drivers

In some cases, you may want to export your Docker logs in order to analyze them later on.

For simple debugging needs, having the logs printed to stdout or stderr is enough.

However, if you plan on running a more complex analysis, you will need to have them stored in logging systems such as Syslog or Logstash.

This is where the concept of logging drivers comes into play.

What are logging drivers?

In Docker, logging drivers are a set of custom plugins that one can activate or install in order to export logs to an external tool such as syslog, Logstash or custom datasources.

For example, you can choose to export your Docker logs to :

• syslog : Docker will write its logs to the syslog daemon. Later on those logs can be centralized and analyzed in an external system such as Kibana;
• journald : on modern distributions, journald is the default logging option. As a consequence, Docker can export logs to it;
• gelf : for administrators using Graylog and the GELF (Graylog Extended Log Format) format;
• awslogs : in order to export logs to Amazon CloudWatch Logs;
• none : if you choose not to have logs for your containers;
• etwlogs : used to write logs to the Event Tracing for Windows;
• fluentd : in order to write Docker logs to the fluentd daemon;
• local : in order to store logs in a minimal format.

Now that you have an idea about logging drivers, you might be wondering what logging driver you are currently using on your system.

To find the default logging driver used in Docker, use the “docker info” command.

$ docker info | grep Logging
  Logging Driver: json-file

In most of the cases, the default logging driver is “json-file” unless you already modified this parameter before.

Defining a default logging driver

In order to define a default logging driver on Linux, you will need to overwrite the log-driver option located in your daemon.log configuration file.

The daemon.log file is located in the /etc/docker configuration folder on your system.

If the “daemon.log” is not existing, make sure to create it and paste the following content in it.

{
  "log-driver": "syslog"
}

Those default settings will be applied whenever you create a new container.

However, you can also create containers with “docker run” and specify the logging driver to be used instead of the default one.

Save your file and restart your Docker service for the changes to be applied.

$ sudo systemctl restart docker
$ sudo systemctl status docker

Finally, run the “docker info” command again in order to check if your logging preferences were changed.

$ docker info | grep Logging
  Logging Driver: syslog

In order to inspect the logs, you will have to navigate to the default log location on Linux : /var/log

By default, your Docker daemon will send its logs to the daemon.log in /var/log.

In order to see the logs associated to the Docker daemon, you can run the following command

$ sudo tail -f daemon.log | grep docker

However, note that defining the default logging driver does not imply that your containers will automatically send their logs to the source you specified.

Redirecting container logs to syslog

In order to make sure that logs are correctly redirected, you have to inspect your containers and the logging driver associated to them.

$ docker inspect <container_id|container_name> | grep -A 5 LogConfig

"LogConfig": {
  "Type": "syslog"
  "Config": {}
}

As you can see, the logging driver is defined to syslog, but what if I used an old container using another logging driver?

To redirect container logs to syslog, you have two options :

• Creating another container, making sure that you defined the logging driver in the previous section
• Run “docker run” and specify the logging driver that you want to use.

To define the logging driver for a container, you have to use “docker run” with the “–log-driver” option.

$ docker container run -it -d --log-driver syslog <image>

Then, make sure that your logging driver was correctly set by running the inspect command again.

$ docker inspect <container_id|container_name> | grep -A 5 LogConfig

In order to view Docker logs, you have to inspect the “daemon.log” file and search for your container id.

$ sudo tail -f -n1000 daemon.log | grep <container_id>

Redirecting container logs to journald

On modern distributions, there is also a way to inspect system logs : by using the journald utility from the systemd environment.

Some administrators may prefer to have syslog in order to implement a centralized logging system for example.

Others may choose to have journald because of the handy commands it exposes.

In order to redirect container logs to journald, you can either define it in the daemon.log configuration or you can specify it at runtime with the “–log-driver” option.

$ docker container run -it -d --log-driver journald <image>

Next, verify that your logging driver was correctly set for the container.

$ docker inspect <container_id|container_name> | grep -A 5 LogConfig

"LogConfig": {
  "Type": "journald"
  "Config": {}
}

In order to inspect logs sent to journald, you have to run the following command

$ sudo journalctl CONTAINER_NAME=<container_name>

For example, for a container named “mycontainer” you would run

$ sudo journalctl CONTAINER_NAME=mycontainer

As you can see, this is not very handy as data shown in the journal is not fully displayed.

To avoid having “blob data” presented when running journalctl, append the “–all” option to your command.

$ sudo journalctl --all CONTAINER_NAME=mycontainer

Great!

You successfully exporter your Docker container logs to journald.

View Docker Compose Logs

In order to inspect Docker Compose logs, you can also run a simple command to have them shown on the standard output.

$ docker-compose logs

You can also choose to have the logs displayed for one service and not for your entire docker compose stack.

$ docker-compose logs <app>

Note that the options detailed before are also available using docker-compose.

For example, if you want to have only 100 lines of logs written for your service in docker-compose, you would write

$ docker-compose logs --tail 100 <app>

reference: https://devconnected.com/docker-logs-complete-guide/