Pipe Log Output to STDOUT with Docker

Mark Shust
InstructorMark Shust

Share this video with your friends

Send Tweet
Published 4 years ago
Updated 3 years ago

Since Docker filesystems are ephemeral, it's important to take into account anything that writes or appends to disk, such as log files. It is standard practice when running Docker containers to pipe log file output to the console, to allow for simple diagnostic and to avoid append-only files from consuming disk space.

In this lesson, we’ll show you how to append output to STDOUT, and how to view log output from your running containers.

Instructor: [00:00] Anything written to append-only files within containers should be piped to standard out. This makes it easy to diagnose log files, and ensure that append-only files don't consume disk space.

[00:14] Some widely used Docker images such as Nginx already write to standard out. However, sometimes you have an application-specific log that you want to apply the same methodology to.

[00:25] Say we have a simple app that overrides the console.log function of Node. Console.log already writes to standard out, so this is a really nonsensical example, but we will use this to get our point across on how to redirect writes from log files to standard out.

[00:43] In this case, we will write all calls to console.log to a file named debug.log. If we run this script for a few moments and then check the output of debug.log, we will see that the script is running and correctly writing to debug.log.

[01:01] There's a pretty simple way we can accomplish this change within our Docker file. We will add a new instruction before our command instruction that is a run.

[01:11] What we will do here is use ln -sf to create a symlink to /dev/standardout from our debug.log file. This simple line will take whatever would normally be written to our debug.log file and pipe it right to standard out.

[01:29] Let's build our image, and then run it in Daemon mode. If we use Docker logs with a -f function to tail the output of our container, we can see that the output normally written to our debug.log file is definitely piping to standard out.

[01:56] We can also confirm that no output is further writing to debug.log within our container, so we don't have to worry about this container accumulating disk space over time.

Wojciech Morkowski
Wojciech Morkowski
~ 3 years ago

I dont't get this lesson. What's the purpose of creating debug.log file if we never write anything to it (at least in Docker environment)?

Mark Shust
Mark Shustinstructor
~ 3 years ago

As noted in the video, this is a non-sensical example. But it demonstrates how you would change an app that normally writes to log files so that they instead pipe to STDOUT, so you can make the app 12-factor friendly.

You don’t need to create a log file if you are already piping things to STDOUT. Note that in this example, we are writing the console.log output directly to the debug.log file.

Wojciech Morkowski
Wojciech Morkowski
~ 3 years ago

Ok thanks, I get it. You can also find great explanation here: https://stackoverflow.com/questions/43968560/how-to-log-the-12-factor-application-way/43968637#43968637

Dr.Emmett Brown
Dr.Emmett Brown
~ 3 years ago

it's good tip. Thank you.

Nehal Soni
Nehal Soni
~ 3 years ago

I can see the value in keeping the logging agnostic. How would you do this with statsd logging? This often requires a client side package with host name and port number. Would you pass those variables into the docker container? Or can you do a similar thing you do here with standard logging for stats?