Save Logs to File Using Python logging.FileHandler() – Python Tutorial

It is easy to save logs to a file using python logging package. Here are some tutorials:

Save Python Message into a Log File with logging

Fix Python logging module not writing to file – Python Tutorial

However, we can find:

We can add a filename by logging.basicConfig() to implement it.

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
                    datefmt='%a, %d %b %Y %H:%M:%S',
                    filename='/tmp/test.log',
                    filemode='w')

Notice: here filemode = “w”, if you want to append logs to a file, you should set filemode = “a”

In this tutorial, we will introduce a new way: using logging.FileHandler() to save logs to a file.

Here is an example code.

import logging
import argparse
import torch

parser = argparse.ArgumentParser(description='main program')
parser.add_argument('--disable_log', action="store_true", help='Disable logging.')
parser.add_argument('--log_file', type=str, default="log.txt", help="Save log to specific file.")

args = parser.parse_args()
#--------------
log_handlers = [logging.StreamHandler()]
if args.log_file is not None:
    log_handlers.append(logging.FileHandler(args.log_file))

logging.basicConfig(format='[%(asctime)s] running: %(message)s',
                    level=logging.WARNING if args.disable_log else logging.INFO,
                    handlers=log_handlers)
logging.info('start to run...')
if torch.cuda.is_available():
    logging.info('NVIDIA CUDA initialized successfully.')
    logging.info('Total %i GPU(s) detected.' % torch.cuda.device_count())
else:
    logging.error(
        'app requires NVIDIA CUDA computing capacity. Please check your PyTorch installation.')
    exit(-1)

Run this code, you will see:

As to logging.FileHandler(), we can find:

The default mode = “a”, which means logs will be appended.

In python 3.8, force parameter is added in logging.basicConfig(), if force== True, any existing handlers attached to the root logger are removed and closed.

        force = kwargs.pop('force', False)
        if force:
            for h in root.handlers[:]:
                root.removeHandler(h)
                h.close()

We can use it as follows:

logging.basicConfig(format='[%(asctime)s] m-LoRA: %(message)s',
                        level=logging.WARNING if args.disable_log else logging.INFO,
                        handlers=log_handlers,
                        force=True)