Lazy Logging
When writing programs, oftentimes during development/production there are lots of logging. We may see a common debug log such as the following. Expensive function may be some form of large information retrieval or I/O process.
import logging
...
logging.debug(f"using {expensive_func()} currently")
...
However, in the case we are using the default log level of INFO in which the log message is not actually output, we are wastefully doing two things:
- Rendering a log message, and for some objects calling their
__str__/__repr__is expensive. expensive_funcis called when not needed.
To address problem 1, we can call logging methods with a string literal instead of an f-string, with the first argument and the pattern-parameters as argument. This prevents f-string rendering and prevents formatting until needed.
import logging
...
logging.debug("using %d currently", expensive_func())
...
To address problem 2, we can use the isEnabledFor() method which
takes the level argument and only returns True if the event would be created
by the Logger for that specific level of call. This prevents the calling of
the expensive function unless absolutely needed.
import logging
...
if logger.isEnabledFor(logging.DEBUG):
logging.debug("using %d currently", expensive_func())
...
This means that unless the threshold is set above DEBUG level, the call to expensive_func()
is never made.
If this is consistently called, you can use LoggerAdapter
instead. This allows you to avoid clutter by removing the consistent need for
isEnabledFor().
from logging import LoggerAdapter
class LazyLoggerAdapter(LoggerAdapter):
def lazy_log(self, level, msg_or_func, *args, **kwargs):
# Only call the function if we're actually going to log
if self.isEnabledFor(level):
if callable(msg_or_func):
message = msg_or_func(*args, **kwargs) # Function called HERE
else:
message = msg_or_func
self.log(level, message)
...
lazy_logger.lazy_log(logging.DEBUG, expensive_operation) # Function not called!
lazy_logger.lazy_log(logging.INFO, expensive_operation) # Function IS called!
Now this will only operate if the DEBUG logging is enabled.
Questions or feedback? Feel free to reach out!