reliure.utils.log

Helper function to setup a basic logger for a reliure app

class reliure.utils.log.ColorFormatter(*args, **kwargs)

Bases: logging.Formatter

BLACK = 0

BLUE = 4

BOLD_SEQ = '\x1b[1m'

COLORS = {'BLUE': 4, 'YELLOW': 3, 'GREEN': 2, 'ERROR': 1, 'DEBUG': 4, 'CYAN': 6, 'WHITE': 7, 'RED': 1, 'INFO': 7, 'WARNING': 3, 'CRITICAL': 3, 'MAGENTA': 5}

COLOR_SEQ = '\x1b[1;%dm'

CYAN = 6

GREEN = 2

MAGENTA = 5

RED = 1

RESET_SEQ = '\x1b[0m'

WHITE = 7

YELLOW = 3

__init__(*args, **kwargs)

format(record)

class reliure.utils.log.SpeedLogger(each=1000, elements='documents')

Bases: reliure.pipeline.Composable

Pipeline element that do nothing but log the procesing speed every K element and at the end.

if you have a processing pipe, for example:

>>> from reliure.pipeline import Composable
>>> pipeline = Composable(lambda data: (x**3 for x in data))

you can add a SpeedLogger in it:

>>> pipeline |= SpeedLogger(each=30000, elements="numbers")

And then when you run your pipeline:

>>> import logging
>>> from reliure.utils.log import get_basic_logger
>>> logger = get_basic_logger(logging.INFO)
>>> from reliure.offline import run
>>> results = run(pipeline, range(100000))

You will get a logging like that:

2014-09-26 15:19:56,139:INFO:reliure.SpeedLogger:Process 30000 numbers in 0.010 sec (2894886.12 numbers/sec)
2014-09-26 15:19:56,148:INFO:reliure.SpeedLogger:Process 30000 numbers in 0.008 sec (3579572.14 numbers/sec)
2014-09-26 15:19:56,156:INFO:reliure.SpeedLogger:Process 30000 numbers in 0.008 sec (3719343.80 numbers/sec)
2014-09-26 15:19:56,159:INFO:reliure.SpeedLogger:Process 9997 numbers in 0.003 sec (3367096.85 numbers/sec)
2014-09-26 15:19:56,159:INFO:reliure.SpeedLogger:In total: 100000 numbers proceded in 0.030 sec (3307106.53 numbers/sec)

__call__(inputs)

__init__(each=1000, elements='documents')

Parameters:

• each – log the speed every each element
• elements – name of the elements in the produced log lines

reliure.utils.log.get_app_logger_color(appname, app_log_level=20, log_level=30, logfile=None)

Configure the logging for an app using reliure (it log’s both the app and reliure lib)

Parameters:

• appname – the name of the application to log
• log_level – log level for the reliure
• logfile – file that store the log, time rotating file (by day), no if None

Parap app_log_level:  

log level for the app

reliure.utils.log.get_basic_logger(level=30, scope='reliure')

return a basic logger that print on stdout msg from reliure lib