Scorer

Implementation of a scorer to calculate precision/recall/f1.

Scorer:__init(gold_log, pred_log)

View source

Constructor.

Arguments:

• gold_log (string): if given, gold labels will be written to this file. Optional.
• pred_log (string]): if given, predicted labels will be written to this file.

Scorer:add_pred(gold, pred, id)

View source

Adds a prediction/ground truth pair to the scorer.

Arguments:

• gold (string): ground truth label.
• pred (string): corresponding predicted label.
• id (string): corresponding identifier for this example

If the scorer was given the gold log and the pred log, then the pair will be written to their respective log files. Optional.

Scorer:reset()

View source

Removes all remembered statistics from the scorer.

Scorer:precision_recall_f1(ignore)

View source

Computes the precision/recall/f1 statistics for the current batch of elements.

Arguments:

• ignore (string): if given, ignore is taken to be the "negative" class and its statistics will be withheld from the computation. Optional.

Returns:

• (table, table, table) micro, macro, and class scores

Example:

local s = Scorer()
s:add_pred('a', 'b', 1)
s:add_pred('b', 'b', 2)
s:add_pred('c', 'a', 3)
local micro, macro, all_stats = s:precision_recall_f1(ignore)

Returns the following

• micro: the micro averaged precision/recall/f1 statistics

• macro: the macro averaged precision/recall/f1 statistics

• class_stats: the precision/recall/f1 for each class