OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	14 / 50 = 0.28
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	18 / 50 = 0.36
Formatter Errors:
	[]
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 14 / 14 = 1.0
	HARD: 0 / 10 = 0.0
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           8 / 23 = 0.3478
Single-Span-Multi-Hop-Extraction 9 / 23 = 0.3913
Multi-Spans-Extraction           0 / 9 = 0.0000
YesNo                            4 / 7 = 0.5714
Arithmetic-Operations            0 / 0 = NaN
Counting                         0 / 0 = NaN
Date-Duration                    0 / 7 = 0.0000
Kinship                          1 / 4 = 0.2500
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           0 / 27 = 0.0000
Single-Span-Multi-Hop-Extraction 0 / 27 = 0.0000
Multi-Spans-Extraction           3 / 41 = 0.0732
YesNo                            0 / 43 = 0.0000
Arithmetic-Operations            2 / 50 = 0.0400
Counting                         0 / 50 = 0.0000
Date-Duration                    9 / 43 = 0.2093
Kinship                          0 / 46 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERROR SAMPLES (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D004Q05', 'D035Q06', 'D042Q07', 'D042Q12', 'D046Q07', 'D071Q11', 'D085Q02', 'D096Q03', 'D097Q09', 'D100Q02', 'D104Q01', 'D120Q03', 'D138Q09', 'D241Q04', 'D314Q03']

Single-Span-Multi-Hop-Extraction:
['D035Q06', 'D042Q12', 'D046Q07', 'D071Q11', 'D085Q02', 'D090Q01', 'D096Q03', 'D097Q03', 'D100Q02', 'D104Q01', 'D138Q09', 'D160Q04', 'D241Q04', 'D314Q03']

Multi-Spans-Extraction:
['D007Q01', 'D008Q06', 'D014Q03', 'D039Q03', 'D045Q02', 'D072Q07', 'D094Q04', 'D107Q01', 'D318Q04']

YesNo:
['D113Q04', 'D117Q08', 'D127Q05']

Arithmetic-Operations:
[]

Counting:
[]

Date-Duration:
['D035Q07', 'D042Q11', 'D082Q01', 'D104Q03', 'D215Q05', 'D243Q07', 'D311Q08']

Kinship:
['D040Q14', 'D040Q15', 'D091Q11']

--------------------------------------------------------------------------------