OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	18 / 50 = 0.36
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	18 / 50 = 0.36
Formatter Errors:
	[]
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 18 / 18 = 1.0
	HARD: 0 / 12 = 0.0
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           10 / 25 = 0.4000
Single-Span-Multi-Hop-Extraction 11 / 25 = 0.4400
Multi-Spans-Extraction           0 / 7 = 0.0000
YesNo                            2 / 4 = 0.5000
Arithmetic-Operations            0 / 1 = 0.0000
Counting                         0 / 3 = 0.0000
Date-Duration                    0 / 4 = 0.0000
Kinship                          5 / 6 = 0.8333
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           0 / 25 = 0.0000
Single-Span-Multi-Hop-Extraction 0 / 25 = 0.0000
Multi-Spans-Extraction           2 / 43 = 0.0465
YesNo                            0 / 46 = 0.0000
Arithmetic-Operations            0 / 49 = 0.0000
Counting                         0 / 47 = 0.0000
Date-Duration                    10 / 46 = 0.2174
Kinship                          0 / 44 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERROR SAMPLES (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D001Q09', 'D033Q04', 'D033Q05', 'D042Q05', 'D064Q04', 'D087Q07', 'D090Q01', 'D116Q03', 'D179Q06', 'D241Q04', 'D245Q01', 'D247Q01', 'D247Q02', 'D247Q05', 'D312Q05']

Single-Span-Multi-Hop-Extraction:
['D033Q05', 'D042Q05', 'D087Q07', 'D090Q01', 'D116Q03', 'D179Q06', 'D241Q04', 'D244Q04', 'D245Q01', 'D247Q01', 'D247Q02', 'D247Q05', 'D312Q05', 'D312Q07']

Multi-Spans-Extraction:
['D008Q06', 'D010Q06', 'D037Q05', 'D088Q06', 'D105Q05', 'D215Q07', 'D318Q04']

YesNo:
['D087Q05', 'D106Q05']

Arithmetic-Operations:
['D248Q04']

Counting:
['D015Q06', 'D069Q13', 'D117Q01']

Date-Duration:
['D076Q10', 'D105Q02', 'D310Q05', 'D311Q07']

Kinship:
['D078Q07']

--------------------------------------------------------------------------------