OVERALL ACC: 459 / 752 = 0.6103723404255319 -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER MODE TOP-1): GRND/PRED SS MS YN ARI CNT CMP CS DD KS Single-Span-Extraction 492 4 0 0 0 0 0 0 0 Multi-Spans-Extraction 0 72 0 0 0 0 0 0 0 YesNo 0 0 71 0 0 0 0 0 0 Arithmetic-Operations 0 0 0 6 0 0 0 0 0 Counting 0 0 0 0 11 0 0 0 0 Comparing-Members 0 0 0 0 0 6 0 0 0 CommonSense 0 0 0 0 0 0 0 0 0 Date-Duration 1 0 0 0 0 0 0 25 0 Kinship 1 2 0 0 0 0 0 0 61 -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER TYPE): GRND/PRED PER D-D LOC ORG NUM YN KIN EVT OBJ MISC Person 78 0 0 0 1 0 0 0 0 0 Date-Duration 0 110 0 0 0 0 0 0 0 0 Location 0 0 110 0 0 0 0 0 0 0 Organization 0 0 0 80 0 0 0 0 0 1 Num-Measure 0 0 0 0 66 0 0 0 0 0 YesNo 0 0 0 0 0 71 0 0 0 0 Kinship 0 0 0 0 0 0 63 1 1 1 Event 1 0 0 0 0 0 0 12 0 1 Object 0 0 0 0 0 0 0 0 138 2 Misc 0 0 0 0 0 0 0 0 0 15 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED): ACC SCORE MEAN SCORE STD-VAR Single-Span-Extraction 0.6431 0.8920 0.1818 Single-Span-Multi-Hop-Extraction 0.6734 0.8979 0.1714 Multi-Spans-Extraction 0.0000 0.0000 0.0000 YesNo 0.5352 0.9402 0.1089 Arithmetic-Operations 0.3333 0.7443 0.2885 Counting 0.0000 0.0000 0.0000 Date-Duration 0.2692 0.8558 0.2561 Kinship 0.9194 1.0000 0.0000 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED): ACC SCORE MEAN SCORE STD-VAR Single-Span-Extraction 0.0482 0.7605 0.2285 Single-Span-Multi-Hop-Extraction 0.0241 0.8158 0.2127 Multi-Spans-Extraction 0.0158 0.0000 0.0000 YesNo 0.0000 0.9914 0.0147 Arithmetic-Operations 0.0091 0.4262 0.3168 Counting 0.0833 0.0000 0.0000 Date-Duration 0.6064 0.8128 0.2643 Kinship 0.0000 1.0000 0.0000 -------------------------------------------------------------------------------- ANSWER MODULE ERROR SAMPLES (WHEN CORRECTLY ACTIVATED): Single-Span-Extraction: ['D001Q11', 'D003Q05', 'D006Q01', 'D006Q02', 'D013Q01', 'D013Q11', 'D014Q09', 'D031Q02', 'D033Q01', 'D033Q03', 'D033Q05', 'D033Q06', 'D033Q14', 'D035Q04', 'D035Q05', 'D038Q02', 'D038Q11', 'D042Q14', 'D042Q15', 'D045Q04', 'D048Q02', 'D048Q03', 'D048Q09', 'D068Q01', 'D070Q01', 'D070Q04', 'D070Q10', 'D070Q11', 'D071Q01', 'D071Q10'] Single-Span-Multi-Hop-Extraction: ['D001Q06', 'D001Q09', 'D001Q11', 'D006Q02', 'D008Q07', 'D008Q08', 'D014Q09', 'D031Q02', 'D033Q01', 'D033Q05', 'D033Q06', 'D033Q07', 'D033Q08', 'D033Q14', 'D035Q02', 'D035Q05', 'D038Q01', 'D038Q11', 'D042Q03', 'D042Q05', 'D042Q06', 'D042Q14', 'D045Q03', 'D048Q03', 'D048Q09', 'D061Q01', 'D061Q03', 'D061Q04', 'D062Q02', 'D068Q01'] Multi-Spans-Extraction: ['D001Q08', 'D003Q03', 'D003Q06', 'D007Q02', 'D007Q03', 'D007Q04', 'D007Q05', 'D008Q06', 'D010Q04', 'D010Q05', 'D010Q06', 'D014Q05', 'D014Q08', 'D015Q01', 'D015Q02', 'D031Q03', 'D031Q04', 'D033Q09', 'D033Q10', 'D042Q09', 'D048Q07', 'D068Q04', 'D068Q05', 'D070Q02', 'D071Q04', 'D071Q05', 'D071Q06', 'D071Q07', 'D071Q08', 'D071Q09'] YesNo: ['D014Q06', 'D088Q02', 'D088Q03', 'D115Q05', 'D116Q05', 'D116Q08', 'D116Q09', 'D313Q03', 'D313Q06', 'D314Q07', 'D252Q09', 'D257Q01', 'D266Q05', 'D266Q08', 'D269Q08', 'D271Q02', 'D273Q02', 'D273Q05', 'D273Q06', 'D273Q08', 'D275Q05', 'D275Q11', 'D278Q12', 'D281Q02', 'D281Q06', 'D282Q03', 'D284Q02', 'D284Q08', 'D290Q06', 'D295Q05'] Arithmetic-Operations: ['D085Q08', 'D113Q05', 'D246Q08', 'D248Q04'] Counting: ['D014Q04', 'D015Q05', 'D015Q06', 'D048Q01', 'D076Q02', 'D076Q03', 'D076Q06', 'D076Q07', 'D076Q08', 'D252Q02', 'D252Q06'] Date-Duration: ['D033Q11', 'D035Q01', 'D035Q09', 'D035Q10', 'D038Q09', 'D042Q10', 'D068Q02', 'D070Q07', 'D071Q14', 'D076Q10', 'D087Q02', 'D104Q02', 'D104Q05', 'D105Q02', 'D105Q04', 'D214Q05', 'D238Q07', 'D311Q07', 'D267Q07'] Kinship: ['D091Q03', 'D091Q12', 'D282Q10', 'D282Q11', 'D288Q12'] --------------------------------------------------------------------------------