OVERALL ACC: 107 / 217 = 0.4930875576036866 -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER MODE TOP-1): GRND/PRED SS MS YN ARI CNT CMP CS DD KS Single-Span-Extraction 148 0 0 0 0 0 0 0 0 Multi-Spans-Extraction 0 19 0 0 0 0 0 0 0 YesNo 0 0 26 0 0 0 0 0 0 Arithmetic-Operations 0 0 0 1 0 0 0 0 0 Counting 0 0 0 0 2 0 0 0 0 Comparing-Members 0 0 0 0 0 3 0 0 0 CommonSense 0 0 0 0 0 0 0 0 0 Date-Duration 0 0 0 0 0 0 0 12 0 Kinship 1 0 0 0 0 0 0 0 5 -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER TYPE): GRND/PRED PER D-D LOC ORG NUM YN KIN EVT OBJ MISC Person 20 0 0 0 0 0 0 0 0 0 Date-Duration 0 47 0 0 0 0 0 0 0 0 Location 0 0 23 0 0 0 0 0 0 0 Organization 0 0 0 14 0 0 0 0 0 0 Num-Measure 0 0 0 0 19 0 0 0 0 0 YesNo 0 0 0 0 0 26 0 0 0 0 Kinship 0 0 0 0 0 0 10 1 0 0 Event 0 0 0 0 0 0 0 2 0 1 Object 0 0 0 0 0 0 0 0 52 0 Misc 0 0 0 0 0 0 0 0 0 2 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED): ACC SCORE MEAN SCORE STD-VAR Single-Span-Extraction 0.5608 0.8822 0.1836 Single-Span-Multi-Hop-Extraction 0.5878 0.8588 0.2183 Multi-Spans-Extraction 0.0526 0.0000 0.0000 YesNo 0.4231 0.9201 0.1519 Arithmetic-Operations 0.0000 0.9946 0.0000 Counting 0.0000 0.0000 0.0000 Date-Duration 0.0833 0.6025 0.3222 Kinship 0.8000 1.0000 0.0000 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED): ACC SCORE MEAN SCORE STD-VAR Single-Span-Extraction 0.0000 0.7412 0.2754 Single-Span-Multi-Hop-Extraction 0.0000 0.8509 0.1986 Multi-Spans-Extraction 0.0208 0.0000 0.0000 YesNo 0.0000 0.9958 0.0007 Arithmetic-Operations 0.0227 0.3860 0.3329 Counting 0.0909 0.0000 0.0000 Date-Duration 0.6027 0.8438 0.2437 Kinship 0.0000 1.0000 0.0000 -------------------------------------------------------------------------------- ANSWER MODULE ERRORS (WHEN CORRECTLY ACTIVATED): Single-Span-Extraction: ['D004Q03', 'D004Q04', 'D004Q07', 'D004Q08', 'D011Q01', 'D011Q03', 'D011Q05', 'D032Q01', 'D032Q02', 'D032Q08', 'D049Q01', 'D049Q04', 'D049Q05', 'D069Q04', 'D069Q06', 'D069Q07', 'D069Q09', 'D069Q12', 'D077Q03', 'D086Q03', 'D092Q03', 'D092Q04', 'D092Q06', 'D092Q07', 'D092Q11', 'D092Q12', 'D100Q01', 'D108Q02', 'D108Q05', 'D108Q07', 'D127Q01', 'D183Q02', 'D241Q07', 'D247Q01', 'D247Q04', 'D247Q05', 'D247Q06', 'D247Q09', 'D312Q05', 'D312Q07', 'D322Q03', 'D322Q05', 'D255Q01', 'D255Q08', 'D255Q09', 'D255Q11', 'D255Q12', 'D263Q05', 'D270Q05', 'D270Q11', 'D276Q05', 'D276Q06', 'D276Q10', 'D276Q14', 'D285Q06', 'D291Q07', 'D291Q09', 'D299Q02', 'D299Q05', 'D299Q06', 'D305Q01', 'D305Q02', 'D305Q05', 'D305Q07', 'D305Q10'] Single-Span-Multi-Hop-Extraction: ['D004Q03', 'D004Q04', 'D004Q06', 'D011Q01', 'D011Q02', 'D011Q05', 'D032Q01', 'D032Q02', 'D032Q06', 'D032Q08', 'D049Q01', 'D049Q04', 'D049Q05', 'D069Q04', 'D069Q06', 'D069Q07', 'D069Q09', 'D069Q12', 'D077Q02', 'D077Q03', 'D077Q05', 'D077Q06', 'D086Q03', 'D092Q03', 'D092Q04', 'D092Q09', 'D092Q12', 'D100Q01', 'D108Q02', 'D108Q05', 'D108Q07', 'D127Q01', 'D183Q01', 'D183Q02', 'D247Q01', 'D247Q02', 'D247Q05', 'D247Q06', 'D247Q09', 'D312Q05', 'D312Q06', 'D312Q07', 'D322Q05', 'D322Q06', 'D255Q01', 'D255Q03', 'D255Q04', 'D255Q12', 'D263Q06', 'D263Q07', 'D270Q05', 'D270Q11', 'D276Q12', 'D276Q14', 'D285Q03', 'D285Q04', 'D285Q05', 'D291Q07', 'D291Q09', 'D299Q06', 'D305Q05'] Multi-Spans-Extraction: ['D004Q01', 'D004Q02', 'D032Q04', 'D039Q01', 'D039Q02', 'D039Q04', 'D039Q05', 'D039Q06', 'D069Q10', 'D086Q01', 'D086Q02', 'D092Q02', 'D100Q06', 'D100Q07', 'D127Q07', 'D241Q08', 'D241Q09', 'D299Q03'] YesNo: ['D004Q09', 'D049Q09', 'D086Q04', 'D086Q05', 'D086Q06', 'D086Q07', 'D127Q06', 'D312Q09', 'D322Q08', 'D255Q07', 'D270Q02', 'D270Q04', 'D270Q09', 'D291Q02', 'D291Q05'] Arithmetic-Operations: ['D247Q07'] Counting: ['D049Q03', 'D069Q13'] Date-Duration: ['D032Q07', 'D032Q09', 'D032Q10', 'D077Q01', 'D092Q05', 'D092Q08', 'D108Q03', 'D247Q08', 'D312Q01', 'D312Q02', 'D312Q03'] Kinship: ['D255Q10'] --------------------------------------------------------------------------------