[Most-ai-contest] Kinship module answer format (1.5)

[alsm]

Dear Dr.Fan,

I've noticed that the answer format for the weeks 1-3 differs 
significantly from the answers in previous datasets. In earlier 
versions, the correct answer was one alternative from several given 
(because of which there were usually several possible correct answers 
and the answer had to be a single term), for example:

"QID": "D036Q20"
"QTEXT": "阿姨的丈夫，我們要稱他為什麼？"
"ANSWER": [{"ATEXT": "姨丈"} {"ATEXT": "姨父"}]

While in the questions for weeks 1-3 the answers are usually all the 
alternatives combined into single string (as a list, conjunction or with 
parenthesis, which makes the problem more of a multi-span extraction), 
as compared below:

"QID": 	"D036Q15"
"QTEXT": "叔叔的妻子，要如何稱呼她？"
"ANSWER": {ATEXT:"嬸、叔母、嬸嬸及娘娘"}
"AFINAL": {ATEXT:"叔母"}


Since the kinship module gives the answers based on the earlier version 
of dataset, it has big performance issues if questions are formatted in 
this manner (it outputs only a substring of the correct answer - like 
the AFINAL above).

I have attached more examples of answer alternatives for kinship 
questions from FGC Dataset 1.5 ("kinship_answer_example.json") as well 
as the error analysis for the questions week 1-3 which shows more 
examples of the issue.

There is also one question which I think is probably incorrectly 
answered in the dataset (it might be an issue of my poor Chinese 
however):

"QID": 	D091Q11
"QTEXT": "先生的奶奶要怎么称呼她?"
"ANSWER":"母親、媽媽、娘及婆婆"
?relevant DTEXT fragments:"...当对象是丈夫的祖母，我们要称呼她为祖姑(奶奶)..." and 
"...当对象是丈夫的母亲，我们要称呼她为母亲(妈妈/娘/婆婆/奶奶(广东人用))..."


Best regards,
Aleksandra Smolka
-------------- 下一部份 --------------
An embedded and charset-unspecified text was scrubbed...
Name: error_analysis1.5.txt
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191225/7e2c7065/attachment-0001.txt>
-------------- 下一部份 --------------
An embedded and charset-unspecified text was scrubbed...
Name: kinship_answer_example.json
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191225/7e2c7065/attachment-0001.ksh>