[Most-ai-contest] The performance of the current integrated system on the dataset ver. 1.5

[kysu]

It is suggested to separate the statistics from the detailed information of those error cases into two different files. For the statistics file of performance, just send them in plain text without compression so that I can directly open it. Thanks for your modification in advance.

KY

-----Original Message-----
From: most-ai-contest-bounces at iis.sinica.edu.tw [mailto:most-ai-contest-bounces at iis.sinica.edu.tw] On Behalf Of 張光瑜
Sent: Wednesday, December 18, 2019 4:10 PM
To: 范正忠 <jjfan at iis.sinica.edu.tw>
Cc: Most-ai Contest <Most-ai-contest at iis.sinica.edu.tw>
Subject: Re: [Most-ai-contest] The performance of the current integrated system on the dataset ver. 1.5

The attached file contains the performance/error reports.

張光瑜


范正忠 於 2019-12-18 15:56 寫到:
> Dear all,
> Enclosed please find train/dev/test dataset (1.5) performance using 1.  
> 廖沛俊 bestBERTFGC_full_em Bert Model 2. Dr. Chang aggregrator 3. Dr. 
> Change single-span-multi-hops enable
> 
> train: 0.65, dev: 0.55, test: 0.52
> 
> Best,
> jjfan
> 
> -------------------------
> 
> FROM: "范正忠" <jjfan at iis.sinica.edu.tw>
> TO: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
> SENT: Wednesday, December 18, 2019 11:30:36 AM
> SUBJECT: Re: [Most-ai-contest] The performance of the current 
> integrated system on the dataset ver. 1.5
> 
> Dear all,
> Enclosed please find train/dev/test dataset (1.5) performance using 1.  
> 廖沛俊 bestBERTFGC_full_em Bert Model 2. Dr. Chang aggregrator
> 
> Best,
> jjfan
> 
> -------------------------
> 
> FROM: "范正忠" <jjfan at iis.sinica.edu.tw>
> TO: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
> SENT: Saturday, December 14, 2019 4:47:37 PM
> SUBJECT: [Most-ai-contest] The performance of the current integrated 
> system on the dataset ver. 1.5
> 
> Dear all,
> Enclosed please find the performance of our current integrated system 
> on train / dev dataset.
> Please refer to the AMODE fields in each question for each module's 
> performance.
> Hope this can help the error analysis of each module and please share 
> your finding in the next week's meeting.
> I am still waiting for 'multi-spans-extraction' & 'supporting 
> evidence' modules to integrate. Please anyone has improved version 
> just send to me.
> Best,
> jjfan
> 
> -------------------------
> 
> FROM: "范正忠" <jjfan at iis.sinica.edu.tw>
> TO: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
> SENT: Monday, November 25, 2019 9:39:07 AM
> SUBJECT: Re: [Most-ai-contest] It's time to start system integration
> 
> Dear all,
> Please don't forget give me a readme.txt or I really don't know what 
> is inside.
> Suggest readme.txt contains at least
> 1. Brief API description / document, function list at minimum 2. 
> System requirements Thanks.
> jjfan
> 
> -------------------------
> 
> FROM: "范正忠" <jjfan at iis.sinica.edu.tw>
> TO: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
> SENT: Monday, November 25, 2019 9:18:05 AM
> SUBJECT: [Most-ai-contest] It's time to start system integration
> 
> Dear all,
> Good Morning!
> It's time to start integration of the system. Please all module owners 
> send me your Python Code. Thanks.
> Best,
> jjfan
> 
> -------------------------
> 
> FROM: "范正忠" <jjfan at iis.sinica.edu.tw>
> TO: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
> SENT: Thursday, November 14, 2019 8:58:50 AM
> SUBJECT: Re: [Most-ai-contest] 科技大擂台討論會(Today,
> 12:30-15:00)
> 
> Dear all,
> 
> Note the following action items should be finished today.
> 
> 3. FGC_Questions_S7.json son 包含系統進行答題的過程中,
> 各個 module 預計輸出.
>    今日會議上討論要修改的部分, 請各位 module owner
> 儘快提出各自修改的部分給我,
> 一起整理完後再寄送給大家 (10/14 以前完成)
> 4. 提供個自 module 所需的 Python library 及使用的 Python version. (10/14 以前完成)
>    建議 pip list
> 7. 為了確保大家沒有誤會各自負責的 module function,
> 煩請簡單描述一下 (input, out, function) 為何. (10/14
> 以前完成)
> 
> ITEM 4. PLEASE ALSO LIST ADDITIONAL INSTALLATIONS IF PIP IS NOT 
> ENOUGH, SUCH AS DATABASE, FILES, ...
> Best,
> jjfan
> 
> -------------------------
> 
> FROM: "范正忠" <jjfan at iis.sinica.edu.tw>
> TO: "Po-Fen Lin" <pofen at iis.sinica.edu.tw>
> CC: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
> SENT: Tuesday, November 12, 2019 5:29:03 PM
> SUBJECT: Re: [Most-ai-contest] 科技大擂台討論會(Today,
> 12:30-15:00)
> 
> Dear all,
> 
> 謝謝大家的參與討論.
> 
> 附上今天討論的內容.
> 1. schedule.ppt 包含每個參與者的 job items
> 及未來計畫進行 milestones
> 2. FGC_release_A_all.json 包含 answer type, answer mode, ...
> 的基礎的 annotation. 提供大家參考
>    若發現 answer type 或 answer mode 有錯的地方, 請 email
> 通知我, 有我這邊統一修正, 在再寄送給大家更新版
> (隨時)
>    請廖沛俊儘快完成 answer keyword spans 的 annotation,
> 並將完成後的 json 檔案提供給我一起整理. (10/15
> 以前完成)
> 3. FGC_Questions_S7.json son 包含系統進行答題的過程中,
> 各個 module 預計輸出.
>    今日會議上討論要修改的部分, 請各位 module owner
> 儘快提出各自修改的部分給我,
> 一起整理完後再寄送給大家 (10/14 以前完成)
> 4. 提供個自 module 所需的 Python library 及使用的 Python version. (10/14 以前完成)
>    建議 pip list
> 5. 1007說明會簡報V9, 國網機器遠端連線操作說明:
> 請郭家鉦幫忙看一下如何處理"自動化介接"及"程式上傳/執行"
> 6. 附件二科技大擂台簡答題之答題規範,
> 科技大擂台簡答題之答題規範v2: 提供給謝尊安參考.
> 7. 為了確保大家沒有誤會各自負責的 module function,
> 煩請簡單描述一下 (input, out, function) 為何. (10/14
> 以前完成)
> 
> 有任何問題, 請隨時讓我知道(email, line, skype, tel, ...).
> 我會一起來協助如何處理.
> 謝謝的家的幫忙!
> 
> Best,
> jjfan
> 
> _______________________________________________
> Most-ai-contest mailing list
> Most-ai-contest at iis.sinica.edu.tw
> https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
> 
> _______________________________________________
> Most-ai-contest mailing list
> Most-ai-contest at iis.sinica.edu.tw
> https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
> 
> _______________________________________________
> Most-ai-contest mailing list
> Most-ai-contest at iis.sinica.edu.tw
> https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
> 
> _______________________________________________
> Most-ai-contest mailing list
> Most-ai-contest at iis.sinica.edu.tw
> https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
> 
> _______________________________________________
> Most-ai-contest mailing list
> Most-ai-contest at iis.sinica.edu.tw
> https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
> _______________________________________________
> Most-ai-contest mailing list
> Most-ai-contest at iis.sinica.edu.tw
> https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest