SciFact

Due to the rapid growth in the scientific literature, there is a need for automated systems to assist researchers and the public in assessing the veracity of scientific claims. To facilitate the development of systems for tmore

Public Submissions Getting Started About

Download

Rank	Submission	Created	Sent+L (F1)	Sent+L (P)	Sent+L (R)	Sent (F1)	Sent (P)	Sent (R)	Abst+R (F1)	Abst+R (P)	Abst+R (R)	Abst (F1)	Abst (P)	Abst (R)

1	+VeriRel(N5)Top5 noname	10/15/2024	0.6767	0.6861	0.6676	0.7479	0.7583	0.7378	0.7146	0.7368	0.6937	0.7285	0.7512	0.7072
2	itsatest no name	07/22/2024	0.6766	0.6775	0.6757	0.7497	0.7507	0.7486	0.7156	0.7290	0.7027	0.7294	0.7430	0.7162
3	vrl Sam	10/14/2024	0.6751	0.7038	0.6486	0.7482	0.7801	0.7189	0.7143	0.7576	0.6757	0.7286	0.7727	0.6892
4	test test	07/09/2024	0.6728	0.6597	0.6865	0.7417	0.7273	0.7568	0.7191	0.7175	0.7207	0.7326	0.7309	0.7342
5	+VeriRel(N5)Top3 cool	10/15/2024	0.6722	0.6914	0.6541	0.7444	0.7657	0.7243	0.7106	0.7438	0.6802	0.7247	0.7586	0.6937
5	wrong mj	10/15/2024	0.6722	0.6914	0.6541	0.7444	0.7657	0.7243	0.7106	0.7438	0.6802	0.7247	0.7586	0.6937
7	+VeriRel(N5)Top10 not cool	10/15/2024	0.6721	0.6739	0.6703	0.7425	0.7446	0.7405	0.7110	0.7243	0.6982	0.7248	0.7383	0.7117
7	test test	05/23/2024	0.6721	0.6739	0.6703	0.7425	0.7446	0.7405	0.7110	0.7243	0.6982	0.7248	0.7383	0.7117
7	aka asd	07/19/2023	0.6721	0.6739	0.6703	0.7425	0.7446	0.7405	0.7110	0.7243	0.6982	0.7248	0.7383	0.7117
7	MultiVerS Allen Institute for AI and Un…	06/04/2021	0.6721	0.6739	0.6703	0.7425	0.7446	0.7405	0.7110	0.7243	0.6982	0.7248	0.7383	0.7117
11	MRE mre	03/05/2022	0.6712	0.6721	0.6703	0.7415	0.7425	0.7405	0.7094	0.7209	0.6982	0.7231	0.7349	0.7117
12	test2 test if performance change	07/09/2024	0.6702	0.6571	0.6838	0.7417	0.7273	0.7568	0.7146	0.7130	0.7162	0.7281	0.7265	0.7297
12	improved_rerank_modified XD from the UoS	07/05/2024	0.6702	0.6571	0.6838	0.7417	0.7273	0.7568	0.6989	0.6824	0.7162	0.7121	0.6953	0.7297
14	T5(Top5)+MultiVerS haha	10/15/2024	0.6694	0.6797	0.6595	0.7407	0.7521	0.7297	0.7086	0.7343	0.6847	0.7226	0.7488	0.6982
15	Longchecker++ DMZ	02/13/2022	0.6694	0.6712	0.6676	0.7398	0.7418	0.7378	0.7064	0.7196	0.6937	0.7202	0.7336	0.7072
16	T5(Top3) + MultiVerS re	10/15/2024	0.6676	0.7003	0.6378	0.7412	0.7774	0.7081	0.7081	0.7551	0.6667	0.7225	0.7704	0.6802
17	qr_test0 anonymous	11/25/2022	0.6649	0.6519	0.6784	0.7391	0.7247	0.7541	0.7088	0.7104	0.7072	0.7223	0.7240	0.7207
18	testing0 annoymous	11/28/2022	0.6649	0.6545	0.6757	0.7420	0.7304	0.7541	0.7088	0.7104	0.7072	0.7223	0.7240	0.7207
19	qr_test0_no_reranking annoymous	11/25/2022	0.6640	0.6478	0.6811	0.7378	0.7198	0.7568	0.7085	0.7054	0.7117	0.7220	0.7188	0.7252
20	rerank_test XD	07/05/2024	0.6580	0.6341	0.6838	0.7334	0.7068	0.7622	0.7004	0.6853	0.7162	0.7137	0.6983	0.7297
21	BGE reranking TEST	05/22/2024	0.6566	0.6676	0.6459	0.7253	0.7374	0.7135	0.6869	0.7136	0.6622	0.7009	0.7282	0.6757
22	ARSJoint (Pre-training) Anonymous	01/27/2022	0.6554	0.6833	0.6297	0.7904	0.8240	0.7595	0.6868	0.7081	0.6667	0.7100	0.7321	0.6892
22	test test	01/30/2022	0.6554	0.6833	0.6297	0.7904	0.8240	0.7595	0.6868	0.7081	0.6667	0.7100	0.7321	0.6892
24	test-prob-ranking xd sheffield	06/17/2024	0.6549	0.6585	0.6514	0.7283	0.7322	0.7243	0.6866	0.7028	0.6712	0.7005	0.7170	0.6847
25	TEST TEST	01/31/2022	0.6537	0.6705	0.6378	0.7895	0.8097	0.7703	0.6895	0.6991	0.6802	0.7123	0.7222	0.7027
25	ARSJoint(Neural Retrieval & P… Anonymous	02/04/2022	0.6537	0.6705	0.6378	0.7895	0.8097	0.7703	0.6895	0.6991	0.6802	0.7123	0.7222	0.7027
27	VerT5erini (2-stage Neural Re… University of Waterloo	02/19/2021	0.6340	0.6059	0.6649	0.6598	0.6305	0.6919	0.6695	0.6285	0.7162	0.6821	0.6403	0.7297
28	Test Elfsong	01/13/2022	0.6339	0.6409	0.6270	0.6557	0.6630	0.6486	0.6608	0.6507	0.6712	0.6829	0.6725	0.6937
29	Test Elfsong	01/24/2022	0.6309	0.6267	0.6351	0.6523	0.6480	0.6568	0.6667	0.6417	0.6937	0.6926	0.6667	0.7207
30	ARSJoint Zhiwei Zhang, Jiyi Li, Fumiyo…	08/13/2021	0.6308	0.6617	0.6027	0.7581	0.7953	0.7243	0.6571	0.6970	0.6216	0.6810	0.7222	0.6441
31	RerrFact Under anonymous review, code …	11/17/2021	0.6209	0.7343	0.5378	0.6771	0.8007	0.5865	0.6631	0.8158	0.5586	0.6738	0.8289	0.5676
32	Rs-Lp JointModel Rs-Lp JointModel	12/05/2021	0.6113	0.7276	0.5270	0.7022	0.8358	0.6054	0.6480	0.7471	0.5721	0.6684	0.7706	0.5901
33	ParagraphJoint PLUS Lab: Xiangci Li (UT Dall…	01/26/2021	0.6094	0.6894	0.5459	0.7059	0.7986	0.6324	0.6716	0.7366	0.6171	0.6912	0.7581	0.6351
34	enhanced-paragraph-joint enhanced-paragraph-joint	12/04/2021	0.6070	0.7422	0.5135	0.6805	0.8320	0.5757	0.6214	0.7391	0.5360	0.6423	0.7640	0.5541
35	Test Will be released soon	12/29/2021	0.5988	0.6978	0.5243	0.6173	0.7194	0.5405	0.6406	0.7005	0.5901	0.6699	0.7326	0.6171
36	VerT5erini (2-stage Neural Re… University of Waterloo	01/26/2021	0.5876	0.6000	0.5757	0.6483	0.6620	0.6351	0.6269	0.6147	0.6396	0.6490	0.6364	0.6622
37	regression Mitchell DeHaven	02/26/2023	0.5831	0.5696	0.5973	0.6174	0.6031	0.6324	0.6834	0.6912	0.6757	0.7380	0.7465	0.7297
38	JM Z	05/14/2021	0.5826	0.6554	0.5243	0.7027	0.7905	0.6324	0.6231	0.7045	0.5586	0.6432	0.7273	0.5766
39	FEVER transfer Mitchell DeHaven	09/28/2022	0.5810	0.5353	0.6351	0.6156	0.5672	0.6730	0.6785	0.6681	0.6892	0.7317	0.7205	0.7432
40	base_rel_(train+dev) Will disclose after final sub…	12/05/2021	0.5796	0.7054	0.4919	0.6529	0.7946	0.5541	0.6125	0.7687	0.5090	0.6233	0.7823	0.5180
41	test n.a.	09/15/2022	0.5792	0.5147	0.6622	0.6099	0.5420	0.6973	0.6780	0.6400	0.7207	0.7161	0.6760	0.7613
42	test test	08/22/2022	0.5711	0.5188	0.6351	0.6075	0.5519	0.6757	0.6711	0.6538	0.6892	0.7237	0.7051	0.7432
43	test test	09/27/2022	0.5697	0.5276	0.6189	0.6119	0.5668	0.6649	0.6577	0.6533	0.6622	0.7069	0.7022	0.7117
44	test test	05/26/2023	0.5660	0.6873	0.4811	0.6550	0.7954	0.5568	0.6349	0.7692	0.5405	0.6667	0.8077	0.5676
45	V-MultiVerS(N20) Xingyu Deng from NLP research…	10/07/2024	0.5630	0.5182	0.6162	0.6198	0.5705	0.6784	0.6195	0.5753	0.6712	0.6694	0.6216	0.7252
46	base-rel 3 prototypes Will be released soon	12/17/2021	0.5626	0.6570	0.4919	0.6584	0.7690	0.5757	0.6080	0.7451	0.5135	0.6293	0.7712	0.5315
47	test test	05/26/2023	0.5609	0.6890	0.4730	0.6474	0.7953	0.5459	0.6330	0.7727	0.5360	0.6649	0.8117	0.5631
48	base-rel prototype submission Currently the solution in tes…	11/14/2021	0.5604	0.6558	0.4892	0.6533	0.7645	0.5703	0.5860	0.7267	0.4910	0.6075	0.7533	0.5090
49	Law & Econ fine-tuning the e-FEVER sytem…	02/11/2021	0.5601	0.5663	0.5541	0.6257	0.6326	0.6189	0.6061	0.6280	0.5856	0.6480	0.6715	0.6261
50	ArgJointModel Zhiyuan Guo from Beihang Univ…	10/05/2021	0.5596	0.6149	0.5135	0.7128	0.7832	0.6541	0.5919	0.6294	0.5586	0.6205	0.6599	0.5856
51	Test test	06/01/2023	0.5577	0.6850	0.4703	0.6474	0.7953	0.5459	0.6273	0.7748	0.5270	0.6542	0.8079	0.5495
52	test test	05/26/2023	0.5570	0.6718	0.4757	0.6456	0.7786	0.5514	0.6296	0.7628	0.5360	0.6614	0.8013	0.5631
53	VerT5erini (BM25 Retrieval) University of Waterloo	01/27/2021	0.5552	0.5833	0.5297	0.6176	0.6488	0.5892	0.5917	0.6028	0.5811	0.6193	0.6308	0.6081
54	QMUL-SDS Xia Zeng and Arkaitz Zubiaga …	03/09/2021	0.5535	0.6617	0.4757	0.6824	0.8158	0.5865	0.5838	0.7297	0.4865	0.5946	0.7432	0.4955
55	base-rel prototype submission Currently the solution in tes…	10/17/2021	0.5521	0.6383	0.4865	0.6564	0.7589	0.5784	0.6027	0.7386	0.5090	0.6240	0.7647	0.5270
56	base-rel Baseline model testing in pro…	08/22/2021	0.5518	0.7237	0.4459	0.6455	0.8465	0.5216	0.6028	0.8045	0.4820	0.6141	0.8195	0.4910
57	test test	06/01/2023	0.5483	0.6471	0.4757	0.6417	0.7574	0.5568	0.6032	0.7308	0.5135	0.6296	0.7628	0.5360
58	first_1 Wangchaochao from PingAn of C…	02/24/2021	0.5477	0.6506	0.4730	0.6072	0.7212	0.5243	0.5835	0.6536	0.5270	0.6135	0.6872	0.5541
59	Multiverse: 3.0 Will release with final submi…	11/12/2021	0.5474	0.6303	0.4838	0.6636	0.7641	0.5865	0.5836	0.7097	0.4955	0.5995	0.7290	0.5090
60	Multiverse 3.1 Will release with final submi…	11/14/2021	0.5466	0.6281	0.4838	0.6626	0.7614	0.5865	0.5820	0.7051	0.4955	0.5979	0.7244	0.5090
61	test test	05/24/2023	0.5455	0.6207	0.4865	0.6606	0.7517	0.5892	0.6150	0.7212	0.5360	0.6253	0.7333	0.5450
62	cogat anonymous	08/26/2023	0.5430	0.6020	0.4946	0.5846	0.6480	0.5324	0.6354	0.7531	0.5495	0.6615	0.7840	0.5721
63	SCIFCHEX Filip J. Cierkosz from Univer…	05/18/2024	0.5372	0.6694	0.4486	0.6084	0.7581	0.5081	0.6000	0.7826	0.4865	0.6167	0.8043	0.5000
64	test test	08/12/2022	0.5330	0.4866	0.5892	0.5819	0.5313	0.6432	0.6386	0.6288	0.6486	0.6874	0.6769	0.6982
65	Test #SFC FJC Sheffield	05/11/2024	0.5304	0.6273	0.4595	0.6022	0.7122	0.5216	0.6133	0.7929	0.5000	0.6188	0.8000	0.5045
66	test anonymous	03/22/2024	0.5291	0.6092	0.4676	0.5657	0.6514	0.5000	0.6092	0.7584	0.5090	0.6307	0.7852	0.5270
67	base-rel prototype (train+dev) Will disclose soon	11/16/2021	0.5285	0.6374	0.4514	0.5759	0.6947	0.4919	0.5778	0.7536	0.4685	0.6000	0.7826	0.4865
68	V-MultiVerS(N10)	10/07/2024	0.5250	0.4788	0.5811	0.6007	0.5479	0.6649	0.5988	0.5376	0.6757	0.6427	0.5771	0.7252
69	aaaaa anonymous	03/20/2024	0.5229	0.5765	0.4784	0.5790	0.6384	0.5297	0.6134	0.7169	0.5360	0.6237	0.7289	0.5450
70	argJointModel-trainset BUAA	10/05/2021	0.5204	0.5911	0.4649	0.6324	0.7182	0.5649	0.5473	0.6111	0.4955	0.5920	0.6611	0.5360
71	TEST 2 test2	05/11/2024	0.5202	0.6466	0.4351	0.5977	0.7430	0.5000	0.5909	0.8000	0.4685	0.5966	0.8077	0.4730
72	ssss anonymous	03/24/2024	0.5138	0.5964	0.4514	0.5785	0.6714	0.5081	0.5946	0.7432	0.4955	0.6324	0.7905	0.5270
73	aaaaaa anonymous	05/22/2024	0.5130	0.5531	0.4784	0.5913	0.6375	0.5514	0.6000	0.6964	0.5270	0.6154	0.7143	0.5405
73	test anonymous	03/21/2024	0.5130	0.5531	0.4784	0.5913	0.6375	0.5514	0.6000	0.6964	0.5270	0.6154	0.7143	0.5405
73	aaaa anonymous	03/23/2024	0.5130	0.5531	0.4784	0.5913	0.6375	0.5514	0.6000	0.6964	0.5270	0.6154	0.7143	0.5405
76	tttt anonymous	03/24/2024	0.5108	0.5978	0.4459	0.5573	0.6522	0.4865	0.5870	0.7397	0.4865	0.6087	0.7671	0.5045
76	aaaa anonymous	03/24/2024	0.5108	0.5978	0.4459	0.5573	0.6522	0.4865	0.5870	0.7397	0.4865	0.6087	0.7671	0.5045
78	Yonky G YonkyG from NEU	06/13/2023	0.5102	0.6059	0.4405	0.5790	0.6877	0.5000	0.5928	0.7698	0.4820	0.6094	0.7914	0.4955
79	SciKGAT Zhenghao Liu from Tsinghua Un…	01/28/2021	0.5048	0.6115	0.4297	0.5524	0.6692	0.4703	0.5833	0.7609	0.4730	0.6000	0.7826	0.4865
80	aaaa no	03/20/2024	0.5024	0.6100	0.4270	0.5469	0.6641	0.4649	0.5922	0.7794	0.4775	0.5978	0.7868	0.4820
80	ssss anonymous	03/25/2024	0.5024	0.6100	0.4270	0.5469	0.6641	0.4649	0.5922	0.7794	0.4775	0.5978	0.7868	0.4820
82	aaa anonymous	03/24/2024	0.5023	0.5842	0.4405	0.5701	0.6631	0.5000	0.5860	0.7267	0.4910	0.6022	0.7467	0.5045
83	bioBert for sciFact bioBert for sciFact	03/16/2021	0.4985	0.5487	0.4568	0.6401	0.7045	0.5865	0.5330	0.5829	0.4910	0.5428	0.5936	0.5000
84	aaaa anonymous	03/20/2024	0.4979	0.5191	0.4784	0.5513	0.5748	0.5297	0.4845	0.4483	0.5270	0.5093	0.4713	0.5541
85	test anonymous	03/22/2024	0.4970	0.5612	0.4459	0.5512	0.6224	0.4946	0.5744	0.6832	0.4955	0.5953	0.7081	0.5135
86	ttttt anonymous	03/23/2024	0.4963	0.5552	0.4486	0.5800	0.6488	0.5243	0.5864	0.7000	0.5045	0.6021	0.7188	0.5180
86	test anonymous	03/24/2024	0.4963	0.5552	0.4486	0.5800	0.6488	0.5243	0.5864	0.7000	0.5045	0.6021	0.7188	0.5180
88	aaaa anonymous	03/23/2024	0.4948	0.5556	0.4459	0.5787	0.6498	0.5216	0.5842	0.7025	0.5000	0.5947	0.7152	0.5090
89	aaa anonymous	03/20/2024	0.4805	0.5171	0.4486	0.5847	0.6293	0.5459	0.5831	0.6746	0.5135	0.6036	0.6982	0.5315
89	aaaa aaaaa	03/25/2024	0.4805	0.5171	0.4486	0.5847	0.6293	0.5459	0.5831	0.6746	0.5135	0.6036	0.6982	0.5315
91	729 anonymous	03/21/2024	0.4794	0.5808	0.4081	0.5778	0.7000	0.4919	0.5391	0.6711	0.4505	0.5660	0.7047	0.4730
92	base-rel Baseline model testing in pro…	09/01/2021	0.4754	0.5747	0.4054	0.6751	0.8161	0.5757	0.5096	0.6503	0.4189	0.5151	0.6573	0.4234
93	Multiverse 2.0 TIET	10/03/2021	0.4753	0.5798	0.4027	0.6635	0.8093	0.5622	0.5179	0.6667	0.4234	0.5289	0.6809	0.4324
94	base-rel Baseline model testing in pro…	09/13/2021	0.4654	0.5564	0.4000	0.6824	0.8158	0.5865	0.4919	0.6149	0.4099	0.4973	0.6216	0.4144
94	Multiverse Deepanshu Khanna from Thapar …	08/22/2021	0.4654	0.5564	0.4000	0.6824	0.8158	0.5865	0.5027	0.6284	0.4189	0.5081	0.6351	0.4234
96	RANLI + MultiVerS Théophile Mandon (LIRMM)	05/22/2024	0.4634	0.4647	0.4622	0.7425	0.7446	0.7405	0.4908	0.5000	0.4820	0.4954	0.5047	0.4865
97	279 anonymous	03/21/2024	0.4606	0.5530	0.3946	0.5584	0.6705	0.4784	0.5260	0.6713	0.4324	0.5425	0.6923	0.4459
98	mmmm anonymous	03/24/2024	0.4482	0.5235	0.3919	0.5719	0.6679	0.5000	0.5161	0.6400	0.4324	0.5538	0.6867	0.4640
99	ASCME Will publish	04/29/2024	0.4455	0.3843	0.5297	0.5477	0.4725	0.6514	0.5085	0.4369	0.6081	0.5461	0.4693	0.6532
100	Multiverse-base Will share soon...	09/07/2021	0.4434	0.5301	0.3811	0.6824	0.8158	0.5865	0.4865	0.6081	0.4054	0.4973	0.6216	0.4144
101	aaaa anonymous	03/24/2024	0.4403	0.5164	0.3838	0.5271	0.6182	0.4595	0.5269	0.6533	0.4414	0.5538	0.6867	0.4640
102	ASCME Computer Science Student @ Un…	03/13/2024	0.4341	0.3745	0.5162	0.5477	0.4725	0.6514	0.4934	0.4239	0.5901	0.5273	0.4531	0.6306
103	test anonymous	03/21/2024	0.4312	0.5579	0.3514	0.5605	0.7253	0.4568	0.4929	0.6641	0.3919	0.4986	0.6718	0.3964
104	AscMe Will publish	04/29/2024	0.4196	0.3574	0.5081	0.5469	0.4658	0.6622	0.4583	0.3954	0.5450	0.4886	0.4216	0.5811
105	pipeline_longformer_test_rera… test rerank with pipeline xd	01/09/2024	0.4143	0.3932	0.4378	0.5090	0.4830	0.5378	0.4528	0.4021	0.5180	0.5000	0.4441	0.5721
106	V-MultiVerS(N5) Xingyu Deng from NLP research…	10/07/2024	0.4142	0.3079	0.6324	0.4726	0.3513	0.7216	0.5055	0.3880	0.7252	0.5400	0.4145	0.7748
107	aaa anonymous	03/24/2024	0.4139	0.5342	0.3378	0.5497	0.7094	0.4486	0.4651	0.6557	0.3604	0.4767	0.6721	0.3694
108	aaa anonymous	03/21/2024	0.4137	0.5628	0.3270	0.5675	0.7721	0.4486	0.4379	0.6379	0.3333	0.4497	0.6552	0.3423
109	new_rerank xd sheffield	01/09/2024	0.4121	0.3872	0.4405	0.5082	0.4774	0.5432	0.4501	0.3979	0.5180	0.4971	0.4394	0.5721
110	AdaptedSciCheck Will publish later	03/31/2024	0.4103	0.3491	0.4973	0.5463	0.4649	0.6622	0.4576	0.3875	0.5586	0.4871	0.4125	0.5946
111	JC_UKP Jorge Cardona at UKP	03/02/2021	0.4102	0.4069	0.4135	0.4879	0.4840	0.4919	0.4896	0.5024	0.4775	0.4896	0.5024	0.4775
112	test4 test	05/11/2024	0.4101	0.4554	0.3730	0.6419	0.7129	0.5838	0.4562	0.5548	0.3874	0.4668	0.5677	0.3964
113	AscMe Will publish	04/29/2024	0.4095	0.3805	0.4432	0.5144	0.4780	0.5568	0.4774	0.4394	0.5225	0.5021	0.4621	0.5495
114	TEST 1 Test.	05/11/2024	0.4087	0.4783	0.3568	0.6409	0.7500	0.5595	0.4505	0.5775	0.3694	0.4560	0.5845	0.3739
115	Test 0 test0	05/11/2024	0.4054	0.4605	0.3622	0.6384	0.7251	0.5703	0.4541	0.5676	0.3784	0.4595	0.5743	0.3829
116	Yang Y	12/10/2022	0.4044	0.3719	0.4432	0.4587	0.4218	0.5027	0.4587	0.4237	0.5000	0.4876	0.4504	0.5315
116	Test Test	05/04/2023	0.4044	0.3719	0.4432	0.4587	0.4218	0.5027	0.4587	0.4237	0.5000	0.4876	0.4504	0.5315
118	sum_rationale Rationale Selection with Summ…	03/02/2021	0.3986	0.3445	0.4730	0.5080	0.4390	0.6027	0.4644	0.4238	0.5135	0.5051	0.4610	0.5586
119	ssss anonymous	03/24/2024	0.3963	0.4686	0.3432	0.5273	0.6236	0.4568	0.4573	0.5887	0.3739	0.4738	0.6099	0.3874
120	Longformer_helps_label test	08/06/2023	0.3954	0.4207	0.3730	0.4900	0.5213	0.4622	0.4522	0.4686	0.4369	0.4662	0.4831	0.4505
121	VeriSci Allen Institute for AI and Un…	01/26/2021	0.3953	0.3856	0.4054	0.4611	0.4499	0.4730	0.4650	0.4661	0.4640	0.4740	0.4751	0.4730
122	#1./script/pipeline.sh open v… Charles	08/02/2021	0.3953	0.3856	0.4054	0.4611	0.4499	0.4730	0.4650	0.4661	0.4640	0.4740	0.4751	0.4730
123	kkecho-model1-test3 kecho	07/26/2023	0.3937	0.4169	0.3730	0.4879	0.5166	0.4622	0.4501	0.4641	0.4369	0.4640	0.4785	0.4505
124	Longformer-pipeline-5 test	08/15/2023	0.3935	0.3925	0.3946	0.4852	0.4839	0.4865	0.4421	0.4150	0.4730	0.4758	0.4466	0.5090
125	test Y	10/10/2022	0.3920	0.3333	0.4757	0.4432	0.3769	0.5378	0.4536	0.4035	0.5180	0.4773	0.4246	0.5450
126	XYD-v1-test2 XD from the University of Man…	07/12/2023	0.3890	0.4167	0.3649	0.4870	0.5216	0.4568	0.4408	0.4545	0.4279	0.4548	0.4689	0.4414
127	Longformer-pipeline-3 XD from the University of Man…	08/15/2023	0.3842	0.4199	0.3541	0.4780	0.5224	0.4405	0.4388	0.4502	0.4279	0.4758	0.4882	0.4640
128	test C	10/10/2022	0.3838	0.3229	0.4730	0.4430	0.3727	0.5459	0.4436	0.3904	0.5135	0.4630	0.4075	0.5360
129	XYD XD from the University of Man…	07/04/2023	0.3795	0.4469	0.3297	0.4479	0.5275	0.3892	0.4570	0.5027	0.4189	0.5061	0.5568	0.4640
130	AdaptedSciCheck test	04/04/2024	0.3771	0.3566	0.4000	0.5172	0.4892	0.5486	0.4412	0.4068	0.4820	0.4784	0.4411	0.5225
131	XD-test-v2 XD from the University of Man…	07/11/2023	0.3763	0.3743	0.3784	0.4328	0.4305	0.4351	0.4340	0.4113	0.4595	0.4638	0.4395	0.4910
132	v3 model uom	07/26/2023	0.3733	0.3684	0.3784	0.4427	0.4368	0.4486	0.4435	0.4286	0.4595	0.4696	0.4538	0.4865
133	v4-test test	07/30/2023	0.3712	0.4868	0.3000	0.4849	0.6360	0.3919	0.4380	0.5287	0.3739	0.4749	0.5732	0.4054
134	sum_rationale Rationale Selection with Summ…	03/01/2021	0.3702	0.2845	0.5297	0.4608	0.3541	0.6595	0.4316	0.3534	0.5541	0.4737	0.3879	0.6081
135	2024_test xd sheffield	01/08/2024	0.3664	0.3678	0.3649	0.4586	0.4605	0.4568	0.3907	0.3695	0.4144	0.4331	0.4096	0.4595
136	prediction_full_pipeline_v1 XD	08/09/2023	0.3627	0.3483	0.3784	0.4352	0.4179	0.4541	0.4235	0.3961	0.4550	0.4528	0.4235	0.4865
137	Multiverse 4.0 Will disclose after final sub…	11/19/2021	0.3562	0.4077	0.3162	0.6606	0.7561	0.5865	0.4211	0.5063	0.3604	0.4421	0.5316	0.3784
138	test7 test	08/09/2023	0.3560	0.3382	0.3757	0.4302	0.4088	0.4541	0.4100	0.3828	0.4414	0.4477	0.4180	0.4820
139	mul-rel-prototype work in progress, will make e…	11/18/2021	0.3498	0.4094	0.3054	0.6533	0.7645	0.5703	0.4086	0.5067	0.3423	0.4301	0.5333	0.3604
140	longformer_v1_test XD from the University of Man…	08/04/2023	0.3395	0.3595	0.3216	0.4879	0.5166	0.4622	0.3852	0.3971	0.3739	0.4084	0.4211	0.3964
141	decontextualization_scifact_v1 Mengfei Lan from School of In…	01/14/2025	0.3227	0.2657	0.4108	0.4225	0.3479	0.5378	0.3640	0.3075	0.4459	0.3824	0.3230	0.4685
142	XYD - v2 XD from the University of Man…	07/26/2023	0.3221	0.3100	0.3351	0.4182	0.4025	0.4351	0.3942	0.3654	0.4279	0.4191	0.3885	0.4550
143	v2-test2-D Sam	07/26/2023	0.3116	0.2954	0.3297	0.4112	0.3898	0.4351	0.3817	0.3538	0.4144	0.4066	0.3769	0.4414
144	GPT Test using ChromaDB Jan Disselhoff from JGU Mainz	03/31/2023	0.3079	0.2953	0.3216	0.3674	0.3524	0.3838	0.3688	0.2921	0.5000	0.4884	0.3868	0.6622
145	First Zhelin Chu	07/31/2024	0.3018	0.2434	0.3973	0.3737	0.3013	0.4919	0.4184	0.3645	0.4910	0.4453	0.3880	0.5225
146	test test	05/16/2023	0.2972	0.2944	0.3000	0.4712	0.4668	0.4757	0.3575	0.3591	0.3559	0.3710	0.3727	0.3694
147	test test	08/22/2022	0.2941	0.2308	0.4054	0.3843	0.3015	0.5297	0.3652	0.3012	0.4640	0.3759	0.3099	0.4775
148	longformer_test_v2 test	08/04/2023	0.2910	0.3082	0.2757	0.4879	0.5166	0.4622	0.3387	0.3493	0.3288	0.3619	0.3732	0.3514
149	Zero-shot FEVER Allen Institute for AI and Un…	01/26/2021	0.2690	0.2371	0.3108	0.3251	0.2866	0.3757	0.3641	0.4226	0.3198	0.4821	0.5595	0.4234
150	test test	05/15/2023	0.2404	0.1527	0.5649	0.2714	0.1724	0.6378	0.2465	0.1529	0.6351	0.2605	0.1616	0.6712
150	test test	05/14/2023	0.2404	0.1527	0.5649	0.2714	0.1724	0.6378	0.2465	0.1529	0.6351	0.2605	0.1616	0.6712
152	ssss anonymous	03/24/2024	0.2395	0.5377	0.1541	0.2983	0.6698	0.1919	0.2898	0.6721	0.1847	0.2898	0.6721	0.1847
153	improved reranker test improved reranker test xd she…	07/05/2024	0.2215	0.1354	0.6081	0.2756	0.1685	0.7568	0.1644	0.0937	0.6712	0.1688	0.0962	0.6892
154	AINLP AINLP	08/22/2022	0.2024	0.4032	0.1351	0.2551	0.5081	0.1703	0.2804	0.4545	0.2027	0.3115	0.5051	0.2252
155	AINLP AINLP	08/07/2022	0.2024	0.1400	0.3649	0.2849	0.1971	0.5135	0.2582	0.1895	0.4054	0.2640	0.1937	0.4144
156	test test	08/06/2022	0.1962	0.1369	0.3459	0.2222	0.1551	0.3919	0.2439	0.1672	0.4505	0.2902	0.1990	0.5360
157	base-rel Prototype WIll disclose soon	11/16/2021	0.1883	0.3106	0.1351	0.2147	0.3540	0.1541	0.2509	0.5538	0.1622	0.2648	0.5846	0.1712
158	Scalable Scientific Verificat… Mikkel Bak Bertelsen, Mikkel …	06/01/2021	0.1859	0.2283	0.1568	0.5096	0.6260	0.4297	0.2434	0.2949	0.2072	0.2540	0.3077	0.2162
159	F+CF Sentence Hybrid XLNet Boey	07/24/2023	0.1751	0.1223	0.3081	0.2442	0.1706	0.4297	0.2254	0.1530	0.4279	0.2894	0.1965	0.5495
160	F+CFP Document XLNet Boey	07/24/2023	0.1373	0.1000	0.2189	0.2169	0.1580	0.3459	0.1957	0.1438	0.3063	0.2388	0.1755	0.3739
161	Test Test	05/14/2023	0.1230	0.0888	0.2000	0.2494	0.1801	0.4054	0.1141	0.0781	0.2117	0.1238	0.0847	0.2297
162	base_3_4 claim verification	03/15/2021	0.1063	0.0587	0.5595	0.1366	0.0755	0.7189	0.1605	0.0954	0.5045	0.2135	0.1269	0.6712
163	roberta_base scifact test Charles	01/02/2022	0.1060	0.1531	0.0811	0.2332	0.3367	0.1784	0.0864	0.0795	0.0946	0.0905	0.0833	0.0991
164	pasic_scibert_tfidf Xiaodong,Wu	03/15/2021	0.1022	0.0795	0.1432	0.2777	0.2159	0.3892	0.1246	0.1011	0.1622	0.1384	0.1124	0.1802
165	ASCME 2 stage Will provide	04/19/2024	0.0833	0.0448	0.5919	0.1114	0.0599	0.7919	0.0763	0.0556	0.1216	0.3644	0.2654	0.5811
166	test C	08/10/2022	0.0730	0.0394	0.5054	0.0984	0.0530	0.6811	0.0890	0.0483	0.5676	0.0925	0.0502	0.5901
167	test test	05/05/2023	0.0696	0.0448	0.1568	0.0780	0.0502	0.1757	0.0896	0.0575	0.2027	0.3586	0.2302	0.8108
168	test test	05/05/2023	0.0558	0.0433	0.0784	0.0577	0.0448	0.0811	0.0410	0.0243	0.1306	0.1921	0.1139	0.6126
169	Retrieval Augmented NLI Théophile Mandon from LIRMM w…	05/07/2024	0.0536	0.0433	0.0703	0.0928	0.0750	0.1216	0.0695	0.0512	0.1081	0.2084	0.1535	0.3243
170	test test	05/04/2023	0.0399	0.2581	0.0216	0.0399	0.2581	0.0216	0.0488	0.2500	0.0270	0.0569	0.2917	0.0315
171	tttta AAAA11	10/13/2022	0.0297	0.0155	0.3486	0.0581	0.0303	0.6811	0.0285	0.0178	0.0721	0.1408	0.0878	0.3559
172	test Carlos Alvarez, Maxwell Benne…	04/22/2024	0.0251	0.0299	0.0216	0.0251	0.0299	0.0216	0.0327	0.0299	0.0360	0.7224	0.6604	0.7973
173	test_our_icl Carlos Alvarez, Maxwell Benne…	05/01/2024	0.0246	0.0286	0.0216	0.0246	0.0286	0.0216	0.0319	0.0286	0.0360	0.7171	0.6429	0.8108
174	Multi-shot Prompting with GPT… Carlos Alvarez, Maxwell Benne…	04/04/2024	0.0238	0.0265	0.0216	0.0238	0.0265	0.0216	0.0305	0.0265	0.0360	0.6756	0.5861	0.7973
175	thamtran Tran Thi Tham from HCMUT	12/12/2023	0.0217	0.0171	0.0297	0.0847	0.0667	0.1162	0.0130	0.0235	0.0090	0.0717	0.1294	0.0495
176	test_no_icl Carlos Alvarez, Maxwell Benne…	05/09/2024	0.0213	0.0243	0.0189	0.0213	0.0243	0.0189	0.0275	0.0243	0.0315	0.6863	0.6076	0.7883
177	test_our_icl_3.5 Contributors: Carlos Alvarez,…	05/27/2024	0.0121	0.0137	0.0108	0.0182	0.0206	0.0162	0.0156	0.0137	0.0180	0.3899	0.3436	0.4505
178	test_no_icl_3.5 Carlos Alvarez, Maxwell Benne…	05/23/2024	0.0103	0.0142	0.0081	0.0137	0.0189	0.0108	0.0138	0.0142	0.0135	0.3502	0.3585	0.3423
179	test_scifact_icl_3.5 Carlos Alvarez, Max Bennett, …	05/27/2024	0.0088	0.0097	0.0081	0.0176	0.0194	0.0162	0.0113	0.0097	0.0135	0.3985	0.3419	0.4775
180	test test	05/14/2023	0.0054	0.5000	0.0027	0.0054	0.5000	0.0027	0.0089	0.5000	0.0045	0.0089	0.5000	0.0045
181	test test	10/18/2022	0.0000	0.0000	0.0000	0.0041	0.0086	0.0027	0.0000	0.0000	0.0000	0.0741	0.1176	0.0541

Leaderboards

SciFact

Sentence Selection+Label (F1) Over Time

Metrics.