This folder contains the benchmark proteins and annotations used in the preliminary assessment of CAFA3.
Benchmark proteins and annotations were generated by comparing the experimental annotation gain in UniProt GOA between two time points t0 and t1. This period is called the annotation growth period.

t0 = 20170213
t1 = 20170605

In this benchmark set we considered experimental annotations as having the following evidence code: EXP, IDA, IPI, IMP, IGI and IEP.
Note that the evidence codes considered to be experimental in this preliminary analysis do not contain TAS and IC.

For more information on CAFA benchmark generation, please refer to the CAFA2 paper[1]: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6.
Code for comparing two UniProt GOA files for experimental annotation differences can be found at https://github.com/nguyenngochuy91/CAFA_benchmark.
If you have any questions regarding the CAFA benchmark generation process, please contact nzhou@iastate.edu.


[1]Jiang, Yuxiang, et al. "An expanded evaluation of protein function prediction methods shows an improvement in accuracy." Genome biology 17.1 (2016): 184.