ResearchCodeBench evaluates AI agents' ability to implement algorithms from academic papers. Contains 212 code implementation tasks across 20 ML/AI research problems from top-tier venues (ICLR, NeurIPS, CVPR, COLM). Tests paper comprehension, algorithm understanding, and precise code implementation skills with 1,449 lines of reference code.
uvx harbor run -d researchcodebench@1.0uvx harbor run -d researchcodebench@1.0uvx harbor run -d researchcodebench@1.0 -t siss_importance_sampling_weightsuvx harbor run -d researchcodebench@1.0 -t siss_subtracted_importance_sampled_scores_importance_sampling_with_mixtureuvx harbor run -d researchcodebench@1.0 -t tabdiff_compute_total_noise_for_categorical_features_with_learnable_kuvx harbor run -d researchcodebench@1.0 -t tabdiff_compute_total_noise_for_numerical_features_with_learnable_rhouvx harbor run -d researchcodebench@1.0 -t tabdiff_initialize_the_learnable_feature-wise_parameter_k_for_categorical_featuresuvx harbor run -d researchcodebench@1.0 -t tabdiff_initialize_the_learnable_feature-wise_parameter_rho_for_numerical_featuresuvx harbor run -d researchcodebench@1.0 -t tabdiff_make_sure_learnable_parameter_ks_for_categorical_features_are_positiveuvx harbor run -d researchcodebench@1.0 -t tabdiff_make_sure_learnable_parameter_rhos_are_greater_than_rho_offsetuvx harbor run -d researchcodebench@1.0 -t tanh-init_identity_matrixuvx harbor run -d researchcodebench@1.0 -t tanh-init_identity_matrix_elseuvx harbor run -d researchcodebench@1.0 -t tanh-init_proposed_weight_initializationuvx harbor run -d researchcodebench@1.0 -t tanh-init_update