🏆 Text2Model Leaderboard

About

This leaderboard tracks AI models' performance in generating optimization models/solutions (e.g. MiniZinc) for the verified problems in the following dataset.

Dataset: skadio/text2zinc

Current Rankings


gpt-4o-mini	cot_with_code_and_grammar_validation	79.09	42.73	60.91	47	110	62.5	12.5	57.47	34.48


gpt-5.2	cot_with_grammar_validation	79.09	42.73	60.91	47	110	90	30	78	44
gpt-4o	cot_with_code_validation	80	41.82	60.91	46	110	90	30	79	43
gpt-5.2	cot_with_code_and_grammar_validation	74.55	40	57.27	44	110	80	30	74	41
gpt-5.2	agents_with_code_validation	77.27	36.36	56.82	40	110	80	20	77	38
gpt-4o	cot_with_code_and_grammar_validation	73.64	40	56.82	44	110	90	30	72	41
gpt-5.2	cot	70	38.18	54.09	42	110	60	20	71	40
gpt-5.2	gala	67.27	39.09	53.18	43	110	60	30	68	40
gpt-5.2	cot_with_code_validation	67.27	36.36	51.82	40	110	70	30	67	37
gpt-5.2	baseline	64.55	37.27	50.91	41	110	50	20	66	39
gpt-4o	cot	60.91	34.55	47.73	38	110	50	10	62	37
gpt-4	cot_with_code_and_grammar_validation	70	24.55	47.27	27	110	80	20	69	25
gpt-5.2	knowledge_graph	57.27	36.36	46.82	40	110	40	10	59	39
o3-mini	global_agents	57.89	33.68	45.79	32	95	62.5	25	57.47	34.48
o3-mini	baseline	56.84	32.63	44.74	31	95	62.5	25	56.32	33.33
gpt-5.2	agents	55.45	33.64	44.55	37	110	60	0	55	37
gpt-4	cot	57.27	28.18	42.73	31	110	70	30	56	28
gpt-4	cot_with_code_validation	57.27	28.18	42.73	31	110	70	30	56	28
gpt-4	cot_with_grammar_validation	62.73	22.73	42.73	25	110	80	30	61	22
gpt-4	knowledge_graph	48.18	25.45	36.82	28	110	60	20	47	26
gpt-4	agents_with_code_validation	44.55	21.82	33.18	24	110	50	0	44	24
gpt-4	agents	44.55	20	32.27	22	110	50	0	44	22
gpt-4o-mini	global_agents	33.64	17.27	25.45	19	110	30	20	34	17
gpt-4	baseline	32.73	17.27	25	19	110	50	20	31	17
gpt-oss:20b	cot	18.37	11.22	14.8	11	98	12.5	0	18.89	12.22
gpt-oss:20b	global_agents	17.27	8.18	12.73	9	110	10	0	18	9
gpt-oss:20b	baseline	13.27	8.16	10.71	8	98	12.5	12.5	13.33	7.78

Per-Dataset Breakdown

Execution and solution accuracy broken down by dataset source. This table shows results from our own evaluation runs.


gpt-4o-mini	agents+code_val	65	38.46	13.85	7	85.71	42.86	5	100	100	11	45.45	18.18	22	40.91	18.18	44.54	20.91


gpt-4	agents	65	38.46	13.85	7	85.71	42.86	5	80	80	11	45.45	18.18	22	40.91	18.18	44.54	20
gpt-4	agents+code_val	65	41.54	13.85	7	28.57	28.57	5	80	80	11	54.55	18.18	22	40.91	27.27	43.64	20.91
gpt-4	baseline	65	32.31	12.31	7	42.86	42.86	5	20	20	11	45.45	27.27	22	27.27	18.18	32.73	17.27
gpt-4	cot	65	50.77	20	7	85.71	57.14	5	100	100	11	63.64	36.36	22	54.55	22.73	57.27	28.18
gpt-4	cot+code+gram	65	69.23	20	7	42.86	14.29	5	100	100	11	81.82	36.36	22	68.18	18.18	70	24.55
gpt-4	cot+code_val	65	50.77	20	7	85.71	57.14	5	100	100	11	63.64	36.36	22	54.55	22.73	57.27	28.18
gpt-4	cot+gram_val	65	60	15.38	7	42.86	14.29	5	80	80	11	81.82	36.36	22	63.64	27.27	62.73	22.72
gpt-4	knowledge_graph	65	44.62	20	7	71.43	42.86	5	80	80	11	54.55	36.36	22	40.91	18.18	48.19	25.45
gpt-4o	cot	65	60	29.23	7	71.43	57.14	5	100	100	11	36.36	18.18	22	59.09	36.36	60	34.54
gpt-4o	cot+code+gram	65	72.31	36.92	7	42.86	14.29	5	100	100	11	90.91	45.45	22	72.73	40.91	73.64	40
gpt-4o	cot+code_val	65	76.92	35.38	7	85.71	42.86	5	100	80	11	81.82	36.36	22	86.36	54.55	80.91	41.82
gpt-4o-mini	gala	65	32.31	13.85	7	57.14	42.86	5	40	40	11	36.36	18.18	22	27.27	13.64	33.64	17.28
gpt-5.2	agents	65	50.77	32.31	7	85.71	57.14	5	60	60	11	63.64	18.18	22	54.55	31.82	55.46	33.64
gpt-5.2	agents+code_val	65	76.92	29.23	7	85.71	57.14	5	100	100	11	81.82	36.36	22	68.18	36.36	77.27	36.36
gpt-5.2	baseline	65	63.08	32.31	7	100	57.14	5	100	100	11	63.64	36.36	22	50	31.82	64.55	37.27
gpt-5.2	cot	65	70.77	35.38	7	100	42.86	5	100	100	11	54.55	27.27	22	59.09	36.36	70	38.18
gpt-5.2	cot+code+gram	65	69.23	32.31	7	85.71	57.14	5	100	80	11	90.91	45.45	22	72.73	45.45	74.55	40
gpt-5.2	cot+code_val	65	64.62	30.77	7	71.43	42.86	5	100	100	11	72.73	36.36	22	63.64	36.36	67.28	36.36
gpt-5.2	cot+gram_val	65	76.92	38.46	7	85.71	57.14	5	100	100	11	90.91	45.45	22	72.73	36.36	79.09	42.73
gpt-5.2	gala	65	66.15	32.31	7	85.71	57.14	5	100	100	11	63.64	45.45	22	59.09	36.36	67.27	39.09
gpt-5.2	knowledge_graph	65	52.31	32.31	7	71.43	57.14	5	100	100	11	36.36	27.27	22	68.18	31.82	57.27	36.36
gpt-oss:20b	baseline	55	10.91	5.45	7	0	0	5	40	40	9	11.11	0	22	18.18	13.64	13.27	8.16
gpt-oss:20b	cot	56	14.29	5.36	7	14.29	14.29	5	100	100	9	0	0	21	19.05	9.52	18.37	11.23
gpt-oss:20b	gala	65	15.38	4.62	7	0	0	5	40	40	11	27.27	18.18	22	18.18	9.09	17.27	8.18
o3-mini	gala	54	57.41	27.78	6	83.33	50	5	80	80	8	37.5	37.5	22	54.55	31.82	57.9	33.69

Upload Evaluation Results Only

Best for: When you only have evaluation metrics and are not interested in submitting the code generated using your strategy

Required: summary.json file with these key metrics:

execution_accuracy: % of problems that executed without errors
solution_accuracy: % of problems with correct solutions

Make sure your model name/strategy name combination is unique or check overwrite if you are updating your results

Refresh leaderboard once the upload is successfull

Upload summary.json

Model Name

Strategy Name

Model Description

Check this to replace existing model/strategy combination

⚠️ Overwrite existing submissions

Upload Status

🏆 Text2Model Leaderboard

About

Current Rankings

Per-Dataset Breakdown

Upload Evaluation Results Only

Upload Complete Zip File

Directory Structure of Submissions and Evaluations