OpenAI wants to retire the leading AI coding benchmark—and the reasons reveal a deeper problem with how the whole industry measures itself.

Abstract: Programming language source code vulnerability mining is crucial to improving the security of software systems, but current research is mostly focused on the C language field, with little ...

Abstract: We performed a comparative analysis of code generation model performance with evaluation using common NLP metrics in comparison to a test-based evaluation. The investigation was performed in ...