Dũng Nguyễn Mạnh

I’m Dũng (Dzung). I graduated with honors degree in Computer Science from University of Engineering and Technology, Vietnam National University (UET).

Since 2022, I’ve been working as a researcher and engineer at FPT Software AI Center’s AI Residency program, under the mentorship of Dr. Nghi Bui from Singapore Management University.

My passion lies in embedding Deep Learning into software systems to tackle code-related challenges, (e.g. code search, code repair and code generation). With the growing impact of Large Language Models (LLMs), my current focus is on integrating these models into coding-assist tools. My projects include automated unit-test generation for testers, developing an all-in-one chatbot to assist developers, and evaluating LLMs’ ability to comprehend and solve programming problems.

My goal is to transform software development by making it more efficient and accessible. I aim to build AI-driven agents capable of understanding, generating, and optimizing source code, reducing the need for human intervention and minimizing errors.

I’m best reached via email.

news

Oct 14, 2024	🚀 A new benchmark for measuring coding understanding capability of CodeLLMs, checkout our homepage CodeMMLU for more details. 📋
Oct 08, 2023	I happy to share that our work on The Vault has been accepted to EMNLP Findings 2023. See more details HERE.
Oct 05, 2022	I got my Bachelor degree in Computer Science from University of Engineering and Technology, Vietnam National Unviersity (CGPA: 3.61). 🧑‍🎓💐
Jul 10, 2022	I was offered to become full-time AI Resident at FPTSoftware AI Center

latest posts

Mar 23, 2024	Mixture of expert explanation
Aug 19, 2023	Alternative dataset for Code Understanding and Generation

selected publications

ArXiv.preprint
CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding Capabilities of CodeLLMs

Dung Nguyen Manh, Thang Phan Chau, Nam Le Hai, Thong T. Doan, Nam V. Nguyen, Quang Pham, and Nghi D. Q. Bui

2024

Abs arXiv Bib PDF Code

Recent advancements in Code Large Language Models (CodeLLMs) have predominantly focused on open-ended code generation tasks, often neglecting the critical aspect of code understanding and comprehension. To bridge this gap, we present CodeMMLU, a comprehensive multiple-choice question-answer benchmark designed to evaluate the depth of software and code understanding in LLMs. CodeMMLU includes over 10,000 questions sourced from diverse domains, encompassing tasks such as code analysis, defect detection, and software engineering principles across multiple programming languages. Unlike traditional benchmarks, CodeMMLU assesses models’ ability to reason about code rather than merely generate it, providing deeper insights into their grasp of complex software concepts and systems. Our extensive evaluation reveals that even state-of-the-art models face significant challenges with CodeMMLU, highlighting deficiencies in comprehension beyond code generation. By underscoring the crucial relationship between code understanding and effective generation, CodeMMLU serves as a vital resource for advancing AI-assisted software development, ultimately aiming to create more reliable and capable coding assistants.
@misc{manh2024codemmlumultitaskbenchmarkassessing, title = {CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding Capabilities of CodeLLMs}, author = {Manh, Dung Nguyen and Chau, Thang Phan and Hai, Nam Le and Doan, Thong T. and Nguyen, Nam V. and Pham, Quang and Bui, Nghi D. Q.}, year = {2024}, eprint = {2410.01999}, archiveprefix = {arXiv}, primaryclass = {cs.SE}, url = {https://arxiv.org/abs/2410.01999}, }
EMNLP’23
The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation

Dung Nguyen Manh, Nam Le Hai, Anh T. V. Dau, Anh Minh Nguyen, Khanh Nghiem, Jin Guo, and Nghi D. Q. Bui

2023

Abs arXiv Bib PDF Code

We present The Vault, an open-source, large-scale code-text dataset designed to enhance the training of code-focused large language models (LLMs). Existing open-source datasets for training code-based LLMs often face challenges in terms of size, quality (due to noisy signals), and format (only containing code function and text explanation pairings). The Vault overcomes these limitations by providing 40 million code-text pairs across 10 popular programming languages, thorough cleaning for 10+ prevalent issues, and various levels of code-text pairings, including class, function, and line levels. Researchers and practitioners can utilize The Vault for training diverse code-focused LLMs or incorporate the provided data cleaning methods and scripts to improve their datasets. By employing The Vault as the training dataset for code-centric LLMs, we anticipate significant advancements in code understanding and generation tasks, fostering progress in both artificial intelligence research and software development practices.
@misc{manh2023vault, title = {The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation}, author = {Manh, Dung Nguyen and Hai, Nam Le and Dau, Anh T. V. and Nguyen, Anh Minh and Nghiem, Khanh and Guo, Jin and Bui, Nghi D. Q.}, year = {2023}, eprint = {2305.06156}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, }