Generative Al in Recruitment: A Comparative Study of Llama2 and ChatGPT for Resume Screening

Generative AI has the potential to enhance hiring efficiency at scale; however, the integration of Large Language Models (LLMs) into such processes requires examination of trade-offs. Thus, we study a popular open-source model, Llama 2-13B, and a close-source model, ChatGPT, in the context of the recruitment process. We focus on their lexical processing, explanation, and decision-making abilities. We aim to address critical concerns, such as enhancing the financial accessibility of open-source LLMs, addressing privacy concerns, improving explainability in hiring processes, and ensuring transparency and reproducibility in their outcomes. We use a resume dataset (Jiechieu & Tsopze, 2020) and a job description dataset from PromptCloud and DataStock. First, we assess the capability of Llama2 and ChatGPT to convert resumes into structured data and effectively align them with job descriptions. Next, we utilize methodologies such as word embedding, BERTopic, and LlamaIndex to determine how well these models match resumes with job descriptions based on their suitability. Additionally, we investigate the key input features that influence decision-making in Llama2 during the resume screening process and employ Integrated Gradients and Chain-of-Thought Prompting to explore the models’ explainability (Parasurama et al., 2022). This study emphasizes the necessity of evaluating Generative AI applications in recruitment to enhance efficiency and ensure fairness in the automated hiring process.

Recommended citation: Your Name, You. (2015). "Paper Title Number 3." Journal 1. 1(3).
Download Paper | Download Slides