Attacks on LLMs
Prompt injection, data leakage, model extraction, privacy evaluation, and secure deployment of foundation models.
AI Security Researcher
Pritha Gupta
I research trustworthy machine learning, LLM security, attacks on LLMs, and automated vulnerability detection for cryptographic systems. My work connects adversarial ML, side-channel analysis, AutoML, and practical security evaluation.
About
Postdoctoral research in trustworthy AI, LLM security, privacy, and cryptographic leakage detection.
I am currently a Postdoctoral Researcher at the Research Center for Trustworthy Data Science and Security, Ruhr University Bochum, where I am a member of the Trustworthy Human Language Technologies (TrustHLT) Research Group. My research primarily focuses on various attacks against Large Language Models (LLMs), including prompt injection, membership inference, model inversion, differential privacy, privacy-preserving fine-tuning, and adversarial evaluation of AI systems. My goal is to better understand the security and privacy risks of modern AI systems and to develop methods for building more trustworthy and robust foundation models.
Previously, I completed my Dr. rer. nat. at the Software Innovation Campus Paderborn (SICP), where my research focused on advanced machine learning methods for information leakage detection in cryptographic systems. During my doctoral studies, I developed machine learning and information-theoretic methods for side-channel analysis, automated attack generation using AutoML and Neural Architecture Search, and security evaluation of cryptographic implementations.
I received my Bachelor's degree in Computer Engineering from Netaji Subhas University of Technology (NSUT), securing admission through the highly competitive engineering entrance examination with an All India Rank (AIR) of 5383 and a Common Entrance Examination (CEE) Rank of 287, achieved without any formal coaching. This experience shaped my interest in computer science and motivated my pursuit of research in artificial intelligence and machine learning.
Before pursuing my PhD, I worked as a Software Engineer at Samsung Research Institute-Noida, contributing to Android framework development and mobile software engineering. This industrial experience continues to influence my research by motivating practical, secure, and deployable AI systems.
My broader research interests include trustworthy artificial intelligence, machine learning, adversarial machine learning, AI security and privacy, information security, privacy-preserving AI, and automated machine learning. I am particularly interested in developing reliable and secure AI systems that can be safely deployed in real-world applications.
Research Focus
Security evaluation for modern AI systems, with a foundation in cryptographic leakage detection and applied machine learning.
Adversarial ML
Gradient reconstruction attacks, membership inference, adversarial machine learning, and privacy-preserving AI.
Automated Vulnerability Detection
Meta-learning, AutoML, and neural architecture search for side-channel and cryptographic analysis.
Experience
Research and engineering roles across AI security, machine learning, Android systems, and applied analytics.
Feb. 2025 - Present
Ruhr University Bochum
- Conduct security evaluations of open-source LLMs against prompt injection, membership inference attacks, reconstruction attacks, and model extraction attacks.
- Develop automated pipelines for benchmarking AI model security and privacy risks.
- Research adversarial machine learning, privacy-preserving AI, gradient reconstruction attacks, and membership inference attacks.
- Investigate automated and LLM-based vulnerability detection for hardware wallets and ledgers.
Feb. 2019 - Jan. 2025
Software Innovation Campus Paderborn, University of Paderborn
- Designed a meta-learning method for selecting anomaly detectors using performance prediction from normal-only training data.
- Improved cryptographic leakage detection by 30-40% with AutoML and neural architecture search for black-box vulnerability analysis.
- Released the open-source AutoSCA tool, adopted by industry partner achelos GmbH.
- Collaborated with achelos GmbH, ABP, and WestfalenWIND on ML and LLM-based solutions.
Jan. 2016 - Nov. 2018
Intelligente Systeme und Maschinelles Lernen, University of Paderborn
- Developed neural-network preference learning models using context-dependent ranking and choice algorithms.
- Deployed Android survey applications to collect user preference data on tablets and smartphones.
Aug. 2015 - Dec. 2015
Diebold Nixdorf, formerly Wincor Nixdorf
- Built a Vaadin-based web application with Java-based statistical analysis for ATM fleet performance in eight weeks.
Nov. 2014 - Mar. 2015
Thapar University
- Implemented a Raspberry Pi-based driverless car system with road detection algorithms and wireless Android app control.
- Published research on autonomous vehicle system design in the International Journal of Computer Applications.
Jun. 2012 - Oct. 2014
Samsung Research Institute-Noida / Samsung India Electronics Pvt. Ltd.
- Developed Android OS, framework, and application features including TouchWiz Home, My Magazine, and Easy Launcher.
- Conducted feasibility studies for new features, collaborated with Samsung HQ and Europe Test Teams, and mentored interns.
Education
Machine learning, computer science, and security research training across Germany and India.
Dr. rer. nat.
University of Paderborn, 2025. Magna cum Laude. Dissertation on advanced machine learning methods for information leakage detection in cryptographic systems.
MSc Computer Science
University of Paderborn, 2018. Grade 1.2, with coursework focused on machine learning and software engineering.
B.E. Computer Engineering
Netaji Subhas University of Technology, 2012. CEE Rank 287 and AIR 5383.
Selected Publications
Full publication list available on Google Scholar.
- From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMs Jiawen Wang, Pritha Gupta, Ivan Habernal, Eyke Huellermeier, Xiaoxue Gao, Nancy F. Chen. arXiv, 2026. Paper
- Information Leakage Detection through Approximate Bayes-optimal Prediction Pritha Gupta, Marcel Wever, Eyke Huellermeier. Information Sciences, 2025. Paper
- Meta-Learning for Automated Selection of Anomaly Detectors for Semi-Supervised Datasets David Schubert, Pritha Gupta, Marcel Wever. International Symposium on Intelligent Data Analysis, 2023. Paper
- Automated Side-Channel Attacks using Black-Box Neural Architecture Search Pritha Gupta, Jan Peter Drees, Eyke Huellermeier. 18th International Conference on Availability, Reliability and Security, 2023. Paper
- Learning Context-Dependent Choice Functions Karlson Pfannschmidt, Pritha Gupta, Bjoern Haddenhorst, Eyke Huellermeier. International Journal of Approximate Reasoning, 2022. Paper
- Automated Information Leakage Detection: A New Method Combining Machine Learning and Hypothesis Testing with an Application to Side-channel Detection in Cryptographic Protocols Pritha Gupta, Arunselvan Ramaswamy, Jan Peter Drees, Eyke Huellermeier, Claudia Priesterjahn, Tibor Jager. Proceedings of the 14th International Conference on Agents and Artificial Intelligence, 2022. Paper
- Pairwise versus Point-wise Ranking: A Case Study Vitalik Melnikov, Eyke Huellermeier, Daniel Kaimann, Bernd Frick, Pritha Gupta. Theoretical Foundations of Machine Learning, 2017. Paper
- Design and Implementation of Autonomous Car using Raspberry Pi Gurjashan Singh Pannu, Mohammad Dawud Ansari, Pritha Gupta. International Journal of Computer Applications, 113(9):22-29, 2015. Paper
Skills
A practical stack for AI security research, prototyping, and experimentation.
Python
PyTorch
TensorFlow
Scikit-learn
AutoML
Hugging Face
Attacks on LLMs
Side-Channel Analysis
Docker
Git
Linux
Java
C++
Android
German A2
English C2
Hindi Native
Contact
Based in Bochum, Germany. Open to research collaborations and applied AI security work.
Academic References
- Prof. Dr. Eyke Huellermeier, LMU Munich, AIML - Artificial Intelligence and Machine Learning, eyke@ifi.lmu.de
- Prof. Dr.-Ing. Ivan Habernal, Trustworthy Human Language Technologies, Ruhr University Bochum, ivan.habernal@ruhr-uni-bochum.de
- Prof. Dr.-Ing. Juraj Somorovsky, University of Paderborn, juraj.somorovsky@uni-paderborn.de
- Assoc. Prof. Arunselvan Ramaswamy, Dept. of Industrial Engineering and Operations Research, IIT Bombay, arunselvan.ramaswamy@kau.se, arunselvan.r@gmail.com
- Prof. Dr. Stefan Sauer, Software Innovation Campus Paderborn, University of Paderborn, sauer@uni-paderborn.de
- Prof. Dr. Daniel Beverungen, Software Innovation Campus Paderborn (SICP), Sprecher - Professor, Sprecher SICP-Steuerkreis, Vorstandsvorsitzender SI-Lab, daniel.beverungen@uni-paderborn.de
- Prof. Dr.-Ing. Shampa Chakraverty, Dean Student Welfare, Faculty of Design, Computer Engineering, NSUT Delhi, shampa@nsut.ac.in
- Prof. Dr.-Ing. Bijendra Kumar, Computer Engineering, NSUT Delhi, bizender@gmail.com
Industrial and Other References
- Prof. Dr. Stefan Sauer, Software Innovation Campus Paderborn, University of Paderborn, sauer@uni-paderborn.de
- Manusheel Gupta, LinkedIn, manusheel.gupta@gmail.com