Benchmark Principles for Academic AI PhDs

To provide orientation to students and institutions, a set of common requirements has been established, for a PhD preparing for a successful career in AI in academia or industry. The benchmark principles shall help spread AI excellence across the world. They are general and apply to both, PhD studies conducted primarily in academia and PhD studies conducted partially or predominantly in industry. The authors of these benchmark principles are convinced that there must not be any difference in quality between a PhD conducted in academia and a PhD conducted in industry. Both should be equivalent.

Members of the AIDA PhD Benchmark Principles Committee can be visualised in the AIDA Committees page.

Find out below the set of requirements that should be met by PhD students.

Research Excellence and Impact

To establish the foundation for a successful career, PhD students should strive for research excellence and impact. As a rule of thumb, a successful PhD student should have authored at least 3 publications in top-tier conferences and/or journals as first author within 3 to 4 years. Note that in contrast to other research areas, in the field of AI and ML, top-tier conferences are typically considered the more important publication venues compared to journals. Top-tier conferences and journals in AI and ML related areas include:

  • Core ML: NeurIPS, ICML, ICLR, UAI, AISTATS, COLT, JMLR
  • Core AI: AAAI, IJCAI, JAIR, AIJ
  • Computer Vision: CVPR, ICCV, ECCV, IJCV, PAMI
  • Natural Language Processing: ACL, EMNLP, TACL
  • AI-related areas: Robotics, Computer Graphics, Data Mining, Multimedia, Affective Computing, HCI, etc.

One publication could be in an equal-level top tier conference/journal in a different but AI-related area. Impact can be measured by the number of citations a publication receives within the first 3 or 5 years. Impactful work may be cited several hundred times within the first 3 years. However, citations vary across subfields, depending on their size. Hence citations alone are not a meaningful measure for success and should be complemented by other metrics, in particular in the case of interdisciplinary work. All research results including papers, source code and data should generally be given the possibility of publication as common in the field of AI and ML. Publication should not be hindered or significantly delayed by potential patenting processes.

Educational Background

A successful career in AI necessitates the required theoretical and practical foundations acquired through coursework or in-depth self-study, during the overall course of (undergraduate and graduate) study. In this context, two areas of study are particularly relevant:

  • Fundamentals: Mathematics: Calculus, Linear Algebra, Logic, Probability Theory, Statistics, Optimization; Computer Science: Programming, Algorithms and Data Structures, Theoretical CS
  • Core Artificial Intelligence: Machine Learning Algorithms and Paradigms (including Supervised Learning, Unsupervised Learning, Reinforcement Learning, Deep Learning), Statistical Learning Theory, Probabilistic Modeling, Causal and Symbolic AI, Knowledge Representation, Search Algorithms, Computer Vision, NLP, Robotics and Planning, Constraint Satisfaction Problems, Reasoning

They can be complemented by courses in interdisciplinary areas at the interface with AI:

  • Neighboring Fields: Data Science, Physics, Psychology, Philosophy, Social Science, Ethics, Biology, Medicine, Economics, Linguistics, Neuroscience, etc.

The above assumes that basic training in computing and programming has taken place during the undergraduate education.

Research Environment

A PhD student should be closely supervised (at minimum monthly) by an expert in AI or ML research and be surrounded by peers (eg, PhD students and PostDocs) conducting research in AI or ML in order to acquire the necessary academic skill set. If the PhD studies are conducted at a lab where such an environment does not exist, eg, in industry or within another field where AI/ML techniques are applied but not developed, the PhD student should be co-supervised by an expert in AI or ML and ideally conduct a research visit of at least 6 months during the beginning of the PhD period in a leading AI or ML research lab to acquire the necessary skill set. In general, it is encouraged that PhD students collaborate or spend time with researchers from other countries to strengthen their network, eg, via joint/collaborative PhD programmes or R&D projects.

Teaching is a significant skill that should be developed during the PhD through TA’ing and supervising Bachelor and Master students. During the PhD studies, the PhD student should spend the majority of time on research related to the PhD and the minority of time on other tasks (applications, industry work, teaching).

For PhD studies conducted in industry, it is desirable to let the problem types be influenced by industry, but to assign maximal freedom to the PhD student in terms of conducting fundamental research in this area. Given the necessary computing power required by many machine learning models, the PhD student should have access to a large enough (GPU) compute cluster during the entire period of study.

Continuous Learning

A successful career requires the candidate to learn about recent advances and developments, for example through participating in short courses or attending conferences and workshops (at least one conference and possibly one short course per year). During the beginning of the PhD, the PhD student should attend at least one summer school in machine learning or a related field, delivered by experts in the respective field.