Mitigation of Reinforcement Learning Algorithms in Changing Environments

The aim of the Rosetrees 2022 Interdisciplinary Award is to encourage collaboration between Clinicians/Scientists and experts in Artificial Intelligence/Machine Learning. We hope that the merging of skills and experience in these areas will lead to innovative research that addresses unmet clinical needs. Applicant proposals must include details on how artificial intelligence and machine learning can result in patient benefit within 5-10 years.

Projects that address logistical problems within the NHS that will result in more efficient services, reduce costs and/or improve patient experience are of interest.

Applications open: NOW
Closes: 31 March 2022
Funding for: UK Students, EU Students, International Students
Qualification: PhD
Hours: Full-time
Location: Manchester

Under this research studentship, the successful candidate will conduct research as part of the EPSRC Industrial CASE Award funded project.

Theme of the PhD project:

The development of (deep) Reinforcement Learning (RL) algorithms to train agents within game environments is well known. Agent training is typically conducted against a known, simplified, or constrained environment. However, the deployed environment is typically more complex, and subject to some change and uncertainly not represented in the training environment. RL algorithms typically characterise performance against probabilistic arenas, rather than being able to cope with an environment that is subject to change over time. The performance of the resulting RL agent can then be expected to become compromised over time, but not necessarily be catastrophic. In this PhD project, we are concerned with (i) understanding this performance degradation and (ii) the development of mitigating strategies. More specifically, the project will focus at creating a train-and-test framework comprising a simulation engine for dynamic environment and a configurable RL approach. In addition to considering changes in the environment, the simulator and RL agent will need to account for real-world challenges, such as multiple conflicting objectives, robustness, and safety issues.

The team at BAE Systems is focused on cutting-edge research in advanced simulation, optimization, and machine learning, and are thus invested in how RL can be extended to support decision making in dynamic environments. The project will therefore contribute directly to BAE Systems’ ongoing research. From a scientific perspective, this project will lead to cross-disciplinary research and output that is of high quality and significance.

Supervision:

The successful candidate will be supervised by Dr. Richard Allmendinger, Dr. Theodore Papamarkou and Dr. Wei Pan from The University of Manchester.  Simon Mettrick and Markus Deittert from BAE Systems will act as external supervisors and lead the industrial input into this research.

Nature of the studentship:

The 4-year studentship will commence in September 2022, covering full tuition fees and a stipend equivalent to UKRI rates (approx. £15,609 tax free for 2021/22; subject to change for future years), plus an industrial top-up stipend from BAE Systems, subject to contract. It also provides travel support for fieldwork, conferences and annual visits to BAE Systems.

Entry Requirements:

Applications are sought from talented and motivated Home and International candidates with an academic background in at least one of these fields: (Deep/multi-agent) reinforcement learning, deep learning, Gaussian processes, Bayesian optimization, transfer/online/meta-learning/safe/multitask learning, dynamic control.

Applicants must hold:

• 1^st or 2:1 Honours degree (or equivalent), and
• Masters degree with Distinction 

English Language requirements (where required) are IELTS 7.0, TOEFL 623 (100 ibt), PTE 76.

How to Apply:   Candidates should submit a PhD application for the PhD Business & Management, and indicate that they wish to be considered for the EPSRC/BAE Systems INDUSTRIAL CASE PhD Studentship.

The application must contain:

• A 3000-word research proposal related to the topic, and
• A written statement clearly indicating how your research competencies and interests to date are aligned with the specific nature of the PhD projects. 

If you do not submit the required supporting documents outlined above by the deadline, your application will not be considered.

Application Deadline: 31 March 2022

Enquiries

Further details about the project, contact richard.allmendinger@manchester.ac.uk with an up-to-date CV including any publication profile.

Making your application: ambs-pgresearch@manchester.ac.uk

Related