Research Interests

3 min read

I develop algorithms that allocate resources before knowing future demands or operating conditions. I am interested in learning from this interaction, exploiting useful predictions, and providing rigorous guarantees when the environment behaves unexpectedly.

Learning while the system runs

In online learning, an algorithm acts, observes feedback, and improves its next decisions. As Cesa-Bianchi notes in his foreword to Orabona’s book, learning and evaluation are interleaved, complementing the familiar separation between training and testing. Observations need not be independent samples from a fixed distribution.

The central performance measure is regret: the algorithm’s accumulated cost relative to a benchmark, such as the best fixed decision in hindsight (for content delivery networks, the files that would have served users best). A suitable algorithm makes its average excess cost vanish over time without ever seeing future requests. This possibility is what draws me to the field.

Several mathematical frameworks capture this problem and form the basis of my work:

  • Online convex optimization (OCO) uses convex costs and feasible sets to design efficient algorithms for sequential decisions, from bandwidth allocation and portfolio selection to updating prediction models as data arrive; see Orabona, Hazan, and Shalev-Shwartz.
  • Bandits capture limited feedback: a recommendation system observes responses only to the items it displays, so learning must balance exploration and exploitation. Stochastic bandits assume stable reward distributions, while adversarial bandits do not; “adversarial” describes the guarantee’s scope, not necessarily a malicious opponent. Lattimore and Szepesvári develop both settings.
  • Markov decision processes (MDPs) and control incorporate lasting consequences: actions affect immediate rewards and future states, such as battery reserves. Learning in unknown MDPs requires reinforcement learning; see Chapter 38 of Lattimore and Szepesvári. Control theory captures the same interplay through system dynamics, and Hazan and Singh study it from an online learning perspective as non-stochastic control.

The broader systems perspective of these frameworks is considered in our survey of pervasive AI, covering federated learning, distributed inference, and bandits under communication and computation constraints.

Using predictions without trusting them

Now, consider a prediction of what happens next: a network operator might forecast demand with a machine learning model. Accurate forecasts could improve allocation, but their reliability may change precisely when good decisions matter most.

My doctoral thesis asks how algorithms can use untrusted predictions while maintaining worst-case regret guarantees: robust to inaccurate forecasts, yet gaining from informative ones. Optimism means adapting to prediction quality, which need not be known in advance.

The protocol of optimistic learning: a predictor, a learner, and an environment interacting over time slots, with the offline, online, and optimistic learning settings outlined.
The protocol of optimistic learning: at each time slot, an untrusted prediction arrives (1); the learner acts on past costs, the prediction, and its accuracy so far (2); and the cost is realized (3) before the next prediction (4).

Caching is a concrete starting point. Our TMC paper develops optimistic algorithms for bipartite caching networks, where topology determines which caches serve each request, and learns which prediction sources are useful. Our SIGMETRICS paper on discrete caching establishes performance limits and algorithms for whole files and, more generally, the online Knapsack problem. The guarantees improve with prediction accuracy while preserving worst-case regret rates. Our monograph on optimistic learning extends these ideas across communication networks.

The thesis then moves beyond a fixed benchmark: if demand changes, so can the best allocation. Through dynamic regret, I study performance against a moving sequence of decisions; our ICML work on optimism with history pruning develops a guarantee for this harder setting.

Finally, I study systems with memory through non-stochastic control. As in an MDP, actions affect future states, but the dynamics and convex costs may be adversarial, and we compete with the best policy in a specified class, chosen in hindsight. My thesis shows how forecasts over an extended horizon help account for the lasting effects of today’s actions.

When changing a decision has a cost

More recently, I have become interested in smoothed online learning, where an algorithm pays both for its decisions and for changing them. In mobile networks, work on smooth handovers balances connectivity against handover costs; for AI inference, recent work applies smoothed online convex optimization to server/GPU provisioning with reconfiguration costs.

These examples motivate algorithms that balance responsiveness with stability; our recent AISTATS paper studies how much an algorithm can moderate its updates while tracking a changing benchmark, and our NeurIPS paper handles non-Lipschitz switching costs.

References for further reading

  1. Francesco Orabona. Online Learning: A Modern Introduction Using Convex Optimization. 2026 manuscript.
  2. Elad Hazan. Introduction to Online Convex Optimization. 2nd edition, MIT Press, 2022.
  3. Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. Cambridge University Press, 2020. Free online edition.
  4. Naram Mhaisen. Optimistic Learning with Applications to Caching Networks. PhD thesis, TU Delft, 2025.
  5. George Iosifidis, Naram Mhaisen, and Douglas J. Leith. Optimistic Learning for Communication Networks. Foundations and Trends in Networking, to appear.