<link>https://naram-m.github.io/</link></image><item><title>Dynamic Regret in Online Convex Optimization with Indicator Switching Costs</h1> <article> <h1>Dynamic Regret in Online Convex Optimization with Indicator Switching Costs</h1> <p>Thu, 01 Oct 2026 00:00:00 +0000</p> </article> <article> <h1>Paper Accepted at NeurIPS 2026</h1> <p>Thu, 01 Oct 2026 00:00:00 +0000</p> <p>Our paper <em>“Dynamic Regret in Online Convex Optimization with Indicator Switching Costs,”</em> with George Iosifidis, has been accepted at <strong>NeurIPS 2026</strong>.</p> <p>The paper studies online decisions that incur a fixed cost whenever they change, capturing startup overheads such as server activation, model deployment, and cache updates. Existing guarantees for such costs only compete against a fixed benchmark. We propose a meta-learning algorithm whose dynamic regret is minimax-optimal, up to logarithmic factors, for tracking piecewise-constant comparators, without prior knowledge of how often the comparator changes.</p> <ul> <li><strong>Preprint:</strong> 📄</li> </ul> </article> <article> <h1>Research Interests</h1> <p>Wed, 09 Sep 2026 00:00:00 +0000</p> <style> /* At desktop widths the theme shows an empty 256px left-hand spacer (purely to balance the right-hand TOC) even though this page has no left sidebar. Below xl it's the real mobile nav drawer, so only hide it at the xl breakpoint where it's pure dead space. */ @media (min-width: 1280px) { .hb-sidebar-container { display: none !important; } } .prose { max-width: 70ch !important; } .prose p, .prose li { line-height: 1.5; } /* Quieter links: body-coloured text with a soft accent underline. */ .prose a { color: inherit; text-decoration-color: color-mix(in srgb, var(--color-primary-500, #3b82f6) 55%, transparent); text-decoration-thickness: 1px; text-underline-offset: 3px; } .prose a:hover { text-decoration-color: var(--color-primary-500, #3b82f6); text-decoration-thickness: 2px; } /* Opening summary. */ .prose .lead { color: var(--tw-prose-headings); font-size: 1.05em; line-height: 1.55; padding-left: 1rem; border-left: 3px solid var(--color-primary-500, #3b82f6); margin-bottom: 2.5rem; } /* One emphasis style: key terms (<dfn>) and bold text look the same. */ .prose dfn, .prose strong { font-style: normal; font-weight: 600; color: var(--color-primary-700, #1d4ed8); } .dark .prose dfn, .dark .prose strong { color: var(--color-primary-300, #93c5fd); } /* Small section numerals (the references heading is left unnumbered). */ .prose { counter-reset: ri-section; } .prose h2:not([id^="references"])::before { counter-increment: ri-section; content: counter(ri-section, decimal-leading-zero); display: block; font-size: 0.8rem; font-weight: 600; letter-spacing: 0.12em; color: var(--color-primary-500, #3b82f6); margin-bottom: 0.25rem; } /* The three frameworks as separated panels instead of dense bullets. */ .prose .frameworks ul { list-style: none; padding-left: 0; margin: 1.25rem 0; } .prose .frameworks ul > li { margin: 0.75rem 0; padding: 0.8rem 1rem; border-left: 3px solid var(--color-primary-500, #3b82f6); border-radius: 0 0.4rem 0.4rem 0; background: color-mix(in srgb, currentColor 5%, transparent); } /* Vector figure from the thesis. Its background is transparent, so in dark mode invert the ink and restore the hues. */ .ri-figure { margin: 2rem 0; } .ri-figure img { display: block; width: 85%; height: auto; margin: 0 auto; } .ri-figure figcaption { font-size: 0.85em; line-height: 1.45; margin-top: 0.75rem; } .dark .ri-figure img { filter: invert(1) hue-rotate(180deg); } .prose ol:last-of-type li { font-size: 0.92em; } /* Reading time is set by hand below: Hugo's estimate (213 words/min, references included) rounds this page up to 4. The theme's own line is turned off in the front matter, so tighten the gap it leaves. */ main > div.mt-4.mb-16 { margin-bottom: 1rem; } .prose .ri-meta { display: flex; align-items: center; gap: 0.45rem; margin: 0 0 1.75rem; font-size: 0.75rem; font-weight: 600; letter-spacing: 0.1em; text-transform: uppercase; color: color-mix(in srgb, currentColor 65%, transparent); } .prose .ri-meta::before { content: ""; width: 0.9rem; height: 0.9rem; background: currentColor; -webkit-mask: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' fill='none' stroke='black' stroke-width='2.2' stroke-linecap='round' stroke-linejoin='round'%3E%3Ccircle cx='12' cy='12' r='9'/%3E%3Cpath d='M12 7v5l3 2'/%3E%3C/svg%3E") center / contain no-repeat; mask: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' fill='none' stroke='black' stroke-width='2.2' stroke-linecap='round' stroke-linejoin='round'%3E%3Ccircle cx='12' cy='12' r='9'/%3E%3Cpath d='M12 7v5l3 2'/%3E%3C/svg%3E") center / contain no-repeat; } </style> <p class="ri-meta">3 min read</p> <p class="lead">I develop algorithms that allocate resources before knowing future demands or operating conditions. I am interested in learning from this interaction, exploiting useful predictions, and providing rigorous guarantees when the environment behaves unexpectedly.</p> <h2 id="learning-while-the-system-runs">Learning while the system runs</h2> <p>In online learning, an algorithm acts, observes feedback, and improves its next decisions. As Cesa-Bianchi notes in his foreword to , learning and evaluation are interleaved, complementing the familiar separation between training and testing. Observations need not be independent samples from a fixed distribution.</p> <p>The central performance measure is <dfn>regret</dfn>: the algorithm’s accumulated cost relative to a benchmark, such as the best fixed decision in hindsight (for content delivery networks, the files that would have served users best). A suitable algorithm makes its average excess cost vanish over time without ever seeing future requests. This possibility is what draws me to the field.</p> <p>Several mathematical frameworks capture this problem and form the basis of my work:</p> <div class="frameworks"> <ul> <li><strong>Online convex optimization (OCO)</strong> uses convex costs and feasible sets to design efficient algorithms for sequential decisions, from bandwidth allocation and portfolio selection to updating prediction models as data arrive; see , , and .</li> <li><strong>Bandits</strong> capture limited feedback: a recommendation system observes responses only to the items it displays, so learning must balance exploration and exploitation. Stochastic bandits assume stable reward distributions, while adversarial bandits do not; “adversarial” describes the guarantee’s scope, not necessarily a malicious opponent. develop both settings.</li> <li><strong>Markov decision processes (MDPs) and control</strong> incorporate lasting consequences: actions affect immediate rewards and future states, such as battery reserves. Learning in unknown MDPs requires reinforcement learning; see . Control theory captures the same interplay through system dynamics, and study it from an online learning perspective as non-stochastic control.</li> </ul> </div> <p>The broader systems perspective of these frameworks is considered in our , covering federated learning, distributed inference, and bandits under communication and computation constraints.</p> <h2 id="using-predictions-without-trusting-them">Using predictions without trusting them</h2> <p>Now, consider a prediction of what happens next: a network operator might forecast demand with a machine learning model. Accurate forecasts could improve allocation, but their reliability may change precisely when good decisions matter most.</p> <p>My asks how algorithms can use <strong>untrusted predictions while maintaining worst-case regret guarantees</strong>: robust to inaccurate forecasts, yet gaining from informative ones. <dfn>Optimism</dfn> means adapting to prediction quality, which need not be known in advance.</p> <figure class="ri-figure"> <figcaption>The protocol of optimistic learning: at each time slot, an untrusted prediction arrives (1); the learner acts on past costs, the prediction, and its accuracy so far (2); and the cost is realized (3) before the next prediction (4).</figcaption> </figure> <p>Caching is a concrete starting point. Our develops optimistic algorithms for bipartite caching networks, where topology determines which caches serve each request, and learns which prediction sources are useful. Our establishes performance limits and algorithms for whole files and, more generally, the online Knapsack problem. The guarantees improve with prediction accuracy while preserving worst-case regret rates. Our extends these ideas across communication networks.</p> <p>The thesis then moves beyond a fixed benchmark: if demand changes, so can the best allocation. Through <dfn>dynamic regret</dfn>, I study performance against a moving sequence of decisions; our develops a guarantee for this harder setting.</p> <p>Finally, I study systems with memory through <dfn>non-stochastic control</dfn>. As in an MDP, actions affect future states, but the dynamics and convex costs may be adversarial, and we compete with the best <strong>policy</strong> in a specified class, chosen in hindsight. My thesis shows how forecasts over an extended horizon help account for the lasting effects of today’s actions.</p> <h2 id="when-changing-a-decision-has-a-cost">When changing a decision has a cost</h2> <p>More recently, I have become interested in <dfn>smoothed online learning</dfn>, where an algorithm pays both for its decisions and for changing them. In mobile networks, balances connectivity against handover costs; for AI inference, applies smoothed online convex optimization to server/GPU provisioning with reconfiguration costs.</p> <p>These examples motivate algorithms that balance responsiveness with stability; our studies how much an algorithm can moderate its updates while tracking a changing benchmark, and our handles non-Lipschitz switching costs.</p> <h2 id="references-for-further-reading">References for further reading</h2> <ol> <li>Francesco Orabona. . 2026 manuscript.</li> <li>Elad Hazan. . 2nd edition, MIT Press, 2022.</li> <li>Tor Lattimore and Csaba Szepesvári. . Cambridge University Press, 2020. Free online edition.</li> <li>Naram Mhaisen. . PhD thesis, TU Delft, 2025.</li> <li>George Iosifidis, Naram Mhaisen, and Douglas J. Leith. . Foundations and Trends in Networking, to appear.</li> </ol> </article> <article> <h1>Partially Lazy Gradient Descent for Smoothed Online Learning</h1> <p>Fri, 01 May 2026 00:00:00 +0000</p> </article> <article> <h1>DRONE-RL: Dynamic Reinforcement Learning for Online Navigation of UAVs in Evolving Environments</h1> <p>Sun, 01 Feb 2026 00:00:00 +0000</p> </article> <article> <h1>Optimistic Learning with Applications to Caching Networks</h1> <p>Mon, 24 Nov 2025 00:00:00 +0000</p> </article> <article> <h1>Successfully Defended My Ph.D. Dissertation</h1> <p>Mon, 24 Nov 2025 00:00:00 +0000</p> <p>On <strong>24 November 2025</strong>, I successfully defended my Ph.D. dissertation, <em>“Optimistic Learning with Applications to Caching Networks,”</em> at Delft University of Technology.</p> <p>The work was carried out in the , under the supervision of <strong>Prof. dr. K. G. Langendoen</strong> (promotor) and <strong>Dr. G. Iosifidis</strong> (copromotor).</p> <style> /* Folded "thesis at a glance" panel: closed by default, opens on click. */ .thesis-glance { margin: 2rem 0; padding: 0.75rem 1rem; border: 1px solid color-mix(in srgb, currentColor 15%, transparent); border-left: 3px solid var(--color-primary-500, #3b82f6); border-radius: 0 0.5rem 0.5rem 0; } .thesis-glance summary { cursor: pointer; font-weight: 600; color: var(--color-primary-600, #2563eb); } .dark .thesis-glance summary { color: var(--color-primary-300, #93c5fd); } .thesis-glance[open] summary { margin-bottom: 0.75rem; } .thesis-figure { margin: 1.5rem 0 0.5rem; } .thesis-figure img { display: block; width: 100%; max-width: 26rem; height: auto; margin: 0 auto; } .thesis-figure figcaption { font-size: 0.85em; text-align: center; margin-top: 0.75rem; } /* Transparent vector figure: in dark mode invert the ink and restore the hues. */ .dark .thesis-figure img { filter: invert(1) hue-rotate(180deg); } </style> <details class="thesis-glance"> <summary>The thesis at a glance</summary> <p>The thesis asks how sequential decision-making algorithms can leverage untrusted predictions while still maintaining worst-case guarantees. Its central application is caching in content delivery networks, where the proposed algorithms are validated experimentally on multiple established datasets and benchmarks. Each chapter addresses this question under a progressively richer benchmark: static regret for coded caching, α-static regret for whole-file caching, dynamic regret for tracking a moving target, and policy regret for systems with memory.</p> <figure class="thesis-figure"> <figcaption>Technical progression of the chapters. Each chapter adopts a distinct regret metric suited to its actions set and choice of comparator(s).</figcaption> </figure> </details> <h2 id="materials">Materials</h2> <ul> <li><strong>Dissertation (PDF):</strong> 📄</li> <li><strong>Defense recording:</strong> 🎥</li> </ul> <h2 id="doctoral-committee">Doctoral Committee</h2> <ul> <li>Rector Magnificus, <em>Chairman</em></li> <li><strong>Prof. dr. K. G. Langendoen</strong>, Delft University of Technology — <em>promotor</em></li> <li><strong>Dr. G. Iosifidis</strong>, Delft University of Technology — <em>copromotor</em></li> </ul> <h3 id="independent-members">Independent Members</h3> <ul> <li><strong>Prof. dr. G. Dán</strong>, KTH Royal Institute of Technology, Sweden</li> <li><strong>Prof. dr. P. Grosso</strong>, University of Amsterdam</li> <li><strong>Prof. dr. M. T. J. Spaan</strong>, Delft University of Technology</li> <li><strong>Prof. dr. T. Spyropoulos</strong>, Technical University of Crete, Greece</li> <li><strong>Dr. T. A. L. van Erven</strong>, University of Amsterdam</li> </ul> <p>I am deeply grateful to my promotors, the members of the committee, my colleagues, and my family for their support throughout this journey.</p> </article> <article> <h1>On the Dynamic Regret of Following the Regularized Leader: Optimism with History Pruning</h1> <p>Sun, 13 Jul 2025 00:00:00 +0000</p> </article> <article> <h1>Multi-Target Path Planning with Probabilistic Detection in Cluttered Environments</h1> <p>Mon, 09 Jun 2025 00:00:00 +0000</p> </article> <article> <h1>Slicing for AI: An Online Learning Framework for Network Slicing Supporting AI Services</h1> <p>Sun, 01 Jun 2025 00:00:00 +0000</p> </article> <article> <h1>An Online Learning Framework for UAV Search Mission in Adversarial Environments</h1> <p>Tue, 01 Apr 2025 00:00:00 +0000</p> </article> <article> <h1>Optimistic Learning for Communication Networks</h1> <p>Tue, 01 Apr 2025 00:00:00 +0000</p> </article> <article> <h1>ELLIS Delft Talk: Optimistic Learning and Applications to Resource Management</h1> <p>Tue, 03 Sep 2024 16:00:00 +0000</p> </article> <article> <h1>An Online Learning Framework for UAV Target Search Missions in Non-stationary Environments</h1> <p>Thu, 01 Aug 2024 00:00:00 +0000</p> </article> <article> <h1>Adaptive Online Non-stochastic Control</h1> <p>Mon, 15 Jul 2024 00:00:00 +0000</p> </article> <article> <h1>Tutorial: Optimistic Learning for Resource Management in Communication Systems</h1> <p>Mon, 10 Jun 2024 00:00:00 +0000</p> </article> <article> <h1>Online Caching with no Regret: Optimistic Learning via Recommendations</h1> <p>Wed, 01 May 2024 00:00:00 +0000</p> </article> <article> <h1>Optimistic Online Non-stochastic Control via FTRL</h1> <p>Thu, 04 Apr 2024 00:00:00 +0000</p> </article> <article> <h1>Reinforcement Learning for Intelligent Healthcare Systems: A Review</h1> <p>Fri, 15 Dec 2023 00:00:00 +0000</p> </article> <article> <h1>Data Storage Methods and Systems</h1> <p>Tue, 05 Sep 2023 00:00:00 +0000</p> </article> <article> <h1>Invited Seminar: Adversarial Online Learning</h1> <p>Thu, 01 Jun 2023 00:00:00 +0000</p> </article> <article> <h1>Federated Learning for Online Resource Allocation in Mobile Edge Computing: A Deep Reinforcement Learning Approach</h1> <p>Sun, 26 Mar 2023 00:00:00 +0000</p> </article> <article> <h1>Data Storage Methods and Systems</h1> <p>Tue, 20 Dec 2022 00:00:00 +0000</p> </article> <article> <h1>Optimistic No-regret Algorithms for Discrete Caching</h1> <p>Thu, 01 Dec 2022 00:00:00 +0000</p> </article> <article> <h1>Exploring Deep Reinforcement Learning-Assisted Federated Learning for Online Resource Allocation in Privacy-Preserving EdgeIoT</h1> <p>Tue, 01 Nov 2022 00:00:00 +0000</p> </article> <article> <h1>Pervasive AI for IoT Applications: A Survey on Resource-Efficient Distributed Artificial Intelligence</h1> <p>Sat, 01 Oct 2022 00:00:00 +0000</p> </article> <article> <h1>Online Caching with Optimistic Learning</h1> <p>Mon, 13 Jun 2022 00:00:00 +0000</p> </article> <article> <h1>Multi-Agent Reinforcement Learning for Network Selection and Resource Allocation in Heterogeneous Multi-RAT Networks</h1> <p>Wed, 01 Jun 2022 00:00:00 +0000</p> </article> <article> <h1>Communication-Efficient Hierarchical Federated Learning for IoT Heterogeneous Systems</h1> <p>Tue, 01 Mar 2022 00:00:00 +0000</p> </article> <article> <h1>On Designing Smart Agents for Service Provisioning in Blockchain-Powered Systems</h1> <p>Sun, 01 Aug 2021 00:00:00 +0000</p> </article> <article> <h1>Rational Contracts: Data-Driven Service Provisioning in Blockchain-Powered Systems</h1> <p>Mon, 14 Jun 2021 00:00:00 +0000</p> </article> <article> <h1>Optimal User-Edge Assignment in Hierarchical Federated Learning</h1> <p>Fri, 01 Jan 2021 00:00:00 +0000</p> </article> <article> <h1>To Chain or Not to Chain: A Reinforcement Learning Approach for Blockchain-Enabled IoT Monitoring</h1> <p>Thu, 01 Oct 2020 00:00:00 +0000</p> </article> <article> <h1>Data Consistency in Multi-Cloud Storage Systems with Passive Servers and Non-Communicating Clients</h1> <p>Tue, 01 Sep 2020 00:00:00 +0000</p> </article> <article> <h1>Novel Techniques for Blockchain-Enabled IoT Systems Leveraging Reinforcement Learning</h1> <p>Mon, 01 Jun 2020 00:00:00 +0000</p> </article> <article> <h1>Real-Time Scheduling for Electric Vehicles Charging/Discharging Using Reinforcement Learning</h1> <p>Sun, 02 Feb 2020 00:00:00 +0000</p> </article> <article> <h1>Secure Smart Contract-Enabled Control of Battery Energy Storage Systems against Cyber-Attacks</h1> <p>Sun, 01 Dec 2019 00:00:00 +0000</p> </article> <article> <h1>A Reconfigurable Multipurpose System on Chip Platform for Metal Detection</h1> <p>Fri, 19 Apr 2019 00:00:00 +0000</p> </article> <article> <h1>Self-Powered IoT-Enabled Water Monitoring System</h1> <p>Sat, 25 Aug 2018 00:00:00 +0000</p> </article> <article> <h1>A Reconfigurable Multipurpose SoC Mobile Platform for Metal Detection</h1> <p>Thu, 01 Mar 2018 00:00:00 +0000</p> </article> <article> <h1>Almost BPXOR Coding Technique for Tolerating Three Disk Failures in RAID7 Architectures</h1> <p>Thu, 01 Mar 2018 00:00:00 +0000</p> </article> </main></body></html>