Applications of Computational Math In Industry (CME 500)

Technology improvements have made it easier than ever to collect diverse telemetry at high resolution from cloud systems. Anomaly detection on the telemetry data is an important sub-routine for many software applications — both for monitoring the health of the application as well as for detecting security or other unintended events. Given the growing complexities of architecting applications on the cloud, devising simple rules for monitoring systems is seldom sufficient, requiring automated algorithms. In this talk, we detail the technical challenges associated with developing and deploying an AD system to detect database security events in the cloud. We observe that a key feature of cloud systems’ is heterogeneity in both space and time — i.e., for the same database at different times or at the same time for different databases, what is anomalous may become benign and vice-versa. A fundamental question to operationalize anomaly detection is then: ‘How to quickly spot anomalies in a data-stream, and differentiate them from either sudden or gradual drifts in the normal behavior?’ To this end, the talk will highlight some algorithmic contributions we made towards studying this problem. In addition, we also share some lessons learnt in deploying these ideas at scale on AWS. We conclude by some interesting theoretical and practical open problems. 

Parts of this talk is based on the following papers: https://proceedings.mlr.press/v162/sankararaman22a.html and https://papers.nips.cc/paper_files/paper/2023/hash/9e15d892c63903ecc278e0dd05536951-Abstract-Conference.html

Abishek Sankararaman is a senior machine learning scientist with Amazon Web Services. He is part of the team that developed the GuardDuty for RDS and works on a  variety of scientific and engineering efforts in anomaly detection and online learning. Prior to  AWS, Abishek was a post-doctoral researcher at University of California, Berkeley. He received is PhD from The University of Texas at Austin, where he was affiliated with the Simons Center for Network Mathematics. He completed his undergraduate degree at IIT Madras.

Machine learning models revolutionize how we interact with technology. However, as models are trained on increasingly large and potentially sensitive datasets, they become subject to privacy concerns, biases, and the retention of outdated information. Machine unlearning emerges as a critical field to address these issues by selectively removing training data's influence from a trained model. This talk will begin by outlining the crucial motivations driving the need for machine unlearning. We'll explore scenarios where unlearning protects user privacy, corrects training data errors, and ensures ongoing model relevance. Next, we'll provide mathematically sound definitions of unlearning, and explore the similarities and differences with the field of differential privacy. Next, the focus shifts to existing approaches to machine unlearning. We'll examine techniques like influence functions, retraining strategies, and knowledge distillation.  Crucially, the talk will address the importance of auditing unlearning algorithms. We'll discuss metrics for evaluating unlearning effectiveness,  guarantees of privacy protection, and the potential for unintended consequences. 

Fabian Pedregosa is a Senior Research Scientist at Google DeepMind, specializing in the field of mathematical optimization and machine (un)learning. Fabian is amongst the 10 most cited researchers in optimization —according to Google Scholar— and has made significant contributions to the field of optimization, as well as the development of popular machine learning software, including leading scikit-learn and optax.

We describe the nuances and challenges present in applying machine learning toward the mission of preventing computer systems breaches for enterprise businesses.  The challenge of defensive cybersecurity is formidable: an adversary only needs to succeed one time in order to compromise a particular system, while a defender needs to enable the defense of all possible attacks.  Fundamental to this challenge is the fact that the prevalence of real malicious attacks is dwarfed by the volume of routine benign activity on the defended computer systems.  That is, classification in cybersecurity suffers from the base-rate problem.  We discuss how in practice the data science team at CrowdStrike navigates the challenges of applied machine learning in cybersecurity.  Topics will include security theory, feature engineering, label noise, information leakage, model generalization, and outcome measurement. 

Dr. Jeff Kaplan is a Director of Data Science at CrowdStrike where he leads a team of machine learning practitioners and researchers.  His team applies supervised an unsupervised learning to build cybersecurity products that help CrowdStrike’s customers identify threats and stop breaches before they occur.  Jeff has a Ph.D. in computational astrophysics from Caltech, and briefly worked at a Fintech startup before moving into cybersecurity at CrowdStrike, where he has worked for the past eight years.

The financial services industry serves millions of customers and is responsible for sensitive, private and personal customer information. As such, robust, comprehensive and scalable testing is crucial to successful technology development and deployment. However, having large volumes of test data readily available can be a challenge. This is especially true when data must be anonymized to protect its sensitive nature or scaled to ensure products perform effectively. Additionally, the current state of test data may not accurately reflect the diversity and nuances of target populations or adequately address edge case scenarios. In this session, Jasmine de Gaia, Head of Customer Data Strategy at Wells Fargo, will discuss how artificially generated data can preserve privacy and expedite technology development. She will share real-world problems and use cases, and will walk through innovative approaches to solve them.

Jasmine de Gaia leads the Customer Data Strategy organization at Wells Fargo. She is responsible for leading enterprise customer data assets and enabling the usage of this data to transform the customer experience through innovative new products and services. Jasmine joined Wells Fargo from JPMorgan Chase & Co., where she led products, strategy, innovation, and transformation for the customer experience. Previously, she led digital transformation and products for Fortune 500 companies and Silicon Valley technology start-ups. Jasmine has been recognized as a data, product and technology leader, is a frequent keynote speaker, and currently has multiple U.S. patents pending.  She serves on multiple boards, including the Global Editorial Board of Chief Data Officer Magazine, the PAST Foundation, an organization dedicated to furthering STEM education for children, as well as the Board of the Center for Innovation Strategy at the Ohio State University. Additionally, she has been recognized as Woman of Influence by Business First magazine. Jasmine holds a Bachelor of Science degree in Industrial Engineering from the University of Windsor and a Master’s degree in Management Science and Engineering from Stanford University.

In this talk, we present three approaches for training multi-objective machine learning models within the context of Information Retrieval. Specifically, we explore target aggregation, loss aggregation, and constraint-based optimization for training rankers in product search. We establish a meaningful connection to Optimal Transport, which provides insights on the relation of the aforementioned approaches. Furthermore, we delve into target generation as a doubly stochastic matrix and present two algorithms designed to find such matrices as solutions within the Birkhoff polytope. Among these, we introduce the Sinkhorn-Newton-Sparse (SNS) algorithm, which empirically demonstrates super-exponential convergence. The SNS algorithm was published at the ICRL 2024 conference. We will also provide insights into the constrained formulation of Optimal Transport, which significantly impacts our task. This formulation enables practitioners to derive approximate transport plans in broader contexts as well.

Michael Shavlovsky is an Applied Scientist at Amazon, where he is part of the Search Science and AI team. His work focuses on developing rankers that power Amazon's product search. The models Michael has developed have been deployed to serve customers in the US, UK, and Germany, generating hundreds of millions of dollars in incremental sales. Prior to joining Amazon, he earned his PhD from UC Santa Cruz, where his research spanned a variety of topics including Game Theory, Machine Learning, and Crowdsourcing. Before that, Michael received his BS and MS in applied mathematics and physics from the Moscow Institute of Physics and Technology.

In the dynamic world of fashion retail, staying ahead of the curve is essential for success. With constantly evolving consumer preferences and market trends, retailers face the challenge of detecting anomalies in their data to identify potential opportunities or threats. In this talk, we delve into the realm of anomaly detection within the context of fashion retail. We explore the intricacies of various anomaly detection models developed and applied in real-world scenarios, highlighting their effectiveness in uncovering unexpected patterns and deviations from the norm.

Javier Diaz is head of AI Research at Inditex Technology. He is focused on developing new ML and AI solutions that are applied over a wide variety of processes across the company. Over the last years he has developed several algorithms of different kinds, predictive models, computer vision or recommender systems that are integrated in different applications. He also holds a PhD in nuclear physics from the University of Santiago de Compostela (Spain) where he studied nucleon-nucleon interactions in nucleus-nucleus collisions.

Javier Diaz
Data Science Manager