CIV-ML's Seminars

Seminars
/main page/Seminars
This page presents work in progress research seminars that are held weekly. These seminars carry on Machine Learning and Bayesian based methods applied to civil engineering. All seminars are open to the public. Previous seminars : 2016-2017 \| 2017-2018 \| 2018-2019 \| 2019-2020 \| 2020-2021 \| 2021-2022
Apr 9, 2024 10:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Intervention on Kalman filter using reinforcement learning Abstract: Training decision-making agents for the SHM anomaly detection task with supervised learning requires a large amount of consistent and objective labels, which are difficult to access in real time series. In this seminar, I will showcase how to leverage reinforcement learning in real time series anomaly detection, where the agents learn by trial-and-error without human labels. The RL agents, driven by maximizing the cumulative long-term prediction likelihood, intervene the hidden states in the Kalman filter. Statistical evaluation of RL-based and switch-Kalman-filter-based agents are compared on synthetic and real time series.
Apr 02, 2024 10:00 \| Polytechnique Montreal		Presenter: Miquel Florensa \| MSc. student, Polytechnique Montreal Title: Diffusion Models with TAGI: review of DM, TAGI-DM formulation and preliminary results. Abstract: In this seminar, we'll first cover the basic principles of diffusion models. Then, we'll delve into how these models can be implemented using BNN/TAGI. Following that, we'll examine some initial results from applying this approach to a 2D toy problem. Finally, we'll discuss the potential benefits and challenges of using "true probabilistic" diffusion models in practice.
Feb 19, 2024 15:00 \| Polytechnique Montreal		Presenter: Lucas Alric \| PhD student, Polytechnique Montreal Title: Latent Constrained Kalman Filter (LCKF) versus Constrained Kalman Filter (CKF): A Comprehensive Study Abstract: The Constraint Kalman Filter (CKF) has been a cornerstone in state estimation, offering robust solutions in various domains. However, its reliance on explicit constraints poses limitations in scenarios where constraints are not rigid and may not adequately capture the dynamic nature of the system. To address this inflexibility, we present the Latent Constrained Kalman Filter (LCKF), a novel methodology that extends the CKF framework to handle more flexible and adaptable constraints effectively.This abstract elucidates the mathematical foundations of LCKF and its implementation intricacies. We delve into the derivation of LCKF equations and elucidate the implementation process, emphasizing its adaptability across diverse applications. Furthermore, we conduct a comparative performance analysis between CKF and LCKF through a toy example, showcasing the efficacy of LCKF in scenarios with latent constraints. Our findings underscore the superiority of LCKF in handling such scenarios, thereby demonstrating its potential as a versatile tool in state estimation tasks.
Feb 19, 2024 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: Using homoscedastic and heteroscedastic AGVI for anomaly detection Abstract: In this seminar, we will see a comparison of using AGVI (both homoscedastic case and heteroscedastic case) and the baseline version for anomaly detection in several of the Hydro-Quebec datasets.
Feb 19, 2024 15:00 \| Polytechnique Montreal		Presenter: David Wardan \| PhD student, Polytechnique Montreal Title: Characterization of the construction period of buildings in Beirut using multi-modal machine learning Abstract: Multiple efforts have contributed to collecting data on the built environment in Beirut. However, these efforts usually rely on time-consuming and labor-intensive field visits that are prone to human errors. Such data are crucial to perform seismic risk and vulnerability assessments on a city scale. This work uses artificial intelligence practices for automated characterization of building construction period, which is related to expected structural performance. The proposed framework is applied to Beirut, where five construction periods are identified and used: pre1935, 1935-1955, 1956-1971, 1972-1990, and post1990. Data consisting of street-view images, construction period, number of floors, and location of buildings in Beirut is compiled from various sources. Three approaches for predicting construction period are explored: (1) the novel transformer-based Swin-T model with an input street-view image; (2) a fully connected neural network model with input tabular data that include the number of floors and socio-economic background of the building; and (3) a late fusion of the Swin-T model and fully connected neural network, thus using both types of inputs. The three approaches were trained on the same dataset and their overall accuracy on the test set was 72.74%, 60.53%, and 78%, respectively. The multi-modality model achieved the best performance and higher confidence in the predictions. It is used in a toy example to predict the construction period of buildings in a Beirut neighborhood and assess its vulnerability to an earthquake scenario. The earthquake consequences are simulated using the Regional Resilience Determination (R2D) tool developed by the NHERI SimCenter. The R2D tool requires as input a description of the earthquake scenario of interest and data on the built environment (the location, plan area, number of stories, year built, occupancy class, structure type, and replacement cost per square meter of each building), and outputs for each building the estimated damage and losses.
Feb 12, 2024 10:00 \| Polytechnique Montreal		Presenter: Alex Immer \| PhD student, ETH Zürich Title: Modern Laplace Approximations for Deep Learning and Heteroscedastic Regression Abstract: Bayesian inference can augment the capabilities of deep learning by quantifying uncertainties of parameters and predictions. While exact inference in deep learning is often intractable, modern Laplace approximations have shown promising results on model selection, continual learning, and model calibration. In this talk, I will discuss recent advances in Laplace approximations that have enabled this with a particular focus on marginal likelihood estimation. Further, I will highlight heteroscedastic neural networks as an interesting application of Laplace approximations.
Feb 09, 2024 13:00 \| Polytechnique Montreal		Presenter: Van-Dai Vuong \| PhD student, Polytechnique Montreal Title: Coupling LSTM Neural Networks and State-Space Models through Analytically Tractable Inference Abstract: In their original form, long short-term memory (LSTM) neural networks (NN) are deterministic models which consider their parameters as having deterministic values, hence failing to take into account the epistemic uncertainties. By contrast, Bayesian LSTMs consider this epistemic uncertainty by placing a distribution over the parameters. Variational inference (VI) is a common method used to create Bayesian LSTMs. However, it relies on gradient descent and backpropagation for inferring the parameters, and it is thus not compatible with close-form probabilistic methods such as state-space models. The Tractable Approximate Gaussian Inference (TAGI) method allows performing analytical Bayesian inference in neural networks. In this work, we present the mathematical formulations for using the TAGI method with the LSTM architecture for obtaining analytically the posterior mean vectors and diagonal covariance matrices for the LSTM's parameters and hidden states. We show through experimental comparisons that our model provides on-par performance compared to the LSTM models trained using backpropagation while enabling to consider the epistemic uncertainties about the model's parameters. This new framework allows to probabilistically couple LSTMs with state-space models because both use Bayesian inference as their learning mechanism. The resulting hybrid model can retain the interpretability feature of SSMs, while exploiting the ability of LSTMs to learn complex patterns automatically with minimal manual setups.
Jan 29, 2024 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Bayesian Online Changepoint Detection Abstract: In this seminar, I will present an anomaly detection method called Bayesian Online Changepoint Detection (BOCD). In comparison to the switching Kalman filter currently employed for detecting regime switches in Bayesian dynamic linear models, BOCD does not require the definition of a non-stationary model and can probabilistically track the number of time steps since the last changepoint detection. BOCD has the potential to provide additional useful hidden states for training decision-making agents.
Nov 15, 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Anomaly labeling in the decision-making framework using imitation learning Abstract: To overcome the limitations of the anomaly detection policy that relies only on the non-stationary regime's probability in switching Kalman filter, we propose a decision-making framework using imitation learning that mimics expert's demonstrations in triggering alarms. The input of the decision-making agents consists of the hidden states preprocessed by the switching Kalman filter, while the output is anomaly label defined by human. The quality of the anomaly labels greatly affects agents' performance. For the synthetic time series, the anomaly labeling is straightforward given that the ground truth of the anomaly is manually defined. Yet, for real time series, the nature of the anomaly, such as the beginning time, the duration, and the magnitude, are not accessible. In this seminar, I will discuss the importance of the anomaly labeling, potential solutions to mitigate the subjectivity in the labeling process, and the preliminary results where an agent tries to reproduce my anomaly detection policy (that is suboptimal for sure).
Nov 08, 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: Is the decaying acceleration model enough to model the uncertainty due to model inadequacy? Abstract: In this seminar, I will discuss the use of the decaying acceleration model alone to model the aleatory uncertainty that arises due to model inadequacy in comparison to using two separate models.
Oct 25, 2023 15:00 \| Polytechnique Montreal		Presenter: Sebastien Jessup \| PhD student, Concordia University Title: Uncertainty integration in heteroscedastic Bayesian model averaging Abstract: Bayesian Model Averaging (BMA) is a widely used tool for model combination using Bayesian inference. Different versions of an expectation-maximisation (EM) algorithm are frequently used to apply BMA, typically in a homoscedastic context. In many situations, such as climate risk modelling or actuarial reserving, the homoscedasticity assumption does not hold. Moreover, the EM algorithm has the well-known issue of convergence to a single model. Considering these issues, we adapt the EM algorithm to a heteroscedastic context. We also propose a numerical error integration approach which considers data uncertainty and addresses convergence to a single model. We further generalise this proposed approach to have flexible weights. We compare the proposed approaches using simulation studies.
Oct 18, 2023 15:00 \| Polytechnique Montreal		Presenter: Aleksandar Jakovljevic \| PhD student, Polytechnique Montreal Title: Investigating the Application of Recurrent Neural Networks and Blocked Cross-Validation for Modelling Conventional Drinking Water Treatment Processes Abstract: In the existing literature, we observe the prevalent use of multilayer perceptrons trained using the holdout method for modelling conventional drinking water treatment processes. In this study, we investigate the use of recurrent neural networks and blocked cross-validation to improve upon these models. The model is tasked with predicting settled water turbidity over the next 15 minutes of operation, based on the previous 30 minutes of operating data. After training and testing, we note that the models trained using blocked cross-validation report much lower test errors than their counterparts trained using holdout, with the GRU network having the lowest errors with RMSE = 0.073 NTU and 0.097 NTU when trained with these methods respectively. However, a simulation of one year of operation after this training/testing phase reveals an inverse correlation between the originally reported test error and the error on this one-year simulation. The result is that the MLP trained using holdout has the best performance on this new set of data. A possible reason for this difference being that the system behaviour in the simulation period has sufficiently changed from the original train/test data and the implications of this result are discussed.
Sep 27, 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Continuously Adaptable Probabilistic Decision-making for Structural Anomaly Detection Abstract: In this seminar, we will walk through the methodologies for developing the decision-making layer in structural anomaly detection. Preliminary results of three relative studies are presented, including a newly developed bounded autoregressive component in Bayesian dynamic linear model, decision-making agents trained with imitation learning algorithms, and time series generation using diffusion models.
Sep 20, 2023 15:00 \| Polytechnique Montreal		Presenter: Van-Dai Vuong \| PhD student, Polytechnique Montreal Title: Unsupervised anomaly detection Abstract: This seminar presents several unsupervised anomaly detection methods and compares their performance on a crack opening dataset from a dam in Canada.
Sep 13, 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: How to run MATLAB code on Compute Canada Clusters Abstract: In this hands-on presentation, I will show how to set up your compute canada cluster account as well as the basics to get started with running MATLAB codes as Slurm files in the cedar cluster.
May 26, 2023 15:00 \| Polytechnique Montreal		Presenter: Zachary Hamida \| Postdoc, Polytechnique Montreal Title: Introducing Ray for Parallel Computing in Python Abstract: This seminar is a short tutorial session that will cover some of the basics in Ray and it's application in machine learning frameworks. The tutorial include example of applications and concepts prepared in python notebook that will be made available online.
May 5, 2023 15:00 \| Polytechnique Montreal		Presenter: Miquel Florensa \| Intern Student, Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Diffusion Models: Overview of Theory and Time Series Generation Abstract: Diffusion models have become a prominent technique for generative modeling of complex data, including images, sounds, and time series. This presentation will offer an overview of the theory and implementation of diffusion models, covering topics such as their architecture, training procedures, and recent advancements. In addition, we will explore the use of diffusion models in time series data by presenting experiments with toy problems that showcase the challenges and potential solutions.
Apr 6 2023 15:00 \| Polytechnique Montreal		Presenter: James-A Goulet \| Professor, Polytechnique Montreal Title: How do neural networks learn? A microscopic look into hidden units through Bayesian inference. Abstract: We are surrounded by applications of neural networks which are capable of achieving complex tasks with an astonishing simplicity. In their basic form, neural networks are known to be universal functions approximators, i.e., they can approximate arbitrary complex functions. In the case of simple feedforward architectures using ReLU activation functions, this approximation is piecewise linear with the number of segments being function of the number of hidden units. With the recent developments related to the TAGI method, we have shown how we can use approximate Gaussian inference as the inference engine for arbitrary large neural networks. Despite the substantial reduction in the number of epochs required to train networks, it remains that even with TAGI, the parameters of neural networks cannot be learnt online in a single epoch. In this presentation, I will present the preliminary work done to identify the root cause of this limitation. The strategy is to proceed from the simplest neural network configuration possible and then build up in order to identify at what point online learning stops to be possible and why it is so.
Mar 31 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka\| Postdoc, Polytechnique Montreal Title: Homoscedastic Aleatory Uncertainty Quantification with AGVI and Self-Tuning Scheduler. Abstract: In this seminar, I will introduce modeling a self-tuning scheduler using a local acceleration model together with learning the homoscedastic aleatory uncertainty using AGVI.
Mar 16 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: How to decide the boundary in Bounded Autoregressive Component? Abstract: To prevent the residual component from capturing the trend in Bayesian Dynamic Linear Model (BDLM), we proposed to constrain the Autoregressive (AR) component in a certain range. Boundary of such a range is controlled by the hyper parameter, gamma, in Bounded Autoregressive (BAR) component. When the boundary is too small, the too strong constrain causes false alarms. On the other hand, when the boundary is too large or close to infinity, no constrain is applied and BAR is close to AR component. In this seminar, we will see how we can tune this hyper parameter to decide a proper boundary according to the desirable false alarm rate and detectability.
Mar 10 2023 15:00 \| Polytechnique Montreal		Presenter: Ali Fakhri \| MSc. student, Polytechnique Montreal Title: Google's MLP-Mixer architecture. Abstract: In this seminar, we will go over Google's MLP-Mixer architecture and observe how it fares against the state-of-the-art methods on image classification tasks. We will conclude with presenting the pros and cons of the MLP-Mixer architecture.
Mar 3 2023 15:00 \| Polytechnique Montreal		Presenter: James-A Goulet \| Professor, Polytechnique Montreal Title: Formulation of probabilistic softmax and remax functions. Abstract: The Tractable approximate Gaussian inference (TAGI) uses the hierarchical softmax for classification tasks. This is because no closed-form formulation exists to propagate uncertainty through the standard softmax, whereas, one exists for the hierarchical version. However, in order to build attention mechanisms as those used in transformers, we need a softmax function as the hierarchical version is unsuited. In this presentation, we will see how can we obtain a close-form solution for a probabilistic softmax as well as for the new remax function that is better suited for an integration with TAGI.
Feb 24 2023 15:00 \| Polytechnique Montreal		Presenter: Miquel Florensa \| Intern student, Polytechnique Montreal Title: Unit Testing on cuTAGI. Abstract: In this seminar I will explain the basis of unit testing and general testing on software development. I will also present some basic information about C++ and CUDA in order to prepare the assistants to use the cuTAGI library. At the end we will run through a demo/tutorial on how to install, compile and run cuTAGI and pyTAGI.
Feb 17 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: Laplace Package for Bayesian Deep Learning. Abstract: In this seminar, we will look through the Laplace package to perform uncertainty quantification for deep learning applied to regression tasks. The package will be tested for both and full Hessian cases, as well as, for single and two hidden layers.
Feb 2 2023 15:00 \| Polytechnique Montreal		Presenter: Van-Dai Vuong \| PhD student, Polytechnique Montreal Title: Bypassing for inference in TAGI. Abstract: During inference In TAGI, the hidden states of the layer (i-1) are updated from those of the subsequent layer (i), bypassing the activation units. In this seminar, we show why we can do it. Furthermore, we show that we can update the hidden states of the layer (i-1) directly from the those of the layer (i+1), bypassing the layer (i).
Jan 19 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Bounded Autoregressive Component in Bayesian Dynamic Linear Model. Abstract: In Bayesian Dynamic Linear Model (BDLM), a pattern change in the time series data will be firstly and mostly perceived as model noise stored in the residual autoregressive component. This usually delay our detection of anomaly or miss the anomaly with small magnitude. My first seminar will present a bounded autoregressive (BAR) component that prevents the residual hidden state from shifting and capturing the changed pattern. Results on a synthetic time series and a real time series from a bridge in Montreal will be presented to show that with BAR, BDLM could potentially detect anomaly at earlier stage and anomaly with smaller amplitude.
Jan 12 2023 15:00 \| Polytechnique Montreal		Presenter: Zachary Hamida \| Postdoc, Ali Fakhri \| MSc. student, Polytechnique Montreal Title: Reunion d'avancement: predire la degradation et comprendre l'effet des interventions (Phase 2). Abstract: This seminar presents a summary of the research work accomplished through the past year. The topics include, the improvements to the deterioration model, the progress in planning intervention activities on a network-scale, and estimating the cost ratio of delaying interventions individual on elements as well as on the whole structures.

Seminars
/main page/Seminars
This page presents work in progress research seminars that are held weekly. These seminars carry on Machine Learning and Bayesian based methods applied to civil engineering. All seminars are open to the public. Previous seminars : 2016-2017 \| 2017-2018 \| 2018-2019 \| 2019-2020 \| 2020-2021 \| 2021-2022
Apr 9, 2024 10:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Intervention on Kalman filter using reinforcement learning Abstract: Training decision-making agents for the SHM anomaly detection task with supervised learning requires a large amount of consistent and objective labels, which are difficult to access in real time series. In this seminar, I will showcase how to leverage reinforcement learning in real time series anomaly detection, where the agents learn by trial-and-error without human labels. The RL agents, driven by maximizing the cumulative long-term prediction likelihood, intervene the hidden states in the Kalman filter. Statistical evaluation of RL-based and switch-Kalman-filter-based agents are compared on synthetic and real time series.
Apr 02, 2024 10:00 \| Polytechnique Montreal		Presenter: Miquel Florensa \| MSc. student, Polytechnique Montreal Title: Diffusion Models with TAGI: review of DM, TAGI-DM formulation and preliminary results. Abstract: In this seminar, we'll first cover the basic principles of diffusion models. Then, we'll delve into how these models can be implemented using BNN/TAGI. Following that, we'll examine some initial results from applying this approach to a 2D toy problem. Finally, we'll discuss the potential benefits and challenges of using "true probabilistic" diffusion models in practice.
Feb 19, 2024 15:00 \| Polytechnique Montreal		Presenter: Lucas Alric \| PhD student, Polytechnique Montreal Title: Latent Constrained Kalman Filter (LCKF) versus Constrained Kalman Filter (CKF): A Comprehensive Study Abstract: The Constraint Kalman Filter (CKF) has been a cornerstone in state estimation, offering robust solutions in various domains. However, its reliance on explicit constraints poses limitations in scenarios where constraints are not rigid and may not adequately capture the dynamic nature of the system. To address this inflexibility, we present the Latent Constrained Kalman Filter (LCKF), a novel methodology that extends the CKF framework to handle more flexible and adaptable constraints effectively.This abstract elucidates the mathematical foundations of LCKF and its implementation intricacies. We delve into the derivation of LCKF equations and elucidate the implementation process, emphasizing its adaptability across diverse applications. Furthermore, we conduct a comparative performance analysis between CKF and LCKF through a toy example, showcasing the efficacy of LCKF in scenarios with latent constraints. Our findings underscore the superiority of LCKF in handling such scenarios, thereby demonstrating its potential as a versatile tool in state estimation tasks.
Feb 19, 2024 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: Using homoscedastic and heteroscedastic AGVI for anomaly detection Abstract: In this seminar, we will see a comparison of using AGVI (both homoscedastic case and heteroscedastic case) and the baseline version for anomaly detection in several of the Hydro-Quebec datasets.
Feb 19, 2024 15:00 \| Polytechnique Montreal		Presenter: David Wardan \| PhD student, Polytechnique Montreal Title: Characterization of the construction period of buildings in Beirut using multi-modal machine learning Abstract: Multiple efforts have contributed to collecting data on the built environment in Beirut. However, these efforts usually rely on time-consuming and labor-intensive field visits that are prone to human errors. Such data are crucial to perform seismic risk and vulnerability assessments on a city scale. This work uses artificial intelligence practices for automated characterization of building construction period, which is related to expected structural performance. The proposed framework is applied to Beirut, where five construction periods are identified and used: pre1935, 1935-1955, 1956-1971, 1972-1990, and post1990. Data consisting of street-view images, construction period, number of floors, and location of buildings in Beirut is compiled from various sources. Three approaches for predicting construction period are explored: (1) the novel transformer-based Swin-T model with an input street-view image; (2) a fully connected neural network model with input tabular data that include the number of floors and socio-economic background of the building; and (3) a late fusion of the Swin-T model and fully connected neural network, thus using both types of inputs. The three approaches were trained on the same dataset and their overall accuracy on the test set was 72.74%, 60.53%, and 78%, respectively. The multi-modality model achieved the best performance and higher confidence in the predictions. It is used in a toy example to predict the construction period of buildings in a Beirut neighborhood and assess its vulnerability to an earthquake scenario. The earthquake consequences are simulated using the Regional Resilience Determination (R2D) tool developed by the NHERI SimCenter. The R2D tool requires as input a description of the earthquake scenario of interest and data on the built environment (the location, plan area, number of stories, year built, occupancy class, structure type, and replacement cost per square meter of each building), and outputs for each building the estimated damage and losses.
Feb 12, 2024 10:00 \| Polytechnique Montreal		Presenter: Alex Immer \| PhD student, ETH Zürich Title: Modern Laplace Approximations for Deep Learning and Heteroscedastic Regression Abstract: Bayesian inference can augment the capabilities of deep learning by quantifying uncertainties of parameters and predictions. While exact inference in deep learning is often intractable, modern Laplace approximations have shown promising results on model selection, continual learning, and model calibration. In this talk, I will discuss recent advances in Laplace approximations that have enabled this with a particular focus on marginal likelihood estimation. Further, I will highlight heteroscedastic neural networks as an interesting application of Laplace approximations.
Feb 09, 2024 13:00 \| Polytechnique Montreal		Presenter: Van-Dai Vuong \| PhD student, Polytechnique Montreal Title: Coupling LSTM Neural Networks and State-Space Models through Analytically Tractable Inference Abstract: In their original form, long short-term memory (LSTM) neural networks (NN) are deterministic models which consider their parameters as having deterministic values, hence failing to take into account the epistemic uncertainties. By contrast, Bayesian LSTMs consider this epistemic uncertainty by placing a distribution over the parameters. Variational inference (VI) is a common method used to create Bayesian LSTMs. However, it relies on gradient descent and backpropagation for inferring the parameters, and it is thus not compatible with close-form probabilistic methods such as state-space models. The Tractable Approximate Gaussian Inference (TAGI) method allows performing analytical Bayesian inference in neural networks. In this work, we present the mathematical formulations for using the TAGI method with the LSTM architecture for obtaining analytically the posterior mean vectors and diagonal covariance matrices for the LSTM's parameters and hidden states. We show through experimental comparisons that our model provides on-par performance compared to the LSTM models trained using backpropagation while enabling to consider the epistemic uncertainties about the model's parameters. This new framework allows to probabilistically couple LSTMs with state-space models because both use Bayesian inference as their learning mechanism. The resulting hybrid model can retain the interpretability feature of SSMs, while exploiting the ability of LSTMs to learn complex patterns automatically with minimal manual setups.
Jan 29, 2024 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Bayesian Online Changepoint Detection Abstract: In this seminar, I will present an anomaly detection method called Bayesian Online Changepoint Detection (BOCD). In comparison to the switching Kalman filter currently employed for detecting regime switches in Bayesian dynamic linear models, BOCD does not require the definition of a non-stationary model and can probabilistically track the number of time steps since the last changepoint detection. BOCD has the potential to provide additional useful hidden states for training decision-making agents.
Nov 15, 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Anomaly labeling in the decision-making framework using imitation learning Abstract: To overcome the limitations of the anomaly detection policy that relies only on the non-stationary regime's probability in switching Kalman filter, we propose a decision-making framework using imitation learning that mimics expert's demonstrations in triggering alarms. The input of the decision-making agents consists of the hidden states preprocessed by the switching Kalman filter, while the output is anomaly label defined by human. The quality of the anomaly labels greatly affects agents' performance. For the synthetic time series, the anomaly labeling is straightforward given that the ground truth of the anomaly is manually defined. Yet, for real time series, the nature of the anomaly, such as the beginning time, the duration, and the magnitude, are not accessible. In this seminar, I will discuss the importance of the anomaly labeling, potential solutions to mitigate the subjectivity in the labeling process, and the preliminary results where an agent tries to reproduce my anomaly detection policy (that is suboptimal for sure).
Nov 08, 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: Is the decaying acceleration model enough to model the uncertainty due to model inadequacy? Abstract: In this seminar, I will discuss the use of the decaying acceleration model alone to model the aleatory uncertainty that arises due to model inadequacy in comparison to using two separate models.
Oct 25, 2023 15:00 \| Polytechnique Montreal		Presenter: Sebastien Jessup \| PhD student, Concordia University Title: Uncertainty integration in heteroscedastic Bayesian model averaging Abstract: Bayesian Model Averaging (BMA) is a widely used tool for model combination using Bayesian inference. Different versions of an expectation-maximisation (EM) algorithm are frequently used to apply BMA, typically in a homoscedastic context. In many situations, such as climate risk modelling or actuarial reserving, the homoscedasticity assumption does not hold. Moreover, the EM algorithm has the well-known issue of convergence to a single model. Considering these issues, we adapt the EM algorithm to a heteroscedastic context. We also propose a numerical error integration approach which considers data uncertainty and addresses convergence to a single model. We further generalise this proposed approach to have flexible weights. We compare the proposed approaches using simulation studies.
Oct 18, 2023 15:00 \| Polytechnique Montreal		Presenter: Aleksandar Jakovljevic \| PhD student, Polytechnique Montreal Title: Investigating the Application of Recurrent Neural Networks and Blocked Cross-Validation for Modelling Conventional Drinking Water Treatment Processes Abstract: In the existing literature, we observe the prevalent use of multilayer perceptrons trained using the holdout method for modelling conventional drinking water treatment processes. In this study, we investigate the use of recurrent neural networks and blocked cross-validation to improve upon these models. The model is tasked with predicting settled water turbidity over the next 15 minutes of operation, based on the previous 30 minutes of operating data. After training and testing, we note that the models trained using blocked cross-validation report much lower test errors than their counterparts trained using holdout, with the GRU network having the lowest errors with RMSE = 0.073 NTU and 0.097 NTU when trained with these methods respectively. However, a simulation of one year of operation after this training/testing phase reveals an inverse correlation between the originally reported test error and the error on this one-year simulation. The result is that the MLP trained using holdout has the best performance on this new set of data. A possible reason for this difference being that the system behaviour in the simulation period has sufficiently changed from the original train/test data and the implications of this result are discussed.
Sep 27, 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Continuously Adaptable Probabilistic Decision-making for Structural Anomaly Detection Abstract: In this seminar, we will walk through the methodologies for developing the decision-making layer in structural anomaly detection. Preliminary results of three relative studies are presented, including a newly developed bounded autoregressive component in Bayesian dynamic linear model, decision-making agents trained with imitation learning algorithms, and time series generation using diffusion models.
Sep 20, 2023 15:00 \| Polytechnique Montreal		Presenter: Van-Dai Vuong \| PhD student, Polytechnique Montreal Title: Unsupervised anomaly detection Abstract: This seminar presents several unsupervised anomaly detection methods and compares their performance on a crack opening dataset from a dam in Canada.
Sep 13, 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: How to run MATLAB code on Compute Canada Clusters Abstract: In this hands-on presentation, I will show how to set up your compute canada cluster account as well as the basics to get started with running MATLAB codes as Slurm files in the cedar cluster.
May 26, 2023 15:00 \| Polytechnique Montreal		Presenter: Zachary Hamida \| Postdoc, Polytechnique Montreal Title: Introducing Ray for Parallel Computing in Python Abstract: This seminar is a short tutorial session that will cover some of the basics in Ray and it's application in machine learning frameworks. The tutorial include example of applications and concepts prepared in python notebook that will be made available online.
May 5, 2023 15:00 \| Polytechnique Montreal		Presenter: Miquel Florensa \| Intern Student, Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Diffusion Models: Overview of Theory and Time Series Generation Abstract: Diffusion models have become a prominent technique for generative modeling of complex data, including images, sounds, and time series. This presentation will offer an overview of the theory and implementation of diffusion models, covering topics such as their architecture, training procedures, and recent advancements. In addition, we will explore the use of diffusion models in time series data by presenting experiments with toy problems that showcase the challenges and potential solutions.
Apr 6 2023 15:00 \| Polytechnique Montreal		Presenter: James-A Goulet \| Professor, Polytechnique Montreal Title: How do neural networks learn? A microscopic look into hidden units through Bayesian inference. Abstract: We are surrounded by applications of neural networks which are capable of achieving complex tasks with an astonishing simplicity. In their basic form, neural networks are known to be universal functions approximators, i.e., they can approximate arbitrary complex functions. In the case of simple feedforward architectures using ReLU activation functions, this approximation is piecewise linear with the number of segments being function of the number of hidden units. With the recent developments related to the TAGI method, we have shown how we can use approximate Gaussian inference as the inference engine for arbitrary large neural networks. Despite the substantial reduction in the number of epochs required to train networks, it remains that even with TAGI, the parameters of neural networks cannot be learnt online in a single epoch. In this presentation, I will present the preliminary work done to identify the root cause of this limitation. The strategy is to proceed from the simplest neural network configuration possible and then build up in order to identify at what point online learning stops to be possible and why it is so.
Mar 31 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka\| Postdoc, Polytechnique Montreal Title: Homoscedastic Aleatory Uncertainty Quantification with AGVI and Self-Tuning Scheduler. Abstract: In this seminar, I will introduce modeling a self-tuning scheduler using a local acceleration model together with learning the homoscedastic aleatory uncertainty using AGVI.
Mar 16 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: How to decide the boundary in Bounded Autoregressive Component? Abstract: To prevent the residual component from capturing the trend in Bayesian Dynamic Linear Model (BDLM), we proposed to constrain the Autoregressive (AR) component in a certain range. Boundary of such a range is controlled by the hyper parameter, gamma, in Bounded Autoregressive (BAR) component. When the boundary is too small, the too strong constrain causes false alarms. On the other hand, when the boundary is too large or close to infinity, no constrain is applied and BAR is close to AR component. In this seminar, we will see how we can tune this hyper parameter to decide a proper boundary according to the desirable false alarm rate and detectability.
Mar 10 2023 15:00 \| Polytechnique Montreal		Presenter: Ali Fakhri \| MSc. student, Polytechnique Montreal Title: Google's MLP-Mixer architecture. Abstract: In this seminar, we will go over Google's MLP-Mixer architecture and observe how it fares against the state-of-the-art methods on image classification tasks. We will conclude with presenting the pros and cons of the MLP-Mixer architecture.
Mar 3 2023 15:00 \| Polytechnique Montreal		Presenter: James-A Goulet \| Professor, Polytechnique Montreal Title: Formulation of probabilistic softmax and remax functions. Abstract: The Tractable approximate Gaussian inference (TAGI) uses the hierarchical softmax for classification tasks. This is because no closed-form formulation exists to propagate uncertainty through the standard softmax, whereas, one exists for the hierarchical version. However, in order to build attention mechanisms as those used in transformers, we need a softmax function as the hierarchical version is unsuited. In this presentation, we will see how can we obtain a close-form solution for a probabilistic softmax as well as for the new remax function that is better suited for an integration with TAGI.
Feb 24 2023 15:00 \| Polytechnique Montreal		Presenter: Miquel Florensa \| Intern student, Polytechnique Montreal Title: Unit Testing on cuTAGI. Abstract: In this seminar I will explain the basis of unit testing and general testing on software development. I will also present some basic information about C++ and CUDA in order to prepare the assistants to use the cuTAGI library. At the end we will run through a demo/tutorial on how to install, compile and run cuTAGI and pyTAGI.
Feb 17 2023 15:00 \| Polytechnique Montreal		Presenter: Bhargob Deka \| Postdoc, Polytechnique Montreal Title: Laplace Package for Bayesian Deep Learning. Abstract: In this seminar, we will look through the Laplace package to perform uncertainty quantification for deep learning applied to regression tasks. The package will be tested for both and full Hessian cases, as well as, for single and two hidden layers.
Feb 2 2023 15:00 \| Polytechnique Montreal		Presenter: Van-Dai Vuong \| PhD student, Polytechnique Montreal Title: Bypassing for inference in TAGI. Abstract: During inference In TAGI, the hidden states of the layer (i-1) are updated from those of the subsequent layer (i), bypassing the activation units. In this seminar, we show why we can do it. Furthermore, we show that we can update the hidden states of the layer (i-1) directly from the those of the layer (i+1), bypassing the layer (i).
Jan 19 2023 15:00 \| Polytechnique Montreal		Presenter: Zhanwen Xin \| PhD student, Polytechnique Montreal Title: Bounded Autoregressive Component in Bayesian Dynamic Linear Model. Abstract: In Bayesian Dynamic Linear Model (BDLM), a pattern change in the time series data will be firstly and mostly perceived as model noise stored in the residual autoregressive component. This usually delay our detection of anomaly or miss the anomaly with small magnitude. My first seminar will present a bounded autoregressive (BAR) component that prevents the residual hidden state from shifting and capturing the changed pattern. Results on a synthetic time series and a real time series from a bridge in Montreal will be presented to show that with BAR, BDLM could potentially detect anomaly at earlier stage and anomaly with smaller amplitude.
Jan 12 2023 15:00 \| Polytechnique Montreal		Presenter: Zachary Hamida \| Postdoc, Ali Fakhri \| MSc. student, Polytechnique Montreal Title: Reunion d'avancement: predire la degradation et comprendre l'effet des interventions (Phase 2). Abstract: This seminar presents a summary of the research work accomplished through the past year. The topics include, the improvements to the deterioration model, the progress in planning intervention activities on a network-scale, and estimating the cost ratio of delaying interventions individual on elements as well as on the whole structures.