Elizabeth Newman | Emory REU Computational Mathematics for Data Science

A Tensor SVD-based Classification Algorithm Applied to fMRI Data

Tue, 14 Dec 2021 00:00:00 +0000

This post was written by Katy Keegan, Yihua Xu, Tanvi Vishwanath, and Vida Jon and published with minor edits. The team was advised by Dr. Elizabeth Newman. In addition to this post, the team has also given a midterm presentation , created a poster , published, code, and written a paper.

Overview: Can we look at a brain scan and know what the brain’s owner is thinking?

Our research attempts to use computers to correctly classify brain scans into 2 groups, depending on what they are doing during the scan.

Functional MRI

We use brain scans called functional MRIs that show us which parts of the brain are using more oxygen and are therefore most active. We have fMRIs of test subjects who, as they are being scanned, are also shown either a picture or a sentence. If computers can classify these study subjects into one of these two categories by only studying their scans, then in a sense we can read their minds.

fMRIs consist of three-dimensional pixels called voxels, and the data are numbers representing colors. Unlike static MRIs which take a scan at one point in time, fMRIs are repeated every few seconds creating a series of images for each trial. Here is an example of an fMRI of one brain during one trial. The different images are of different slices of the brain. The abbreviations on the right refer to brain regions, for example “SMA” stands for “Supplementary Motor Area” located at the top center of the head.

If we have three-dimensional fMRI brain voxel data for many patients, multiple scans in sequence, then we need to analyze a quantity of data unwieldy even for modern computers.

Image Classification

Image classification is using a computer to figure what what an image represents. Computers can’t see images, so they use features of the images that it can understand. For example, we can train a computer to match an image to a numerical digit. Computers learn by training on many images, for example using the MNIST database of handwritten images. MNIST contains a wide variety of images that can represent a 0 or a 1:

We can construct a “basis”, which can be thought of as a collection of the most relevant features shared by all of the images belonging to that class.

We see that the basis for Class 0 shows more curved features, while the basis for Class 1 contains traces of more straight and vertical features. To choose the basis the image more closely matches, we compute a “projection”, not unlike what you may have calculated with vectors. The larger the projection, the better the match to a particular basis.

The computer tries to classify this test image as either a zero or a one:

This image represents the projection of the test image onto the basis for numeral one:

The image projection onto the basis for zero, shows differences and inconsistencies:

The projection shows that the “distance” between our test image and our two classes is smaller for Class 1 than for Class 0, and so our method classifies our test image as a 1.

By learning from a training set of images a computer can examine the data in a new image and figure out which digit it most resembles. Similarly, we are training our computer to learn how to use the data in an fMRI to classify our study subjects into those who are shown an image and those who are shown a sentence.

Tensors and Singular Value Decomposition

Typically, large data sets like the fMRI voxels are stored in matrices, which have some powerful tools for extracting the most relevant components.

When we store fMRI data in a matrix we lose important relationships between the data points. For example a computer does not know that a particular voxel representing a part of the brain at one moment is that same part of the brain a few seconds later.

In our work, we study how to store our data in a tensor which is like matrix but with more than 2 dimensions. Our tensor of fMRIs have a total of 5 dimensions, shown in the figure below. The green slices consist of voxels of the brain in 3 spatial dimensions: x,y,z (yellow). For each trial multiple images are taken over several seconds (blue), and there are multiple trials (red).

<img width=“600” alt=“fmri_tensors”

When we store our voxel data in tensors instead of matrices, we retain all the information about the 3 dimensional location in the brain, the sequence of the image in time, and the trial. Now we want to decompose our tensor data using a method analogous to the way matrices can be decomposed, so we can use far less data and still extract an accurate prediction of what the subject is thinking. In matrices this is called Singular Value Decomposition; we call our approach tensor-SVD or tSVD.

Here is a diagram of matrix decomposition using SVD:

This what we imagine a tensor SVD would be:

You’ll notice that we use matrix multiplication in working with matrix SVDs. We are searching for the equivalent tensor multiplication.

Other Applications

Our research is not only useful for fMRIs; many datasets have multiple dimensions. Streaming entertaining companies have data on thousands of viewers and what movies they’ve watched. Hospitals track thousands of patients, each of whom has had multiple lab tests and other studies. If our research enables us to classify our fMRI subjects, then we may also be able to predict whether someone will want to watch Terminator, or whether a patient is likely to have cancer.

References

Tensor tensor products with invertible linear transforms

Tensor-tensor algebra for optimal representation and compression of multiway data

Tensor decompositions and applications

Image classification using local tensor singular value decompositions

Fast & Fair: Efficient Training of Fair Neural Networks

Fri, 01 Dec 2023 00:00:00 +0000

This blog post was written by Allen Minch, Hung Anh Vu, and Annie Warren and published after some minor edits. The team was advised by Dr. Elizabeth Newman. More information about this project can be found in our paper, poster blitz video, and poster.

(Un)Fairness in Machine Learning

Our project focuses on a key problem in machine learning: unfairness. How do we define unfairness, and where does it come from?

Classifiers in machine learning, such as deep neural networks, can be very good at classifying complex data, such as images. However, machine learning classifiers trained through supervised learning are designed to minimize the average classification error in the data they are trained on. This means they are inherently designed to be very “faithful” to the data they are trained on and to reflect patterns and correlations in that data. This can be a problem when such correlations involve a sensitive attribute - for instance, a correlation of a person’s race with an attribute we believe should not be related to race.

Suppose the real-world data set that a machine learning classifier is trained on is biased with respect to some sensitive attribute, like race. In that case, the classifier will, in trying to maximize its accuracy on the data it is being trained on, be “faithful” to the bias that it is given and be trained to make predictions that reflect this bias. While it could be fairly stated that these biased predictions are not the fault of the machine learning algorithm itself but the data on which it is trained, that doesn’t change the fact that, as a society, we would generally view it as unfair to base real-world decisions on using a machine learning classifier that makes biased predictions. This is especially true when the stakes are high, where a biased classifier prediction could mean someone getting unfairly arrested, unfairly not being hired for a job, or unfairly being denied a loan. It is thus in our interests as a society to be careful about using machine learning classifiers that might be biased in their predictions and try to tweak how they are trained to reduce this unfairness.

Improving Fairness Through Adversarial Training

We develop efficient adversarial training techniques to reduce the unfairness of classifiers in machine learning in the context of a sensitive attribute like race. In our project, we are dealing with supervised machine learning, where a classifier is trained on a dataset with many samples of input features and a corresponding true label. Because the context is supervised, we have true labels as information to help us evaluate the classifier’s performance. To assess our success in improving the fairness of a classifier, we need to have fairness metrics by which to evaluate a classifier.

Why would we think that adversarial training would help improve fairness? There are a couple of lenses for understanding why we might expect this to improve fairness. One lens has to do with the idea of overfitting. Intuitively, a primary reason we expect a classifier to be unfair is that it is overfitting to the bias in the particular data it is trained on. Adversarial training, by nature, tries to combat overfitting to the training data with its efforts to make a classifier more robust by giving perturbations to the training data points. Thus, we might expect combating overfitting with adversarial training also to combat unfairness, making the classifier less sensitive to bias in the given data. A second lens is that intuitively, a situation we would tend to view as most flagrantly unfair is when two individuals are quite similar yet happen to be part of different sensitive groups and turn out to be classified differently by a classifier. Perhaps some unfairness we observe in a classifier manifests in this kind of scenario. If we can make our classifier more robust - so that two nearby points are more likely to be classified the same way - perhaps this kind of unfairness could be reduced, which could minimize the unfairness of our classifier overall. Since adversarial training improves robustness, this would be another lens to understand why we might expect adversarial training to improve fairness.

We develop a second-order optimization method for solving the adversarial training problem. When solving an optimization problem, first-order methods are used when we only need to compute the gradient. This means we rely solely on first-order information to optimize our function. In the unconstrained case, we can take a step as large as necessary in the direction of the gradient for maximal increase/decrease. However, for our approach, we must satisfy a constraint requiring us to project our solution onto the constraint set. Computing the gradient and projecting the solution can become very computationally expensive. Therefore, with some mathematical speculation, we could utilize second-order information (computing the Hessian) to solve the optimization problem more efficiently.

Evaluating Fairness

We used multiple metrics for evaluating fairness with respect to sensitive attributes. To describe these intuitively, imagine that we had a dataset of individuals whose race is either white or non-white and with true labels as to whether or not they have committed a crime in the past. Suppose we used this data to train a classifier designed to predict whether or not an individual will commit a crime in the future and then used a test dataset to see how the classifier performs.

Independence This means that the prediction of our classifier is uncorrelated with the sensitive attribute; a positive prediction is equally likely for all groups. In our example, independence being satisfied in a test dataset would mean that if 15% of white individuals in the dataset are predicted to commit a crime in the future, then 15% of non-white individuals are also. (Note: Y represents the true label, Y_hat the predicted label, and S the sensitive attribute in the graphic below. The graphic shows two probabilities that would have to be equal for independence to be satisfied).
Separation This means that the prediction of our classifier is conditionally independent of the sensitive attribute, given an individual’s true label.
Sufficiency: This means that our classifier’s predictive usefulness is equal across sensitive groups. If sufficiency is satisfied, then for each $(x, y)$, the probability that an individual’s true label is $x$ given the value of their predicted label is $y$ is the same across sensitive groups.

We implemented these fairness metrics in Python with PyTorch tensors. We worked with binary classification and sensitive attributes in our project, which meant that being given PyTorch tensors containing each individual’s sensitive attribute, true label, and predicted label respectively, we could obtain the proportions needed for evaluating our fairness metrics leveraging some boolean indexing tricks.

Coding Strategies and Main Findings

Our adversarial training problem consists of two optimization problems: an inner one and an outer one. The outer optimization problem is the typical machine learning problem of finding optimal classifier parameters. The inner optimization problem consists of, for each point in the training data set, finding a perturbation to that point within some radius that maximizes the value of the loss function. In practice, to computationally solve a two-layered optimization problem like this, the approach we take is to solve the inner optimization problem at each point, update parameters in the outer optimization problem based on these solutions, solve the inner optimization problem again, and continue going back and forth between the two problems.

As it turns out, robust training, because it requires solving the inner optimization problem at every training data point in every epoch of the outer optimization problem, can be pretty slow computationally compared to simply doing non-robust training. Thus, as the FastNFair team, we focused on solving the inner optimization problem as quickly and efficiently as possible.

Our numerical results on three datasets show that our method is more efficient than first-order methods. We also observe that adversarial training can improve fairness but potentially at the cost of accuracy. If robust training with a certain radius improves fairness, it appears to improve fairness by larger margins than random perturbation; solving the optimization problem well is worthwhile.

Interested in Learning More?

Please see our poster.

Elizabeth Newman

Fri, 01 Dec 2023 00:00:00 +0000

Elizabeth Newman is an Assistant Professor in the Department of Mathematics at Emory University. Her research focuses on multilinear algebra and deep learning algorithms for scientific applications.