- Google Research Award, "Sentiment Classification Using Multimodal Data", 2011-2012.
In this study, we deal with several challenges in the sentiment classification problem such as; developing an efficient multi-model learning algorithm, processing the online data and the computational complexity of the develloped algorithms. For this purpose, we propose a hybrid model, which uses information fusion techniques for the combination of the fMRI, text and face expression data. The ensemble learning architectures are usefull tools for this purpose also and fMRI data will be employed in the design phase of the algorithm, to optimize the correlation parameters between the query text and facial expression data, received from the webcams.
Our proposed architecture enables;
- Distributed and parallel processing of data,
- Self-organizing and dynamic optimization of the models and
- Goal oriented adaptive fusion of data.
This 4-layered information fusion framework includes (see figure below);
- Experimentation and Data Collection,
- Feature Fusion,
- Information Extraction Fusion and
- Decision Fusion
This project was supported by Google Inc.
Developing ways to study mental representations has been a long-standing and intriguing challenge to computer scientists. MVPA is a very recent approach that few researchers have implemented in neuroimaging research. MVPA uses machine learning algorithms to train on fMRI data, and can be used to classify the type of information being processed or cognitive operation that is being carried out, based on distributed patterns of neural activity in the brain. In its current state, MVPA is a very advanced and promising approach that can enable researchers to answer questions about cognitive operations and brain function that could not be assessed with other approaches. However, given its recent nature, the algorithms researchers use for MVPA has much room to improve. In this project, we propose an interdisciplinary collaboration that aims to fill this gap in the literature. Specifically, we propose several fMRI studies, whereby the feasibility of the current machine learning algorithms in the literature can be tested, and the algorithm with maximum performance can be identified. With this approach, via seeking out the machine learning approach that can best describe neural activity in the brain associated with cognitive processes, our aim is to build a model, called Local Voxel Networks that can be used to classify neural activity assessed by fMRI during cognitive operations. The suggested approach has theree main steps:
- Step 1: Acquiring the fMRI data from experiments
- Step 2: Enhancing the fMRI data by image preocessing techniques to prepare for MVPA techniques
- Step 3: Developping a new model for fMRI image and comparing it to the available techniques
This project is currently supported by Tübitak
Researchers in Computer Science discipline put a great effort for developing machine intelligence techniques, inspired from human intelligence and to cultivate elegant mathematical tools to design intelligent machines. In this project, by using these tools, we will design a mathematical model for human cognitive processes (such as memory, learning and emotion). In other words, we reverse inspire from the machine intelligence techniques to model the natural intelligence. For this purpose, we will initially record neural activity during encoding and retrieval of neutral and emotional study material using functional magnetic resonance imaging (fMRI). We next develop a hierarchical learning-memory model based on the recorded fMRI data. The modules of this model will be integrated in web-based information and communications (ICT) tool, so that cognitive neuroscientists all around the world can employ the proposed model to their own data. The major goal of this project is to develop an ICT based learning memory research platform, which provides a hierarchical model for cognitive processes using brain connectivity. This easy-to-use platform will provide a new tool for researchers to analyse and visualize neural data for their ongoing researches. In order to achieve the above goal, we suggest a simplified model, called Cognitive Learner, which creates a bridge between “high-level” learning-memory tasks and fMRI data. We, also, propose an information theoretic approach to estimate the degree of connectivity among the voxels of the fMRI data. This approach provides a Hierarchical Brain Network (HBN) for the underlying cognitive process based on the acquired neural data. The input to the overall system is the fMRI signal, recorded during encoding and retrieval of neutral and emotional study material. The outputs are
- a representation of “homogenous” brain regions in Hierarchical Brain Network
- the label of the cognitive process