Projects

  1. Google Research Award, "Sentiment Classification Using Multimodal Data", 2011-2012.

    2. In this study, we deal with several challenges in the sentiment classification problem such as; developing an efficient multi-model learning algorithm, processing the online data and the computational complexity of the develloped algorithms. For this purpose, we propose a hybrid model, which uses information fusion techniques for the combination of the fMRI, text and face expression data. The ensemble learning architectures are usefull tools for this purpose also and fMRI data will be employed in the design phase of the algorithm, to optimize the correlation parameters between the query text and facial expression data, received from the webcams.

    Our proposed architecture enables;

    • Distributed and parallel processing of data,
    • Self-organizing and dynamic optimization of the models and
    • Goal oriented adaptive fusion of data.
    figure1

    This 4-layered information fusion framework includes (see figure below);

    • Experimentation and Data Collection,
    • Feature Fusion,
    • Information Extraction Fusion and
    • Decision Fusion
    This project was supported by Google Inc.


  2. Tubitak Project 112E315, "Local Voxel Networks for Modeling and Classification of Brain Activity During Cognitive Processing, Using Brain Signals"

    3. Developing ways to study mental representations has been a long-standing and intriguing challenge to computer scientists. MVPA is a very recent approach that few researchers have implemented in neuroimaging research. MVPA uses machine learning algorithms to train on fMRI data, and can be used to classify the type of information being processed or cognitive operation that is being carried out, based on distributed patterns of neural activity in the brain. In its current state, MVPA is a very advanced and promising approach that can enable researchers to answer questions about cognitive operations and brain function that could not be assessed with other approaches. However, given its recent nature, the algorithms researchers use for MVPA has much room to improve. In this project, we propose an interdisciplinary collaboration that aims to fill this gap in the literature. Specifically, we propose several fMRI studies, whereby the feasibility of the current machine learning algorithms in the literature can be tested, and the algorithm with maximum performance can be identified. With this approach, via seeking out the machine learning approach that can best describe neural activity in the brain associated with cognitive processes, our aim is to build a model, called Local Voxel Networks that can be used to classify neural activity assessed by fMRI during cognitive operations. The suggested approach has theree main steps:

    • Step 1: Acquiring the fMRI data from experiments
    • Step 2: Enhancing the fMRI data by image preocessing techniques to prepare for MVPA techniques
    • Step 3: Developping a new model for fMRI image and comparing it to the available techniques
    We aim to classify cognitive processes using developed model with a high accuracy, acquired during neural activity by fMRI. Proposed project framework, Local Voxel Networks, has potential to build the infrastructure of cognitively guided information technologies. Attaching a forehead band or wearing a helmet in order to acquire brain signal are ongoing studies which will make it possible to train machines with our “thoughts” during daily life. Thus a user interface composed of “thoughts” can be developed. Achieving this project has implications for several distinct fields. Specifically, developing a machine learning algorithm to study mental processes using neural data can bring together the fields of Computer Science, Psychology, and Neuroscience, and as a result benefit all fields. In addition, via improving our understanding of how information and cognitive operations are represented in the brain, this project can have a more broad impact in the field of Cognitive Sciences. Thus, the interdisciplinary nature of this project can potentially have a high impact that can benefit independent but complementary areas of science, that could not be otherwise achieved with sole efforts from each of these areas.

    This project is currently supported by Tübitak


  3. Tubitak Project 114E045, "Multi-layered Cognitive Learning Model"

    4. Researchers in Computer Science discipline put a great effort for developing machine intelligence techniques, inspired from human intelligence and to cultivate elegant mathematical tools to design intelligent machines. In this project, by using these tools, we will design a mathematical model for human cognitive processes (such as memory, learning and emotion). In other words, we reverse inspire from the machine intelligence techniques to model the natural intelligence. For this purpose, we will initially record neural activity during encoding and retrieval of neutral and emotional study material using functional magnetic resonance imaging (fMRI). We next develop a hierarchical learning-memory model based on the recorded fMRI data. The modules of this model will be integrated in web-based information and communications (ICT) tool, so that cognitive neuroscientists all around the world can employ the proposed model to their own data. The major goal of this project is to develop an ICT based learning memory research platform, which provides a hierarchical model for cognitive processes using brain connectivity. This easy-to-use platform will provide a new tool for researchers to analyse and visualize neural data for their ongoing researches. figure1 In order to achieve the above goal, we suggest a simplified model, called Cognitive Learner, which creates a bridge between “high-level” learning-memory tasks and fMRI data. We, also, propose an information theoretic approach to estimate the degree of connectivity among the voxels of the fMRI data. This approach provides a Hierarchical Brain Network (HBN) for the underlying cognitive process based on the acquired neural data. The input to the overall system is the fMRI signal, recorded during encoding and retrieval of neutral and emotional study material. The outputs are

    • a representation of “homogenous” brain regions in Hierarchical Brain Network
    • the label of the cognitive process
    We predict that the Cognitive Learner will correctly recognize the type of information participants are encoding and retrieving from memory, based on distributed patterns of neural activity in the brain.

    This project is currently supported by Tübitak