ポスター
G. モデリング、ハードウェア、応用
G. Modeling, Hardware Implementation, and Applications |
|---|
| 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-310
高次運動野機能分化を再現する動的状態空間強化学習モデル
A Reinforcement Learning Model with Dynamic State Space Reproduces Functional Differentiation of the Higher Motor Cortices
*田村 尚己(1)、虫明 元(1)、坂本 一寛(1,2)
1. 東北大学大学院医学系研究科 生体システム生理学分野、2. 東北医科薬科大学 神経科学教室
*Naoki Masamune Tamura(1), Hajime Mushiake(1), Kazuhiro Sakamoto(1,2)
1. Department of Physiology, Tohoku University School of Medicine, 2. Department of Neuroscience, Faculty of Medicine, Tohoku Medical and Pharmaceutical University
Keyword: HIGHER BRAIN FUNCTION, REINFORCEMENT LEARNING, DECISION UNIQUENESS, DYNAMIC STATE
Humans adapt to the environment by appropriately associating sensory information and prior action histories with a new action. However, in the real world, a huge variety of information is available. The information that should guide an action is not obvious; learners must find such data unaided. Theoretical models of such cognitive function deepen our understanding of higher-level brain function. Conventional reinforcement learning models are given the state space and action clues as prior knowledge. In contrast, our model finds de novo clues and defines new states using the concept of decision uniqueness, i.e., criteria that aid selection of a deterministic action. Specifically, the model generates multiple Q-tables depending on the types of available clues and then selects the Q-table exhibiting the highest level of decision uniqueness. The model includes Q-tables in which state definitions are not fixed, and dynamically generates states that refer to information for an arbitrary number of time steps. We evaluated our model by training it to perform two different tasks: a delayed response task which requires visually guided action selection by the premotor cortex (PM), and a sequential movement task assessing neuronal activity reflecting memory-guided action sequences in the supplementary motor area (SMA). The model found clues in prior sensory information and action histories, and linked them to appropriate new actions using the decision uniqueness criterion. Furthermore, when one of the two dynamic Q-tables (those detailing prior visual information and action history) was individually suspended, the performance deterioration reflected damage to the PM and SMA, respectively. Our model increases our theoretical understanding of how higher-level motor cortices function, provides a novel framework for reinforcement learning, and serves as a foundation for application of the “computational higher brain dysfunctionology” approach that uses theoretical models to individualize patient treatments. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-311
A Mathematical Model for Stimulus Discrimination in Conditioned Taste Aversion
*Shu-Ting Chang(1), Jay-Shake Li(1)
1. Department of Psychology, National Chung-Cheng University, Chiayi, Taiwan(R.O.C.)
Keyword: Conditioned taste aversion (CTA), Instrumental conditioning, Rescorla and Wagner’s model (RW model), Competing attention
Conditioned taste aversion (CTA) is a learned association of taste and visceral distress, which is traditionally viewed as an instance of Pavlovian conditioning. It can also be regarded as instrumental punishment where the act of drinking is the operant behavior and the reinforcement is a delayed aversive stimulus. Li et al. (2013) presented two bottles of water with different flavors sequentially to rats with an explicit positive contingency between the magnitude of the aversive visceral stimulus and the amount of one of the two flavored water (tastants) drunk on that day. After several days of training, rats could detect the subtle relationship between one of the two tastants and the magnitude of punishment and resumed drinking of the uncontingent tastant. The traditional models of conditioning failed to explain the result of this complicated experiment. In the present study, we proposed a mathematical model of CTA based on Rescorla and Wagner’s model (1972), which is an implementation of the Hebbian learning rule (1949). In the model, the amount of drinking is determined by the antagonism between a drink-suppressing activity and a thirsty drive. The drink-suppressing activity is proportional to the amount of tastant rats drank multiplied by an associating weight. To adjust the weight, a pair of competing attentions were incorporated in addition to the Hebbian learning rule, which associated the tastants with visceral distress. We successfully simulated the two-tastants CTA experiment of rats described above. The results showed that a two-tastants CTA learning is more complicated than what a traditional conditioning model could handle. We need to incorporate competitions between the two tastants to enable the model to efficiently discriminate the two stimuli and their subtle relationship to the unconditioned stimulus. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-312
物体識別の深層ニューラルネットワークが獲得する情報表現-画像テクスチャ情報に基づく解析-
Analysis using image textures for understanding the artificial representation for object classification in the trained AlexNet model
*伊藤 和真(1)
1. 東邦大学
*Kazuma Ito(1)
1. Toho University
Keyword: DEEP CONVOLUTIONAL NEURAL NETWORK, STYLIZED IMAGE, OBJECT CLASSIFICATION, VISUAL SYSTEM
Physiological studies have reported that neurons in intermediate visual (V4) and inferior temporal (IT) cortices of primates selectively respond to shape, image texture, and materials of natural objects as crucial cues for the object perception (Pathpathy & Connor, Nature Neurosci., 2002; Goda et al., J. Neurosci., 2014). Recently, the deep convolutional neural network (DCNN) has been used as a powerful modern tool for achieving and developing advanced computer algorithms (Silver et al., Nature, 2016; Vaswani et al., arXiv, 2017). Especially, DCNN model such as the AlexNet (Krizhevsky et al., NIPS, 2012; Krizhevsky, arXiv, 2014) obtained the suitable mechanism for the object classification (He et al., CVPR, 2016). Interestingly, for the trained AlexNet model, texture-based representations of natural images might underlie the decision of the object classification (Geirhos et al., ICLR, 2019). However, it remains unclear the crucial cue and the characteristics of artificial representations in each layer for the object classification of the AlexNet model. For understanding the mechanism underlying the object classification of the trained AlexNet, we analyzed the modulation of the responses in the model neurons induced by the changes of the texture representation in natural images. In this study, we applied the natural images and stylized images by changing the texture to the AlexNet model and computed the correlation coefficient based on responses of model neurons to these images. Irrespective of distinct texture information, the activities of model neurons to these natural and stylized images were markedly correlated at the early-level layers. By contrast, the magnitudes of the correlation decreased as the level of model layers increased. In addition, the classification results of the model have been markedly altered due to the changes of the texture representation. These results not only imply that the trained AlexNet model might establish texture-based representation for natural images through the hierarchy of their network like the biological visual system but also suggested the important contribution of a texture-based representation to the object classification of the DCNN model. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-313
表情認識時における眼球運動変調を行う扁桃体の計算論的モデル
A computational model of the amygdala for eye movement modulation during facial expression recognition
*山田 泰輝(1)、栗山 凛(1)、山﨑 匡(1)
1. 電気通信大学
*Taiki Yamada(1), Rin Kuriyama(1), Tadashi Yamazaki(1)
1. University of Electro-Communications
Keyword: Amygdala, Computational model, Eye movement modulation, Emotional valence
The amygdala plays a central role in emotional information processing in the brain. Lesions in the amygdala affect eye movements to eyes in the face and impair facial expressions recognition. It is also known that the amygdala encodes neural signals of ambiguity of both positive and negative facial expressions. These findings suggest that the amygdala modulates eye movements to resolve ambiguity in recognized facial expressions.
However, local circuit mechanisms in the amygdala for the eye movement modulation remain unclear because previous studies do not aim to reproduce this modulation.
To address this problem, we implement computational models of the amygdala composed of the lateral nucleus receiving eye positions, the basolateral nucleus representing emotion ambiguity, and the central nuclei whose neural signals correspond to the eye movement.
In our model, the amygdala iterates receiving the signals of eye positions and emotion ambiguity as inputs and driving an eye movement as an output. In particular, we assume our amygdala model learns eye movements through this iteration.
First, to examine contributions of emotion ambiguity to the eye movements, we carried out two simulations. In one simulation, as signals for emotion ambiguity, only a positive one is used, and in the other only a negative one is used. We show that our model successfully learns eye movements to eyes in the face when using only negative ambiguity signals. Thus, although the uncertainty of both positive and negative valence is expressed in the amygdala, it is implied that negative emotional valence ambiguity is vital for the eye movement. In other words, the amygdala might contain a local circuit responsible for negative valence. This observation is consistent with the traditional view of the amygdala as a center of fear conditioning.
We also investigate appropriate model parameters for the eye movement.
In summary, our model provides a valuable means to examine functional roles of distinct nuclei in the amygdala for eye movements towards emotional information processing. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-314
超立方体上の擬似ビリヤードダイナミクスに基づくレザバー計算の性能評価とその改善
Benchmarks for reservoir computing based on pseudo-billiard dynamics in hypercube and its improvement.
*崎野 也真人(1)、田向 権(2)、森江 隆(2)、香取 勇一(1,3)
1. 公立はこだて未来大学、2. 九州工業大学、3. 東京大学生産技術研究所
*Yamato Sakino(1), Hakaru Tamukoh(2), Takashi Morie(2), Yuichi Katori(1,3)
1. Future University Hakodate, 2. Kyushu Institute of Technology, Kitakyusyu, 3. Institute of Industrial Science, The University of Tokyo
Keyword: Pseudo-Billiard Dynamics, Chaotic Boltzmann Machine
This study aims to evaluate the performance of reservoir computing based on pseudo-billiard dynamics in hypercube and improve memory. Reservoir computing (RC) is a kind of recurrent neural network and can be trained with a low computational cost since the training is limited only to output connections. RC has many potential applications, including time-series prediction, pattern recognition, and robot control. Thus, RC attracts much attention in the field of computer science. Reservoir computing based on pseudo-billiard dynamics in hypercube is suitable to be implemented on hardware. The proposed model is based on the chaotic Boltzmann machine (CBM), which behaves chaotically. To control the chaotic motion in the CBM, which is inappropriate to ensure the echo state property of the reservoir computing, can be suppressed with the reference clock. Since the units of the reservoir layer of the proposed model interact with each other in binary signals, the proposed model is expected to realize an efficient hardware implementation. However, the proposed model lacks validation. Thus we evaluate the performance of the proposed model. The proposed model is evaluated by comparing the proposed model and echo state network using the information processing capacity (IPC) task. IPC task takes as input a series of random numbers corresponding to polynomials, and as target output, an orthogonal basis function with degree and delay is introduced. We found that the proposed model is superior for nonlinearity and inferior for memory in the IPC task. As a future plan, we address to improve the memory performance of the proposed model. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-315
部分観測マルコフ決定過程におけるリザバーコンピューティングを基にしたTD学習モデルの短期記憶性能
Short-term memory ability of reservoir-based temporal difference learning model in partially observable Markov decision process.
*吉野 遊(1)、香取 勇一(1,2)
1. 公立はこだて未来大学、2. 東京大学生産技術研究所
*Yu Yoshino(1), Yuichi Katori(1,2)
1. Future University Hakodate, 2. Institute of Industrial Science, The University of Tokyo
Keyword: RESERVOIR COMPUTING, REINFORCEMENT LEARNING , AUTONOMOUS ROBOT
In this study, we investigate the short-term memory performance of a temporal difference (TD) learning model based on reservoir computing(RC) in an autonomous robot. Autonomous robots use information obtained from the environment to determine actions based on tasks and instructions. Robots may have restrictions on the number of sensors that can be mounted and restrictions on the computing power of a small microcomputer. One proposed method for implementing a control system for autonomous robots is to use a combination of RC and reinforcement learning. RC is a framework for processing complex time series based on recurrent neural networks. Reservoirs have short-term memory capacity and are less computationally expensive because they limit the learning portion. In reinforcement learning, agents (robots) and the environment interact through rewards and actions. TD learning, one of the reinforcement learning methods, learns by the difference between temporally successive predictions. The proposed RC-based TD learning model trains the weights of the connections between the reservoir and the output layer. The connection weight is changed sequentially by online reinforcement learning so that the agent can learn and follow an ever-changing environment. In previous studies, I used a simple task to examine short-term memory capacity, which is an important characteristic of the model. The current study will examine whether we can use short-term memory ability to control behavior when we set more complex tasks. Specifically, we investigate whether the system can distinguish between multiple signals given in the past and reach the destination in an extended T-time maze task. The results show that the model is effective for partially observable Markov decision process (POMDP) problems using complex past histories. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-316
統計的帰無モデルとの比較による安静時脳活動の時空間構造解析
Spatiotemporal characteristics of resting-brain activity investigated using statistical null models
*保坂 祐輝(1)、地村 弘二(2)、瀧 雅人(3)、松井 鉄平(1,4)
1. 岡山大学理学部生物学科、2. 慶応大学理工学部生命情報学科、3. 立教大学人工知能科学研究科、4. JSTさきがけ
*Yuki Hosaka(1), Koji Jimura(2), Masato Taki(3), Teppei Matsui(1,4)
1. Okayama Univ, 2. Keio Univ, 3. Rikkyo Univ, 4. JST-PRESTO
Keyword: RESTING STATE, FMRI, NULL MODEL, MACHINE LEARNING
Resting state brain activity can be measured relatively easily and non-invasively in humans. In addition, it has the advantage of high reproducibility among different institutions, making it an important research target with potential clinical applications such as diagnosis of mental disorders. However, despite the intensive studies over the last 15 years, the basic spatiotemporal structure of resting brain activity is still poorly understood. A current leading hypothesis suggests that resting brain activity is the result of the brain transitioning between multiple stable states. To demonstrate the dynamics of such resting brain activity, analytical methods such as the Sliding Window Correlation (SWC) and the Coactivation Pattern Analysis (CAP) have been proposed (Hutchison et al., Neuroimage 2013; Liu and Duyn, PNAS, 2013). These methods have been used to characterize multiple stable states of resting brain activity and the transitions between them. Recently, however, it was pointed out that these analysis methods may not be able to determine whether resting brain activity has multiple stable states (Laumann et al., Cerebral Cortex, 2016; Liegeois et al., Neuroimage, 2017; Matsui et al., Neuroimage, in press), suggesting that previous spatiotemporal structural analyses of resting brain activity based on SWC and CAP have not been able to substantiate the main hypothesis: the transition between multiple stable states of resting brain activity. In the present study, we used Energy Landscape Analysis (ENA) (Watanabe et al., Front. Neuroinfo.,, 2014; Ezaki et al., Phil. Trans. Roy. Soc. A, 2017), which is another promising analysis method. In ENA, stable states can be extracted by fitting the data using the maximum entropy method. We applied this method to both the actual resting state brain activity data provided by the Human Connectome Project and the surrogate data created based on it (Liegeois et al., Neuroimage, 2017). We found that the result obtained by ENA was almost the same for both the real resting brain activity and the surrogate data created from it, suggesting that ENA is unlikely to be able to provide evidence that the resting brain activity has multiple stable states. Currently, we are investigating whether it is possible to discriminate surrogate data from real data using machine learning techniques such as deep learning. | | 2022年6月30日 13:00～14:00 沖縄コンベンションセンター 展示棟 ポスター会場1
1P-317
行列分解を用いたfMRIデータのエンコーディングに関する研究
Encoding in fMRI data through matrix factorization
*遠藤 優介(1)、竹田 晃人(1)
1. 茨城大学
*Yusuke Endo(1), Koujin Takeda(1)
1. Ibaraki University
Keyword: functional magnetic resonance imaging, sparse coding, matrix factorization
For information representation in brain, various models are proposed based on the compression of redundant information. In particular, it is widely recognized that sparse coding (Olshausen and Field, 1996) will be realized in information representation of visual or auditory stimulus. In addition, recent study by two-photon microscope calcium imaging (Ohki and Yoshida, 2020) revealed that sparse coding is realized in primary visual cortex. Their conclusion is based on the fact that visual stimulus can be reconstructed from activities in only a small number of neurons. However, despite of high spatial resolution of two-photon microscope calcium imaging, its observation field is typically one square millimeter and not large. As another method, functional magnetic resonance imaging (fMRI) offers relatively high spatial resolution among non-invasive methods and wide observation field over the whole region of brain. Based on the background above, we aim to reveal whether sparse coding is realized or not in information processing over the whole region of brain. For the purpose, we use public neuroactivity data acquired by fMRI under visual stimulus. In our work, we attempt feature extraction from neuroactivity data by matrix factorization. In matrix factorization, one should decompose d-by-n matrix A to d-by-k matrix B and k-by-n matrix C. These matrices are related by A=BC+E, where the matrix E represents algorithmic error. In particular, this problem is called sparse matrix factorization if either decomposed matrix B or C is sparse. In our work, we try to model sparse coding by sparse matrix factorization: famous sparse matrix factorization algorithms such as Method of Optimal Directions, Sparse-PCA, and Lp-norm minimized ICA are applied to fMRI data. For verification of sparse-coding hypothesis in information processing in brain, we also apply non-sparse matrix factorization algorithms such as standard PCA and ICA. As easily imagined, the properties of extracted feature will depend on matrix factorization algorithm. Therefore, through quantitative analysis of matrix factorization result, we discuss which method of matrix factorization is the most suitable for modelling sparse coding in fMRI data. | |
|
|
