Brain tumor is a fatal central nervous system disease that occurs in around 250，000 people each year globally and it is the second cause of cancer in children. It has been widely acknowledged that genetic factor is one of the significant risk factors for brain cancer. Thus， accurate descriptions of the locations of where the relative genes are active and how these genes express are critical for understanding the pathogenesis of brain tumor and for early detection. The Allen Developing Mouse Brain Atlas is a project on gene expression over the course of mouse brain development stages. Utilizing mouse models allows us to use a relatively homogeneous system to reveal the genetic risk factor of brain cancer. In the Allen atlas， about 435，000 high-resolution spatiotemporal in situ hybridization images have been generated for approximately 2，100 genes and currently the expression patterns over specific brain regions are manually annotated by experts， which does not scale with the continuously expanding collection of images. In this paper， we present an efficient computational approach to perform automated gene expression pattern annotation on brain images. First， the gene expression information in the brain images is captured by invariant features extracted from local image patches. Next， we adopt an augmented sparse coding method， called Stochastic Coordinate Coding， to construct high-level representations. Different pooling methods are then applied to generate gene-level features. To discriminate gene expression patterns at specific brain regions， we employ supervised learning methods to build accurate models for both binary-class and multi-class cases. Random undersampling and majority voting strategies are utilized to deal with the inherently imbalanced class distribution within each annotation task in order to further improve predictive performance. In addition， we propose a novel structure-based multi-label classification approach， which makes use of label hierarchy based on brain ontology during model learning. Extensive experiments have been conducted on the atlas and results show that the proposed approach produces higher annotation accuracy than several baseline methods. Our approach is shown to be robust on both binary-class and multi-class tasks and even with a relatively low training ratio. Our results also show that the use of label hierarchy can significantly improve the annotation accuracy at all brain ontology levels.