[cs-talks] Upcoming CS Seminars: IVC (Tues) + Student Sem (Thurs)

Greenwald, Faith fgreen1 at bu.edu
Tue Oct 13 15:46:33 EDT 2015

IVC Seminar
Understanding and Improving the Internal Representation of CNNs
Aditya Khosla, MIT
Tuesday, October 20, 2015 at 2pm in MCS 148

Abstract: The recent success of convolutional neural networks (CNNs) on object recognition has led to CNNs becoming the state-of-the-art approach for a variety of tasks in computer vision. This has led to a plethora of recent works that analyze the internal representation of CNNs in an attempt to unlock the secret to their remarkable performances and provide a means to further improve upon these performances by understanding the shortcomings. While some works suggest that the CNN learns a distributed code for objects, others suggest that they learn a more semantically interpretable representation consisting of various components such as colors, textures, objects and scenes.

In this talk, I explore methods to deepen the semantic understanding of the internal representation of CNNs and propose methods for improving it. Unlike prior work that relies on the manual annotation of each neuron, we propose an approach that uses existing annotation from a variety of datasets to automatically understand the semantics of the firings of each neuron. Specifically, we classify each neuron as being one detecting color, texture, shape, object part, object or scene and apply this to automatically parse images at various levels in a single forward pass of a CNN. We find that despite the availability of ground truth annotation from various datasets, the task of identifying exactly what a unit is doing turns out to be rather challenging. As such, we introduce a visualization benchmark containing the annotations of the internal units of popular CNN models, allowing for further research to be conducted in a more structured setting.

We demonstrate that our approach performs well on this benchmark and can be applied to answering a number of questions related to CNNs: How do the semantics of neurons evolve during training? Do they latch on to specific concepts and stick to them, or do they fluctuate? Do the semantics learned by a network differ when training from scratch or fine-tuning? How does the representation change if the image set is the same but the label space changes?

Related paper:


Aditya Khosla is a PhD student at MIT working on deep learning for computer vision and human cognition. He is interested in developing machine learning techniques that go beyond simply identifying what an image or video contains, but instead predict the impact visual media has on people e.g., predicting whether someone would like an image or not, and whether they would remember it. He is also interested in applying computational techiques to predictably modify these properties of visual media automatically. He is a recipient of the Facebook Fellowship, and his work on predicting image popularity and modifying face memorability has been widely featured in popular media like The New York Times, BBC, and TechCrunch. For more information, visit: <http://mit.edu/khosla> http://mit.edu/khosla

Student Seminar
Bitcoin Attacks!
Ethan Heilman, BU
Thursday, October 15, 2015 at 12pm in MCS 148

Topic: Attacking Bitcoin's network for fun and profit.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs-mailman.bu.edu/pipermail/cs-talks/attachments/20151013/66d4d488/attachment.html>

More information about the cs-talks mailing list