• Use cases
  • How it works
  • Developers

A complete open source AI platform to predict emotion from images and video 

Access training data and a fully trained deep learning model

Data set = 5 million images

Infrastructure = Tensor flow 
Model type = Inception V3 

Neon Open's model architecture

Architecture 

Modified Inception v3
Uses PReLUs instead of ReLUs for stability
One auxiliary head for regularization1024-feature final layer, followed by softmax to get prediction

Training

Train on NxNxD dense submatrices of full ranked comparisons matrix
Apply Laplacian smoothing across demographics
Batch normalization
Gradient clipping 
Modified RankNet loss

Behavioral task used to collect training data

Data was collected on Amazon's Mechanical Turk from US participants over the age of 18. Age and gender information for each participant was collected.
Task
Flash triplets of images at subject
Locations of images randomized on screen
Force subject to make a quick decision
Images are repeated to ensure consistency
Provides a weak sample that A > B, A > C
Images sampled by scraping videos off the internet
Trial images randomly chosen, weighted so that images with fewer labels are more likely to be surfaced in task
Controls to minimize cheaters
Required training process for task
Internal inconsistency in the task is flagged
Interacting too quickly is flagged
Demographics for a subject are validated by asking for the demographics again a few days later

Get model and code on GitHub 

Picture

Selected papers


Lebrecht, S., Bar, M., Feldman Barrett, L., & Tarr, M.J. Micro-valences: Perceiving Affective Valence in Everyday Objects. Frontiers in Psychology 3. 2012: doi:10.3389/fpsyg.2012.00107

​Lebrecht, S., & Tarr, M. J. Can Neural Signals for Visual Preference Predict Real-World Choices? BioScience. November 2012 / Vol. 62 No. 11

​
See more papers

Helpful resources

We find the following resources useful for people working in computer vision, video, and deep learning and so wanted to share!
Training data
ImageNet
Image Aesthetics: AVA
8M YouTube videos

Deep learning infrastructure 
TensorFlow
Caffe

Computer vision toolkits
Open CV
Get in touch

About

Team
Medium
​GitHub
License
Patents



Contact

Google Group
Twitter
​Instagram
Email
© NEON OPEN 2017
  • Use cases
  • How it works
  • Developers