FERNeuralNets

class neuralnets.ConvolutionalLstmNN(image_size, channels, emotion_map, time_delay=2, filters=10, kernel_size=(4, 4), activation='sigmoid', verbose=False)[source]

Convolutional Long Short Term Memory Neural Network.

Parameters:
  • image_size – dimensions of input images
  • channels – number of image channels
  • emotion_map – dict of target emotion label keys with int values corresponding to the index of the emotion probability in the prediction output array
  • time_delay – number of time steps for lookback
  • filters – number of filters/nodes per layer in CNN
  • kernel_size – size of sliding window for each layer of CNN
  • activation – name of activation function for CNN
  • verbose – if true, will print out extra process information

Example:

net = ConvolutionalLstmNN(target_dimensions=(64,64), channels=1, target_labels=[0,1,2,3,4,5,6], time_delay=3)
net.fit(features, labels, validation_split=0.15)
fit(features, labels, validation_split, batch_size=10, epochs=50)[source]

Trains the neural net on the data provided.

Parameters:
  • features – Numpy array of training data.
  • labels – Numpy array of target (label) data.
  • validation_split – Float between 0 and 1. Percentage of training data to use for validation
  • batch_size
  • epochs – number of times to train over input dataset.
class neuralnets.ConvolutionalNN(image_size, channels, emotion_map, filters=10, kernel_size=(4, 4), activation='relu', verbose=False)[source]

2D Convolutional Neural Network

Parameters:
  • image_size – dimensions of input images
  • channels – number of image channels
  • emotion_map – dict of target emotion label keys with int values corresponding to the index of the emotion probability in the prediction output array
  • filters – number of filters/nodes per layer in CNN
  • kernel_size – size of sliding window for each layer of CNN
  • activation – name of activation function for CNN
  • verbose – if true, will print out extra process information

Example:

net = ConvolutionalNN(target_dimensions=(64,64), channels=1, target_labels=[0,1,2,3,4,5,6], time_delay=3)
net.fit(features, labels, validation_split=0.15)
fit(image_data, labels, validation_split, epochs=50)[source]

Trains the neural net on the data provided.

Parameters:
  • image_data – Numpy array of training data.
  • labels – Numpy array of target (label) data.
  • validation_split – Float between 0 and 1. Percentage of training data to use for validation
  • batch_size
  • epochs – number of times to train over input dataset.
class neuralnets.TimeDelayConvNN(image_size, channels, emotion_map, time_delay, filters=32, kernel_size=(1, 4, 4), activation='relu', verbose=False)[source]

The Time-Delayed Convolutional Neural Network model is a 3D-Convolutional network that trains on 3-dimensional temporal image data. One training sample will contain n number of images from a series and its emotion label will be that of the most recent image.

Parameters:
  • image_size – dimensions of input images
  • time_delay – number of past time steps included in each training sample
  • channels – number of image channels
  • emotion_map – dict of target emotion label keys with int values corresponding to the index of the emotion probability in the prediction output array
  • filters – number of filters/nodes per layer in CNN
  • kernel_size – size of sliding window for each layer of CNN
  • activation – name of activation function for CNN
  • verbose – if true, will print out extra process information

Example:

model = TimeDelayConvNN(target_dimensions={64,64), time_delay=3, channels=1, label_count=6)
model.fit(image_data, labels, validation_split=0.15)
fit(image_data, labels, validation_split, epochs=50)[source]

Trains the neural net on the data provided.

Parameters:
  • image_data – Numpy array of training data.
  • labels – Numpy array of target (label) data.
  • validation_split – Float between 0 and 1. Percentage of training data to use for validation
  • batch_size
  • epochs – number of times to train over input dataset.
class neuralnets.TransferLearningNN(model_name, emotion_map)[source]

Transfer Learning Convolutional Neural Network initialized with pretrained weights.

Parameters:
  • model_name – name of pretrained model to use for initial weights. Options: [‘Xception’, ‘VGG16’, ‘VGG19’, ‘ResNet50’, ‘InceptionV3’, ‘InceptionResNetV2’]
  • emotion_map – dict of target emotion label keys with int values corresponding to the index of the emotion probability in the prediction output array

Example:

model = TransferLearningNN(model_name='inception_v3', target_labels=[0,1,2,3,4,5,6])
model.fit(images, labels, validation_split=0.15)
fit(features, labels, validation_split, epochs=50)[source]

Trains the neural net on the data provided.

Parameters:
  • features – Numpy array of training data.
  • labels – Numpy array of target (label) data.
  • validation_split – Float between 0 and 1. Percentage of training data to use for validation
  • epochs – Max number of times to train over dataset.