Back to overview
Matrix Product
Neural Net Problems - Exercise 4
March 26, 2020

Here are my solutions to exercise 4.

Implementing Our Network to Classify Digits

Part 1


Write out a=σ(wa+b) in component form, and verify that it gives the same result as the rule, 1+exp(jwjxjb)1, for computing the output of a sigmoid neuron.


Let it be stated that I have not yet taken linear algebra at college, so I have very limited experience with it.

Let us say that layer 2 has 2 nodes and layer 1 has 3 nodes.

The weights from layer 1 to layer 2 can be expressed as the following (wji, where j is the neuron in the second layer and i is the neuron in the first layer):


This is exactly the same as 1+exp(jwjxjb)1 but computed for both neurons at once with matrices!

Say we wanted to compute the output of the first sigmoid neuron in layer 2.


Also, I wanted to note that I found this website called the ml cheatsheet and it has been really useful in describing the mathematic concepts.

The header image was taken from Khan Academy.