Perceptron

the perceptron is the simplest model of a neuron: takes a weighted sum of inputs, fires if it exceeds a threshold. foundation of everything that follows.

by “firing”, we simply mean that after all the inputs have each been multiplied by the bias term, they are all added together. This sum is then passed through an Activation Function, there are many but this often either just squishes the range of possible values to a scale between 0 and 1 (Sigmoid Function) or sets to 0 if less than 0 ( ReLU )

$$z=\sum _i{w}_i{x}_i+b$$ $$\hat{y}=\{\begin{array}{ll}1& z\ge 0\\ 0& z<0\end{array}$$

$"\\begin{document}\n\\usetikzlibrary{arrows.meta, positioning}\n\\begin{tikzpicture}[\n input/.style={circle, draw, thick, minimum size=10mm, fill=blue!8, text=black},\n sumnode/.style={circle, draw, thick, minimum size=14mm, fill=orange!12, text=black},\n actnode/.style={rectangle, draw, thick, minimum width=16mm, minimum height=10mm, rounded corners=3pt, fill=green!10, text=black},\n output/.style={circle, draw, thick, minimum size=10mm, fill=red!10, text=black},\n bias/.style={circle, draw, thick, minimum size=10mm, fill=yellow!15, text=black},\n weight/.style={font=\\small, midway, fill=white, inner sep=2pt, text=black},\n arr/.style={-{Stealth[length=2.5mm]}, thick},\n lbl/.style={font=\\small\\bfseries, text=black!40}\n]\n\n$

source code