GameMaker (2.3)
Icon

Multi-Layer Neural Network

TeroHannula

You must be logged in to obtain assets

Description

Incoming updates: There are few upcoming changes and additions to this asset, which I want to mention as heads-up as they can affect asset behaviour compared to current version. I have no planned date when these will be ready, as some parts are tricky to make. In future update I will introduce Layer-types, which will be building blocks for two new models: "Sequential" and "Graph". Layers have different behaviours, and can be connected to other layers, giving ability to create more versatile architecture for neural networks. The current MLP-model can be thought as subset of Sequential-model, where all layers happen to be dense-layers. However I will keep MLP as separate model, as it has intrinsic value of written differently in GML, which can be more easier to understand. More information of upcoming changes at the bottom of description.

Multi-Layer Neural Network in GML with backpropagation!

  • Feedforward neural network in pure GML
  • Also includes alternative C++ extension for more speed.
  • Asset includes scripts for neural network, but also examples shown in this page and video.
  • GML solution should theoretically be multi-platform.
  • Derivatives are written by hand and uses look-up tables.
  • Simplified free version can be found here: link

What asset will let you do.

  • Create multi-layer neural network
  • Layers are fully-connected and signal propagates forward.
  • Train with Gradient Descent for supervised learning
    • Use backpropagation
    • Choose gradient descent optimizer.
    • Give lot of examples so network can learn.
  • Train with Genetic Algorithm for unsupervised learning
  • Save and load as Buffer or JSON-string.
    • Define precision of Buffer to reduce file-size.
  • Includes way of storing example input/output pairs in arrays.
  • Includes MNIST-reader
    • Lets you open MNIST-dataset and easily train networks with it.
    • Same reader also supports MNIST Fashion dataset.
    • MNIST is dataset of 60.000 hand-drawn digits, with test-set 10.000.
    • MNIST Fashion is similiar dataset, but has images of clothings.
  • Asset has three versions of scripts: Array, Grid, Plus
    • Array is for readibility and learning purposes.
    • Grid is for "pure GML performance" version.
    • Plus is for actual speed using C++ extension.
    • Extension is self-made, (my first C++ code). Works similiarly to Array.

List of supported Gradient Descent Optimizers

  • Stochastic
  • Momentum
  • Nesterov
  • Adam
  • AdaGrad
  • AdaDelta
  • AdaMax
  • Nadam
  • RMSprop
  • AMSgrad

List of supported activation functions

  • Tanh
  • TanhLecun
  • TanhShrink
  • ArcTan
  • sin
  • cos
  • Sigmoid
  • BipolarSigmoid
  • LogSigmoid
  • LogLog
  • Relu
  • ReluCap
  • LeakyRelu
  • PRelu
  • Elu
  • Gelu
  • Selu
  • Celu
  • Swish
  • Elish
  • SoftSign
  • SoftPlus
  • SoftClipping
  • SoftExponential
  • Sinc
  • Gaussian
  • SQRBF
  • ISRU
  • ISRLU
  • SQNL
  • BentIdentity
  • BinaryStep
  • Absolute
  • Threshold
  • HardTanh
  • HardSigmoid
  • HardShrink
  • HardSwish
  • HardElish

List of supported cost functions

  • Mean Squared Error
  • Mean Absolute Error
  • Mean Squared Logarithmic Error
  • Root Mean Squared Error
  • Root Mean Squared Logarithmic Error
  • Huber loss
  • Hinge
  • Smoothed Hinge
  • Quadratically Smoothed Hinge
  • Log-Cosh
  • Cross Entropy
  • Categorial Cross Entropy
  • Binary Cross Entropy
  • Multi-Label Cross Entropy
  • Exponential
  • Hellinger
  • Kullback Leibler Divergence
  • Generalized Kullback Leibler Divergence
  • Itakura-Saito Distance

About upcoming changes:

Asset will be renamed as "Mieli Framework - Neural network scripts for GMS2"

  • All related scripts will be have "mieli_" at beginning.
  • Will support three neural network models: MLP, Sequential, Graph
  • Seq. and Graph uses upcoming Layers as building blocks

Multi-layer Perceptron / Fully-connected neural network

  • Scripts will be renamed as "mieli_mlp_"
  • One bug in bias-calculation will be fixed.
  • I try implement multi-threading for MLP plus, to further improve performance of extension.
  • General organizing of code and commenting more.

New Sequential model

  • Layers are connected sequentially -> layers are connected like chain.
  • Can create Multi-layer Perceptron by using only Dense-layers.

New Graph model

  • Layers can be connected as 'nodes'
  • Layer can be connected to several other layers, or take several layers as input
  • Might be able to have loops in structure, I'll have that checked.
  • Layers can be ordered sequentially which makes Sequential model subset of Graph model.

Upcoming Layer-types

  • Input
  • Convolution - wip
  • Conv.Transpose - wip
  • Conv.1x1
  • MaxPool
  • MeanPool
  • UpSample
  • Activation
  • Relu
  • DropOut
  • Dense
  • Recurrent - planned
  • LSTM - planned
  • GRU - planned
  • Flatten
  • Reshape
  • Combine
  • MaxOut
  • SoftMax

End User Licence Agreement (EULA).

Age Rating: 4+

Version

GMS2.3 - Version 1.0.0. Published March 17, 2021

Loading, please wait

Package contents

Loading, please wait

What is the issue?

Loading, please wait