Machine Learning

• How to Improve Your Network Performance by Using Curriculum Learning
• L2 Regularization
• The Evolution of AlphaGo to MuZero
• Different Conceptions of Learning
• Beyond Gradient Descent

From Inferential Learning:

• Sigmoid
  • \(\sigma(x) = \frac 1 { 1 + \exp(-x)}\)
  • Range \((0,1)\)
  • Useful for producing probability for binomial distribution
• Softplus
  • \(\zeta(x) = \log(1+\exp(x))\)
  • Range \((0, \infty)\)
  • A smooth version of positive-part function \(x^+ = max(0, x)\)
  • Useful for producing \(\beta\) or \(\sigma\) of normal distribution

Relations:

• \(\frac {d} {dx} \zeta(x) = \sigma(x)\)
• \(\frac {d} {dx} \sigma(x) = \sigma(x)(1- \sigma(x))\)
• \(\log \sigma(x) = - \zeta(-x)\)
• \(\zeta(x) - \zeta(-x) = x\)