Vomitting Out Some Machine Learning with Torch

Don’t know anything about Lua or Torch, and not so much about machine learning. Little project to get going.

Torch is to Lua what Numpy is to python. Never done any lua before, although for a while it was the main language on the esp8266. Torch seems like a popular base for machine learning in competition with theano and tensorflow. Lua is like if python and javascript has a slightly retarded baby.

Thought I’d give a simple tic tac toe playing guy a go. The structure is play a bunch of totally random games, collect up all the winning games. Then the problem is a classification problem where the categories are the next move (1-9).

Then used the stock nn neural network package to learn on it. Had a tough time finding clear docs. I am unimpressed.

Then use trained neural network to play against the random component.

The win stats increased from ~28% to ~45% (with some fluctuations run to run of a couple percent). Not bad. Especially since going second is disadvantageous. Okay, as I wrote that I realized it’s easy to try flipping that. Going first the stats go from 59% to 69%.

Hmmm. Maybe I should look at draws?

Also, a smart strategy for the moves would be to use the suggested moves according to their rank, not using the top suggested move then if that is invalid using a random move.

math.randomseed(os.time())

function won(board,x)
  --diagonals
  if board[1][1] == x and board[2][2]==x and board[3][3] == x then
    return true
  end

  if board[1][3] == x and board[2][2]==x and board[3][1] == x then
    return true
  end

--rows
  for i=1,3 do
    if board[i][3] == x and board[i][2]==x and board[i][1] == x then
      return true
    end
  end
--columns
  for i=1,3 do
    if board[1][i] == x and board[2][i]==x and board[3][i] == x then
      return true
    end
  end

  return false

end


function full(board)
  for i=1,3 do
    for j=1,3 do
      if board[i][j] == '' then
        return false
      end
    end
  end

  return true
end


function mapBoardtoNum(board)
  newboard = {{},{},{}}
  for i=1,3 do
    for j=1,3 do
      if board[i][j] == 'x' then
        newboard[i][j] = 1
      end
      if board[i][j] == '' then
        newboard[i][j] = 0
      end
      if board[i][j] == 'o' then
        newboard[i][j] = -1
      end
    end
  end
  return newboard
end

--[[
print(won({
{'x','',''},
{'x','o',''},
{'x','o',''}}, 'o'))
]]


mymoves = {}
myboards = {}
wins = 0
gamenum = 10000
for k=1,gamenum do

  board = {{'','',''},
  {'','',''},
  {'','',''}}

  move = 'o'

  game = {}
  choices = {}

  turn = 1

  while not won(board,'x') and not won(board,'o') and not full(board) do

    if move == 'x' then
      move = 'o'
    elseif move == 'o' then
      move = 'x'
    end

    repeat
      i = math.random(3)
      j = math.random(3)
    until board[i][j] == ''

    if move == 'x' then
      game[turn] = mapBoardtoNum(board)
      choices[turn] = i -1 + 3 * (j-1) +1
      turn = turn + 1
  end
    board[i][j] = move

  end

  if won(board,'x') then
    wins = wins +1
    for i = 1,#game do
      table.insert(myboards, game[i])
      table.insert(mymoves, choices[i])
    end
  end

end

--print(mymoves)
--print(#myboards)
print('won ' .. wins ..' out of ' .. gamenum)

training = {}
--[[
training.data = torch.Tensor(myboards)
training.labels = torch.Tensor(mymoves)
training.size = function() return (#mymoves) end
]]

training.size = function() return (#mymoves) end
for i=1,training:size() do
  training[i] = {torch.Tensor(myboards[i]), torch.Tensor({mymoves[i]})}
end

ninputs = 9
nhiddens = 30
noutputs = 9
require 'nn'
model = nn.Sequential()
model:add(nn.Reshape(ninputs))
model:add(nn.Linear(ninputs,nhiddens))
model:add(nn.Tanh())
model:add(nn.Linear(nhiddens,noutputs))
model:add( nn.LogSoftMax() )

criterion = nn.ClassNLLCriterion()


trainer = nn.StochasticGradient(model, criterion)
trainer.learningRate = 0.01
trainer.maxIteration = 7

trainer:train(training)




--[[
print(board)
print(won(board,'o'))
print(won(board,'x'))
print(choices)
print(game[1])
]]


board = {
{'x','o',''},
{'x','o',''},
{'x','o',''}
}

logprobs= model:forward(torch.Tensor(mapBoardtoNum(board)))
print(logprobs)
max, pred =torch.max(logprobs,1)
print(max)
print(pred)
--[[
-- Basic format
{
{'x','o',''},
{'x','o',''},
{'x','o',''}
}
]]

print('random won ' .. wins ..' out of ' .. gamenum)


mymoves = {}
myboards = {}
wins = 0
for k=1,gamenum do

  board = {{'','',''},
  {'','',''},
  {'','',''}}

  move = 'o'

  game = {}
  choices = {}

  turn = 1

  while not won(board,'x') and not won(board,'o') and not full(board) do
    --print('yo')
    if move == 'x' then
      move = 'o'
      repeat
        i = math.random(3)
        j = math.random(3)
      until board[i][j] == ''
      board[i][j] = move


    elseif move == 'o' then
      move = 'x'
      --print(board)
      --print(torch.Tensor(mapBoardtoNum(board)))
      local probs = model:forward(torch.Tensor(mapBoardtoNum(board)))
      maxs, pred = torch.max(probs,1)
      --i -1 + 3 * (j-1) +1
      pred = pred - 1
      i = pred % 3 + 1
      j = (pred - pred%3) / 3 + 1
      i = i[1]
      j = j[1]
      --print(i[1])
      --print(j)
      --print(board)
      --print(board[i][j])
      if board[i][j] == '' then
        board[i][j] = move
      else
          repeat
            i = math.random(3)
            j = math.random(3)
          until board[i][j] == ''
          board[i][j] = move
      end



    end

  end

  if won(board,'x') then
    wins = wins +1
  end

end

print('learned won ' .. wins ..' out of ' .. gamenum)

 

Annihilation Creation with Wick Contraction in Python

Trying an alternative approach to dealing with the algebra of qft computationally based on wick contraction. We can easily find an algorithm that finds all possible pairs. Then we just reduce it all accordingly. Some problems: The combinatorics are going to freak out pretty fast.

I think my use of functional stuff like maps and lambdas is intensely unclarifying the code.

import numpy as np

#Needs to return a list of lists of pairs
def pair(mylist):
    #Base case
    if len(mylist) == 2:
        return [[(mylist[0], mylist[1])]]
    #popoff the first element
    element1 = mylist[0]
    pairs = []
    for i in range(1,len(mylist)):
        #Pick one
        element2 = mylist[i]
        #get all the pairs expluidng the already picked pair
        subpairs = pair(mylist[1:i] + mylist[i+1:])
        #Put the picked pair back in x is a list of pairs.
        pairs = pairs + map(lambda x: [(element1,element2)] + x ,subpairs)
    return pairs

#For Fermionic pairing, it might be conveneitn to extend pairs to (element1,element2,+-1)
#With the sign depending on wheter i is even or odd

'''
pairing = pair([1,2,3,4])
print pairing
print len(pairing)
'''

#Here's an idea, I can use a and adag as objects with indices
#and return functions (two point green's functions)
# or could return matrcies that are discretized green's functions.
def contract(twoguys):
    if twoguys[0] == 'a' and twoguys[1] == 'adag':
        return 1
    else:
        return 0

#Confusing, but what
pairings = pair(['a','a','a','adag','adag','adag'])
print pairings
#could abstract out multiplication as a function passed in. Then if contraction returns functions, I could
#return a new function that multiplies the inner functions
#or could replace multiply with np.dot if returning matrices
def multiplyup(pairing):
    return reduce(lambda acc, val: acc * val,map(contract,pairing))

print sum(map(multiplyup,pairings))

 

Attaching the Jordan Wigner String in Numpy

Just a fast (fast to write, not fast to run) little jordan wigner string code

import numpy as np
from numpy import kron, identity
import numpy.linalg as la


# sigma functions

sigma_x = np.array([[0, 1],[1, 0]])
sigma_y = np.array([[0, -1j],[1j, 0]])
sigma_z = np.array([[1, 0],[0, -1]])

# standard basis

spin_up = np.array([[1],[0]])
spin_down = np.array([[0],[1]])

# spin ladder operators

sigma_plus = sigma_x + 1j * sigma_y
sigma_minus = sigma_x - 1j * sigma_y

# pauli spin



N = 3
def chainify(mat, pos):
    if pos == 0:
        newmat = mat
    else:
        newmat = identity(2)
    for j in range(1,N):
        if j == pos:
            newmat = kron(newmat,mat)
        else:
            newmat = kron(newmat,identity(2))
    return newmat


def sx(i):
    return chainify(sigma_x,i)
def sy(i):
    return chainify(sigma_y,i)
def sz(i):
    return chainify(sigma_z,i)
def sp(i):
    return chainify(sigma_plus,i)
def sm(i):
    return chainify(sigma_minus,i)


#print sz(0)
#print sz(1)
#print sz(2)


#print np.dot(sp(0),sp(0))
# sp sm = 2 + 2 sz
#print np.dot(sp(0),sm(0))- 2*identity(2**N) - 2*sz(0)

I = identity(2**N)

fdag = lambda i: sp(i)/2
f = lambda i: sm(i)/2

def stringify(mat, pos):
    if pos == 0:
        newmat = mat
    else:
        newmat = sigma_z
    for j in range(1,N):
        if j == pos:
            newmat = kron(newmat,mat)
        elif j<pos:
            newmat = kron(newmat,sigma_z)
        else:
            newmat = kron(newmat,identity(2))
    return newmat

def cdag(i):
    return np.mat(stringify(sigma_plus/2, i))

def c(i):
    return np.mat(stringify(sigma_minus/2, i))

#print np.dot(cdag(1),c(1)) + np.dot(c(1),cdag(1)) # This is 1
#print np.dot(cdag(1),c(2)) + np.dot(c(2),cdag(1)) # This is 0

#It does appear to work.

print cdag(1)*c(1) + c(1)*cdag(1) # This is 1
print cdag(1)*c(2) + c(2)*cdag(1) # This anticommutator is 0.

What fun!

A little Automatic Differentiation in Python

Givin this a shot as I understand it.


class Differentiable:
 def __init__(self, func, deriv=None):
 self.func=func
 self.deriv = deriv
 def __mul__(self, b):
 a = self
 return Differentiable(lambda x: a.func(x) * b.func(x),deriv = lambda x: a.func(x) * b.deriv(x) +a.deriv(x) * b.func(x))
 def __add__(self, b):
 a = self
 return Differentiable(lambda x: a.func(x) + b.func(x), deriv = lambda x: a.deriv(x) + b.deriv(x))
 def __pow__(self, b):
 a= self
 return Differentiable(lambda x: a.func(x)**b, deriv = lambda x: b*a.deriv(x)* a.func(x)**(b-1))
 def compose(self, b):
 a = self
 return Differentiable(lambda x: a.func(b.func(x)), lambda x: b.deriv(x)* a.deriv(b.func(x)))
import math

cos = Differentiable(math.cos,lambda x : -1*math.sin(x))
sin = Differentiable(math.sin, math.cos)
x = Differentiable(lambda x: x, lambda x: 1)

print (x**2).deriv(2)
print (x**2).func(2)
print (x+(x**2)).deriv(3)

chain = (x**2).compose(x**2)
print chain.deriv(2)

Seems to work. Coo.

 

Quantum Harmonic Oscillator Algebra in Sympy

This is kind of garbage, but it does work.

from sympy import *
a = Symbol('a', commutative=False)
adag = Symbol('adag', commutative=False)
ket = Symbol('|0>', commutative=False)
bra = Symbol('<0|', commutative=False)

expr = bra * a * a * a * adag  * adag * adag * ket
print expr
rules = [(a * adag, adag * a + 1), (a * ket, 0), (bra*adag, 0), (bra * ket, 1)]
expr22 =  expr.subs(rules).expand()

for i in range(10):
    expr22 = expr22.expand()
    expr22 = expr22.subs(rules)
print expr22

Need to loop over it because the substitution rules aren’t smart enough to distribute the commutators themselves.

Still, seems to work. Kind of a hack, but seems to work.

 

Here’s the same thing built out of not much. Not elegantly done particularly

def evalexpr(expr):
    if expr == []:
        return 1
    if expr[-1]=='a':
        return 0
    elif expr[0]=='adag':
        return 0
    else:
        for i in range(len(expr)-1):
            if expr[i]=='a' and expr[i+1]=='adag':
                head = expr[0:i]
                if i+2 < len(expr):
                    tail = expr[i+2:]
                else:
                    tail = []
                return evalexpr(head+tail) + evalexpr(head+['adag','a']+tail)
                break

print evalexpr(['a','a','a', 'adag', 'adag','adag'])