Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Torch Tutorial for PlantVillage Challenge

Torch Tutorial for PlantVillage Challenge.

To refresh your memory, in Torch model:forward(input) computes the output of neural network. This is the forward pass of the back-propagation algorithm while model:backward(input,gradOutput) is the backward pass of the back-propagation. Forward and backward passes for criterion are also very similar.

We’ll use stochastic gradient descent with momentum (optim.sgd) to make the updates to the network. Let us also use the learning rate scheduler in AlexNet paper: divide the learning rate by 10 every 30 epochs.

Our training script will look like:

    require 'nn'
    require 'datasets/plantvillage.lua'
    require 'models/alexnet.lua'

    ------------------[[Configuration]]-----------------
    -- create model, criterion and data generator
    model = createModel()
    criterion = nn.CrossEntropyCriterion()
    dataGen = DataGen('path/to/folder/with/train-val-directories/')

    -- learning rate and other parameters
    nbEpochs = 120
    learningRate = 0.01
    momentum = 0.9
    batchSize = 32

    ------------------[[Training]]-----------------
    -- params are parameters of the network that need to be optimized
    -- gradParams are gradients of these parameters
    params, gradParams = model:getParameters()

    -- for optim solver
    optimState = {
            learningRate = learningRate,
            momentum = momentum,
            nesterov = true,
            dampening = 0.0,}

    local function feval()
        return criterion.output, gradParams
    end

    -- confusion matrix to calculate accuracies
    confusion = optim.ConfusionMatrix(38)

    nEpoch = 1

    -- function to train model for an epoch
    function train()
        optimState.learningRate = learningRate*math.floor((nEpoch - 1) / 30)
        nEpoch = nEpoch + 1
        model:training()
        for input, target in dataGen:trainGenerator(batchSize) do
            -- Forward pass
            output = model:forward(input)
            criterion:forward(output, target)

            -- Backward pass
            model:zeroGradParameters() -- clear gradients from previous backward
            critGrad = criterion:backward(output, target)
            model:backward(input, critGrad)

            -- Make updates using adam
            optim.sgd(feval, params, optimState)
        end
    end

    -- function to validate current model on validation data
    function validate()
        confusion:zero()
        model:evaluate()
        for input, target in dataGen:valGenerator(batchSize) do
            -- Forward pass
            output = model:forward(input)
            confusion:batchAdd(output, target)
        end

        confusion:updateValids()
        valAcc = self.confusion.totalValid*100
        print('Validation accuracy = ' .. valAcc)
    end

    for i = 1, nbEpochs do
        train()
        validate()
    end

If you look at the code in the repo, you’ll find that I have divided training into main.py and train.py scripts for configuration and training respectively.

In main.lua, we manage the configuration of the neural network and criterion. In train.lua, I wrote a Trainer class with Trainer:train() and Trainer:validate() methods very similar to train() function above except with some logging. This allows us to experiment with different configurations easily.