Main Page | Namespace List | Class Hierarchy | Alphabetical List | Class List | Directories | Namespace Members | Class Members

NeuronalAgent Class Reference

#include <src/NeuronalAgent.h>

Inheritance diagram for NeuronalAgent:

Inheritance graph
[legend]
Collaboration diagram for NeuronalAgent:

Collaboration graph
[legend]
List of all members.

Detailed Description

An agent using a two layer perceptron neural network for its AI decisions.

First, train it to mimick another agent with an explicit AI. This will give a non-random starting point for the later evolution, and at least gives this agent a chance to survive. Then, use a genetic algorithm to evolve of population of neuronal agents into something else.

If you're really lucky, the offsprings may be more clever than the original AI, and escape into the Internet to take over the world. Please mail me if this happens :)

For each neighbor agent, the network is run with the following inputs. Parameters marked 'c' are common to all neighbors computations, but provided nonetheless on a per-neighbor basis. Parameters marked 'n' are neighbor-dependent, and set to respectively 0,0,0,-2 when there is no neighbor. Parameters marked 'e' are only used during the evolution phase, and not the training phase (TODO: work in progress). They represent the acquired knownledge part of the agent, and are reset when creating a new offspring.

The network outputs 'n' and 'c' are all coefficients to apply to a vector. See Agent.h for steering vector definitions. All neighbor-dependent vector are set to 0 when there is no neighbor. The network outputs 'e' are scalar values only used after training (not part of the error function)

The neural network is run for each neigbhor, then the computations are aggregated according to the following policies. For each result vector:

Once a steering force is computed for each output, the forces are then once again aggregated according to one of the following policies for the final steering force:

Average should in principle give smoother results and allows training all network weights. Winner take all is useful for more natural results (ex: avoid the closest), to decide only one clear action in a case where multiple choices could be made, instead of an average and not so good result. Unfortunately as gradients are 0 for unused weights in this case, the training phase may not go so well. Averaging both neighbor computations and result vectors is thus the default, but you're encouraged to play with other policies.

During training phase, the Error function is computed as follow: E = 1 - (ResultForce dot TeacherResultForce) where ResultForce is the output of all previous computations, and TeacherResultForce is what the teacher would have done in the same situation.

Note: The training phase is stateless, without memory, so the neural network can be registered on many teachers at the same time to get data from many different situations

Author:
Nicolas Brodu


Public Types

enum  AggregateStrategy { Average, WinnerTakeAll, User }
 Strategies to aggregate the steering force vectors from each neighor computations: + For each steering force resulting from a steerForXXX behavior: - Average: average vectors from all neighbor computation contributions to get final steering vector - Winner take all: get largest norm output*force vector as the only one for this steering force + For the final steering force - Average: average contributions from all individual steering results, computed as defined above - Winner take all: get largest norm individual steering force, computed as defined above, as the only one.

Public Member Functions

 NeuronalAgent (int nhidden, int specie, AggregateStrategy neighborsStrategy=Average, AggregateStrategy forcesStrategy=Average, bool addToGlobalAgentList=false)
virtual void addTeacher (CrogaiAgent *teacher)
virtual void removeTeacher (CrogaiAgent *teacher)
 For learning subclasses to be notified when this agent leaves the game.
virtual void stopTraining ()
 stops training and remove us from all teacher's students lists As soon as the training stops, this agent is added to the global agent list and may play the game! You can safely position the agent and prepare its avatar before calling this function, it won't interfere with the training or other agents so long as this function is not called.
virtual void setMutationParameters (double mutationRate=0.0, double mutationJitter=0.0)
 Specifies the parameters for the spawning "updateChild" function.

Public Attributes

AggregateStrategy aggregateAgents
 See comment above about policy: - aggregateAgents: For getting each individual steering force from neighbor computation contributions - aggregateForce: For getting the final steering force from each individual steering force.
AggregateStrategy aggregateForces
 See comment above about policy: - aggregateAgents: For getting each individual steering force from neighbor computation contributions - aggregateForce: For getting the final steering force from each individual steering force.
TwoLayerPerceptron neighborTLP
 Our brain. Not very much, but more than most living things in the real world out there!
TwoLayerPerceptron aloneTLP
 Our brain. Not very much, but more than most living things in the real world out there!
bool monitorError
 During training, error can optionally be computed and displayed.

Static Public Attributes

static const int ninput = 10
 number of inputs and outputs are common to all agents.
static const int noutput = 10

Protected Member Functions

virtual void updateChild (CrogaiAgent *child)
 Subclasses may call this function when spawning to set base class parameters.
virtual void displayError (double errorValue, bool alone)
 Dislay the argument as the current training error for this network Default is to display it on cout, but subclasses may change this behavior.
virtual void updateAI (const double elapsedTime, const double currentTime)
 Perform AI update: The position is constrained to be above the terrain, and attitude is updated accordingly.
virtual void learnAI (const double elapsedTime, const double currentTime, CrogaiAgent *teacher)
 Students registered on this teacher should have overloaded this.
virtual void learnOrUpdateAI (const double elapsedTime, const double currentTime, CrogaiAgent *valueProvider)

Protected Attributes

double * input
 avoid creating input and output parameters each time
double * output
double * gradout
osg::Vec3d * outputForces
osg::Vec3d * aggregateOutputForces
double * outputForcesLenCache
std::list< CrogaiAgent * > teachers
 Our teachers. Unused after the training phase.
OpenThreads::Mutex teacherAccessMutex
double mutationRate
 setSpawnParameters
double mutationJitter
 setSpawnParameters

Static Protected Attributes

static const double noutputInverse = 0.1


Constructor & Destructor Documentation

NeuronalAgent::NeuronalAgent int  nhidden,
int  specie,
AggregateStrategy  neighborsStrategy = Average,
AggregateStrategy  forcesStrategy = Average,
bool  addToGlobalAgentList = false
 

Parameters:
nhidden Number of hidden units for the two layer perceptron
specie Specie, or team, this agent belongs to. It's up to you to set one specie for all agents with the same number of hidden units, or to do other weird mixes.
teacher Give this agent a teacher to show it what to do. You can explicitely set the teacher to 0 to de-activate the training phase and start with random weights if you really want so, but I would advice against it if you want the agents to have a reasonable chance to survive. Also the reason why 0 is not a default parameter. Use setTeacher(...) later on to deactivate/reactivate training;
neighborsStrategy How to aggregate the results of computations done for each neighbor. Default is average, but this may not be the most useful for collision avoidance. See below.
forcesStrategy How to aggregate the results of computations done for each steering force. Default is average, but this may not be the most useful for collision avoidance. See below.
addToGlobalAgentList Whether or not this agent should be taken into account for playing the game. Default is false, contrary to parent default, because the network needs training first.


Member Function Documentation

void NeuronalAgent::learnAI const double  elapsedTime,
const double  currentTime,
CrogaiAgent teacher
[protected, virtual]
 

Students registered on this teacher should have overloaded this.

This is the function called during teacher updateAI to notify all students. It does nothing by default, and is never called if your object is not registered as a student.

Reimplemented from CrogaiAgent.

void NeuronalAgent::setMutationParameters double  mutationRate = 0.0,
double  mutationJitter = 0.0
[virtual]
 

Specifies the parameters for the spawning "updateChild" function.

These parameters are inherited by the child, so setting them once for the first generation is enough.

Parameters:
mutationRate The network weights are added a random number between -mutationRate*currentValue and +mutationRate*currentValue. 0 thus makes a perfect copy, which may be useful at the beginning to build a large population from only a few teachers. Default is 0.
mutationJitter The network weights are also added a random number between -jitter value & +jitter value, then normalized by the network layer dimensions. This allows to give a chance to null weights, which would otherwise not be affected by the mutation rate. Default is 0.

void NeuronalAgent::updateAI const double  elapsedTime,
const double  currentTime
[protected, virtual]
 

Perform AI update: The position is constrained to be above the terrain, and attitude is updated accordingly.

Two new internal variables are set for subclasse, which can be checked together with other state variables (position, attitude, etc...)

  • altitude: This variable is set to the position Z delta before calling the stubs If there was a terrain collision, it is strictly negative. Otherwise, it represents the altitude of this agent above the floor. At exact floor level, there is nothing done by default (consider no collision). The position field is not updated, this is done during integration update.
  • floorNormal: the floor normal at the agent location.

Subclasses should call this function at the beginning of their AI update routine

Reimplemented from TerrainAgent.

void NeuronalAgent::updateChild CrogaiAgent child  )  [protected, virtual]
 

Subclasses may call this function when spawning to set base class parameters.

Each agent has an initial genetic payload consisting of neural network weights, which are NOT its current weights. Indeed, each agent can use its memory and message communication skills during its life, and can also learn and train its network during its life.

A new child inherits only the initial weights as genetic information, and does not inherit the parent learnt changes. At the very begining of your program, the first parent may use a teacher to get a reasonable AI and have a chance to survive. As time passes, the new children can evolve these initial weights using genetic algorithm and the hope is that they get better than the initial parent.

Error monitoring, aggregate policies, and spawn parameters are transfered to the child unchanged.

Reimplemented from CrogaiAgent.


Member Data Documentation

bool NeuronalAgent::monitorError
 

During training, error can optionally be computed and displayed.

This class display it on cout, but subclasses may change this behavior. See the displayError function. monitorError is false by default

const int NeuronalAgent::ninput = 10 [static]
 

number of inputs and outputs are common to all agents.

They are respectively the number of parameters for the AI algorithm and the number of steering forces to get a coefficient for.


The documentation for this class was generated from the following files:
Generated on Mon Mar 28 11:27:31 2005 for Crogai by  doxygen 1.4.1