Issue with nextAction(Observation observation) on beta7

RL4J
1

Hi, I’m having trouble with nextAction(…) from DQNPolicy on 1.0.0-beta7.
I build an observation from a double array > new Observation(Nd4j.create(states)) where states is double of length 19.
There is no problem while training (I use it in the MDP implementation), but the following error arises when when the same observation is used in policy.nextAction(observation):
“Input that is not a matrix; expected matrix (rank 2), got rank 1 array with shape [19]. Missing preprocessor or wrong input type? (layer name: layer0, layer index: 0, layer type: DenseLayer)”

2

I guess that is a bit of an unintended consequence that models always expect to be given batches, even if in the context of reinforcement learning it doesn’t make quite as much sense.

But you can fix this problem easily by replacing policy.nextAction(observation) with policy.nextAction(Nd4j.expandDims(observation, 0)).

3

Thanks Paul for the reply.
This worked but I had to adjust it a bit since I’m using “observation” as an instance of class Observation, so I applied policy.nextAction(Nd4j.expandDims(observation.getData(), 0)).