DatasetIterator for music generation

What you are trying to do here is known as an auto-regressive problem, as you want to predict the next step from the previous steps.

I’ve talked about some of the details of setting up the data for such a problem in this post: Training data shapping - #2 by treo

When it comes to composing music with deeplearning, and even dl4j in particular, there is this example: deeplearning4j-examples/dl4j-examples/src/main/java/org/deeplearning4j/examples/wip/advanced/modelling/melodl4j at master · eclipse/deeplearning4j-examples · GitHub

It is reading Midi instead of the very simplified model that you propose, and unfortunately it isn’t working quite correctly.