Hello, I’m making a model for music generation and looking for suggestions on how to improve it. The dataset consists of a feature and label file, respectively.
A single feature is a 32 timestep long sequence, and each timestep is a 24-d vector with elements -1, 0 or 1.
A label is a single timestep, identical as those from the features.
In CSV a feature/label pair looks as such:
--feature-- 0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,-1,0,0,1,0,0,0,1,0 ... ... 29 more rows ... 0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0 --label-- 0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0
This is my current model setup:
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder() .weightInit(WeightInit.XAVIER) .updater(new Adam()) .gradientNormalizationThreshold(1) .list() .layer(new LSTM.Builder().activation(Activation.RELU6).nIn(24).nOut(100).build()) .layer(new RnnOutputLayer.Builder(LossFunctions.LossFunction.MEAN_ABSOLUTE_ERROR) .activation(Activation.TANH).nIn(100).nOut(24).build()) .build();