Access preprocessed data after transformation?

DataVec
1

In DL4J/Datavec, is there a way to access the columns of preprocessed data after at the scoring step? I have a case where I have a csv of data that includes fields that are not used by the neural net for prediction, but they are important to include in my output after the predictions are made.

Before I train my model, I have been using this step for preprocessing to remove the columns prior to training:

TransformProcess transformProcess = 
    new transformProcess.Builder(schema).removeColumns(columnsToOmit).build());
RecordReader transformProcessRecordReader = 
    new TransformProcessRecordReader(recordReader, transformProcess);

The problem I am running into is that after this transformation, I can of course train or make predictions, but I can no longer access those columns that are removed.

Is there a way to “ignore” the columns rather than removing them so that I can access them after the model makes a prediction?

In my debugger I can see some protected fields that show the data is still there but i’m really trying to avoid a custom implementation of iterator if there is a simpler way to do this.

2

In principle, the transformation that is applied is all about transforming it to be input for the network.

However, there is a metadata feature that is pretty handy for your case. Put the data you want to keep for output purposes into the metadata of the dataset, and you’ll be able to get it without confusing the network.