Create NDArray from CSV

1

(Question originally posted in Gitter)

Hi, is there a simple way to read a CSV file containing numbers into a NDArray?

Anything simpler than the following Scala code?

val recordReader = new CSVRecordReader(0, ',')
recordReader.initialize(new FileSplit(file))
val records = new ListBuffer[Array[Int]]()
while (recordReader.hasNext)
  records += recordReader.next.toArray.map(_.toString.toInt)
records.toArray.toNDArray
2

Use a RecordReaderDataSetIterator instead, if you set the miniBatchSize to be exactly the number of rows in the CSV file, the next DataSet you get from it will have the entire file loaded in an INDArray.

Do you want to load it like that for some custom operations on it, or do you want to use it for training? Because for training you don’t actually need to load everything into an NDArray manually, you can use RecordReaderDataSetIterator directly.

3

It is a matrix representing a convolution. The number of rows or columns is not fixed and so, unknown until the full CSV has been read.