Single node SPARK cluster

DL4J
1

I have an off-site single node SPARK cluster that I want to utilize… It’s running SPARK 2.4.3 and HADOOP 2.7.7 on YARN… It has (24) cores and (32) GB of ram… I wanted to try any example just to make sure everything is configured correctly before I start my project…

Looking through the examples, I noticed there are (2) different SPARK examples… dl4j-spark-cluster and dl4j-spark… Does it matter which one I try? Personally, the easier the better…

https://github.com/eclipse/deeplearning4j-examples/tree/master/dl4j-spark-examples

2

This would be the best place to get started.
https://github.com/eclipse/deeplearning4j-examples/tree/master/dl4j-spark-examples/dl4j-spark/src/main/java/org/deeplearning4j/patent

3

I finally had a chance to look into this example and I dont have an Azure account… I am running my own Hadoop node… Is there a different example I can try? I tried running the tinyimagenet example, but I got a ton of errors… I was able to correct most of them, but got stuck on one that said the data was corrupted when its not…

I’m going to try the AnimalModelByHdfsSparkCluster and will keep you posted… If there is a better one, please let me know!

AnimalModelByHdfsSparkCluster.java