Quickstart using GPS trajectories file from UCI

Hi,
I would like to experiment/learn how to use DL4J using an input dataset containing GPS locations.

I found the following dataset which meets my requirements:

https://archive.ics.uci.edu/ml/machine-learning-databases/00354/GPS%20Trajectory.rar

I could be (probably am) wrong. My reading of the quickstart code I was advised to use to start with:

Is designed to take
synthetic_control.data from the UCI archive
as input.

In order to make it work with the GPS trajectories dataset, is the only solution to modify UCISequenceClassification.java
?

What am I missing? Any suggestions would be greatly appreciated.

Thanks,

Alex Donnini

@adonnini I would start from there if possible. A pre done neural net and iterator are there for you. You should just need to point it at the text file.

The main point is to isolate a label per row. That label will be an index representing a class.

Feel free to ask follow up questions and understand bits at a time. I can step you through the necessary concepts.

Thanks for the quick response Adam.

I will follow your advice. However, I will need to change

downloadUCIData()

, not just point it at the right file (Line 180), wouldn’t I?

Thanks,

Alex

Yes adapting it is a good idea. You can change it to keep the download or to just use a local file path. That’s up to you.

My main goal first is to get you to understand how we create csv data with classifier labels then you can adapt it yourself as necessary.

Note that for design purposes it’s up to you whether you want to add the time element in there or just build a simpler model that tries to learn patterns on particular locations.

Usually GPS data has a time element so I figured that would be a good start for you.

I adapted the download method to the GPS track points file and ran

UCISequenceClassification

a typical line in a features .csv file looks like this

292 -10.91861335 -37.06844958 1410945904545

After fixing a couple of errors caused by number formatting, I ran into
the following error:

"
Exception in thread “main” java.lang.IllegalStateException: Cannot
convert sequence writables to one-hot: class index 148 >= numClass (6).
(Note that classes are zero-indexed, thus only values 0 to nClasses-1
are valid)
"
On this one, I think I need your help.

Here is the complete log:

o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
o.n.n.NativeOpsHolder - Number of threads used for linear algebra: 6
o.n.l.c.n.CpuNDArrayFactory - Binary level Generic x86 optimization
level AVX512
o.n.n.Nd4jBlas - Number of threads used for OpenMP BLAS: 6
o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Linux]
o.n.l.a.o.e.DefaultOpExecutioner - Cores: [12]; Memory: [30.0GB];
o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
o.n.l.c.n.CpuBackend - Backend build information:
GCC: “7.5.0”
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS
Exception in thread “main” java.lang.IllegalStateException: Cannot
convert sequence writables to one-hot: class index 148 >= numClass (6).
(Note that classes are zero-indexed, thus only values 0 to nClasses-1
are valid)
at
org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.convertWritablesSequence(RecordReaderMultiDataSetIterator.java:685)
at
org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.convertFeaturesOrLabels(RecordReaderMultiDataSetIterator.java:362)
at
org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.nextMultiDataSet(RecordReaderMultiDataSetIterator.java:326)
at
org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.next(RecordReaderMultiDataSetIterator.java:206)
at
org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator.next(SequenceRecordReaderDataSetIterator.java:361)
at
org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator.next(SequenceRecordReaderDataSetIterator.java:340)
at
org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator.next(SequenceRecordReaderDataSetIterator.java:44)
at
org.nd4j.linalg.dataset.api.preprocessor.AbstractDataSetNormalizer.fit(AbstractDataSetNormalizer.java:106)
at
org.deeplearning4j.examples.quickstart.modeling.recurrent.UCISequenceClassification.main(UCISequenceClassification.java:126)

Process finished with exit code 1

Please let me know if you need anything from me.

Thanks,

Alex

@adonnini this looks like you don’t have a label index? How many labels is your dataset supposed to have? Sorry I haven’t dug much in to it but I’ll try to guide you if you can give me the information I need. I’ll try to be a bit more thorough if I get more time but I don’t have the bandwidth to dissect the whole dataset atm.

Hi Adam,

I appreciate your response and any help you can give me. I reverted to
the original GPS track points file where there was a track id column.
Now a typical entry looks like this:

"
292 -10.91861335 -37.06844958 4 1410945904457

"
The column headings in the GPS track point files are:

"
id latitude longitude track_id time
1 -10.9393413858164 -37.0627421097422 1 2014-09-13 07:24:32

"

The file has 18108 rows.

I ran UCISequenceClassification with the new GPS track points file. The
result was the same.

I hope the information above helps shed some light on the problem.

Thanks. Have a good day,

Alex

@adonnini what class are you hoping to give each one? The track_id? If so you need to isolate that as your label. Set that up then show me what you did and I can teach you next steps. The UCI template
follows a sequence classification problem so that should give you something to experiment with.

If that’s not your label you will need to process it and ensure you have a label.

As for the rest of this some feature engineering may need to happen. What are you hoping to predict with GPS? I Think on that and then think about what other variables you may or may not need to add to find a pattern.

Adam,

Thanks.

I am not sure I understand what you mean by class given to each one.
Could you point me to a starting point for doing this?

My goal is to predict the next location a user will visit. Given this, I
assume that track_id is the parameter for the class. So track_id should
be my label. Makes sense?

I also don’t understand what you mean by “If so you need to isolate that
as your label.” is this a manual process? How do I do this?

I am pretty clear on what I want to achieve (and have been from the
start). I am still very much learning the terminology and processes
involved in implementing a neural network to help me achieve my goal.

Thanks,

Alex

Hi Adam,

I think I made some progress.

I made the following changes:

  1. I changed the value of records in

contentAndLabels (arrayList<Pair<K, V>) from the default 0 to 449 to the
value of the track id field in the GPS track file.

Consequently, the number of records in contentAndLabels is now equal to
the number of records in the GPS track file, >18,000

  1. I applied the 75/25 train/test split (which I do not believe was
    applied in the original code.

  2. I changed the maximum size of files to be held in the train and test
    features and labels directories correspondingly. Now the nuber of files
    in the train/test features/labels directories is in the thousands.
    I checked the values of files in the labels directories. They are
    correct, i.e. they correspond to the value of the track id filed in the
    train/test features files.
    By the way, something strange is that the features files in the
    train/features directory are not always created although the
    corresponding label files are always created. I do not understand why.
    It could be a question of timing changes due to the additional work I do
    inside the loop where features and labels files are written.

  3. I changed the value of

numLabelClasses

from 6 to 50000 (although the maximum number of the track id field is
under 39000, I believe).

I did not change the value of miniBatchSize which is set to 10, although
probably I should change it (to what value?)

Bottom line, UCISequenceClassification now fails with the following
error below.

What do you think? Should I undo any of the changes I made? Which other
changes should I make?

Please let me know if you need additional information from me.

Thanks,

Alex

o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
o.n.n.NativeOpsHolder - Number of threads used for linear algebra: 6
o.n.l.c.n.CpuNDArrayFactory - Binary level Generic x86 optimization
level AVX512
o.n.n.Nd4jBlas - Number of threads used for OpenMP BLAS: 6
o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Linux]
o.n.l.a.o.e.DefaultOpExecutioner - Cores: [12]; Memory: [30.0GB];
o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
o.n.l.c.n.CpuBackend - Backend build information:
GCC: “7.5.0”
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS
o.d.n.m.MultiLayerNetwork - Starting MultiLayerNetwork with
WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode
set to [NONE]
o.d.e.q.m.r.UCISequenceClassification - Starting training…
Exception in thread “main”
org.deeplearning4j.exception.DL4JInvalidInputException: Received input
with size(1) = 5 (input array shape = [1, 5, 1]); input.size(1) must
match layer nIn size (nIn = 1)
at
org.deeplearning4j.nn.layers.recurrent.LSTMHelpers.activateHelper(LSTMHelpers.java:171)
at
org.deeplearning4j.nn.layers.recurrent.LSTM.activateHelper(LSTM.java:145)
at org.deeplearning4j.nn.layers.recurrent.LSTM.activate(LSTM.java:110)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.ffToLayerActivationsInWs(MultiLayerNetwork.java:1147)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2798)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2756)
at
org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:174)
at
org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:61)
at org.deeplearning4j.optimize.Solver.optimize(Solver.java:52)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fitHelper(MultiLayerNetwork.java:1767)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1688)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1675)
at
org.deeplearning4j.examples.quickstart.modeling.recurrent.UCISequenceClassification.main(UCISequenceClassification.java:175)

Process finished with exit code 1

@adonnini each line (I guess gps recording in this case) will have a class associated with it. That class is a label for the model to learn the pattern you’re looking for.

Whatever you do please have a problem definition that is achievable before continuing. Machine learning problems fall in to 2 buckets: classification (eg: this picture is a cat or the direction is up) or regression (this is the price of a house given the inputs)

I’ve been assuming you’re doing classification this whole time (it’s what most people are doing) but if it’s regression please do tell me. That would require some re configuration.

Regarding the second post: yes track_id is likely your label based on the problem I’m seeing there.

What I mean by “isolate” is configure the iterator to use that as your label. The iterator needs a column index to treat as special. Otherwise it will treat it as a feature.

You would need to do the same for any problem you’re trying to perform.

Thanks Adam. This is useful.

Did you see my last post? I think I did at least some of what you
recommended, using the track id as the label.

After making that change and increasing the number of labels allowed,
processing went further up to the start of the training phase where it
failed due to some problems with parameter values, I think. Please let
me know when you get a chance. In the meantime, I will take a look at
the iterator code.

I am quite familiar with regression (statistics and math major). Based
on the two problem buckets, I would say that having to choose one of
them, my problem falls into the regression bucket, although that is not
quite true. It certainly is not a classification problem.

A bit of background, In my app I already have functions that process
user location information to determine likely next location, typical
paths (trajectories), and “typical” locations (stay points). I use
clustering (KMeans++ and DBScan modified to also use time as a distance
measure in addition to latitude and longitude), and my own algorithms to
determine stay points and trajectories.

Now, as a next step, I want to implement the same functions using a
neural network because the approach I am using now lacks a learning
component. Adding one is feasible in theory but not practical (keeping
in mind that the app runs on Android devices).

To a certain extent, the neural network I need to use is one based on
(semi)unsupervised learning as I do not know the output in advance, and
there isn’t a formula (regression) to calculate the output. I hope you
can see why I don’t think the regression bucket is quite right form my
problem.

I fond a number of papers which discuss implementation of neural
networks to determine a user’s next location. The one which most
interested me, at least in part because they have an open source
implementation, is

Although mieLab seems to have done a very nice job, I decided to not use
their neural network mainly for two reasons:

  • Coded in Python (I would need to learn it nearly from scratch, and
    find a way to run Python on an Android device, not impossible)
  • The project owners are not interested in others using their neural network

Essentially, I need to understand and implement feature extraction,
selection/use of activation functions, embeddings, and attention
functions as they apply to my problem.

I hope the above was not too long and helps place what I am trying to do
in a clearer context.

Thanks,

Alex

@adonnini for regression you’ll want to look at our examples there then as well. Take a look here:

The example is a bit older but shows how to evaluate regression, set the objective (in this case you’ll probably want to do an RMSE and change the output)
There’s also threads on the forums:

Regression instead of a classification - #6 by treo
LSTM Regression Example - #10 by andrew-selvia

It’s up to you whether you want to do classification or regression. Just make sure you frame the problem correctly.

Usually with GPS data you want to predict something like a kind of action where people are or if they’re making a certain kind of action based on something like speed in addition to location.

If you can’t frame it in terms of a set of concrete labels you probably need to look at doing regression.

Without reading through your examples and just your description here predicting the user’s next location in terms of GPS coordinates might be possible if you had something like the speed and the neural net could predict a target coordinate/direction based on that.

Predicting direction (north, south,…) could be possible if you label your data correctly.

Regarding not knowing in advance you can probably create a dataset based on that. Accelerometer data + a location that can show a direction should be possible.
What have you investigated in this direction?

I strongly recommend you know how to frame the problem and figure out what’s possible before writing too much code.

Hi Adam,

Thanks for your feedback.

The issue(s) I am still struggling with are not related to the “next
location” problem itself. I am very familiar with all the possible
approaches to estimating a next location and have implemented a solution
in my application. The main problem is that it lacks a learning mechanism.

What I still don’t quite understand, and perhaps this is what you mean
by knowing how to frame the problem, is how given inputs of track points
(latitude, longitude and time) a neural would learn to deduce
trajectories and stay points, and a potential next location.

I know how to do this using more traditional approaches (as I have
implemented in my app). I don’t understand how to teach a neural network
to use those approaches (e.g. clustering, distance measurements, and
speed of movement). I really don’t get it.

I think I understand the concepts of feature extraction, activation
functions, embeddings, attention function, etc. but when it comes to
applying them to my problem I am stuck. I really feel I am missing
something pretty fundamental/basic.

Thanks,

Alex

@adonnini this sounds like you want potentially multiple models. How are you setting up your clustering? If you’re doing clustering you should know how to setup a set of feature vectors. What you’d put in to a clustering algorithm you can put in to a neural net.

The main problem is the label you want to use.
Let’s focus on your goals:

  1. trajectories: this is a regression problem - you’d need a trajectory at a given point to use as the label
  2. distance measurements: this is a multi point regression problem. You’ll again want to have a set of inputs and distance measurements as targets
  3. speed of movement: same thing. You need to have a speed at a given time + some data points.

Classification problems would be:

  1. predict “fast/slow”
  2. up/down/direction

Try to clearly segment out 1 objective to start from. You might be able to build a common data pipeline for most of this then build out different models depending on your use case.

Thanks Adam,
A good starting point. The point you make about clustering is very
helpful. Thanks.
One question, what if my input is track points, and not trajectories? In
my app, I produce trajectories (I call them paths) starting from track
points. Neural net apps which produce next location estimates use Python
torch to produce trajectories (I think).
Thanks,
Alex

If you’re doing path prediction then just make sure the path can be clearly identified in your data somehow. Think in terms of potential patterns the net is supposed to learn from the data. For example if you’re predicting where someone’s going to go next there’s 2 variables:

  1. direction
  2. speed

Both of those should indicate how far you will go next.

Hi Adam,

I am attempting to follow the regression example you pointed me to a few
days ago and modify the code I have been using. My initial goal si to
get a “simple” “regression” network run successfully be fore proceeding
with the full implementation of my model. By the way, my understanding
is that in Python torch produces trajectories from track points. Is
there something similar that you know of in the Java world?

Here are the details of my work on the regression model to date.

I have 122 feature and 122 label files where the value in each label
file is equal to the value of the track_id column in the feature file
(id,latitude,longitude,track_id,time) ending with the same number (e.g.
0.csv in the labels directory has value of 1 which is the value of
track_id in csv.0 in the features directory).

My dataset iterator is deifned as follows (using the info in
https://deeplearning4j.konduit.ai/deeplearning4j/reference/recurrent-layers#example-4-many-to-one-and-one-to-many-data)

     DataSetIterator trainData = new 

SequenceRecordReaderDataSetIterator(trainFeatures, trainLabels,
miniBatchSize, numLabelClasses,
true,
SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END);

     //Normalize the training data
     DataNormalization normalizer = new NormalizerStandardize();
     normalizer.fit(trainData);              //Collect training data 

statistics
trainData.reset();

     //Use previously collected statistics to normalize on-the-fly. 

Each DataSet returned by ‘trainData’ iterator will be normalized
trainData.setPreProcessor(normalizer);

My network is defined as follows

     MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
             .seed(123)    //Random number generator seed for 

improved repeatability. Optional.
.weightInit(WeightInit.XAVIER)
.updater(new Nadam())
.gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)
//Not always required, but helps with this data set
.gradientNormalizationThreshold(0.5)
.list()
.layer(new
LSTM.Builder().activation(Activation.TANH).nIn(5).nOut(10).build())
.layer(1, new
RnnOutputLayer.Builder(LossFunctions.LossFunction.MSE)
.activation(Activation.IDENTITY).nIn(10).nOut(numLabelClasses).build())
.build();

     MultiLayerNetwork net = new MultiLayerNetwork(conf);
     net.init();

numLabelClasses is set to -1 as indicated here

Execution fails when attempting to run net.init() producing the log below.

I think the reason lies in the fact that I don’t quite understand the
use of the nIn, nOut, miniBatchSize, and related parameters, and what
they should be set to given the number of train and test feature and
label files I have. Part of my problem is that the only way to try and
understand these parameters is by reading the code which has not been
very successful.

When you get a chance, please let me know what you think and any
pointers towards the solution of my most recent problems.

By the way, just FYI, I was able to run UCISequenceClassification to
complettion without errors, producing meaningless (non)results as expected.

Thanks,

Alex

Sorry. Here is the error log:

o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
o.n.n.NativeOpsHolder - Number of threads used for linear algebra: 6
o.n.l.c.n.CpuNDArrayFactory - Binary level Generic x86 optimization
level AVX512
o.n.n.Nd4jBlas - Number of threads used for OpenMP BLAS: 6
o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Linux]
o.n.l.a.o.e.DefaultOpExecutioner - Cores: [12]; Memory: [30.0GB];
o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
o.n.l.c.n.CpuBackend - Backend build information:
GCC: “7.5.0”
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS
o.d.n.m.MultiLayerNetwork - Starting MultiLayerNetwork with
WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode
set to [NONE]
Exception in thread “main” java.lang.IllegalStateException: Indices are
out of range: Cannot get interval index Interval(b=0,e=640,s=1) on array
with size(0)=629. Array shape: [629], indices: [Interval(b=0,e=640,s=1)]
at org.nd4j.linalg.api.ndarray.BaseNDArray.get(BaseNDArray.java:4239)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.init(MultiLayerNetwork.java:712)
at
org.deeplearning4j.nn.multilayer.MultiLayerNetwork.init(MultiLayerNetwork.java:605)
at
org.deeplearning4j.examples.quickstart.modeling.recurrent.UCISequenceClassification.main(UCISequenceClassification.java:202)

Process finished with exit code 1