Does NormalizerStandardize.transform(dataSet) support multi-threads?

Tuning Help
1

hi, I have very big data
and want to normalize and save them in advance to avoid repeated normalization during every epoch.
so I want to use multi-threads to reduce normalization time.
but one question is “Does NormalizerStandardize.transform(dataSet) support multi-threads?”.

And one more question:
Does existing NormalizerStandardize.fit(iterator) can be called repeatedly, every time when I get more data?

thanks.

2

@baedaron sorry for the late reply. For the future: it’s stateful so you should synchronize it.

3

so, you mean “not thread-safe and NormalizerStandardize.fit(iterator) from the begining every time”. right?

4

Yes. Sorry, please @ me if you don’t hear from me. Thanks!