# Does NormalizerStandardize.transform(dataSet) support multi-threads?

**URL:** <https://community.konduit.ai/t/does-normalizerstandardize-transform-dataset-support-multi-threads/3075>\
**Category:** Tuning Help\
**Created:** [February 17, 2024, 10:22am UTC](https://community.konduit.ai/t/does-normalizerstandardize-transform-dataset-support-multi-threads/3075 "2024-02-17T10:22:20Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![baedaron](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/baedaron/32/1132_2.png) [@baedaron](https://community.konduit.ai/u/baedaron)\
**Post date:** [February 17, 2024, 10:22am UTC](https://community.konduit.ai/t/does-normalizerstandardize-transform-dataset-support-multi-threads/3075/1 "2024-02-17T10:22:20Z")

</div>

hi, I have very big data  
and want to normalize and save them in advance to avoid repeated normalization during every epoch.  
so I want to use multi-threads to reduce normalization time.  
but one question is “Does NormalizerStandardize.transform(dataSet) support multi-threads?”.

And one more question:  
Does existing NormalizerStandardize.fit(iterator) can be called repeatedly, every time when I get more data?

thanks.

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [February 29, 2024, 9:13pm UTC](https://community.konduit.ai/t/does-normalizerstandardize-transform-dataset-support-multi-threads/3075/2 "2024-02-29T21:13:47Z")

</div>

@baedaron sorry for the late reply. For the future: it’s stateful so you should synchronize it.

---

<div class="post-metadata">

**Author:** ![baedaron](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/baedaron/32/1132_2.png) [@baedaron](https://community.konduit.ai/u/baedaron)\
**Post date:** [March 2, 2024, 2:33am UTC](https://community.konduit.ai/t/does-normalizerstandardize-transform-dataset-support-multi-threads/3075/3 "2024-03-02T02:33:56Z")

</div>

so, you mean “not thread-safe and NormalizerStandardize.fit(iterator) from the begining every time”. right?

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [March 8, 2024, 2:11pm UTC](https://community.konduit.ai/t/does-normalizerstandardize-transform-dataset-support-multi-threads/3075/4 "2024-03-08T14:11:01Z")

</div>

Yes. Sorry, please @ me if you don’t hear from me. Thanks!