# Transfomer neural network (TNN)

**URL:** <https://community.konduit.ai/t/transfomer-neural-network-tnn/2868>\
**Category:** DL4J\
**Created:** [October 24, 2023, 6:23pm UTC](https://community.konduit.ai/t/transfomer-neural-network-tnn/2868 "2023-10-24T18:23:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gerrie-myburgh](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/gerrie-myburgh/32/1221_2.png) [@gerrie-myburgh](https://community.konduit.ai/u/gerrie-myburgh)\
**Post date:** [October 24, 2023, 6:23pm UTC](https://community.konduit.ai/t/transfomer-neural-network-tnn/2868/1 "2023-10-24T18:23:19Z")

</div>

Are there any samples somewhere of how to implement a transformer neural net (TNN) in DL4J

---

<div class="post-metadata">

**Author:** ![Ujjwal2805](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/ujjwal2805/32/1201_2.png) [@Ujjwal2805](https://community.konduit.ai/u/Ujjwal2805)\
**Post date:** [October 24, 2023, 6:46pm UTC](https://community.konduit.ai/t/transfomer-neural-network-tnn/2868/2 "2023-10-24T18:46:18Z")

</div>

@gerrie-myburgh PFA

[partarstu/transformers-in-java: Experimental project for AI and NLP based on Transformer Architecture (github.com)](https://github.com/partarstu/transformers-in-java)

@agibsonccc@treo please share any more resources for the same,

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [October 28, 2023, 2:09am UTC](https://community.konduit.ai/t/transfomer-neural-network-tnn/2868/3 "2023-10-28T02:09:36Z")

</div>

@Ujjwal2805 I’ll publish some better examples after the release. Unfortunately still in the middle of a clean up of the code base yet.

---

<div class="post-metadata">

**Author:** ![AllenWGX](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/allenwgx/32/35_2.png) [@AllenWGX](https://community.konduit.ai/u/AllenWGX)\
**Post date:** [October 31, 2023, 1:32am UTC](https://community.konduit.ai/t/transfomer-neural-network-tnn/2868/4 "2023-10-31T01:32:08Z")

</div>

look forward to it, transformer is really popular right now.

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [November 5, 2023, 9:48am UTC](https://community.konduit.ai/t/transfomer-neural-network-tnn/2868/5 "2023-11-05T09:48:31Z")

</div>

Yeah of course! Unfortunately cuda technical debt has been the main concern lately.  
I’ve also been trying to make the test suite more robust. I ran in to many limitations when running the tests. They couldn’t reliably run even with a 4090 due to the number of kernel launches. I want to make sure M3 won’t have any surprises. I want to make sure that’s under control then a lot of focus will go in to transfrormers.