# Decision Transformer **URL:** \ **Category:** Uncategorized\ **Created:** [April 11, 2024, 11:19am UTC](https://community.konduit.ai/t/decision-transformer/3119 "2024-04-11T11:19:24Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![kgoderis](https://avatars.discourse-cdn.com/v4/letter/k/278dde/32.png) [@kgoderis](https://community.konduit.ai/u/kgoderis)\ **Post date:** [April 11, 2024, 11:19am UTC](https://community.konduit.ai/t/decision-transformer/3119/1 "2024-04-11T11:19:24Z")
I was wondering if anyone has already tried to build a decision transformer using DL4J? And, or, tried to extend GPT-2, for example, from a Hugging Face model ---
**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\ **Post date:** [April 15, 2024, 6:04am UTC](https://community.konduit.ai/t/decision-transformer/3119/2 "2024-04-15T06:04:04Z")
@kgoderis we import and finetune BERT. I am finishing out the next release which will have a lot of improvements around import and the like. You’d have to build my development branch from source right now though. I need to separate that branch out in to a series of major PRS once I’m done with testing. ---
**Author:** ![AllenWGX](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/allenwgx/32/35_2.png) [@AllenWGX](https://community.konduit.ai/u/AllenWGX)\ **Post date:** [April 17, 2024, 2:31am UTC](https://community.konduit.ai/t/decision-transformer/3119/3 "2024-04-17T02:31:41Z")
Will transformer be available in next realease ? I mean it in the dl4j/samediff way, rather then importing tf/pytorch file. ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [April 17, 2024, 9:42am UTC](https://community.konduit.ai/t/decision-transformer/3119/4 "2024-04-17T09:42:21Z")
If it can be imported, it can also be built from scratch, but it may be a bit cumbersome to do so. You can use the model’s `summary()` function to see how it is structured and reimplement it on your own if you want to. ---
**Author:** ![kgoderis](https://avatars.discourse-cdn.com/v4/letter/k/278dde/32.png) [@kgoderis](https://community.konduit.ai/u/kgoderis)\ **Post date:** [April 17, 2024, 10:19am UTC](https://community.konduit.ai/t/decision-transformer/3119/5 "2024-04-17T10:19:36Z")
what is the name of your dev branch? there are many ab\_\* branches in existence ---
**Author:** ![Arasaka](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/arasaka/32/1190_2.png) [@Arasaka](https://community.konduit.ai/u/Arasaka)\ **Post date:** [April 17, 2024, 11:06am UTC](https://community.konduit.ai/t/decision-transformer/3119/6 "2024-04-17T11:06:44Z")
It seems like @agibsonccc he mentioned that he wants to pay attention to transformers. ---
**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\ **Post date:** [April 19, 2024, 11:20pm UTC](https://community.konduit.ai/t/decision-transformer/3119/7 "2024-04-19T23:20:09Z")
Yeah I need to look at the low level ops there first. I have mostly been focusing on a dev branch fixing some minor errors we have been having. We technically have all the transformer ops but I want to add some optimized routines and better pretrained models in samediff. ---
**Author:** ![Arasaka](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/arasaka/32/1190_2.png) [@Arasaka](https://community.konduit.ai/u/Arasaka)\ **Post date:** [April 21, 2024, 11:04am UTC](https://community.konduit.ai/t/decision-transformer/3119/8 "2024-04-21T11:04:54Z")
It will be good if it appears in a future (or slightly later) release.