# Generic windowing functions for 2D images?

**URL:** <https://community.konduit.ai/t/generic-windowing-functions-for-2d-images/689>\
**Category:** DL4J\
**Created:** [July 6, 2020, 1:24am UTC](https://community.konduit.ai/t/generic-windowing-functions-for-2d-images/689 "2020-07-06T01:24:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aewhite](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/aewhite/32/263_2.png) [@aewhite](https://community.konduit.ai/u/aewhite)\
**Post date:** [July 6, 2020, 1:24am UTC](https://community.konduit.ai/t/generic-windowing-functions-for-2d-images/689/1 "2020-07-06T01:24:27Z")

</div>

My question is virtually identical to [this SO question](https://stackoverflow.com/questions/62431258/how-to-create-a-sliding-2d-autoencoder-in-keras) but for DL4j. In short, can I define a sliding autoencoder/operator in DL4J? This could apply to the input layer or even the result of a Conv2d layer.

The proposed solution for Tensorflow was to use the extract\_image\_patches function. I see that is available for DL4J as well, so that is my fallback.

Is that the idiomatic way to simulate a windowing operation? Seems like it would have non-trivial performance implications; though I haven’t tested it to verify.

I am unsure if this or stackoverflow is the correct place to ask this. Sorry if this is the wrong place to post general API questions.

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [July 8, 2020, 8:08am UTC](https://community.konduit.ai/t/generic-windowing-functions-for-2d-images/689/2 "2020-07-08T08:08:04Z")

</div>

Usually this is the better place to ask questions to anything related to DL4J, as it allows for a better conversation than Stackoverflow.

Do I understand your question correctly, that you in principle want to apply a window over a picture, and use each possible position over the entire picture for unsupervised training?

There are a few ways I can think of doing that, but to suggest something useful, I need to know a little bit more:

- Are you going to train just on top of a single picture?
- If not, what kind of mini batch composition are you trying to get?
- What are you trying to achieve with that overall? What is the final goal there?

---

<div class="post-metadata">

**Author:** ![aewhite](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/aewhite/32/263_2.png) [@aewhite](https://community.konduit.ai/u/aewhite)\
**Post date:** [July 8, 2020, 3:13pm UTC](https://community.konduit.ai/t/generic-windowing-functions-for-2d-images/689/3 "2020-07-08T15:13:35Z")

</div>

> Do I understand your question correctly, that you in principle want to apply a window over a picture, and use each possible position over the entire picture for unsupervised training?

Exactly. But I do want to apply this method over multiple layers. In principle, the input could be an image with N channels or the output of the previous layer with N features. The key is preserving locality.

I think you understand core of what I want to do. This is a purely an academic/learning experiment where I am trying to see if it’s possible to build a hierarchy of “grouped” features in order to discover structure. Something akin to capsule networks, but I want to use a number of alternative methods for defining how those capsules are expressed.

As for training, I expect there is some flexibility here but ultimately, each image would map to a tree of sparse features where each feature has a small “pose matrix” (if I were to use the capsule terminology). As for how many images to process in a batch; I don’t think I have any restrictions there other than performance.