Modifying SD variables

SameDiff
1

I would like to understand if and how SD variables once created can be modified.

I have a couple of specific questions.

if I define an SD variable like this:

SDVariable x = new SDVariable();

is

x.setArray(y);

(y is an INDArray)

a valid statement, and will it set/reset the content of x?

When creating an SD variable should/can it be done without defining it, i.e.

SDVariable x = new SDVariable();

and then later in the code defining x, e.g.

x = x.setArray(Nd4j.ones(features));

does the above sequence, creation, followed by definition work?

Last question, for now. As I understand it from the documentation, if x is an SD variable, x.getArr() extracts the underlying content of the SD variable. Is this right?

Thanks

2

@adonnini Don’t create them manually. Use the samediff apis. The issue here is samediff itself tracks state internally and if it’s not aware of those changes then it doesn’t matter what you call on SDVariable itself. Use sd.associate if you need to update something.

3

Forgive me Adam. I am slow and ignorant. In your answers you tend to give responses not to my questions but solutions to my problems. I appreciate that. However, it does not help me understand why what I try to do is wrong. I know you are super busy and responding with (perceived) solutions is more time efficient.

Are you saying that the code I wrote doesn’t work or that it is clumsy? What is the purpose of setArray and getArr? Why would Saemdiff’'s state machine not be aware of a change made to a samediff variable if that variable was created correctly?

Again, forgive my ignorance, I do not understand why it is so complicated to make changes to SD variables once they are defined.

I looked for documentation on sd.associate and could not find any. Probably, I looked in all the wrong places. Could you please point me to a location where I could find it? I found mention of associateArrayWithVariable is this what you meant by sd.associate? are there other “types” of associate?

Does associateArrayWithVariable replace an existing array with its argument? I am not sure about the second argument in the call. What about the opposite operation, extract an array from an SD variable? Is getArr() the right call? If not which call should I use?

Thank you

4

@adonnini No need to worry. I was operating under the assumption you’ve experimented with these apis already. I remember us briefly discussing this on the forums once before which is why I just mentioned what I did.

Regarding the rest of your frustration with SDVariables…it’s because your’e fighting the framework.
A samediff instance provides all the necessary apis you need. If it’s not there then it’s probably for a reason.

I already mentioned why it’s like that. It’s because we have internal state. SDVariables need to be registered with the samediff instance, otherwise they aren’t useful for anything.

If you need to create a variable use sd.var(…). We handle creating the variable.

set andgetArr(…) are mainly internal apis. Those should probably be better encapsulated. Unfortunately, the package definitions prevent that.

I’ve asked you multiple times not to invent new solutions to problems. If you don’t see an example for it, there’s probably a reason and you are welcome to ask how to do something.

Now please accept that samediff has internal state, it works like that for a reason. That reason being, arrays get updated, variables can have multiple values. The reason for that is because each samediff .output creates a new “session” internally. That session does a feed forward for each op in the graph at the time of definition. This is to prevent values from being overridden.

Samediff being declarative means you can randomly call .output(…) and then if you want add another variable to the graph. That means that the graph for execution is recreated each time.

If you’re randomly changing things then you’re breaking the expected behavior of the framework.
Variables for the most part are meant to be immutable.

If you absolutely need to (despite having placeholders as well as a fit method which are meant to provide this behavior in the first place) then yes you use associateArrayWithVariable.

Please try to understand here that when I assume you know something I just try to point you in the right direction. If you need more information, ask. That’s what you did. Thus, I’m replying.

Beyond that, please accept this advice from one programmer to another. If you have to fight a library to get something done it’s probably because it’s not the intention of the library to be used like that. There might be reasons for that that are obvious to the end user.

With that out there, whether its’ correct or not is a separate discussion I’m more than willing to have.

5

I realize that I am still “fighting” the SD framework, i.e. not quite getting it, so to speak.

Two examples of how I use samediff where I could not find an alternative to using getArr and setArray:

I need to “unsqueeze” mask, an SD variable. This is what I do:

            INDArray maskArray = mask.getArr();
            NDBase base = new NDBase();
            base.expandDims(maskArray, 1);
            mask.setArray(maskArray);

I need to apply mask, an SD variable, to scores, another SD variable

        if (mask != null) {
            INDArray scoresArray = scores.getArr();
            applyMask(scoresArray, mask.getArr());   //this is a method I found in org.nd4j.linalg.lossfunctions
            scores.setArray(scoresArray);
        }

I looked in the documentation including tests and examples. I could not find ways to apply masks or expandDims for SD variables.

I also do this:

I need to do the linear projections in batch from dModel => h x dK. linear applies a Linear operation

        SDVariable[] qkv = new SDVariable[]{query, key, value};
        for (int i = 0; i < 3; i++) {
            qkv[i] = linear(qkv[i], weights, bias).reshape(nbatches, -1, h, dModel / h).permute(0, 2, 1);
        }

Is it OK to first create qkv and then define its content?

Thanks for your help.

6

@adonnini any reason why a mask wouldn’t be a placeholder like the rest? Our multidataset and datasets support adding a mask in there.

Your expandDims should also be apart of the graph as well.

7

Thanks.

I know that datasets support adding a mask. In fact, I use that capability when preparing the input data for my app.

This, however, is different. Adding a mask is an option given while constructing layers. It would not work to add a mask a priori to a dataset.

What is your answer to the last question?

8

@adonnini that should all be placeholders then. Anything dynamic should be a placeholder variable or a constant. Otherwise use associateVariable. Note that that will break certain things if you’re not careful.

Please focus on trying to re design your idea to be a placeholder. I don’t see any reason your app wouldn’t know how to do that.

Note that if your’e doing training, on the TrainingConfig we have variable to index maps. We specify those so we can automatically figure out what placeholders to map your data set features, labels, and associated masks to. That should be enough.

9

Thanks!

Sorry to bug you again could you answer my last question? Or did you?

By the way,

Did you mean associateArrayWithVariable?

Thanks

10

Yes. Just try my suggestions and let me know what you find.