How to pass variables from python step to the following non-python step in Konduit pipeline?

1

Is it possible to pass variables from a python step to the following step that is not a python step? In the configuration yaml of pipeline of Konduit we configure some python step before a non-python step, e.g. after some processing the python step has produced an image as a variable, and then we wish to show this image so we want use the show image step. But the ShowImageStep is written in Java obviously not python, is it possible to do so?

2

Yes it is possible to pass things from python to Java. If the python output is a numpy ndarray it’ll be converted to an INDArray on the Java step. But I’m not 100% sure what the ShowImage step needs for input…

3

It seems show image step will try to get something as type of ai.konduit.serving.pipeline.api.data.Image not the type of ai.konduit.serving.pipeline.api.data.NDArray .

4

@TempKonduitUser1 That’s a very good question. Unfortunately, there’s no option for sending image data from python and convert it to ai.konduit.serving.pipeline.api.data.Image in Java. What are you trying to achieve exactly? If you’re creating bounding boxes to images then you can just take the bounding boxes from the python runner and then merge the output from that with a show image step. There’s an example for that in the tests over here: https://github.com/KonduitAI/konduit-serving/blob/5e6b1002e075dbb9234a11254b3408b03ce35da2/konduit-serving-models/konduit-serving-tensorflow/src/test/java/ai/konduit/serving/models/tensorflow/TestTensorFlowStep.java#L373-L470

Let me know if this is something that’s going to work for you.

5

@ShamsUlAzeem1 Thank you! Do you mean by using PythonStep, I can send something as bounding box list to the following step with the image from the previous step? OK. But if I wish to draw something customized other than bounding boxes, what should I do in the pipeline afterward?

6

@TempKonduitUser1 I see, well in that case if showing the image output is your main goal then maybe you can do that directly from the python script itself… Something like

from PIL import Image

image = Image.open('passport.jpg')
image.show()

Also, I’m working on sending images from a python step to java. Also, working on a new pipeline step that will convert data to images from the previous step.

7

@TempKonduitUser1

I’ve just checked and there’s a way to convert data to images from Python to Java. The way to do it is to encode the image into bytes and describe another variable with a pattern of len_ + <variable_name>. So, for example, if your encoded image is out_image, then your byte length variable should be len_out_image. Then you’ll get an instance of ai.konduit.serving.pipeline.api.data.Image in Java from the PythonStep

The configuration may look something like this:

---
host: "0.0.0.0"
port: 0
pipeline:
  steps:
    - '@type': "PYTHON"
      python_config:
        append_type: "BEFORE"
        extra_inputs: {}
        import_code_path: "init_script.py"
        python_code_path: "run_script.py"
        io_inputs:
          image:
            python_type: "str"
            secondary_type: "NONE"
            type: "IMAGE"
        io_outputs:
          out_image:
            python_type: "bytes"
            secondary_type: "NONE"
            type: "IMAGE"
          len_out_image:
            python_type: "int"
            secondary_type: "NONE"
            type: "INT64"
        job_suffix: "konduit_job"
        python_config_type: "CONDA"
        python_path: "1"
        environment_name: "base"
        python_path_resolution: "STATIC"
        python_inputs: {}
        python_outputs: {}
        return_all_inputs: false
        setup_and_run: false
    - '@type': "SHOW_IMAGE"
      imageName: "out_image"
      displayName: "Output Image"
      width: 1280
      height: 720
      allowMultiple: false

and run_script.py will have the output configured something like:

import cv2
from sys import getsizeof

success, buffer = cv2.imencode(".jpg", img) # Where img could be a numpy array containing image data

out_image = buffer.tobytes()
len_out_image = getsizeof(out_image)
8

@ShamsUlAzeem1 So, by your instruction, we think we can solve the problem of processing video by setting up steps as: VideoFrameCaptureStep → PythonStep → ShowImageStep . Do you think we are right this time?

9

Yup, there’s no need to create a complex graph. A sequence of steps would work fine as well.

10

@ShamsUlAzeem1 Are you sure it is correct? I have read the class ai.konduit.serving.data.image.step.show.ShowImageStep, and haven’t found the member named “lenOutImage” .

11

So the instruction isn’t about the ShowImageStep, it’s for getting bytes data out of a PythonStep: https://github.com/KonduitAI/konduit-serving/blob/4cf2b70618718ccbf0430b663ab11c5b86fd0413/konduit-serving-python/src/main/java/ai/konduit/serving/python/util/KonduitPythonUtils.java#L491

You don’t have to worry about that. I’ve shared an example configuration and python script with you as well