@agibsonccc Hey Adam, I want to know if I can use DL4J , java to build a chatbot providing it questions and answers in csv file and when I ask a particular question it respond with the answer that I have inserted in answer column of the same CSV and if Possible then How to do the same can you guide , I m new to Machine learning and DL4J.
Someone who knows what they are doing can use DL4J for that.
However, as you are new to those things, I would highly suggest that you first learn about the things your question entails.
If you simply want to ask questions and look up answers based on the question, you are looking for something like a vector database + vectorization of the question.
In the simplest case you could use something like vecto.ai, which is a database that includes the vectorization part. Or if you want to do it yourself, you could select a pretrained model and do the vectorization yourself and then use any other vector database too.
If you want to have something that can summarize multiple lookup results into one answer, you will have some more work on your hands and the keyword you should look up for that is “retrieval augmented generation”.
I would go with what Paul said. Anyone thinking at the “chat bot” level shouldn’t be building their own models and should be looking at a product instead. The recommended route would be using large language models fine tuned on your own data. You need a vector database among other components to make it work correctly.
Is it possible to tokenize the data of csv convert to NdArray and then feed it to model.
@agibsonccc @treo
@Ujjwal2805 you can yes. You would have to write a custom tokenizer for that and then plug it in.