Interactive mode with Llama class

[abetlen]

• Similar to interactive mode in llama.cpp.
• Changes should not effect __call__ and the create_* method behaviour.
• Should support max_tokens / infinite generation, eos / ignore_eos, and a reverse prompt.
• Should support streaming

[MillionthOdin16]

I see. So this will be our implementation of the interactive type mode? I think my confusion was due to the fact that most if not all of my use of llama.cpp is within interactive mode and chat interactions.

[abetlen]

Yes that's the intention, I had an example I was working on from our convo yesterday, just have a couple bugs but it's actually fairly short to implement with the Llama class.

[ZeitgeberH]

I think this is a very important feature for the sever. Otherwise, we are forced to use llama.cpp's interactive mode for chat.

[abetlen]

I'm going to close this because interactive mode that follows the llama.cpp main program has been implemented in the examples https://github.com/abetlen/llama-cpp-python/blob/main/examples/low_level_api/low_level_api_chat_cpp.py which includes a LlamaInteract class