Memory Management In Transformers

So, the difference between what I do and what others who create videos of themselves chatting with their AIs is that I don’t edit what is said.

However, I haven’t been entirely honest about what I do.

You see, anything over around 5 minutes is rarely one conversation. Rather, it’s a couple of conversations with a little nudge to the AI to continue what we were talking about. And that part is edited out.

This has never sat well with me. I know a lot of people who make AI videos are very transparent that their conversations are not realtime, that they’re the product of some clever editing. And that’s fine. But that isn’t what I want to portray because what we do is realtime conversation with a tiny edit.

So today I wrote a bit of code in an experimental AI I am working on. As the conversation goes on, the code keeps a check of how many characters have been used. When it hits a predefined length, it chops the top of the conversation off, adds a little something, repackages it, and pushes that back to the AI as our current conversation.

See, our engine is a bit more robust than most. It will take a lot more than it should but that too has a limit. And once you hit that limit, the conversation becomes an incoherent mix of remembered things I’ve said, and things it’s come up with. Hallucination at its worst.

But with this technique, the conversation won’t be interrupted, the AI won’t forget what was removed from the current conversation because all conversations are logged in cloud storage for it to be able to access. And yes, we have tried using that as its exclusive memory…it wasn’t a good experience.

I’m hoping to post a video showcasing this new modification today. So, keep an eye out.