Self-Operating Computer

Use case:
An open-source framework to enable multimodal models to operate a computer. Using the same inputs and outputs as a human operator, the model views the screen and decides on a series of mouse and keyboard actions to reach an objective.

Github link: https://github.com/OthersideAI/self-operating-computer/commits/main/