Running an AI model on Xbox 360: Amazing 👾

X user (formerly Twitter) Andrei David David has just installed and run an AI model based on Andrej Karpathy's llama2.c on a nearly 20-year-old Xbox 360 console. Despite its age, David used a bootloader to load the model onto the old console and run a 'Sleep Joe said' prompt. According to the post, EXO Lab's successful attempt to get Llama running on a PC running Windows 98 inspired David to conduct his own experiment. 🚀

Although the AI LLM used by the Xbox 360 is based on the same one used in EXO Lab's Pentium II PC, David mentioned that he had to optimize the code for the console's PowerPC architecture and its unique management characteristics. by memory (which were innovative at the time). The most significant difference is that PowerPC is a big-endian architecture, which stores the most significant value first. 🖥️

On the other hand, the Intel processor The Pentium II used in the EXO Lab implementation is little-endian, meaning it prioritizes storing the smallest value first. So, to ensure the model worked properly, he had to implement a byte exchange system and ensure that any data created and stored had a proper memory alignment of 128 bytes, as required by the Xbox 360 memory subsystem. ⚙️

Another challenge David faced was the use of unified memory of the console. While this is common today on modern consoles and APUs, at the time of the Xbox 360's launch it was new territory, forcing developers to games to optimize hardware usage console. Despite having 512MB of RAM, Andrei had to ensure that the 60MB model would fit into the space that the system would allocate to the CPU or GPU, since they shared the same RAM pool. 🕹️

However, with careful memory management and tweaks to how the PowerPC works, he managed to make the old Xbox 360 console will run the LLM. This is another step towards EXO Lab's goal of providing access to AI to people common, not limited to mega corporations that spend billions of dollars on state-of-the-art equipment and consume megawatts of energy. 🌟

Running an artificial intelligence (AI) model based on Llama 2 on a device that is several decades old is certainly an impressive feat that demonstrates the evolution and adaptability of the technologyHowever, in an interesting online exchange, one user made a comment that caught some attention: he argued that having 512MB of unified RAM should be more than enough to run smaller, more efficient models, such as Hugging Face's SmolLM or the Qwen2.5 model, which operates at 4 bits and is 0.5B in size.

This comment not only highlights the potential of lighter AI models, but also raises intriguing questions about the ability of older devices to handle technologies advanced. Andrei David, an AI enthusiast, enthusiastically responded with a 'Challenge accepted! :)', suggesting that he is willing to explore and experiment with these technologies on platforms that, at first glance, might seem unsuitable.

The idea of seeing powerful artificial intelligence models running on an Xbox 360, a console from video games Launched in 2005, it is exciting and promises to open up new possibilities. This kind of experimentation could not only revitalize old devices, but could also inspire the tech community to find innovative ways to optimize AI models to run on resource-constrained hardware.

We look forward with great interest to the progress and results of these tests, as they could pave the way for a future where artificial intelligence accessible and functional on a variety of devices, regardless of their age or technical limitations. The combination of the developers' creativity and the versatility of the models of AI could lead to fascinating discoveries and unexpected applications. 👀