Signed in as:
filler@godaddy.com
Artificial Intelligence in Microchips, Computers, Clouds
Many of us are in awe of the advances of AI and machine-learning algorithms. Recent foundation models can write, plan, create photorealistic images, and interpret large documents as well or above human level. In some cases, the most advanced language models can pass lawyers exams, physician medical exams, not to mention acquiring knowledge at a level that few fellow humans can master.
But what powers all these AI advancements? What technological marvels fuel that AI revolution every time you submit a query to ChatGPT? At the core, there are AI processors, bundld together into a single computer, or servers. Many servers work together in a data-center to process users request "in the cloud", meaning in a remote computer somewhere, not on the users own devices.
AI Microchips
At the core of the AI revolution are microchips capable of enormous computational feats. When you see the response from ChatGPT streaming on your cell-phone, you may not be aware of the amount of calculations performed by AI processors. For every part of words you recieve (called "tokens"), the AI processor multiplies from 150 to 1,000 or more BILLIONS of multiplications and additions. And it all has to happen fast, within 1-10 milli-seconds, as we are accustomed to searching on the internet and getting. near-instant responses.
It all happens seamlessly within powerful AI microprocessors designed to crunch numbers at blazing speeds - the most recent can perform close to 4,000,000 Billion operations per second. But it is not just number-crunching that is important, memory is equally if not even more important. A computer processor needs memory to gather data for its operations. And to perform a large number of operations it needs to read a large number of data from memory and fast. This memory specification, called MEMORY BANDWIDTH is today the most important bottleneck in AI processors. If the memory cannot provide data fast enough, the AI processor is STARVED, and is just sitting idle waiting. This means that it cannot use all of its computational power. Since memory and processor have been traditionally designed in two different fabrication processes and are sitting on two sepatae microchips, there is a physical distance that makes memory communication slower. Pascaline Systems is working on ways to bring the processors and memory closer together, so we can surpass the memory limitations of today's AI microchips.
AI Computers
Once you have a processor and some memory around it, you have part of a single computer. Multiple AI microchips are usually wired together into a large computer or server, usually containing 4 or 8 AI processors. These processors each have 80-196 GB of memory today, and they need to be wired together with fast communication busses to be able to share information.
Outside the processing core, communication to the outside world is provided with standard ethernet cards, the backbone of all internet today. But some AI computers need more bandwidth to communicate with other nearby AI computers. In this case more advanced form of computer communications are needed. Pascaline Systems is working to co-locate AI processors and their memory closer together, so that the communication bandwidth is optimized and increased. This makes AI computers faster and more capable.
AI Data Centers
Many AI computers are needed by the multitude of daily users, These computers are housed in a data center.
Connecting each computer to the outside world requires special networking equipment and advanced new network cards. A large number of networking switches is used to route internet traffic and users requests to AI computers efficiently. Infiniband and Gigabit Ethernet are typical setups used today, together with bundles of fast optical fibers to connect computers in different areas and to the outside world.
An AI data center has large power needs to run all its computers, and to keep them at their ideal operating temperature. Computer running at high speed can generate the same amount of heat of a hairdryer or small personal heater at low setting. Hosting thousands of them requires a complex HVAC system that can use 40% of the entire data center operational cost.
Data centers also require security and administration to maintain safe operation during emergencies or other rare events.
Pascaline Systems mission is to create green data centers that lower the footprint of AI operations. This can only be done at scale and with continuous assessment of technological advances, combining the most efficient equipment, and optimizing operations across a large number of AI computers.
