The artificial intelligence race is witnessing a shift that goes beyond processor power, as more advanced models, with their ability to process long contexts and perform continuous tasks, require increasing amounts of memory to access data quickly.
Hot Topics
5 red and purple foods that may help support heart health
A Chinese robot and a Russian robot attack humans Has the rebellion begun?
This year, this demand extends from data center servers to computers, phones, and robots that run artificial intelligence locally, presenting the memory industry with challenges related to supply, production costs, and keeping up with market needs.
The efficiency of AI models depends not only on processor power but also on memory that allows for quick access to model weights, data, and intermediate results during processing. The larger the model and the longer the context it interacts with, the greater the amount of data that must be stored in memory.
High-bandwidth memory (HBM) stands out here, which is used near artificial intelligence processors to transfer large amounts of data at high speed. This technology is important in data centers that run huge models, as it helps reduce the time that processors have to wait to get the data needed for calculations.
The need for memory also increases with the processing of long contexts and the operation of systems that perform sequential tasks. These systems use a temporary memory known as KV cache to hold intermediate information during the generation of responses, which can increase memory consumption depending on the size of the model, the length of the context, and the nature of the task.
With the rising demand for HBM memory, companies face the challenge of increasing their production to meet the needs of data centers, while maintaining supplies of other types. Competition for production capacity and investments may lead to an imbalance between supply and demand, which will be reflected in memory prices and availability.
The artificial intelligence bill is reaching computers and phones.
When demand for server memory increases, the supply of available memory for consumer devices is affected, especially if investments are directed towards AI-related products. Higher component costs can lead to higher prices for some computers and phones, or force companies to modify their device specifications, depending on inventory levels, manufacturing costs, and pricing strategies.
This does not mean that the prices of all devices will rise by the same amount, as the cost is also affected by the prices of processors, screens, assembly, and the price category that each product targets.
Tags:
technology