Rudina’s AI Atlas #4:
TinyML
Rudina Seseri
While we have heard a lot about large language models, like GPT-4, this week, there is an entire practice of machine learning dedicated to smaller, highly efficient machine learning models.
Large language models require immense computational power and are accessed by users via an API. However, what if we could do the entire ML computation on a local device like an IoT controller or camera? Enter TinyML, a field that explores the types of AI models that can be run on small, low-powered devices.
🗺️ What is TinyML?
TinyML is a field of study in machine learning that is focused on exploring the types of models one can run on small, low-powered devices with limited hardware capabilities.
It is a form of edge computing, the deployment of computing and storage resources at the location where data is produced rather than connecting to a centralized server/cloud for those purposes.
🤔 Why TinyML Matters and Its Shortcomings
The primary advantages of TinyML include:
Real-time Performance: Data is not sent to a server for computations, which reduces the latency of the output.
Power Efficiency: TinyML models run on edge devices that consume an order of magnitude less power than a consumer CPU.
No Bandwidth Requirements: An internet connection is not required because the computation happens at the point data is collected.
Privacy: No transfer of information to servers increases data privacy, unlocking use cases where machine learning is applied to personal or private data.
The primary shortcomings of TinyML revolve around the sacrifices engineers must make to develop machine learning models that can run on intentionally low-powered hardware. These include:
Limited Processing Power: ML engineers must make sacrifices in terms of the complexity of machine learning models. These limitations can be challenging for applications that require real-time processing, such as video or audio analysis.
Limited Memory: The small devices that typically run TinyML models have limited memory, making it difficult to process large amounts of data at once. This can be especially challenging for applications that require large datasets, such as image or speech recognition.
Limited Data: Edge computing is often leveraged in cases where the data is sensitive or difficult to collect, meaning the model will have limited data to be trained on. One way this is address sparse data is to leverage transfer learning.
Limited Developer Tools: TinyML is still a relatively nascent field in machine learning with limited developer tools, the most common of which are TensorFlow Lite and TensorFlow Lite Micro.
🛠 Uses of TinyML
TinyML is most valuable in use cases with limited internet connectivity, sparse or private data, and high latency demands. Examples of use cases that meet these characteristics include:
🏭 Industrial IoT: Industrial software deployments are often characterized by low data quality and low-bandwidth infrastructure. Thus, it is often favorable to perform the computations on the device rather than pumping data to the cloud.
⌚️ Wearables: These devices collect significant biometric data and among their value propositions is to provide real-time analytics. These real-time analytics are made possible by TinyML models running on the devices themselves.
🏠 Smart homes: The wake words in smart home devices such as “hey Siri” are examples of TinyML that have been used for years. These devices are using machine learning, on the device itself, to detect your voice and identify the prompt.
🚜 Environmental/agricultural sensors: By the nature of the industry, devices are often deployed to areas with limited connectivity and a lack of a centralized power source such as large fields.
⚙️ Data Pre-Processing: Developments in internet-of-things (IoT), edge computing, and intelligent edge devices have substantially increased our collection of data, but not all data at the edge is useful. Using TinyML, the data is analyzed locally to separate the signal from the noise.
As the AI model sizes shrink and the hardware capability improves, more and more tasks will move to the edge and new use cases for AI will be unlocked.
Learn more about the ecosystem and community around TinyML at tinyml.org