3.3.2. Measure The Performance of the IoT Devices¶
| ID | Priority | Best Practice |
|---|---|---|
| BP 3.3.2.1 | Highly Recommended | Capture device diagnostic data into the IoT platform |
| BP 3.3.2.2 | Recommended | Measure, evaluate, and optimize firmware updates |
| BP 3.3.2.3 | Required | Limit the number of messages that devices receive and filter out |
Architecture Notes - BP 3.3.2.1 - Capture device diagnostic data into the IoT platform¶
As the number of devices increases watch out for performance bottlenecks when all the devices connect to the cloud-side. These devices will be generating a large aggregate amount of data, and obtaining device diagnostics is critical for ensuring the understanding of the area of improvement. Using different types of device diagnostics, the immediate health of a device and likely those in proximity to that device can be obtained.
Recommendation 3.3.2.1.1 - Deploy an agent to the device to start capturing the relevant diagnostic data
- For microprocessor-based applications, consider deploying the AWS Systems Manager Agent (SSM Agent) so that you can continuously monitor your device’s performance metrics.
- There are sample agents provided to use on the device-side (device or gateway). If device-side diagnostic metrics cannot be obtained, then it is possible to obtain limited cloud-side metrics. Below are some sample metrics:
- TCP Connections
- TCP_connections
- Connections
- Local_interface
- Listening TCP/UDP Ports
- Listening_TCP/UDP_ports
- Interface
- Network Statistics
- Bytes_in/out
- Packets_in/out
- Network_statistics
- TCP Connections
- Sending Metrics from Devices
Architecture Notes - BP 3.3.2.2 - Measure, evaluate, and optimize firmware updates¶
Firmware updates are critical to ensure that the IoT devices remain performant over time, but might sometime not have the expected impact. As you deploy firmware updates to your devices, monitoring your KPIs will ensure that the updates do not have any unintended impacts to the performance of your hardware devices or to your IoT applications.
Recommendation 3.3.2.2.1 - Implement canary deployment for device firmware
- Deploy new firmware to a limited set of devices and monitor the impact on performance before rolling out to the entire fleet. Abort deployment if degradation is detected.
- Example:
- Use AWS IoT Jobs to manage OTA updates and configure it to deploy to a limited set of devices.
- After the update, evaluate end-to-end performance of the system using your previously identified KPIs.
- If performance characteristics of firmware release appear to have been impacted, use AWS IoT Secure Tunneling, a feature of AWS IoT Device Management, to remotely troubleshoot the device.
- Release firmware updates to remediate identified issues.
- For more:
- The Internet of Things on AWS – Official Blog – Using Continuous Jobs with AWS IoT Device Management
- The Internet of Things on AWS – Official Blog – Using Device Jobs for Over-the-Air Updates
- The Internet of Things on AWS – Official Blog – Introducing Secure Tunneling for AWS IoT Device Management, a new secure way to troubleshoot IoT devices
Architecture Notes - BP 3.3.2.3 - Limit the number of messages that devices receive and filter out¶
Devices might have constrained resources, and filtering messages at the edge might subject them to unnecessary load. This could be counterproductive from a power and memory consumption perspective. Sending only messages that the device make use of, reduces the load on the resources and ensures better performances.
Recommendation 3.3.2.3.1 - Structure the topics using the scope/verb approach
- In this way, you can subscribe to all messages for a given scope (for example, a device) or refine the subscription on a given scope and verb.
- Designing MQTT Topics for AWS IoT Core