Advanced Image Processing
chevron down
back search icon cancel search

Advanced Image Processing

This is a guest blog post written by Matthew Clark, Fitbit Community MVP. As such, the views and opinions expressed in this guest contribution reflect those of the author and do not necessarily reflect the views and opinions of Fitbit.

For years I have enjoyed programming smartwatch apps, starting with the Pebble Watch and then eventually developing for the Samsung Galaxy Gear, Tic Watch, Android Wear, Apple Watch, and eventually Fitbit. One of my more popular apps is the Map application for Fitbit. It supports several mapping services (Google Maps, HERE Technologies, Microsoft Bing, MapQuest) and users can track their current location on their wrist.

The Fitbit Versa 2 has a feature called Always-On Display (AOD), which allows clock faces and apps to activate up to 20% of the pixels when the Fitbit device is not in an active display state. It is terrific - we can now easily see the time and other information without quick wrist flicks or pressing a button. Fitbit has generously allowed a few developers to program for this feature, and some of the solutions are quite clever.

My first attempt to extend the Map app just had the time displayed when AOD was active. I submitted the app for approval by Fitbit, but it was rejected because just displaying the time wasn't unique. Clock faces would have been approved with just the time displayed when in AOD mode, but for apps the bar is higher.
I thought that if I could create an image that was map-like for the AOD feature then it would be approved. I tried taking a map image and applying a Sobel Filter to highlight the edges of an image. Not bad, but some map images have many, many edges so the image did not necessarily have fewer pixels to display.

This processed image had to be sent to the Fitbit watch. The easiest method was to create and transmit a JPEG image. Sounds simple, but with one major flaw: JPEG is a lossy image compression scheme, so when the image is decompressed then many pixel values have very low intensity values. Therefore this solution will not work for Fitbit's AOD feature, since at least 80% of the pixels must have a zero intensity value.

Perhaps I could just send the image without compression? This is not complicated, but sending large images using Bluetooth takes too long and uses too much battery power. A Fitbit Versa 2 watch has 300x300 pixels, so sending red-green-blue (RGB) pixel data in raw format would require 270 KBytes. Ouch. I needed to find a smart image format with lossless compression.

I recalled learning about an image format called TXI that Fitbit uses internally for images within its smartwatch platform. However the TXI format is not publicly available. Thanks to some sleuthing, I found some relevant code used by Fitbit when their apps are compiled. A key file was within the folder node_modules/@fitbit/image-codec-txi/mod/encoder.js, and I later learned that the source code was available. This file had important constants, methods, and referenced other libraries with relevant functions.

While exploring the TXI libraries I learned that I could do a few things to reduce the image size. TXI supports grayscale images, so I could switch from RGB to monochrome, reducing the data requirements by about 60%. Also, TXI supports a lossless compression technique called run-length encoding (RLE). To increase the likelihood of having repeated pixel values for higher compression, I quantized the remaining pixel limiting the intensities to just eight values.

This solution worked! I could now process a map image and turn it into a crude representation of a map that uses just 10% of the pixels. A raw image may be 270KBytes in size, but my reduced edge map was only about 15 KBytes. For a final touch I decided to color the grayscale edges blue, and overlay a red location marker for one's current GPS location. When AOD is activated, the map displays a screen like the one displayed.

Here is a summary of the steps implemented for this solution:

  • retrieve 300x300 map image from mapping service
  • decompress image into two-dimensional array of pixels
  • convert from RGB to grayscale
  • Perform Sobel edge detection on grayscale image
  • calculate histogram of new edge-pixels image
  • compute 92.5% intensity value based on histogram
  • set pixel values to zero if they are below the threshold
  • quantize remaining pixel values to a limited set of 8 values
  • create TXI image using run-length encoding compression
  • transmit TXI image from smartphone to Fitbit device
  • display this new image when always-on display is activated

Fitbit has approved the updated Map app, and it is available for free in the Fitbit app gallery.

For variety I also added a feature that displays a dim image of the map image. The custom settings screen allows switching between the edge map, a faint map, or no map when AOD is activated.

I have applied this technique to other apps and clock faces I have developed, including these:

Slideshow
Stores and displays up to 20 of your images. The AOD feature shows edges or a faint view of your photo.
Web Images
Store unlimited URLs of web images and webcams, and display them on your watchface. This app is pending and will be published soon.

In summary, the Always-On Display feature is terrific and I'm sure other developers will also take advantage and create different clever solutions.

About the author

Matthew Clark is a professional software developer, specializing in web, mobile, and wearable applications. https://GORGES.app

Until Next Time

Follow @fitbitdev on Twitter, and join our Fitbit Community Forum to keep in touch, 24/7. Curious to see the amazing work Fitbit Developers have done so far? Keep tabs on #Made4Fitbit on Twitter.