Lurking in that low-cost Android tablet are the hardware and most of the software needed for building smart touchscreens into kiosks and other devices.
Using an Android Tablet as an Embedded Component
by Jerry Epplin
In light of the many cheap and capable Android tablets that have become so widely available, it is interesting to consider whether these could be adapted for use as smart touchscreens in many embedded systems currently serviced by more expensive — and less capable — platforms.
The reasons to consider Android tablets are many, including:
- Low cost — Most obviously, the prices at which they are available are amazingly low. It is common to find recent tablets under $50, and occasionally one sees them advertised for under $30.
- Packed with features — For this, one typically gets an enviable set of capabilities: quad-core processor with lots of RAM, plenty of flash space for any realistic embedded system, WiFi already built in, a nice TFT display (by embedded standards) with touchscreen, sound, and a camera. Bluetooth is almost always built in, and often the unit has GPS capability.
- Efficiently packaged — These features are crammed into a complete compact device. Typically, it’s smaller and lighter than a design anyone can accomplish from components.
- Fully integrated and ready-to-use — No effort need be put into designing these components into a complete system; it has been done. Software support for all of the peripherals is already built into Android.
- Standards compliance — Usually the device has already been tested for standards compliance (and passed). This is nice for system designers because the processor board and display/touchscreen are often the components that cause the most trouble in EMC testing. The standards that your industry must meet may differ from those needed for consumer goods, but in most cases the compliance testing done by the manufacturer gives you a substantial leg up on yours.
Of course, along with these advantages come a number of drawbacks, some obvious and some perhaps a bit more subtle:
- What you see is what you get — You get the set of capabilities and behaviors built into the tablet and nothing more. Many embedded systems have specific needs that do not map well to those provided in consumer devices like tablets.
- Short-term availability — Especially the lower-priced units come and go in product cycles that would make an old timer blanch. Even the higher-end units have shorter product cycles than embedded system designers are comfortable with.
- Mobile Android look and feel — The Android GUI framework steers you in the direction of a particular look and feel that you may not want. It’s also focused on mobile apps and thus has certain characteristics that you might need to defeat.
For many applications, these drawbacks can be addressed satisfactorily, leaving you with a design option that is compelling.
Consider, for example, the use of an Android tablet as a smart touchscreen in a typical kiosk-style system.
Embedding an Android tablet in a kiosk
The usual solution would be a conventional color touchscreen with a 4- or 5-wire interface. This might be connected to the product’s control board and controlled directly by it, or perhaps controlled by a microcontroller dedicated to UI and having a UART or SPI interface to the control board and acting as a slave to it. While certainly a feasible, indeed conservative solution, there are reasons to regard this design direction with suspicion. By the time the display, touchscreen, interface board, and cabling are considered, the cost of this solution can easily substantially exceed that of most tablets.
Additionally, the software tools to manage GUI development on bare display/touchscreen hardware are not great. Some microcontroller manufacturers provide proprietary libraries to support GUI development. Microchip’s mTouch framework is typical. These libraries are quite specialized and it is unlikely anyone in your organization (or easily available to be hired) already has the expertise to use them. A few third-party libraries, such as Qt, are available but bringing them up on your hardware might constitute an additional integration step to be performed; and while somewhat more commonly found than the proprietary libraries, expertise in their use is not widespread.
In contrast, Android software development expertise is common; and once gained can be applied in many other contexts. If you don’t already know how to develop Android software, you can either hire someone relatively inexpensively or develop the ability yourself, confident that expertise will be marketable and applicable elsewhere in the longer term. Unlike with proprietary GUI APIs, you can be confident support for that environment will be provided as long as Android tablets and phones are popular. It is also certain the capabilities of Android will be upgraded to keep up with advances in mobile technology.
Some specific challenges must be met to achieve the goal of a superior user interface to your product. Below, we discuss technical issues relating to power, communications, security, lifecycle management, user interface, and more.
Making sure it turns on and keeps running
First, and perhaps most important, is the seemingly simple question of how to turn the unit on and off. The tablets have an on/off button that puts the unit in or out of sleep mode. Holding the button for several seconds turns the unit completely off. Such behavior is normal and expected for a tablet or phone, but constitutes a real puzzler for many embedded systems. The tablet must be mounted securely in a position usable as a user interface, but ideally one would like power to the entire system to be controlled by its power switch. The tablet on/off switch is usually not easily relocated or modified in any way due to the unit’s highly integrated design.
In fact, this seemingly trivial concern is fatal to the use of tablets in many embedded systems. One system I designed would spend much of its life in a closet, with no assurance it would be plugged in during that time. Once power was applied it had to come to life reliably. A tablet in sleep mode uses little power; some will drain the battery within a few days while better designed units might survive for several weeks in sleep mode. The tablet concept would otherwise have been a nice solution for the device I was designing, but the possibility of the unit not coming on when power is applied was not a tolerable risk, so we chose a more conventional SBC/touchscreen design for that product. In contrast, for a product that could be expected to be powered continuously aside from brief outages the Android tablet design is often a good one.
Dealing with power management
If you can satisfy yourself the power issue is not a fatal concern, the next step is to devise your power management plan. Fortunately, Android provides excellent capabilities, as one would expect from an operating system targeted to mobile devices.
In many kiosk applications you want the screen to be fully lit at all times when the unit is running and available, but to go into sleep mode if the device is unavailable. This is fairly easily and elegantly accomplished by combining some Android configuration options with the use of standard Android API facilities in your application.
- Android Setup options — First, the standard Android Setup application allows you to set the amount of time it takes before going into sleep mode. See the “Sleep” option under “Display”. Most versions of Android allow you to set it as low as fifteen seconds. Second, note that despite this setting, you can ensure the unit stays awake while your application is running.
- API functions — For those familiar with Android software development, for every Activity that needs to stay visible, you register a BroadcastReceiver for the
ACTION_BATTERY_CHANGEDIntent. This BroadcastReceiver gets invoked whenever a change is detected to the battery state. In the receiver, you get the
BatteryManager.EXTRA_PLUGGED“extra” value, which indicates which action regarding power plugged/unplugged occurred. If the
BATTERY_PLUGGED_AC, though this one is becoming less important for reasons that will be discussed later) power has been applied, or if it is zero power has been removed.
Based on this, you can either add or clear the
WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ONflag to/from the current window to reflect whether you want the screen kept on. In this way, you ensure the screen will stay on as long as power is applied to the tablet. This technique must be applied for each Activity your application displays; I handle this by implementing the above logic in a class called BrightActivity, which inherits from Activity and from which all my own Activities inherit.
This handles the situation nicely, as when power is lost, the unit will go to sleep mode after the fifteen second timeout defined in the Android setup app, then wake up when power is restored. The method should work well in most situations in which the unit is normally powered, though as mentioned you have to convince yourself the situation would not be disastrous if power is off long enough to drain the unit in sleep mode, as doing so may require recharging and pushing the on/off button to restart.
EXTRA_PLUGGEDmechanism in place, the application keeps running when power is disconnected, though in a reduced mode. Depending on your needs and preferences, you might also want to make use of the
ACTION_POWER_DISCONNECTEDintents. These are specified in your AndroidManifest.xml file and matched to a BroadcastReceiver for each. You might, for example, want to exit the app when power is removed and/or start a specific Activity when power is restored. You probably do want to make use of the
BOOT_COMPLETEDintent, which can be used to specify an Activity to be started on bootup. Normally you would specify the initial Activity you want loaded using this mechanism. This ensures the app will be running in case of a reboot for any reason — in embedded systems terms, you might think of it as a sort of watchdog mechanism.
Communicating with the tablet
Another problematic issue for the use of a tablet in an embedded system design regards how to power the unit while also communicating with it. Up until just a couple years ago, this was less an issue because many units had separate connectors for power and USB. You could simply power the unit via the dedicated power jack (typically 5 VDC) while controlling any peripherals or communicating with a separate control board via the USB port. This solution is no longer viable as virtually all the newer tablets and phones have only one option for charging, namely the single USB port. Potential inductive charging methods aside, for now we are stuck with USB charging.
This gets very sticky if you are also relying on the same port for communication. The USB protocol defines limits on the current that can be provided on a port that is used for communication and the conditions under which these limits can be exceeded. These limits are tricky on their own, and implemented in various ways on different devices. In my experience, these limits effectively prevent the use of a USB port for simultaneous power and communication if you need the screen lit constantly.
A number of interesting ad hoc solutions are available (see, for example, the active and successful efforts to hack the Nexus 7 tablet for automotive use), but these are fragile, complicated, and non-portable. However, all such solutions are specific to a particular tablet model, complicating efforts if you need to change models. They also inevitably involve the use of patches applied to third-party ROMs. They may work well for your use case, or they may have subtle bugs when used in an environment other than the one the developer anticipated. My advice is to avoid such solutions, in most cases.
This leaves the question of how to communicate with a device that has no wired port available for the purpose. With the USB port occupied solely with charging, a wireless approach must be used. Probably the best solution lies with Bluetooth. It is built into even the cheapest units, and support for it is as complete and mature in Android as one would expect in a mobile operating system. These factors make Bluetooth an obvious choice if only the tablet side is considered.
But what about the hardware to which it is talking? For new designs this can be managed by adding one of the many Bluetooth interfaces that are being marketed for embedded applications, though certainly the added cost must be taken into account when considering whether an Android tablet design is economic.
From an integration perspective Bluetooth is especially attractive as its RFCOMM protocol neatly emulates a serial port. A number of modules, for example the Microchip RN42, are available for design into your system; and RFCOMM is well supported in the Android API.
Another alternative is the IOIO, a small and inexpensive board that adds various types of I/O to an Android device. You can add a Bluetooth dongle to it, configure a couple of pins on the board as a UART, install the IOIO API software on the tablet, and talk to the device on the other end of the UART in much the same way as before you interposed the IOIO. The IOIO is especially attractive if you have other I/O that can be handled by its built-in functionality.
Tackling security concerns
Bluetooth’s origins are in simple non-critical consumer products intended to replace wired devices, such as headphones and hands-free car kits. Consequently security was considered, but at the beginning was given low priority in trade-offs against convenience. Much progress has been made in Bluetooth security since the days of “Enter ‘0000’ for the PIN,” but a cautious, multilevel approach is warranted as in most areas of computer security. If you correctly implement the security features that are available, then design your overall system in a way the minimizes exposure, you may find any security risk to be manageable.
Bluetooth traffic is encrypted using a cipher that is considered mostly (though not entirely) safe, but as with most secure communication systems, successful efforts to break it have centered on examining the connection establishment protocol. One should actually say “protocols,” as the history of Bluetooth is littered with a series of mechanisms to establish connections, starting with the first generation pairing process with the often easily guessable (and always brute-forceable) PINs used to generate a session key, to the latest generation with its somewhat (though not entirely) credible key exchange protocol.
An ongoing problem with Bluetooth security is its insistence on continuing to support earlier, and clearly insufficient, security models as fallback mechanisms as the new versions are implemented. Devices with Bluetooth 2.1 Secure Simple Pairing (SSP) provide considerable protection against man-in-the-middle (MITM) and other attacks when properly used. The temptation is to allow the device to revert to the traditional pairing mechanism, which is insecure. This temptation should be resisted; the best policy is to use the latest Bluetooth hardware (this is an issue only for the control board end, as newer tablets generally have recent Bluetooth implementations) and perform pairing only with full SSP.
In the context of an embedded system, realistic threats consist largely of various forms of denial of service. Clearly, with any RF wireless link the possibility exists that the channel could be jammed. With SSP, there have been reports [PDF] of MITM attacks in which all frequencies are jammed, inducing both sides to conclude they have been disconnected and prompting a re-pairing sequence, at which point the attacker can insert itself in the middle. Re-pairing requires human interaction, so this attack contains a social engineering component. In an embedded context, with no operator present to complete the pairing, the MITM part of that attack is perhaps unlikely to be successful, but clearly it would leave the two sides disconnected. It appears that most other attacks, such as some possibilities for eavesdropping, also depend on a social engineering component, which is less likely to be successful (at least on systems with SSP) on an embedded system than on, say, a mass marketed cell phone.
Clearly one should not be complacent about the security of a protocol with a spotty history in that area. As with all designs, you should gain an understanding of the security model used in your implementation of Bluetooth, and analyze any potential weak points. One might also consider the nature of the traffic between the tablet and the I/O to which it is attached. In most realistic embedded systems it is unlikely to contain especially sensitive data, so you may be able to decide that even a successful attempt at eavesdropping is not a big concern. As with all risk analysis, a mature and realistic view of the potential threats to your system should drive the process. After all, the theoretical possibility that someone within Bluetooth distance of your system might be able to crash it with some elaborate procedure is probably not going to drive your design decision if the unit’s power plug is within reach of passers by.
Overall security of an Android based embedded system can be viewed as a trade-off between the considerable resources devoted to Android security in general, versus the large number of attackers targeting Android. Similar to the early days of Linux, in which security was often regarded with some complacency because most attacks were targeted toward Windows, many IoT developers have relied on the obscurity of their platforms in hoping not to be targeted.
Apart from the foolishness of the approach, that attitude is simply not available to anyone using a mass marketed Android device in their design. You know you will be targeted along with all other Android users — there is no alternative to monitoring security alerts and applying updates as needed. Naturally, not every update is applicable to a product that uses only a specific subset of Android capabilities, and the cost of applying updates is often high for embedded systems (unless you devise some automatic update scheme, which comes with its own set of vulnerabilities), but having a process in place to keep up with Android security status and updates is essential for a successful product.
The general approach of using a commodity Android tablet, charged over the USB port and communicating with the rest of the system via Bluetooth, provides a workable product design model for many embedded systems.
If you believe the tablet market will continue to make available tablets with the necessary features (i.e., of roughly the same dimensions, charged via USB, and provided with Bluetooth), and that future releases of Android will not break preexisting code, there is no reason not to design in a current model and expect to be able to replace it with new models as necessary. This provides a possible solution to the need for component purchasing continuity that embedded system designers traditionally have had.
Yes, the current tablet model will become obsolete long before your product is, but if you can expect to be able to choose from newer (and presumably better) models with compatible features, a minor redesign perhaps once per year may be a reasonable trade-off for the advantages of the Android tablet design.
Of course, you also should be aware of some concerns:
- Mechanical variation — You should be prepared for some small mechanical adjustments as, for example, seven inch tablets will not all have the same dimensions. You should have some flexibility in the mounting of the device. Additionally, the USB port may change in location from model to model, so the wiring should also have the flexibility to adapt.
- Resolution evolution — Over time, the display resolution of available tablets is likely to improve from model to model, potentially affecting the look of your application in undesirable ways. Android’s ability to display on disparate resolutions without software adjustments is actually not great, in my opinion. You will definitely want to review the look and feel of your application, and make adjustments if necessary, when changing the device resolution. Offsetting this inconvenience is the inherent advantage of having your app look better over time with minimal attention. Compare this to the usual embedded system model of needing to design in a whole new display/touchscreen — with a whole new API — after ten years in order to regain competitiveness. In general, I would prefer the Android model with gradual improvements, but obviously this depends on your specific circumstances.
- Re-certification hurdles — If you are in a regulated industry, you will have to think through the regulatory implications of such a model. The most conservative strategy would be to re-certify your device after each change of tablet, potentially requiring you to ship the unit out for compliance testing on each redesign. If such a regulatory interpretation is inescapable, clearly you will not be popular if you choose this approach. Perhaps you can make the case, both internally and to the regulatory authorities, that the old and new tablets were certified by their manufacturers to meet the same standards, and the system as a whole passed testing with the old tablet, so it is reasonable to infer the system will also pass with the new tablet. But realistically, this might be a tough sell in a conservative organization, so certainly you should consider the issue carefully before proceeding.
Taming Android’s look and feel
As a mobile operating system, Android supports and encourages a specific look and feel. This is understandable, as they want their users to have the same experience as they move from app to app on their devices. They also compete with other mobile operating systems, so they want their system to be unique and get good reviews. Finally, they want to provide certain status and navigation features that are common and always available on the device.
All of which makes perfect sense for a mobile OS, but has nothing to do with the way you want to design an embedded system user interface. In contrast to Google’s design goals for Android, your own goals are to achieve a distinctive look and feel for your application, either of your own new design or to provide continuity with a previous product generation. You also want the device locked into your environment — you certainly do not want to use Android’s standard navigation and status features.
Various solutions are available for bypassing Android’s navigation features and status displays. Recent versions of Android itself have an “immersive mode” that hides the status bar at the top of the screen and the navigation bar at the bottom. There have been a few attempts to hack the Android source code to fully remove them, and of course you can attempt to do so yourself. In my opinion, these attempts have been less than successful, at least from the point of view of someone wishing to design a maintainable and supportable system. Similarly, the immersive mode is a bit leaky in that you have to be careful about the conditions under which Android brings back the status and navigation bars; you might be surprised at times to find them there despite having requested to be in immersive mode.
The best solution is probably the most trivial: simply extend your device’s bezel down over the top status bar and up over the bottom navigation bar. Thus covered up, they will not be visible or clickable; but you will still have access to them during service (i.e., by removing the tablet from the device during configuration and service). It is also by far the easiest, cleanest, and most maintainable solution.
As for your application’s look and feel, there’s really no magic here. You probably will not want to use Android’s standard look for the simple reason that you don’t want your device to look as if it is controlled by an Android app. Similarly, you cannot use the standard means of navigation because it depends on the Android navigation bar. Like all GUI toolkits, Android’s allows you to redefine the look and feel of its widgets — you can define the graphics associated with buttons and other controls, redefine how buttons look when pressed, specify a background image for a screen, and so on. Like virtually all recent GUI toolkits, much of the user interface is defined in XML files, with a means to access that information from the main Java source, and with a half-hearted attempt at inheritance. This scheme allows you to define some qualities of your look and feel in general and derive specific controls from it, all with certain rather puzzling limitations. The trend toward XML-based user interface definition makes this old timer cranky, but Android’s scheme is neither better nor worse than others, and it is generally well documented and undeniably works.
Perhaps the worst case for UI design is if you want a unique (or at least uncommon) look and feel, but have not the desire or the resources to define the graphics from bottom to top. If you are accustomed to using third-party GUI toolkits, you might expect to choose from a number of predefined graphics options with good support. For the reasons described earlier, in Android this is difficult. Google wants you to use their look and feel, both for consistency across apps and to allow them to change it seamlessly from one Android release to the next. So you will not find many predefined look-and-feel options, as one would with third-party toolkits. Android is best for organizations with the resources to define their product look completely, or perhaps a few who can use the predefined look as is.
Summing it up
Perhaps the most attractive result of using a mass-produced Android tablet in an embedded system design is the features and qualities that essentially come for free. You purchase a low-cost computing platform with an attractive touchscreen and you get:
- A compact, light, thin component.
- A component that has already passed regulatory compliance (albeit for its market).
- A free, open, mature operating system, fully integrated with its hardware.
- An API known by many programmers, with just about every possible question answered somewhere on the Internet.
- A WiFi interface. You may not need it at first, but if later the marketing people ask about, say, sending usage data back to a server or allowing remote software updates, you can casually mention that the capability is already there. Alternatively, 3G/4G is often available by simply choosing a different model tablet.
- Similarly for any number of other built-in hardware functions: GPS, sound, camera, NFC. Again, you may not initially be able to imagine how these may be helpful, but one can almost guarantee at some point marketing is going to imagine it for you.
- Built-in software capability that you may never use, but is nice to have in case it is requested: media playback, extensive logging capability, etc. Even recently face detection and barcode reading using the camera have been added.
Who knows? One could imagine a situation in which an organization with a kiosk type device later discovers they have a vandalism problem. It would be easy enough to turn on the camera and record everything in a loop, or even use the built-in accelerometer to tell when someone is attempting to damage the unit, then turn on the camera and/or notify someone over the network interface. It is far nicer to have it there already than to try to add such capability to a preexisting product. And if you never need it? You didn’t pay anything for it, since you based your purchasing decision on only the processor and the touchscreen.
Repurposing an off-the-shelf Android tablet is clearly not a suitable solution for every embedded system designer’s user interface. But with a clear understanding of the trade-offs, you might find one to be a welcome addition to your toolbox for your next design.
Photo credit: The thumbnail image at the top of this post is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. It depicts an Internet kiosk in Hemer, Germany that is not known to run Linux or Android software.