The beauty of Android lies in the many different ways that third-party applications can interact with the system. Password manager apps such as LastPass provide the ability to automatically feed relevant username/password data to almost any login screen. Text Aide allows you to significantly shorten your time texting your friends by allowing you to create text expansion macros. Native Clipboard decreases the hassle involved with frequently switching between apps to copy large amounts of text by allowing you to double-tap any input field to bring up a clipboard. Who can forget Greenify, perhaps the #1 most recommended app by enthusiasts, which keeps rogue background apps in check and can thus enhances battery life? Finally, albeit less familiar with most users, there's AutoInput – a Tasker plug-in designed to automate screen taps, text input, swipe gestures, and much more. These apps all serve vastly different use cases, but each of these apps rely on a very misunderstood part of core Android functionality: Accessibility.
To the average Android user, it might seem odd that many of these amazing features utilized by your favorite app are controlled by a setting under the accessibility submenu. Making an app accessible is typically supposed to mean that an Android app is usable to a person with disabilities. So why in the world do LastPass, Native Clipboard, Text Aide, Greenify, or AutoInput have an accessibility service? Furthermore, why does enabling an accessibility service seem to cause so much UI lag? It doesn't seem to matter what version of Android you're on – whether it be Android 5.0 Lollipop or Android 7.0 Nougat – because the lag caused by certain accessibility services can affect your experience. A simple solution to this problem is to merely disable accessibility services you might have enabled – but in doing so, we lose so much useful functionality. Another solution is to petition Google to "fix" Android's accessibility lag, but Google claims that Android Accessibility is working as intended. We've spoken to a few developers intimately familiar with accessibility services and have researched how the functionality works, and we're here to test that claim: is Android's accessibility lag a bug or is it a feature?
Understanding Android Accessibility
As you might imagine by the name, Accessibility is mostly intended for developers to provide additional functionality for any users with disabilities. Indeed, a quick peek over on the official documentation pages for Accessibility reveals that Google has a pretty narrow view on what kinds of services should be provided by Accessibility Services.
Many Android users have different abilities that require them to interact with their Android devices in different ways. These include users who have visual, physical or age-related limitations that prevent them from fully seeing or using a touchscreen, and users with hearing loss who may not be able to perceive audible information and alerts.
Android provides accessibility features and services for helping these users navigate their devices more easily, including text-to-speech, haptic feedback, gesture navigation, trackball and directional-pad navigation.
Google's TalkBack, which comes pre-installed on every Android phone, is a great example of what the 'typical' Accessibility Service is supposed to be like. Voice Access takes accessibility a step further and allows for almost complete control of your phone using only your voice. But the fact that Google intended Accessibility Services to be used in this manner does not prevent developers from implementing them in whatever way they want – and that's exactly what developers have done. It's exactly because of the way that Accessibility works that makes the feature incredibly useful to users with or without disabilities.
To simplify things a bit, here's a basic rundown of how Android's Accessibility works. A developer creates an Accessibility Service that subscribes to various Accessibility Events that are sent by the system to the Service depending on whether or not certain criteria are met. When all Services are disabled under Settings –> Accessibility, Android does not collect or send any Accessibility Events. But when the user starts enabling Accessibility Services, Android will begin monitoring and collecting only those Accessibility Events that the Accessibility Service requests. For example, an Accessibility Service that subscribes to the Accessibility Event TYPE_WINDOW_CONTENT_CHANGED will be notified by the system every single time that a change in the current window occurs. Another Accessibility Event called TYPE_VIEW_CLICKED fires off every single time the user clicks on a button of some kind.
Android Accessibility Demonstration. In this video, I've enabled the app Tasker to monitor for changes in the Window title. This requires enabling Tasker's Accessibility Service. You can replicate this by creating a new profile in Tasker with the 'Event' context set to 'Variable Set' and choosing %WIN as the variable to monitor. In total, this approximately 1 minute video captured 107 changes in the current Window.
These kinds of Accessibility Events occur with great frequency during normal user interaction. So imagine what happens when a user enables multiple Accessibility Services that request high frequency Accessibility Events be fired off. That's right – lag. To mitigate this, developers can more narrowly define what kinds of Accessibility Events their Service should react to and in what context, such as the ability to limit the Service to only react when in certain apps or to limit the polling period between Events. But other than that the amount of overhead generated by an Accessibility Service is dependent mostly on what kinds of Accessibility Events it subscribes to. In essence, not every Accessibility Service will cause lag. A single Accessibility Service that requires a high frequency Event may cause lag, especially if said Service is coupled with another Service that requires another high frequency Event to be monitored.
Diving Deep into Accessibility with APK Teardowns
As you could tell from the video posted above, an Accessibility Service that monitors for changes in the window content can result in fairly noticeable changes in UI performance due to the sheer amount of captured Accessibility Events fired off by the system. However it's quite difficult to determine exactly how much overhead is caused by a particular Accessibility Service. Monitoring LogCat will generally get you nowhere, as Accessibility Events are only printed to LogCat if the developer of the Accessibility Service chooses to do so. Thankfully, the daddy of all Android Accessibility Services, AutoInput, does exactly that. And the LogCat output is exactly as messy as you would imagine.
AutoInput doesn't hide the truth from us. The overhead caused by the app can be quite enormous depending on what Events you monitor. But this overhead is necessary for the app to function. In order for AutoInput to intercept every key press, every screen gesture, every UI update, and every button press, it needs to monitor the respective Accessibility Events. Without these Events, AutoInput cannot hook into the system and provide the almost unlimited UI automation that it currently allows for. Thus, all of AutoInput's functions make perfect sense within the context of Accessibility. But for other apps, we need to look a bit deeper to understand how their Accessibility Services are handled.
An Accessibility Service's attributes are defined in an XML resource file within the APK. Therefore, we can perform an APK teardown on an app with an Accessibility Service to figure out the Service's attributes. Each app functions differently, so I will try to explain how their Service's attributes relates to the specific function it performs.
Native Clipboard
Native Clipboard is my go-to when it comes to clipboard managers. If you are looking for a highly customizable clipboard manager, Native Clipboard is a pretty great app. It even has an Xposed Module component to allow you to long-press on the 'Paste' button to bring up the clipboard manager! Unfortunately, if you don't have access to the Xposed Framework (such as every user on Nougat) then you'll have to settle for enabling the Accessibility Service which will allow you to double-tap on any text input to bring up the clipboard manager. Here's what that entails.
<?xml version="1.0" encoding="utf-8"?> <accessibility-service android:description="@string/access_decs" android:accessibilityEventTypes="typeViewClicked|typeViewFocused|typeViewLongClicked|typeWindowStateChanged" android:accessibilityFeedbackType="feedbackGeneric" android:notificationTimeout="100" android:accessibilityFlags="flagReportViewIds|flagRetrieveInteractiveWindows" android:canRetrieveWindowContent="true" xmlns:android="http://ift.tt/nIICcg" /> Native Clipboard's Accessibility Service requests firing off an Accessibility Event each and every time that a view is clicked, long-clicked, focused, or if there is a change in the window state. Without having access to the source code, I cannot say exactly how Native Clipboard works, but it's likely that Native Clipboard waits for the Window state to indicate that the soft keyboard is currently open, and then it monitors for taps on the input field. The app has a polling period of 100ms, so that is definitely quick enough to react basically immediately to changes in the soft keyboard visibility as well as double taps. This could result in some UI overhead whenever the user is using the soft keyboard to type any text, potentially resulting in lag.
Greenify
Next up is everyone's favorite battery saver, Greenify. Greenify uses Accessibility Events to power its non-root functions.
<?xml version="1.0" encoding="utf-8"?> <accessibility-service android:description="@string/accessibility_service_description" android:settingsActivity="com.oasisfeng.greenify.accessibility.AccessibilitySettings" android:accessibilityEventTypes="typeAnnouncement|typeNotificationStateChanged|typeWindowStateChanged" android:accessibilityFeedbackType="feedbackGeneric" android:notificationTimeout="0" android:accessibilityFlags="flagReportViewIds" android:canRetrieveWindowContent="true" xmlns:android="http://ift.tt/nIICcg" /> It uses changes in the Window State to determine when the phone's screen has turned off, and it requires that you delay the lock screen activation by changing an option in security settings. Greenify will also receive Events of type Announcement or Notification State changed, the latter which is unnecessary on Android 5.0+ devices thanks to the Notification Access feature. It will, however, still receive these events regardless of that fact. Greenify should not cause much overhead by itself, but the possibility remains.
Nova Launcher
Probably the most popular third-party launcher app on the market, Nova Launcher is an excellent example of an app using an Accessibility Service with minimal to no overhead. The only reason for the Service's existence is to aid certain devices in performing gestures.
<?xml version="1.0" encoding="utf-8"?> <accessibility-service android:description="@string/accessibility_service_description" android:accessibilityEventTypes="" android:packageNames="com.teslacoilsw.launcher" android:accessibilityFeedbackType="" android:notificationTimeout="10000" android:canRetrieveWindowContent="false" xmlns:android="http://ift.tt/nIICcg" /> As you can see, there is no Accessibility Event defined in the XML file. All that is mentioned is the name of a package – Nova Launcher. What happens here is a workaround for certain devices for which Nova Launcher's gestures do not work. This service will provide Nova Launcher all Accessibility Events fired off from only within Nova Launcher. It sounds odd, but it's apparently a way to fix Nova's homescreen gestures if your device doesn't work with them. Since this only requests Events from Nova itself, the Service poses very little overhead.
LastPass
Finally, perhaps the most infamous Accessibility Service which causes lag (probably due to its immense popularity) – LastPass. The issue of lag within LastPass is so noticeable that the company has an official FAQ page describing the issue. As the FAQ states, there is nothing you can do about the lag except to disable the Service. Why does LastPass's Service seem so egregious when it comes to lag? Let's take a look at the Service's attributes.
<?xml version="1.0" encoding="utf-8"?> <accessibility-service android:description="@string/accessibility_service_description" android:accessibilityEventTypes="typeViewFocused|typeWindowContentChanged" android:accessibilityFeedbackType="feedbackGeneric" android:notificationTimeout="200" android:accessibilityFlags="flagReportViewIds" android:canRetrieveWindowContent="true" android:canRequestEnhancedWebAccessibility="true" xmlns:android="http://ift.tt/nIICcg" /> The truth is, there's nothing really out of the ordinary with LastPass's Service. It only requests two Event types to monitor – TYPE_VIEW_FOCUSED and TYPE_WINDOW_CONTENT_CHANGED. It does this because it needs to know when an app/webpage's content changed/comes into focus, and then it retrieves the current window content to look for any password input fields. But since the service constantly does this on two extremely frequently firing Accessibility Events, it results in lag. That's the unfortunate truth.
Living with the Lag
When we first read that Google was closing bug reports about Accessibility lag because the feature was "working as intended", we were just as perplexed and upset as many of you. But rather than accept the explanation at face value, we decided to look into the matter ourselves to determine the truth. So when the Googler on the bug report page said this:
Hi this issue is persistent over Android releases, Also there will always be an additional lag when an accessibility service is enabled. That's because the device, in addition to the standard UI, is providing a lot of information to accessibility services so they can provide an alternative user experience to those users.
We have come to understand why this is intended behavior. Apps that use Accessibility Services in a manner that was unintended by Google will always incur some performance overhead; this cost is simply necessary to provide Services with the plethora of information that Android Accessibility fires off in the background. Android's lag with Accessibility Services is not a bug, but a feature. A feature that we will have to live with unless the entire system is reworked, and I cannot imagine how that would be done to accommodate so many different feature sets from so many different apps.
At the very least, the LastPass developers would not take this sitting down. Their developers have worked with the Chromium developers to optimize accessibility support, perhaps by enabling LastPass support through the use of APIs rather than enabling an Accessibility Service. Optimizing around the overhead incurred by Accessibility Services is one possibility, but as many developers have implicitly noted on the Chromium forums, it's simply a bandaid that won't resolve the fact that unintended uses of Accessibility Services may result in lag.
Special thanks to the developer of AutoInput, joaomgcd, for answering many of my questions regarding Accessibility!
from xda-developers http://ift.tt/2dKFEaF
via IFTTT
No comments:
Post a Comment