android | Read the Tea Leaves

Posts Tagged ‘android’

13 Jun

How to de-Google your Android phone

Posted by Nolan Lawson in Android. Tagged: android, satire. 11 comments

First, download a ROM from this Russian message board. It’s okay! You can totally verify the GPG signature. Allow yourself 30 minutes to remember how GPG works, then verify that forum poster LeetAndrej420 has indeed signed the file.

Next, root your Android phone. You will need to hold the volume-up and power buttons for ten seconds, then unplug from USB, then reboot a few times after you mess it up, then give up and download the Android dev tools.

After you figure out the Android adb and fastboot commands, you should see a friendly UI with green Courier text on a black background. Press the button that says, “I void my warranty and completely exonerate the OEM in the likely event that I am actually pwning myself by installing random software from the internet onto a tracking device I carry in my pocket every day.” But it’s okay. You trust Andrej, right?

Next you will need to install the “recovery” tool. Despite the name, this is actually the best way to brick your device. Luckily it is incredibly feature-rich, boasting 12 buttons on the home screen, including an “Advanced” button containing more buttons. These buttons will invite you to do things like “clear the Dalvik/ART cache,” which you totally know what that means.

When you download the recovery tool, make sure you get the right version for your phone! Of course, it’s not named after your phone’s brand name, but rather a cheeky internal name chosen by the OEM, like “bacon”, “cheeseburger”, or “mahimahi”. The professionalism on display from all parties should fill you with confidence.

You will download the recovery tool from a site called SickWarez.biz. Use GPG to ensure that it’s signed by Andrej.

Once downloaded, go into recovery mode and install the ROM, being careful to press the one correct button out of 12, like a game of Minesweeper that will brick your phone if you lose. This will also factory-reset your device, which is fine because all your photos and contacts are backed up to your Google account… ah, right. You’ll want to do something about that.

Assuming you have successfully installed the ROM without turning your phone into a $700 doorstop, you can now install apps. Thankfully there is F-Droid, which hosts all your favorite open-source apps. Wait, your favorite apps aren’t open-source? Well, at least it has Signal. Wait, it doesn’t have Signal?

Once you’ve installed the Yalp Store, which sideloads apps from Google Play in a way that may or may not be totally illegal and will get blocked by Google once they read this blog post and realize that it exists, you can now download some actually useful apps.

Thankfully, though, your personal data will be safe and secure from third-party developers, because these apps will not work. Be prepared for error messages like, “Please install Google Maps,” “Google Play Services required,” or “What kind of sicko has a Google phone without Google? What is wrong with you?”

After all this ceremony, you can now relax and enjoy your Google-free Android device. Note, though, that weather widgets, GPS, push notifications, and the majority of Android apps you rely on will not work. That said, there are some great note-taking apps! Plus SMS will still work. Good old SMS.

So now that you’ve successfully turned your $700 Android device into a glorified $30 Nokia flip phone, which may or may not be siphoning your passwords to a Ukrainian teenager, you can finally have a Google-free smartphone experience. Or you could just buy an iPhone.

27 Nov

Living with an open-source phone

Posted by Nolan Lawson in Android. Tagged: android, lineageos. 31 comments

A few months ago, I decided to make a radical change to my smartphone lifestyle. I wanted a phone that would limit the amount of personal data I was broadcasting to third parties – namely Apple or Google – and decided to run a more-or-less vanilla version of Android, without any Google Play Services (i.e. no Google Maps, GMail, Google Play, or Google Search apps).

I’ve been living with this setup for a few months, and to my surprise, it’s really not that bad. It doesn’t feel like a return to the Stone Age, nor does it feel like I’ve sacrificed all the niceties of a smartphone to the point where I’m carrying around a glorified flip phone.

However, it’s a bit non-obvious how to get all of this stuff to work, and especially how to get by with a Android phone that doesn’t have the Google Play Store. So in this post, I’d like to talk a bit about my smartphone setup, in the hope that it might help someone who’s looking to make a similar switch.

Choosing an Android ROM

There are various flavors of Android out there: you can choose LineageOS (the spiritual successor to CyanogenMod), Paranoid Android, CopperheadOS, or you can even just build AOSP (the Android Open Source Project) from source.

After fumbling around with building AOSP from source (which was much more difficult than I thought – lots of requirements for the build environment, CPU horsepower, and storage), I also tried LineageOS and CopperheadOS, and eventually settled on LineageOS.

Why LineageOS? Well, mostly because it was easy, it offered fast security updates, and I like the minimalist interface and built-in apps. LineageOS was also familiar to me, as I had previously used Cyanogen for many years. (I forgot how much I enjoyed the small creature comforts, such as long-pressing the volume up/down keys to skip tracks!)

Default homescreen on LineageOS

I also needed a recovery image in order to install the ROM, and it seems that TWRP has become the de-facto standard these days, replacing the venerated ClockworkMod.

As it turns out, none of these options will result in a 100% open-source phone, as you’ll still be running the vendor binaries for your particular phone. But this is about as close as you can get to a smartphone that runs only free/libre open-source software.

Choosing a phone and installing a ROM

This isn’t going to be a guide to unlocking or rooting your Android phone. The process is different for every phone, and it would take too long to describe all the various steps. Suffice it so say that it’s still a painful process, and your choice of phone can either make it easier or harder.

I’ve been rooting, unlocking, and tinkering with Android phones for a long time, since the days of the HTC Dream and HTC Magic around 2009/2010. Since then, I’ve worked as an Android and mobile web developer, and I’ve become very comfortable with tools like the Android SDK, adb, fastboot, and Android Studio. And yet, to this day I still find installing custom ROMs to be a frustrating and time-consuming experience. I wish it were easier for casual folks to do this stuff.

The Nexus line of phones (now Pixel) have always been the easiest to customize, and for many years I stuck with the tried-and-true Nexus 5, which is still a surprisingly capable phone despite having been released in 2013. (I swear it ran faster than my 5X, which also kicked the bucket after only a year or two of use.) However, my aging Nexus 5 just couldn’t cut the mustard anymore due to hardware issues (the USB port was too loose; it had become tricky to charge it), and so I decided to buy a new phone instead.

I settled on the Samsung Galaxy S5, mostly because I could get it for cheap (<$200 on Amazon) and with full support for my carrier (T-Mobile). In retrospect, choosing a non-Nexus device made my life a lot harder, and after several hours of research on unlocking Samsung phones (including building Heimdall from source on a Windows machine, because the prebuilt binaries were out of date but the software was too old to build on a Mac), I finally had my LineageOS phone up and running.

The second thing you'll want to do is ensure that your device is encrypted, which you can enable in the security settings. Unfortunately this ended up making my phone unable to start the OS, but after booting into recovery and doing a factory reset, I had both encryption and the OS up and running just fine.

My LineageOS home screen

Getting apps

Without the Google Play Store, you'll have to use F-Droid, which has the added benefit of only hosting free and open-source apps. In fact, a couple of my own apps were previously on there (Catlog and KeepScore), and as far as I can tell, they were built directly from source. (I’m not sure why they were removed; possibly because I stopped maintaining them.)

To get F-Droid, you just download it directly from f-droid.org. You'll also have to allow installation from "unknown sources" in the security settings.

For the odd app that isn't available on F-Droid, you can also use the Yalp Store, which can either use your Google account or an ephemeral account to download apps from the Play Store. It’s not clear to me whether or not this violates the Google Play Terms of Service, though, so proceed with caution.

Messaging

I use Signal as my default SMS app and for most messaging with family and friends. Unfortunately it isn't available on F-Droid, but you can download it directly from the Signal website, or use the Yalp Store technique above.

Update: if you’re wondering how to verify the SHA256 fingerprint of the APK downloaded from the Signal website, here’s an example.

Like most Android apps, Signal normally uses Firebase Cloud Messaging (FCM, formerly GCM) to send push notifications. If you don't have Google Play Services installed, Signal will offer to switch to a less battery-efficient background polling mechanism.

I've used this mechanism for months and only had occasional problems where a large backlog of old messages was suddenly delivered all at once. It also hasn't had a visible impact on battery life. (I usually end the day with half a charge left; the GS5 has a great battery!)

Web browser

For a web browser, I mainly use Firefox via the FFUpdater tool, which is clunky but gets the job done. I’ve set DuckDuckGo as my default search engine, and I have uBlock Origin installed as an add-on, which undoubtedly makes my browsing faster and easier on the battery.

Occasionally though, I do find sites that don’t run so will with mobile Firefox, and for that I use Auto Updater for Chromium, which automatically installs Chromium and updates it. This is a bit nicer than FFUpdater because it does its work silently in the background, rather than requiring you to manually check for updates.

The built-in browser doesn’t support “Add to Homescreen,” so I find it fairly useless.

Maps

I use OsmAnd~, which is nice because it allows you to download maps in advance for offline use. Unfortunately it’s still nowhere near as feature-complete as Google Maps, so if you’re looking for something to help you navigate your car, you may be out of luck. (Update: actually, OsmAnd~ supports turn-by-turn navigation.)

I also frequently use maps.google.com in Firefox, which works surprisingly well. Besides some minor performance issues, it’s fairly indistinguishable from the native app for basic directions, bus timetables, and store lookups.

Google Maps running in Firefox

Living in Seattle, one of my favorite apps is also OneBusAway, which provides up-to-date arrival times for Seattle buses. Unfortunately this app requires Google Play Services in order to work, so I’ve had to do without.

Weather

Speaking of Seattle, it’s also important for me to know how much it’s going to rain in a given day. (Spoiler alert: a lot.)

This happens to be one of those things about LineageOS that’s a bit non-intuitive. To get weather to work, you have to go to the “extras” page and download a weather provider. (In my case, I’m using OpenWeatherMap.)

Unfortunately, though, even after installing the weather provider, I couldn’t get the built-in “cLock” widget to show the weather. (It keeps saying “network geolocation is disabled,” even though it’s not.) So I ended up installing Forecast widgets, which gets its data from the National Weather Service rather than the built-in weather provider.

Maybe this is just some bug with my installation or with the hardware itself, but in any case I’m satisfied with this workaround. The Forecast widget looks and works fine.

Weather and time widget on my homescreen

Keyboard

The only non-standard features I really want from a keyboard are 1) emoji support, and 2) swipe input.

Unfortunately I couldn’t find any open-source keyboard that can do both of these things, and the AOSP keyboard doesn’t seem to support either. The closest I found was AnySoftKeyboard, which at least has an emoji screen. It doesn’t allow you to search for emoji, though, which is a bit frustrating.

Sending an emoji via AnySoftKeyboard and Signal

Overall, the keyboard experience has been my least favorite part of the LineageOS experience. I didn’t realize how much I had become accustomed to swipe input until I had to revert back to tapping, which feels to me like hunt-and-peck. Input is extremely laborious and slow, although on the plus side I’m spending less time texting, so maybe that’s a good thing.

Update: actually, AnySoftKeyboard does support searching emoji! You just need to type : and then the search term.

Music and podcasting

I’m a fairly old-school music consumer. I prefer to listen to albums from start to finish, and I never really got into services like Pandora or Spotify. Instead, I buy my MP3s from Amazon (old habit, my entire library is there) and then sync them from my desktop computer to my phone using adb-sync.

For browsing and listening to my music library, I’ve found Vanilla Music to be fantastic. It has a cover art downloader, the interface is minimal and clean, and it can play an album from start to finish, which is all I really ask.

Vanilla Music with some of my albums

Oh, and of course Vanilla Music sets the cover art on the lock screen. It’s the little things that count.

I’m also something of a podcast addict, so I use AntennaPod to download and listen to podcasts. I’ve tried several podcast apps on F-Droid, and I found this one to be the easiest and most reliable overall. I particularly like that it allows me to search on iTunes, since some podcasts can be hard to find elsewhere.

Social media

I stopped using Twitter, but if you absolutely must, their mobile website is not bad in either Firefox or Chromium. I’m told Facebook’s mobile website is also pretty functional.

I spend most of my social media time these days on Mastodon, and as it turns out the mobile websites work perfectly fine in both Firefox and Chrome, so you can just pin it to your homescreen. There’s also Tusky if you prefer the native app experience.

Ridesharing

I don’t use Uber, but Lyft has a mobile webapp that works just as well as their native app. So when I need a ride, I just open ride.lyft.com in Firefox. It’s a PWA, and as far as I can tell it works just as well as the native Android app.

Somewhat bafflingly, though, if you go to lyft.com directly, the site may try to route you to their driver portal or to the app store instead.

Screenshot of Lyft's website offering "Lyft in Firefox,"

Lyft’s website may offer “Lyft in Firefox,” which is the PWA

In case you get confused, you may need to look for the “Lyft in Firefox” link. To me, this seems like a bizarre way of saying “Lyft for the web,” but you do you, Lyft.

Email, calendar, and contacts

I switched to FastMail recently for my primary email account, and I’ve been really happy with the service. It’s fast, it’s simple, and it has a clear business model where I pay them 5 bucks a month and they host my email. Simple.

To get my email on my phone, I use K-9 Mail, as I’ve found it a bit more reliable and feature-rich than the built-in Email app. If you prefer a simpler interface, though, the built-in app works fine too.

FastMail also supports calendar and contact sync, and this is where it gets a bit trickier. If you’ve ever connected an email client to your email server by manually typing the IMAP and SMTP settings (which, incidentally, you’ll have to do for K-9 email), then it’ll feel pretty familiar. FastMail supports CalDav and CardDav, and so to hook these up to my phone I used DavDroid.

Honestly this was probably the most tedious process of setting up my LineageOS phone. It involved creating app passwords for each client (I use separate passwords for email, calendar, and contacts, in the interests of security), and manually typing in the server names and ports for the various FastMail endpoints. This was a long and error-prone process, but in the end I do have full email, calendar, and contact sync, so I can’t complain.

Passwords and two-factor auth

For nearly a decade I’ve stored my passwords using the Joel Spolsky method, which is to use PasswordGorilla on desktop and PasswdSafe on Android. I like this method because it’s simple, it works cross-platform, and I maintain control of the password file.

For two-factor authentication (other than basic SMS), there’s FreeOTP, which essentially takes the place of Google Authenticator.

The web works for everyone

One of my main reasons for switching to a non-Google Android phone was to see how capable the web is as an application delivery system. And aside from the native apps listed above and some minor utilities (e.g. a barcode scanner and a notes widget), I do pretty much everything in a web browser.

Media sources like Hacker News, Ars Technica, and others don’t need an app – you can just pin a website to your home screen. (Although for Hacker News, I use the excellent hn.premii.com.) The main exception for me is NewsBlur, which I access via the open-source client app.

For videos, both YouTube and Vimeo also work great as mobile webapps. In Firefox, they can run both in fullscreen mode and in the background.

For file sync, I use Dropbox’s mobile webapp for quick file downloads from my Dropbox account and Firefox Send for sending more ephemeral files. I need a better solution to backing up photos, though; for now I’m just using adb-sync to sync to my Dropbox folder on the desktop.

I also do work at Microsoft, and although I have a personal policy of not syncing my work email to my phone, I can make a temporary exception by loading outlook.office.com in Firefox, which works surprisingly well. For those odd moments when I need to send an email or check the location of a meeting, it gets the job done.

As a guitarist, I also needed a way to tune my guitar, and sadly the excellent g-strings is both paid and closed source, so it was a no-go. So I use Paul Lewis’ guitar tuning webapp instead, which is a good substitute.

Me singing a bit off-key into a guitar tuner

Turns out the web of 2017 is capable of quite a bit, from hailing a taxi to tuning your guitar!

Conclusion

I hope I’ve demonstrated in this blog post that it’s not only possible to use an open-source phone without any of the pervasive tracking, spyware, or bloatware that we’ve come to expect from most smartphones, but it’s actually quite viable and even enjoyable.

When I booted up my Galaxy S5 for the first time, I was immediately greeted with a barrage of ads for Samsung and T-Mobile services and upsells, all in the 5 minutes it took to turn on USB debugging so I could install custom software and mercifully silence all of the nagware. My clean and stripped-down LineageOS setup is about as far from that horrendous out-of-the-box experience as you can get.

Note though, that this is the setup that works for me, and your mileage may vary depending on how much you rely on various software and services. If you use GMail and GDocs, you may be better off just using a Google-flavored Android phone. And if you’re bought into the iTunes and iCloud ecosystem, it may be more trouble than it’s worth to switch from an iPhone.

Also, despite my attempts to de-mystify some of the less obvious parts of this setup, I’m in no way claiming that any of this is accessible to the non-geek, non-hacker crowd. The mere process of installing LineageOS is so far beyond the capabilities of the average non-techie that it actually fills me with a kind of despair.

This is probably a subject for an entire post, but I’m becoming concerned that my friends and I in the open-source, pro-privacy hacker scene are building a two-tiered world, where the tech elite can successfully install and configure software that maintains their security and privacy, whereas the average person is stuck either paying a premium for a privacy-respecting but closed-source iPhone, or paying a reasonable price for an Android phone where their security and privacy are far from assured.

That said, maybe if more of us inhabit this (admittedly neophyte-unfriendly) world, then maybe we can work to make it more accessible to those of us who don’t know how to use a command line and have no patience to type out IMAP server URLs into their smartphone. Let’s hope so.

13 Aug

Creating a contact with multiple fields in Android

Posted by Nolan Lawson in Android. Tagged: android, code, contacts, development. 2 comments

For the impatient: skip the article, download the code.

Recently, when writing a physician directory for the Canton of Geneva, I wanted to include a feature for adding a new contact. That is, I wanted a button that would pop up the “Add a new contact” screen, with various fields (such as phone number, postal address, and email address) already filled in. Piece of cake, right?

Adding a contact in the physicians app.

Unfortunately, it turns out that the Android docs and Stack Overflow are pretty bereft of clear, concise instructions for creating a contact with multiple fields of various types, e.g. work phone, mobile phone, or home fax (if such a thing still exists).

Plus, the entire ContactsContract changed in API level 11 (Honeycomb), meaning that anything written for ICS or Jelly Bean wouldn’t work in Gingerbread, and vice-versa. Oh joy.

Luckily for you — assuming you stumbled across this post after a frustrated trip to Google — I’ve written a helper class to do all the heavy lifting. It provides a simple, fluent API that works for Android version 2.1 (Eclair) through 4.2 (Jelly Bean), and it’s open source.

You create a contact like this:

Intent intent = new AddContactIntentBuilder("Joe Blow")
    .addFormattedAddress("123 Fake Street, Springfield USA",
        StructuredPostal.TYPE_HOME)
    .addPhone("555-867-5309", Phone.TYPE_HOME)
    .addPhone("555-123-4567", Phone.TYPE_WORK)
    .addPhone("555-987-6543", Phone.TYPE_FAX_WORK)
    .addEmail("joe.blow@gmail.com", Email.TYPE_HOME)
    .addEmail("joe@blow.com", Email.TYPE_WORK)
    .build();

startActivity(intent);

And here’s what this code produces, in both Jelly Bean and Gingerbread:

Adding a contact in Jelly Bean and Gingerbread.

Happy contact creating!

Download or fork the code from GitHub.

25 Sep

KeepScore version 1.2: more style, more substance

Posted by Nolan Lawson in Android. Tagged: android, keepscore, ui design. 14 comments

KeepScore v1.2

I had always thought of KeepScore as a fairly simple app. Functional, yes. But beautiful? Meh.

It’s a counting app. Counting apps only have to do one thing right, and that’s count. This is not brain surgery, people. Just keep it simple, and you’re already 95% of the way there.

A few weeks ago, though, I decided to make KeepScore my guinea pig for trying out some new design elements from the “Holo” theme, introduced in Android 3.0. At the same time, I also added some fit-and-finish features that were sorely needed, giving the app a much more coherent feel.

The result is KeepScore version 1.2, probably the biggest update I’ve ever written for the app. It looks and functions so differently now, I feel like I barely recognize my own app.

What I like most about this update, though, is that it adds a fresh coat of paint without subtracting anything from the usability. In fact, I think KeepScore is actually much easier to use than it was before, to the point where I feel a little embarrassed for having bragged about it in previous posts.

New home screen

The new home screen is a design I’ve been wanting to do for awhile. Here’s a side-by-side comparison of the old and new looks:

Out with the old, in with the new.

The new design basically takes the “Load Game” screen and transplants it onto the welcome screen. I find it’s a huge improvement. There was a ton of wasted space with the old design, and plus it took two clicks to get to your saved games. Now everything the user needs is front and center, without sacrificing any usability or app branding.

I also wanted to make sure that the new design wasn’t so cluttered that it would confuse first-time users. Their experience is still pretty streamlined: there’s a big “New Game” button the size of a barn that you can’t possibly miss.

Big gray squares. Your thumb is drawn to them.

The new home screen also makes use of the “Action Bar” paradigm, which was introduced in Android 3.0 and back-ported thanks to the wonderful Action Bar Sherlock library.

The in-game view

In-game, not a whole lot has changed. If it ain’t broke, why fix it? All I added was a very small graphical flourish:

Never change, KeepScore. Never change.

Did ya miss it? The last score in the score history now has a “fade-out” gradient, to indicate that the list has been cut off at the bottom.

This is to solve a common problem I heard from users, which is that they could never remember whether the list was ordered top-down or bottom-up. Hell, I kept forgetting myself! So hopefully this subtle change will make that clearer.

Rematch button

This is something I struggled with for a long time. In early versions of KeepScore, I had a “Reset” button, which prompted the user with “Overwrite game or start new game with same players?” Knowing that users don’t read anything, though, I was unsatisfied with this dialog.

The old, confusing dialog.

In later versions, I replaced it with “Reset” and “Copy Game”:

“Reset” and “Copy Game” buttons.

Now I’ve combined them both into “Rematch”:

New “Rematch” button.

What I realized about “Reset” and “Copy Game” is that they’re an inelegant solution to a common problem. 99% of the time, if you’re still using the app after the game is over, it’s because you want to start a new game with the same players. However, I didn’t want users to overwrite their old scores, because then they’d lose all their history from the previous game. Hence the option of copying the game before resetting it.

“Rematch” captures this concept much more succinctly than “Copy Game” and “Reset.” And plus, it makes it more difficult for users to shoot themselves in the foot, i.e. by overwriting their scores.

Unfortunately I can’t take the credit for this idea. I borrowed it from Rounds, which is a pretty slick round-based score keeper that was actually originally built on KeepScore’s source code. When I saw the “Rematch” button in that app, I slapped myself on the forehead and wondered how I’d never thought of it.

Edit Players

This is a pretty nifty new feature. In the previous versions, I had an “Add Player” button and a “Shuffle” button, but there was no way to manually reorder players or delete players.

Every day I’m shuffling, and adding, players.

Now all of that is handled in a separate “Edit Players” screen, which makes it a breeze to change players mid-game. You can even touch and drag to get the order exactly right.

Clearly, Storm Eagle should go after Mega Man.

This screen also makes it easier to change players’ names. Previously, the only way to do that was to long-press on a player’s name, which is kind of low on discoverability. But hopefully the button with the pencil icon is a lot easier to figure out.

A tip of my hat goes to Carl Bauer for the drag-and-drop list implementation.

History chart

One occasionally-requested feature was a line chart to show the players’ scores over time. Well, ask and ye shall receive:

Fact: nerds love data. And gamers are all nerds.

Players’ scores are on the Y axis, rounds on the X axis. It’s probably useless for any non-round-based game, but kind of neat nonetheless.

I realize the history chart is probably the most unpolished out of the new visual features I added. The colors are pretty bland, and it’s all very MS Paint-esque, because Android has no native graphing library, so I had to whip this up from scratch. But I’m not too concerned, since most people don’t bother going into the History anyway. And for those that do, I think it’s a nice little feature.

Other new features

Besides all the UI changes, I also added some new functionality:

Backup/restore. Back up your games to an XML file on USB storage, and load them later. Duplicates are handled automatically based on unique game IDs.
Undo/redo. Self-explanatory. Any action in-game can be undone or redone, i.e. scores subtracted, scores added, etc.
Better German translations. Germany is the Mecca of modern-day board gaming, so this has got to be worth something. The app is already available in French and Japanese.
Dropped support for pre-Eclair devices. Android 1.5 and 1.6 only account for 0.5% of the user base, and the new backup/restore feature required some XML libraries from Eclair. Sorry, Cupcake and Donut! You were delicious while you lasted.

So there you have it. KeepScore v1.2 has a fresh new look, a better UI, and it’s still free and open-source. So go grab it from the Google Play Store!

2 Sep

CatLog jives with Jelly Bean, goes open-source

Posted by Nolan Lawson in Android. Tagged: android, catlog, logcat, open source. 35 comments

CatLog

CatLog is an app I’ve always been immensely proud of. I wrote the initial version in the span of a weekend, and yet it grew to be my second-biggest Android app, after the now-defunct Pokédroid. Even though it’s a pretty esoteric app, and nobody except developers will find it very useful, I’m glad I could contribute something valuable to the Android community and help make Android development a bit less of a pain. It’s cool to see fan-made instructional videos on YouTube and all the forum posts where people say, “Just download CatLog and send me a log report.”

But lo, all is not well in CatLog Land. As of the newest version of Android (4.1 Jelly Bean), Android apps can no longer read each other’s logs using the READ_LOGS permission. You’re limited to your own logs, unless you’re a system app or you gain root privileges. Uh oh.

Now, this is a defensible position on Google’s part. After all, there was a pretty high-profile security hole found in the Facebook Android SDK due to developers carelessly writing sensitive information to the system log. And in general, most apps don’t need to read each other’s logs, so the change is understandable. Stay in your own sandbox and all that.

This change is going to have a big impact on certain varieties of apps, though. Not only will it affect log-reading apps (like CatLog and aLogcat), but also apps that rely on log-reading in some way. For instance, you can say goodbye to the various “app lock”-type programs that rely on reading the system log to determine when other apps are being launched. If you don’t believe me, check out the permissions page for those apps. See where it says “read sensitive log data”? That’s the death knell for these types of apps, unless somebody figures out a smarter way to detect when another app is launched. (My own AppTracker works in the same way. So it’s toast as well.)

So what does this mean for CatLog? Well, in the future, it means it will only work on rooted phones, which basically limits its appeal to developers and root-happy techies. Until now, it had also come in handy for end users, since it gave them a way to easily submit bug reports (in cases where, for whatever reason, the default reporting mechanism wasn’t available). But starting with Jelly Bean, CatLog will require root access, which means it’s basically worthless for Joe Android User now.

So given that this is more or less CatLog’s swan song, I’m taking a pretty radical step with it. I’m open-sourcing it. Yep, CatLog is now free to remix and re-use, released under the ultra-permissive WTFPL license, just like my other apps.

Why such a permissive license? Well, because I honestly don’t care. CatLog was always a free app, and although I’m grateful for the nice pocket change I make from the donate version (about $20 per month), I doubt open-sourcing it will affect the donations much, and anyway the app was never about making money for me. So there’s really no reason to lock down the source code. I mean, yeah, there are already some copycat apps out there that stand to benefit, but they’re not really doing anyone any harm hanging out in sixth or seventh place in the search results. CatLog’s main advantage is its reputation on the Google Play Store.

On the other hand, if you do want to re-use CatLog’s code, the only thing I ask for is attribution. Sure, the WTFPL doesn’t require it, but this is just one of those “don’t-be-a-jerk” requests.

I have another strong reason for wanting to open-source CatLog: I’m bored of it. Frankly, I haven’t been able to give it much attention lately, because I think 99% of its useful features are finished, and everything that’s left is just flourishes and fine-tuning. It needs a facelift and probably some tweaks to the filter syntax, but with the enthusiasm I’ve shown for the app lately, I’m obviously just not the one to do it.

Also, I find myself turning away from Android development in general. I started writing Android apps when the system was still in its infancy, with only two phones available – the HTC Dream and the Magic. I found it a lot more fun when Android was still simple and untamed, when the market wasn’t flooded with glitzy, polished apps all competing for users’ mind-share. Back in those days, you could even write a simple Pokémon app with an ugly UI and people would love you for it. Development was easy, and the exposure was fun.

Nowadays the Play Store is much more crowded, and Android development is more difficult in general, what with supporting hundreds of devices with multiple form factors (including tablets), and multiple Android versions stretching from 1.5 Cupcake to 4.1 Jelly Bean. The APIs have grown incredibly complicated, and I can’t count the number of times I’ve discovered bugs that only appeared on a certain Android version or on a certain phone. It’s a huge headache trying to maintain all this compatibility, which is why I still haven’t updated any of my apps to the new “Holo” theme from ICS.

However, my lack of enthusiasm shouldn’t limit CatLog’s potential. When you’ve lost interest in a software project, I think it’s your duty to make it open-source, so that somebody else has a chance to grab the baton and run with it. And that’s exactly what I’m doing with CatLog. So if you have any features or bugfixes you’d like to write, fork me on GitHub and go nuts!

30 Mar

A slight makeover for KeepScore

Posted by Nolan Lawson in Android. Tagged: android, keepscore, ui design. Leave a comment

Recently I went to the trouble of de-uglifying the “Load Games” screen for KeepScore. The whole screen is just one big ListView, so taking a cue from my own recent post, I added some fast scroll sections divided by date. I think the effect is more pleasing to the eye, and it also makes it easier to navigate through your past games.

The old version of the UI is on the left, and the new one is on the right:

There. Isn’t that much nicer? The important information (i.e. the player names) pops right out, whereas the other stuff is banished to a light gray subtitle. The icons to the left give the user the feeling that each row refers to some tangible object, saved somewhere, and the checkmarks on the right are useful for doing bulk-delete operations.

Here are some more screenshots:

I’m especially proud of the little row of buttons there at the bottom. They pop up when any boxes are checked, and gracefully recede when the boxes are unchecked, similar to the Gmail app. It was really tough to get them to actually hover over the ListView as they animate upwards, and then have the ListView concede screen space once the animation is complete. I report with some satisfaction that even the Gmail app (version 2.3.5.2) doesn’t do this – when the animation starts, the ListView jumps upward, leaving an awkward little white space for the buttons to pop over.

Awkward white space in Gmail

No awkward space in KeepScore

Overall, the new UI is cleaner, prettier, and more usable. And the code is open source for anyone who wants to borrow it.

19 Mar

Spruce up your ListView by dividing it into sections

Posted by Nolan Lawson in Android. Tagged: android, development, ui design. 5 comments

If there’s one piece of the core Android framework that every Android dev struggles with, it’s ListView. ListView is incredibly flexible and complex, and you’ll probably find you need it more than once in any decent-sized app. If you haven’t already slammed your keyboard and screamed at ListView before, you probably haven’t been writing Android apps very long. It’s so important, Google even had a whole session about it at their I/O conference in 2010.

ListView is the crucible, the teeth-cutting, the rite of passage for all aspiring Androidians. It’s like Luke seeing Darth Vader in the cave on Dagobah. Once you’ve battled with ListView and emerged from the cave victorious, you’ll know you’re a true Android developer.

This is just one story about ListView.

When I was writing Pokédroid, I came across an interesting problem. The first screen of the app was just a huge list of creatures, but it was too difficult to navigate through. Depending on what game you had, you were only interested in the ones numbered 1-151 (first generation), 152-251 (second gen), 252-386 (third gen), 387-493 (fourth gen), or 494-649 (fifth gen). This meant that the newer (and therefore more interesting) Pokémon were at the bottom, where they were hard to get at. But assuming the National Pokédex numbering, this was just the proper order.

Problem: there were too many goddamn Pokémon.

Too goddamn many.

The solution I came up with was to make the list more navigable by showing “fast scroll” overlays with the names of the various Pokémon generations. Named after the games’ regions, they go “Kanto,” “Johto,” “Hoenn,” etc. That way, the user could immediately know what section of the list they were in, and they could quickly scroll between sections.

Lots of Android apps do a similar thing. The Contacts and Music apps, for instance, show overlays to let you know which part of the alphabet you’re on:

This is made possible by the use of the “fast scroll thumb,” i.e the little grooved square to the right. It allows you to zoom through your list contents and hone in on the item you want. It’s like blasting down the highway and watching the exit signs, versus crawling down a suburban street, inspecting each house number one-by-one. It’s a much better user experience.

So the fast scroll thumb is awesome. And to use it, all you have to do is add fastScrollEnabled=”true” to your ListView’s XML. The only catch? If you want to use it for anything other than alphabetical sorting, your section overlays are going to look like this:

Bleccch.

Yup, the overlay has a fixed width, so you can only really use it for single characters. What’s a poor Android developer to do?

As it turns out, the only way to fix this problem is to implement your own version of the Contacts app’s internal FastScrollView and hack it yourself. I wasn’t the first to discover this, but I did post some snippets of the solution to Stack Overflow back when I first implemented it in Pokédroid. Since then, I’ve been getting some questions and clarification requests on the original post, so I decided to go ahead and write a full demo app to show how it works. After all, Pokédroid is and will probably always remain closed-source, but this code at least is probably worth sharing.

The demo app is on GitHub. Since Pokémon is kind of an esoteric subject, I decided to go with the topic of countries and continents instead. In this example, we’ve got a big list of countries, sorted either by continent or by country name. When you use continent-sorting, you can see overlays of the continents:

…and when you sort by the country name, you see alphabetic overlays instead:

Of course, if you wanted to get really fancy, you could vary the width of the overlay based on what kind of sorting you’re using. But it should be clear enough how to do that from the source code. In any case, with Pokédroid, I had a handful of different sorting mechanisms, but the most common ones had rather long titles, so I just kept the width the same for all of them. In the end, it looked like this:

That’s Pokémon sorted by generation, type, and base HP. The possibilities are pretty endless. You can take your ListView and sort it, divide it, slice-n-dice it however you want.

The important thing is that “fast scroll” sections make for a better user experience. ListViews can hold a lot of data, but that doesn’t mean you should let your list get bloated and then leave all the hard scrolling up to the user. I have an app on my phone where the developer uses an unsectioned ListView with over 200 items. Two hundred! It takes almost five seconds just to scroll from top to bottom! That may not sound like much, but in the UI world, five seconds is an eternity.

Just imagine your poor users, holding their phone in one hand and flipping your ListView with the other hand, over and over again, like they’re trying to light a wet match. Then reflect on how much you could improve that experience with some fast scroll sections.

Well, ListView-abusing Android developers (you know who you are): now you have no excuse. The CustomFastScrollView code is public and open-source, so go use it. Get cracking!

1 Feb

One-star reviews are lousy bug reports

Posted by Nolan Lawson in Android. Tagged: android, bug reports, keepscore. Leave a comment

Puzzling over cryptic bug reports is a frustrating and unavoidable part of being a developer. When users want to complain to you about a bug, they just usually don’t think through all the pieces of data that might help solve the problem.

What OS are you using? What version of the software? What were you doing to cause the bug? When users are angry, they don’t want to deal with such tedious details. They just want to vent.

This happens with large software companies, small software companies, and indie developers alike. It’s such a common gripe among developers that it’s not even worth describing any further. Any seasoned dev knows what I’m talking about.

In the Android world, dealing with bug reports is even more frustrating, because they usually come in the form of 1-star reviews on the Android Market. 1-star reviews provide all the cathartic venting that users desire, without any of the useful information that could actually solve the problem.

Here are some actual 1- and 2-star reviews I’ve gotten on the Android Market:

didn’t open… gutted
There is no sound on moment would give higher rating when fixed
always forced close on samsung galaxy s. I have to uninstall it.

Yeah, not so helpful. Figuring out a bug from comments like these is like trying to solve a detective story with half the pages torn out.

Worst of all, these kinds of comments are dispiriting for developers, because star ratings are so crucial to getting your application to be highly ranked in the Market. My own recent app KeepScore had only 4- and 5-star reviews, and was starting to get ranked pretty highly, before receiving an onslaught of these nasty little comments:

Angry? Yes. Helpful? No.

KeepScore is designed to save scores automatically. In particular, it’s supposed to automatically save your scores whenever the app leaves the foreground, as shown here in the source code.

I couldn’t reproduce the data loss described in these reviews. Even when an incoming call disrupts an ongoing game, KeepScore gracefully exits and displays a comforting message saying, “Game automatically saved.” I’ve never seen it lose data.

So what happened here? Did the Android system kill the app before it could call the onPause() method and save the data (which, according to the Android Activity Lifecycle, shouldn’t happen)? Did the users just accidentally create a new game, so that it replaced the old one in the “Resume Last Game” section, making them think that the data had been lost? Who knows. Without a proper bug report, I have no idea what to make of this.

Bad reviews make the author feel better, but they rarely lead to better applications. I’m going to try not to let these reviews sour my experience with KeepScore, though, or discourage me from putting more effort into it. I want to get to the root of this problem.

So, loyal KeepScore users, have any of you run into this particular issue? If so, please report it on the GitHub page, and let’s squash this bug! Oh, and if I do manage to fix it, please leave a nice little comment for my trouble, will ya? It’d be nice to have some good reviews to offset all these bug reports.

30 Jan

CatLog now supports external Intents

Posted by Nolan Lawson in Android. Tagged: android, catlog, developers, logcat. 10 comments

As of version 1.3.2, you can now start up the main CatLog activity using an external Intent, with parameters for filter text and log level.

For Android developers, the idea is that you can just put a switch in your app where, if some debug variable is enabled, you can press a button or access a menu item to start up CatLog and search for text related to your app. This should make it less painful to do debugging, in situations where you don’t have access to adb logcat.

I’ve written a simple demo app on GitHub to show how to use the new Intent. But the basic gist is that you want to paste something like this into your code:

Intent intent = new Intent("com.nolanlawson.logcat.intents.LAUNCH");

intent.putExtra("filter", myFilterText);
intent.putExtra("level", myLevelText);

startActivity(intent);

That’s it! Full documentation is below.

Intent

com.nolanlawson.logcat.intents.LAUNCH

Parameters

filter

Text to filter by. Case doesn’t matter, and you can search for either process ids, tags, or log text.

level

Log level to set CatLog to, case insensitive. One of:

E (error)
W (warn)
I (info)
D (debug)
V (verbose)
F (what a terrible failure)

22 Nov

CatLog is #1!

Posted by Nolan Lawson in Android. Tagged: alogcat, android, android market, catlog, logcat. 1 comment

My goal with CatLog was to write the best darned Logcat app for Android, and in that regard I think I succeeded. But as long as the adequate but inferior aLogcat was ahead in the search results for “logcat,” I felt like my work was incomplete. After all, most people will just download the first app in the list without trying any others. How can I really say that I’ve written the “best Logcat app for Android,” when it’s not most people’s first choice?

Starting sometime this month, though, it finally happened – CatLog now shows up first in a search for “logcat” on the Android Market:

I’m ecstatic that my app is finally getting the recognition I think it deserves, but, to be honest, I’m also kind of puzzled as to why it suddenly managed to nudge ahead of aLogcat. Comparing the Market statistics of the two apps side-by-side, it’s not clear what makes CatLog stand out:


	CatLog	aLogcat
Released:	Aug. 2010	Nov. 2009 (?)
Downloads:	10,000-50,000	100,000-500,000
Reviews:	587	1,683
Rating:	4.7	4.6
Updated:	August 14, 2011	March 6, 2011
Android Version:	1.5 and up	1.5 and up
Category:	Tools	Tools
Size:	323k	39k
Price:	Free	Free
Content Rating:	Everyone	Everyone

There doesn’t seem to be a big difference in the ratings (4.7 vs. 4.6), and aLogcat has a considerably higher number of downloads and reviews. So what changed? I think this blog post might provide a clue. It seems that, besides downloads and ratings, Google’s ranking algorithm also takes into consideration the retention rate of an app – i.e. how many users actually keep the app installed, as opposed to those who just download it.

It’s impossible for me to know what aLogcat’s retention rate is, because Google doesn’t make that information public. But I do know that CatLog has 40,834 downloads and 15,487 active users, which gives it a retention rate of 38%. This is the highest retention rate out of my most popular apps (30% for Chord Reader, 18% for Japanese Name Converter, and 20% for Pokédroid), so I’m guessing it’s also higher than whatever aLogcat has. Considering that aLogcat was released almost a year before CatLog, maybe it initially attracted a large user base that later started flocking to my app? Who knows.

Alternatively, it could be the fact that I’ve recently updated CatLog, whereas aLogcat hasn’t been updated since March of 2011. If that’s the case, then aLogcat could quickly regain the lead by just releasing an update. This seems unlikely, though, given that such a system would be easily gameable by just releasing a new update every day. As I noted in a previous post, those kinds of shenanigans made the “Just In” section of the Android Market practically useless, so Google eventually nipped that practice in the bud.

Whatever the reason, it’s nice to see that quality apps do eventually drift to the top. Similarly, I’ve watched one of my other apps, KeepScore, jump from 11th to 3rd in a search for “score keeper.” I’m hoping that, by just being the quiet valedictorian in the back of the class, it can eventually make it to the top. CatLog proves that that’s possible.

« Older Entries

Read the Tea Leaves Software and other dark arts, by Nolan Lawson