Colorizing adb output

Although I quite like most of the IDE functionality Eclipse has to offer, there’s one thing I positively hate about it: under linux the UI elements are HUGE, leaving less room for the code you’re writing. The things that bother me most are the editor-tabs (they’re mostly too high for my taste) and the status bar at the bottom (why is that thing even there?).

I haven’t yet figured out how to fix the above things, but I did find a nice solution for another thing that crept up recently: the logcat output window. The ratio between UI elements and actual content is way off here, especially since there’s no way to hide the search bar and icons:

But log output is important: you really want to know what’s going on when you’re running/debugging your program. Since I had an old PowerBook G4 lying around I figured I look into running adb logcat from an ssh shell and use that as a replacement for the Eclipse window. Turns out that the readability of logcat’s output stinks. Without resorting to *nix trickery it won’t even let you filter on package (TAGs are not packages!!)!

So how to deal with this dire situation? Apparently Jeff Sharkey was bothered by the same thing a while back and wrote a nice Python script that colorizes the logcat output (thanks Jeff!). Combine that with a little grep and you’ve got a nice logging setup:
adb logcat | grep your.package.name | ~/coloredlogcat.py

Update:
While using the above command, I noticed some irregularities: entries not appearing on the console where they should be. After some digging (“tee” for the win!) I found that this was due to the lack of line buffering in grep. Which is easily fixed once you know what to look for:
adb logcat | grep --line-buffered your.package.name | ~/coloredlogcat.py

BoundedMapView – A MapView with limits

In the source I attached to my previous post I included another class I recently created which is based on a patch from Marc Kurtz and Zoran Nikolic (see here for details). Actually, it’s more than just based on their code: I copied quite a bit of their work, so credits where credit’s due.

Anyway, essentially the class allows you to set limits to the area that can be viewed. This is useful if you supply an offline map database that only contains a specific area, a city for example. The class itself is called BoundedMapView.java. A class that illustrates the usage is below the break (for information on MBTileProvider see the previous post as well).
Continue reading

Using MBTiles in osmdroid

A couple of days ago I posted an example of getting osmdroid to work with an offline MBTiles database. The example is quite elaborate, mostly because 80% of it is comment, not code. I found three issues with this:

  1. The amount of text between the lines of actual code is so big, that it makes it hard to follow, but …
  2. This is necessary because the steps don’t really speak for themselves and …
  3. Even if you remove all the comments, you still end up with more lines of code than you should

So I thought I’d remedy this by extending the framework. As a result I updated RouteMapActivity.java and created three new classes:

  1. MBTileProvider.java
  2. MBTileModuleProvider.java
  3. MBTileSource.java

Below you’ll find the code to these classes. Please note that you have to make sure that they are put in the correct package for things to work. You can also download the sources.
Continue reading

Integrating osmdroid part 2 – Using MBTiles as offline data source

Yesterday I wrote an article about how to integrate the osmdroid library into an Eclipse project. You may have noticed I didn’t mention anything about actually creating an Activity that actually does something with the freshly installed library.

Let’s change that :-). Below you’ll find a heavily commented example on how to instantiate a MapView using an offline MBTiles map source. Since creating an mbtiles file is also a project on its own, I’ll see if I can write something up about that process soon.

Questions? Let me know!

package com.example.yourproject;

import java.io.File;

import org.osmdroid.DefaultResourceProxyImpl;
import org.osmdroid.ResourceProxy;
import org.osmdroid.tileprovider.MapTileProviderArray;
import org.osmdroid.tileprovider.modules.IArchiveFile;
import org.osmdroid.tileprovider.modules.MBTilesFileArchive;
import org.osmdroid.tileprovider.modules.MapTileFileArchiveProvider;
import org.osmdroid.tileprovider.modules.MapTileModuleProviderBase;
import org.osmdroid.tileprovider.tilesource.XYTileSource;
import org.osmdroid.tileprovider.util.SimpleRegisterReceiver;
import org.osmdroid.views.MapController;
import org.osmdroid.views.MapView;

import android.app.Activity;
import android.os.Bundle;
import android.os.Environment;
import android.view.Menu;

public class RouteMapActivity extends Activity {

  @SuppressWarnings("unused")
  private static final String TAG = "RouteMapActivity";

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    /**
    * This whole thing revolves around instantiating a MapView class, way,
    * way below. And MapView requires a ResourceProxy. Who are we to deny
    * its needs? Let's create one!
    *
    * It would have been nice if this was taken care of in the MapView
    * constructor. Interestingly MapView *has* a constructor that creates a
    * new DefaultResourceProxyImpl but unfortunately that one doesn't allow
    * us to specify the parameters we *do* need to set ...
    */
    DefaultResourceProxyImpl resProxy;
    resProxy = new DefaultResourceProxyImpl(this.getApplicationContext());

    /**
    * A class that implements the ITileSource interface knows how to
    * convert an InputStream or a file path into a Drawable. It doesn't do
    * much more than that. The real 'sourcery' is performed by
    * MapTileFileArchiveProvider which will be introduced shortly.
    *
    * What we need is really a BitmapTileSourceBase instance, but this
    * class is defined as abstract. XYTileSource is not and comes closest
    * to what we want.
    *
    * Comment: I don't quite get why BitmapTileSource base is abstract; it
    * doesn't contain any abstract methods.
    */
    XYTileSource tSource;
    tSource = new XYTileSource("mbtiles",
    ResourceProxy.string.offline_mode,
    8, 15, 256, ".png", "http://who.cares/");

    /**
    * Don't think the name SimpleRegisterReceiver is particularly well
    * chosen. SimpleReceiverRegistrar would have been better because the
    * only thing SimpleRegisterReceiver does, is wrap the methods
    * Context.registerReceiver(..) and Context.unregisterReceiver(..). Have
    * a look at the source if you don't believe me ;-).
    *
    * So why does it exist then?? Don't know, but it's quite possible to
    * just ignore this step and state the Activity implements
    * IRegisterReceiver and replace the 'simpleReceiver' variable with
    * 'this' further down (no additional implementation required).
    */
    SimpleRegisterReceiver sr = new SimpleRegisterReceiver(this);

    /**
    * The following looks complicated, but really only creates an
    * iArchiveFile[]. Apparently Marc Kurtz and Nicolas Gramlich, the
    * authors of MapTileFileArchiveProvider, figured it might be useful to
    * support multiple files/sources. I guess that might make sense if
    * you're providing separate files for, for example, cities.
    *
    * They also provided quite a bit of logic in MapTileFileArchiveProvider
    * for handling SD Card inserts and ejects. Additionally, if files are
    * not explicitly specified they can be dynamically loaded from the
    * /mnt/sdcard/osmdroid directory, which is a nice feature.
    */
    String packageDir = "/com.example.yourproject";
    String p = Environment.getExternalStorageDirectory() + packageDir;
    File f = new File(p, "TileDatabase.mbtiles");
    IArchiveFile[] files = { MBTilesFileArchive.getDatabaseFileArchive(f) };

    MapTileModuleProviderBase moduleProvider;
    moduleProvider = new MapTileFileArchiveProvider(sr, tSource, files);

    /**
    * So at this point we have a MapTileModuleProvider that provides
    * MapTileModules: a MapTileModule looks at one or more sources, which
    * are *not* ITileSources but IArchiveFiles and provides MapTiles. What,
    * then, does MapTileProviderArray do? Well, it just adds another layer
    * to the complexity cake: this makes it possible to set multiple
    * MapTileModuleProviders, such as an on- and offline source. I'm sure
    * it's useful for someone, but for simple applications it's probably
    * too much.
    */
    MapTileModuleProviderBase[] pBaseArray;
    pBaseArray = new MapTileModuleProviderBase[] { moduleProvider };

    MapTileProviderArray provider;
    provider = new MapTileProviderArray(tSource, null, pBaseArray);

    /**
    * Are we there yet??? Create the MapView already!
    */
    MapView mapView = new MapView(this, 256, resProxy, provider);
    mapView.setBuiltInZoomControls(true);

    // Zoom in and go to Amsterdam
    MapController controller = mapView.getController();
    controller.setZoom(12);
    controller.animateTo(new LatLonPoint(52.373444, 4.892229));

    // Set the MapView as the root View for this Activity; done!
    setContentView(mapView);

  }

  @Override
  public boolean onCreateOptionsMenu(Menu menu) {
    getMenuInflater().inflate(R.menu.activity_route_map, menu);
    return true;
  }

}

Integrating osmdroid into your Android app

The library osmdroid is a great replacement for your standard Google Maps. Now why’d you wanna go and replace Google Maps? Well, until recently Google didn’t support offline usage of their data. Most of the time this isn’t an issue, but if you want to take your (m)app abroad and avoid heavy roaming charges, you’re looking at offline maps. The good news is that osmdroid supports this. Even better is dat there’s a data type that can be used both by osmdroid and the iOS map-framework route-me: that format is called MBTiles. So if you’re building a cross-platform app, there are still some benefits in using the library over Google’s upcoming offline maps support. Also, one of the nice things about osmdroid is that it uses pretty much the same API as Google’s MapView, which, in theory, should make it a drop-in replacement. As with any library, however, there are always a few bumps in getting things to work. In the remainder of this post, I’ll detail the steps necessary for getting up and running in no time!

To source, or not to source …

Basically there are two ways of using the framework: you can choose to include the pre-built .jar which can be obtained from the downloads section) or you can choose to include the library from source.

Whichever route you choose, osmdroid depends on SLF4J (Simple Logging Facade for Java) which can be found here. As they state on their website: just add the file to the build path. The easiest way to accomplish this is by finding the “libs” directory in your Eclipse project (you are using Eclipse, right!?) and copying the file there. Refresh the directory contents and you should be good to go!

The same applies if you’re using the osmdroid jar: just copy it to the “libs” directory and refresh.

However, if you’re like me and like to be able to browse the sources while using the library, you’ll want to import osmdroid as a separate project and refer to it from your main project. Considering the fact that the documentation for this, otherwise excellent, library is quite terse, I highly recommend this option. This wiki page is the original source for how to install the sources under Eclipse, but you’ll probably find these steps a bit easier to follow:

Compiling osmdroid from source requires a few additional libraries. These are:

  • android.jar (part of the Android SDK, we’ll get to this in a second)
  • httpmime.jar (in debian this file is part of the package libhttpmime-java and located in /usr/share/java)
  • slf4j-android-1.5.8.jar (you should have already downloaded this :-))

Ensure that you have found the above dependencies and proceed to download the sources via subversion:

http://osmdroid.googlecode.com/svn/trunk/ osmdroid

This will create a new directory osmdroid in your current directory. Switch to Eclipse and right-click in the package explorer and select “Import…” –> “Existing files into workspace”. Select the directory you just downloaded the sources to as “root directory” and make sure that you only tick the “osmdroid-android” project.

As you’ll probably have noticed, Eclipse has identified quite a number of unresolved dependencies. Let’s go fix that. First add the variable ANDROID_SDK_PLATFORM to your classpath by selecting:

Window --> preferences --> Java --> Build path --> Classpath variables

Once you’ve added a new variable, the result should look similar like the image below (the path should, obviously be changed to match your installation).

We’ve already taken the first step in resolving them, but there’s a few additional things to do. Right click the project in the package explorer and select

Build path --> Configure build path --> Libraries

Here add “httpmime.jar” and “slf4j-android-1.5.8.jar” via “Add External JARs” and “android.jar” via “Add Variable” (select ANDROID_SDK_PLATFORM which you defined earlier and click “Extend” to select the jar). Then finally add JUnit4 via the “Add library” button. The result should look similar to this:

If you’ve followed these steps, you should now be able to compile the library, but I suppose that doesn’t help you much. So go back to your Android project and right click it from the package explorer. Here too navigate to the “Java Build Path” but this time select the “Projects” tab. Here you can add osmdroid-android as a required project. Doing so will allow you to refer to the lib’s classes from your own code.

Still, if you don’t want your app to crash on load, there is one final click to be made: check the box in front of osmdroid-android in the “Order and Export” tab and you’re done!!

Up & running!

Well, it took almost an entire day, but I finally have my linux (debian) box, including a fully functional Android SDK and Eclipse environment, up and running again. Can’t believe how much easier configuring X and compiz has become since, what, 2002? Yaiks, 12 years!?

For some reason compiling a new kernel gave me more trouble than expected. Apart from the fact that a single compile takes quite a bit of time, the 3.2 kernel (the one that ships with Ubuntu) wouldn’t boot. Not sure what the reason was. Backtracked and got 2.6.32 working, which enabled compilation of the NVIDIA kernel modules. Then tried 3.4.7 with the config file from quantal quetzal. This worked fine, but it turns out the NVIDIA drivers don’t agree with the new Nouveau driver (which I didn’t even know existed).

Btw, why does make dep-pkg create a package over 500MB big?? Seems rather excessive for a kernel image. Really should to see what the “old” way (make-kpkg) leads to, but let’s save that for tomorrow.

Anyway, after following the steps on this page I could also launch Android projects on my HTC Desire. Huzzah! Ow, and many thanks to Heiko Behrens for the emulator skins!

But off to bed now! Got a fresh day of programming ahead tomorrow!

DB Versioning (cont’d)

Well, the articles I found seemed straightforward enough, but things get a bit tricky when you introduce relationships between tables. The article on codeproject does mention how to handle foreign keys: create a unique identifier per ‘object’ that you want to version (they call it the ‘PermanentRecordId’). This identifier cannot be used as a proper foreign key, since the Audit (or version) table contains multiple rows sharing the same key.

But of course the above would not keep us from using it as one in a JOIN construct.  However, a side effect is that if two tables are teamed up in such a way, they are versioned independently from each other. This wouldn’t pose a problem for the specific example that the article gives, but could pose one in different circumstances.

I modified SQLAlchemy to support versioned tables. Took a bit of work, but managed to get a number of basic tests working. Didn’t see one thing coming, though. Let me illustrate in (pseude)code:

> versioned_object = VersionedObject(attr='hello')
> VersionedObject.query().all()
[<VersionedObject@0x01>]
> versioned_object.attr = 'goodbye'
> VersionedObject.query().all()
[<VersionedObject@0x01>, <VersionedObject@0x02>]
> versioned_object
<VersionedObject@0x01>;

In words: once you modify an attribute and commit to the DB your ORM object (instance) will be unusable (since SQLAlchemy uses the PrimaryKey to identify the thing in the DB).

Argh!

I suppose that if you modify an object only once after retrieving it from the DB –which might be a frequent occurrence for webservices– the thing is still somewhat useful.