Archive for the ‘Programming’ Category
So, a word about the environment I’m going to use. I’m going to do my general hacking in Haskell on Ubuntu 12.04, running on either my MacBook Air or a Nokia Booklet 3G, depending on what’s closer. I’ll be using the haskell-platform installed from the stable debian repository. As I want to push things into a “production” instance, I’m going to be hosting things on Amazon’s EC2 — if I get serious about this project and want to share data, I’ll scale out on EC2.
Getting Started: Ubuntu on the MacBook Air
Installing Ubuntu Desktop i386 12.04 on my MacBook Air was a bit of a trial, and I didn’t document all the steps thoroughly. The general approach was to follow the steps at the Ubuntu site here, although one thing I did that gave me problems was to create a separate partition on the drive for the installation media, because doing that meant that I couldn’t repartition the disk when I did the install. Eventually, I was able to make a bootable USB stick, boot the USB stick using rEFit, format the new partitions, and I was set. Dual-booting between Ubuntu and Mac OS X using both the built in ssytem and rEFit works great.
Getting Started: Ubuntu on the Nokia Booklet 3G
I have a special place in my heart for the Nokia Booklet 3G, because the darned thing is indestructible, and sports a 3G modem for anywhere-wireless connectivity. Unfortunately, it feels about as slow as the VAX 11/70 I shared with sixty university students the week before finals, so it doesn’t get as much love from me as it otherwise might. But I prefer its keyboard over that on the MacBook Air, actually, and the pretty blue cover just looks sweet.
Earlier releases of Ubuntu on the Booklet were pesky to set up because of the stupid integrated graphics driver, but that’s been well-documented for some time. Here’s what I did to get things running:
- Use UNetBoot to create a bootable USB stick with Ubuntu Desktop i386 12.04.
- When booting from the stick, switch to the console using ctrl-alt-f1, log in, and
sudo lightdm restartto get past the black screen.
- Run the installer as normal.
- When the system reboots, switch to the console using ctrl-alt-f1, log in, and
sudo lightdm restartto get past the black screen.
- Fix the video issue permanently on boot by editing the grub configuration: change
GRUB_CMDLINE_LINUX_DEFAULTin /etc/default/grub to
"quiet console=tty1 acpi_backlight=vendor acpi_osi=Linux mem=896mb"and update grub using sudo update-grub.
- Fix a problem with the trackpad during standby by editing /etc/pm/config.d/gma500 and adding the line
Getting Started: Getting Haskell and Yesod on Ubuntu
Once you’ve got Ubuntu all set up and updated with the latest packages, open a shell and run:
sudo apt-get update
sudo apt-get install emacs23
sudo apt-get install haskell-platform
sudo apt-get install haskell-mode
$HOME/.cabal/bin to your path in your .bashrc or the shell configuration of choice, and reload the shell configuration file.
cabal install yesod-platform
to install yesod.
Getting Started: Getting Haskell and Yesod on EC2
I couldn’t get a base AMI instance running with a later version of the haskell platform — I chose an instance that was too small and cabal failed to build. Picking a bigger instance, I couldn’t get the Haskell platform to build with the version of ghc that I chose. I then moved to using a RedHat instance backed by ebs, and as root:
yum install gmp-devel
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm
yum install haskell-platform
To get Yesod running, as your development user:
cabal install yesod-platform
Next up: setting up my first site!
I’ve gotten some feedback on the kd-trees implementation for Qt — I was leaking the nodes as they were added to the tree.
I’ve fixed the code in the archive on the original post, and attached it again here.
If you’re using it, I apologize for the inconvenience!
I’ve been dabbling in Haskell since at least 2005, but I’ve never gained fluency. The reason is simple: I’ve never programmed anything of reasonable size in it. I use it a lot for little functional calculation programs, things like utilities for computing the great-circle distance between two points, stuff like that, but haven’t taken the plunge and done anything big enough to really push my understanding forward.
But that leaves me at loose ends; I spent some time working on a book project this winter that’s sort of stalled out (my fault, need to find a publisher and finish it), and looking to sink my teeth into Haskell again. And I think I’ve found the project. There’s a collection of documents from Quakers I want to collect and put in a database, first and foremost for me, but others may benefit as well. Slapping a web front end on the database makes for an easy UI to the database, and would scale well once I collect the data if I decide to share it with other people.
Enter Yesod, a Web framework written in Haskell. It looks to be stable and highly performant, and well-documented, too — some good Web tutorials, and even a book (which of course I’ve already ordered.)
So I’ve decided that rather than using a framework like I know a little about and knock together something in an evening, I’ll stretch a bit and do it in Yesod instead. Building a front end to the database for the data I want to collect in Yesod should get me past the baby steps of writing a few type classes and a bunch of simple functions, with the occasional higher-ordered function thrown in. And, once I get some data in the database, if I decide to open up the data to other users, it’ll need a more robust front end, which requires more bits of Haskell, I’m sure.
In the process, I’ll capture what I do on the project here. It doesn’t pertain to the last book I wrote, or the book before that, but hopefully it’ll be of use to others, and writing for an audience (no matter how small) always forces me to understand what I’m doing better than if I do it alone.
Expect a post or two a week, some weeks more, some weeks less, as time permits. I have some good chunks of time in the next couple weeks to get started, then I go on vacation for a bit — during which either I’ll post a lot less or a lot more, depending on how I feel about things.
I recently needed a spatial index for some data I was working with at work, and decided to rough out both a quad tree and a k-d tree implementation for Qt.
Indexing data spatially is important if you have a lot of objects organized spatially, and need to do queries by an object’s location, rather than a single-dimensional key such as an ID. Qt has a framework for doing this in the context of the QGraphicsFramework, but doesn’t have generalized container classes that index along multiple dimensions — think, for example, of a multi-dimensional QMap.
I implemented the quad tree largely from scratch, looking at wikipedia and a couple implementations such as the one here. My quad tree uses C++ templates to abstract the data type, so you can stick anything in it (as long as it can be put in a QList; the underlying store uses QList for object management). Quad trees come up a lot in collision detection in computer graphics and gaming, where you want to know if an object is close enough to another object to collide, but don’t want to search through lots of objects and do collision detection on each object. It works by recursively subdividing the plane containing the objects, performing additional subdivisions each time a region contains more than a specific number of objects.
I cheated when it came to the k-d tree, because there’s a great C implementation on code.google.com that did precisely what I needed. Because I needed to store QVariants in the k-d tree, I just wrapped the entire C library in a Qt class, making the C code static to my implementation. It’s not quite as sexy as a templated version, because it’ll only store objects you can wrap in a QVariant, but for my application it was enough, and in the future if I have a different kind of object I want to store, it’s easy enough to make it storable as a QVariant, too. k-d trees are more general than quad trees, operating in more dimensions than two as they subdivide the search space.
Neither of these implementations are truly “high performance” — for clarity and ease of debugging, they use Qt’s data structures under the hood, which are fairly performant but not as high performance as managing all the data out of the application’s heap using pointers alone. This is especially true for the
find methods, which return copies of the data stored in the trees rather than pointers to the actual data. This is, I believe, correct in my case, where I don’t want to expose the internal structure of my data, and where I’m using Qt classes that use references and copy-on-write semantics anyway. However, if you’re looking to squeeze every last bit of performance from your spatial index, you probably don’t want to start with the code you’ll find here. Nonetheless, for general computational tasks, these should serve you quite well.
You can find both of the implementations and a little standard I/O app for using them here.
What do you do if you think you’ve correctly connected a signal to a slot and yet your slot’s not being fired? I ran into that Friday at 5:30pm, just about the time I hoped I’d call it a day and go home. Fortunately in my case, I was simply passing
null to connect, and I got a message in the debugging log about trying to connect a signal from a null object. (Unfortunately for me, I didn’t see the log right away, because of all the other stuff I was dumping to the log.)
If you’re in the same boat, what do you do to debug the situation? Here’s a bunch of things to try:
- Check the compile log for error messages about undefined signals and slots.
- Check the run-time log for errors on connection. If need be, ensure that connect succeeds by testing its return value (it should return
- Check to make sure that the connect code is reached, that the emit code is reached, and the slot code. (Odds are one of them isn’t.)
- Make sure you declare the connection like this:
connect(sender, SIGNAL(someSignal(type)), receiver, SLOT(received(type)))
The signal and slot specifications should have the argument types, but not the arguments themselves. Moreover, you can omit
constand reference specifications in the arguments; the meta-object compiler strips them anyway.
- Check the parameters of the signal and slot and ensure that they match precisely
- Check to be sure that you’ve correctly declared the
slotsportions of your header correctly.
- For that matter, be sure that your sender and receiver both inherit from
QObject, and that you have
Q_OBJECTdeclared in your class definition. (Remember, you need to do both.)
- If you’ve forgotten part of the
QObjectdeclaration in your header, re-run qmake and rebuild.
- Make sure you make the connection with
connectbefore you invoke any functions that fire the signal. Signals may fire synchronously.
- Make sure you’re not disconnecting the signal anywhere with
Usually, the problem’s pretty easy to track down, especially if you check the log and the signal and slot declarations closely. A common mistake is to try to wire a signal to a slot that’s not been declared a slot, too, so check your headers closely!
I don’t usually see my writing and blogging as an avenue for my political beliefs and interests, but like many, I feel a need to speak out against SOPA and PIPA. This blog and the other content at lothlorien.com will be unavailable during tomorrow’s “Strike against SOPA”.
I apologize for any inconvenience this may cause.
Whew! Some things seem to take forever for me to figure out!
Here’s the scenario: we’re working on a Qt Quick (QML and C++) application at the lab to run on Microsoft Windows. We’d like to deploy the app to a bunch of study participants to use for a week or so. It’s obviously not practical for me to go around and set everybody up with Qt, Qt Mobility, and build our app. (Especially not this app; we’re using the Qt 4.8 prerelease at this point!). So we need to build a Windows installer that includes the Qt DLLs, our app and its QML, and the Qt Mobility plugins we’re using. We’re using Qt Mobility in our QML — so we need to include the Qt Mobility plugins for Qt Quick, not just the Qt Mobility libraries.
Packaging a Qt application for Windows is straightforward — just follow these instructions for packaging as a shared library. But what about the Qt Quick plugins for Qt Mobility we need?
Well, Qt Quick supports modules for that, as described here. In fact, if your application required QtWebKit’s Qt Quick plugin, you’d create a QtWebKit directory, stick a
qmldir file with the line
qtwebkitplugin.dll into the directory you made and Bob’s your uncle!
We use QtMultimediaKit, so at first I created a
QtMultimediaKit directory, created a
qmldir file in that directory, and copied
QtMultimediaKit1.dll from my Qt Mobility build output directory to the
QtMultimediaKit directory I’d created.
No luck — in fact, the app just launches and shows nothing. (To add insult to injury, my app uses Qt’s support for Open GL, so not only did I not get the black screen characteristic of a QML error, but I got junk in the application window and no errors on the console.)
Turns out that
QtMultimediaKit1.dll goes in the same directory as the application, and the declarative plugin file is somewhere else! If you look in the
plugins directory, you’ll see a
declarative directory, and the actual Qt Quick declarative plugin you’re looking for is in there! In my case, I needed to copy
plugins/declarative/multimedia/release/declarative_multimedia.dll to a new directory
QtMultimediaKit, create a single
qmldir file that read
and put that adjacent to my application executable.
So, the resulting files and directories for me look something like:
Now off to learn how to make a Windows installer. I think I’ll try Inno Setup.
So, I’ll happily admit to anyone that asks that I’m terrible about self-promotion; if anything, this blog is an extension of my books: writing about what matters to me for you, hoping to find a match. I don’t pick topics because they sell, or because I think they have wide appeal; I’ve always been drawn to writing about problems that I’ve encountered that I think others may encounter as well.
That’s why I found this chart from StatPress so interesting tonight — it’s a look at the most commonly read entries on this blog.
I love Lisp — no denying it. And I’ve spent some time plinking away at using Lisp on various machines; my time spent coercing CLIM to work on Mac OS X is clear testimony to that fact. Unfortunately, I’ve never been able to turn my affection for Lisp into a paying concern; in fact, I’ve barely been able to integrate it with my day job (this paper is as close as it gets, folks!) because my primary career focus is mobile, not any of the problems Lisp typically targets.
Unfortunately, given the amount of time I have for coding, Qt wins out over Lisp and CLIM, or even plain Lisp; the day job is currently focused on research on mobile UIs at Nokia, and Qt (or Windows Phone 7, which I’m just getting started with on my own time) is where it’s at.
I’ll admit some frustration, though — I’d like to be of more use to the community that finds these pages useful, especially because I think it’d be more fun sometimes than just quickly knocking out a few pages about problems I’ve solved on the day job for other Qt developers to refer to. So, open questions to all you people who got here looking for Lisp on Mac OS X: what else can I help you with? Where are your pain points? And what the heck are you using Lisp on Mac OS X for, anyway?
Two more for you!
Like Qt’s copy-on-write shared data semantics? Want to do the same for your own class? It’s easier than you think. Simply inherit from
QSharedData, which provides the necessary reference-counting semantics. See here for more info.
Running a Slot on a Different Thread
Sometimes you want to schedule a single operation on a different thread; you can do this using
QObject::moveToThread to move an object’s thread affinity to a different thread, and then trigger the slot as a queued connection. See here for a discussion of how Qt’s threads and Qt’s object system cooperate.
Following on the heels of last week’s post, here are a few more thoughts gleaned from the experience of attending last year’s Qt Developer Days. These are all old news, especially if you’ve been doing Qt for a while, but putting them here may be useful to you, and clears out a page of thoughts in my engineering notes!
QImage is the general-purpose class for image manipulation whose data lives in main memory, while
QPixmap objects are usually backed by the GPU. This has some ramifications for your application:
QImageis optimized for I/O and direct pixel access.
QPixmapis optimized for direct screen display.
- On constrained platforms like mobile devices, there may be relatively little GPU RAM available, so you should consider using
Regardless of which you choose, both classes use Qt’s copy-on-write (COW) semantics, so if you have multiple instances of the same image, you pay a small price penalty—basically the cost of the object overhead, not the memory footprint of the image itself.
QStandardItemModel and Performance
Qt’s support for model-view-controller is great, and one of the handiest things about it is that the
QStandardItemModel class provides everything you need in a model to get going. It is, however, not necessarily the most efficient way to do so.
If you have your own model already—say you’re porting an existing application, or working with a library with its own data representation—it really is a good idea to implement your own model class from
QAbstractItemModel. I’m not bad-mouthing
QStandardItemModel, but it’s based on Qt’s collection classes, and odds are that you know better ways to store your data (especially if there’s a lot of it!) than Qt does.
Personally, my approach is that if I don’t have a data container and I’m only dealing with a handful of data items — say, up to a hundred or so objects with a half-dozen roles or so — I use
QStandardItemModel and don’t look back. (Qt’s collection classes are, after all, quite good for general applications). But if I’ve already got a data model, or I know I’m going to be dealing with lots and lots of objects, it’s time for a custom model that leverages my knowledge of the data set I’m wrapping.
QtConcurrent for Parallel Computation
I’ve not tried this personally, but
QtConcurrent looks really cool for doing any kind of concurrent map/reduce style programming. If you need to write code that leverages multiple cores for concurrent computation, it’s best to look here before trying to roll your own framework using