GSoC Wrap-Up Review

Hello!

This year’s summer has just come to and end, and so has the Summer of Code. It’s time to go over what I did while working on a performance testing framework for DUNE.

First, I wrote a Python program that measures the run time and resource consumption of an external program. It stores quite a lot of data, but the most useful are definitely time spent, top memory consumption, and computer parameters such as number of CPUs. The tool can then output this data into a temporary log file, store into a sqlite database, or upload it to a central server. Furthermore, it can generate nice graphical representations of the data in the form of HTML pages with javascript graphs. It is written in a very modular way, so while there is a script to tie it all together, each of the described actions can be done separately. This also minimizes external dependencies, so if the user doesn’t have the Python SQLite module installed, the database part is skipped.

Then, with help from my mentors, I tied the measurement tool into the DUNE build system, or rather, both of DUNE’s currently available build systems, autotools and cmake. This allows you to set minimal configuration, and then just run “make perftest” from the build directory, and all performance tests are performed, measured, stored, uplodaded and visualized. By using the build system directly, we can get information about the compiler used and its flags. This is important if you ever want to compare compilation with different compiler options. When run this way, the tool separately measures both the compilation time and run time. The compilation time may get quite long with a lot of templates, unnecessary includes and in general large code files, so such a test comes in handy for identifying compilation bottlenecks.

For displaying the results, I used Twitter Bootstrap, Dygraphs and Table.js, so the generated pages look quite nice. Graphs are interactive, and some table columns can be filtered for easier browsing. Some examples are shown here.

Results of a single text run

Graphical representation of results of repeated runs of the same test

Finally, I added a server component, implemented as a number of CGI scripts in Python. One of these endpoints receives uploaded log files, while another stores them into a ‘proper’, PostgreSQL database. This two-step process is used, so that processing can be done in batches and separately from uploading. For example, one could easily upload the data in some other way, like the secure copying with SSH. The current uploading setup is not completely open, as it requires a username and password, but these are stored in plaintext on the server.

With a server that accepts data from multiple computers, I could add some additional views. For example, there is a page that identifies outliers with adjustable tolerance. Outliers are data points with considerable deviation from the mean, which in our cases means unusually long run or compile times.

Server overview of all collected data

Results of a single run on a server

Aggregated results of a single test on a server

A page for finding outliers on a server

All in all, I would say my summer project was a success. It started a little slow, at first with a two-week pause because of a summer school I attended, and then when had to finish my master’s thesis a month before I expected. However, despite not always following the set schedule, I tried really hard to complete everything I set out in the timeline. This project had several different components, from the DUNE libraries in C++, the two build systems, Python, database programming and websites, so I had to learn some new things over the summer. For this I am grateful, and I would like to thank the DUNE developers again for giving me this chance.

I’m sorry I can’t attend the DUNE developer meeting this week, even though the developers invited me and even offered to cover my expenses. I’m giving a talk on the optics of liquid crystals at a conference in Kranjska Gora that happens to take place the exact same three days. However, I can say I enjoyed working on this project, and can only hope that my contributions will help others.

Thank you!

Typing with the enemy

I ordered a new keyboard thin week, and today it finally arrived. But even though I’m a firm believer it free software, I bought the Natural Ergonomic 4000, made by Microsoft. It is about half the price of any comparable Logitech product available in Slovenia, despite it costing 50€. In comparison, the American Amazon sells it at 27$. This disparity (which is noticeable everywhere, but especially with tech products) made even more happy about Spark coming out first in Europe.

First thing I noticed, this thing is monstrous. My previous keyboard is rather old, so it’s mechanical and larger than newer keyboards, but in pales in comparison with MS’s product. The desk I use at home has a keyboard drawer, and it fits in perfectly, both in width and height (The keyboard has a large and elevated wrist-rest). My second thought, after trying to type on it, is that it’s going to take some time getting used to it. Having been using the Dvorak layout for several years, I’m used to touch typing, but I’ve never tried a curvy ergonomic keyboard before. It’s quite different, but I haven’t used it enough to formulate any king of informed opinion.

And as to apologize for buying from Microsoft, I feel a comparison between our favourite evil empires is in order. In Slovenia, 50€ will get you this mighty keyboard with split keys, a soft wrist-rest, all kinds of shortcut button and a scroller between the key sets. On the other hand, this costs 75€:

I know that both are cheaper in USA, and even some places in Europe, but the ratio of their official recommended prices is similar. And still I have no doubt many people will claim how typing anything on the Apple product is so much easier and more comfortable. Sorry for turning this post into a rant, but it got me quite angry while shopping.

Ok, writing this post on the new toy went better than expected, now I’ll get back to homework coding. Yes, I have to write code for school as well, but it also involves less interesting parts, like writing sentences. Ugh.

Zabava ob izidu KDE 4.8

Po enoletnem premoru se tradicionalno druženje ob izidu proste programske opreme skupnosti KDE spet vrača. Ob izidu različice 4.8 si bomo najprej ogledali, kaj se je v zadnjih dveh letih dogajalo v drugi največji odprtokodni skupnosti in predstavili nekaj največjih novosti namizja Plasma in ostalih programov. Sem spada tudi posebno novo okolje Plasma Active, ki bo letos na voljo na tabličnih računalnikih. Predstavili bomo tudi nekaj izmed načinov, kako se lahko svojimi prispevki in izkušnjami pridružite skupnosti KDE. Osredotočili se bomo predvsem na prevajanje in programiranje.
Po predstavitvah sledi druženje s pijačo in kakim prigrizkom. Po načelu »kdor prvi pride, prvi melje« bomo razdelili tudi nekaj daril.

Predavatelji:

• Jure Repinc je študent računalništva, ki v svojem času poskuša čimveč prispevati k raznim odprto-kodnim projektom, v največji meri pri KDE.
• Andrej Vernekar je zaposlen kot vodja pisanja tehnične dokumentacije, v prostem času pa prevaja KDE-jevo programsko opremo in je koordinator slovenjenja KDE ter predsednik društva LUGOS.
• Miha Čančula sem študent fizike, ki pri projektu KDE razvijam šahovski program Knights in Cantor, ki je vmesnik za matematične programe

Lokacija:

Kiberpipa (zemljevid)

Povezave:

http://community.kde.org/Promo/Events/Release_Parties/4.8#Ljubljana

English version

Ice Cream Sandwich on my Galaxy Ace

Today, I installed the first beta of CyanogenMod 9 (based on ICS) on my phone, beating the official distribution channels for an unknown but long and possibly infinite amount of time.

Why

The coolness factor, of course. I’m quite sure I’m the first of all my friends to have ICS. Also, it looks very slick. It is also heavily hyped, so I had to try it out.

CM9 welcome screen, still fresh and empty

The user interface is about as responsive as with 2.3, which is not perfect, but good enough. It’s actually quite fast, but the animations are not smooth, damaging the polished feel of the UX.

There is also the fact that the app development API has got many additions since Android 3.0, so it’s easier and more portable to write apps for ICS than for 2.3 that was installed by default.

CM9 app drawer.

How

There is a thread on the XDA-developers forum with news and releases. I downloaded the zip and installed it from ClockWorkMod recovery. A complete data wipe is needed, so make sure that your Google account is fully integrated and synced before trying the installation, or backup everything.

As the first post warns, if you don’t know how to install it, you probably shouldn’t. So I won’t post complete instructions here, but feel free to ask in the comments.

What now

The camera still doesn’t work. If this is something important to you, don’t use CM9 on your phone. For me, it’s an inconvenience, but the development possibilities are more important for me.

What you may or may not notice on the previous picture is that the icons are quite small on SGA’s 3.5-inch screen. The same is true for text in menus, fortunately there is an option to enlarge it in Settings > Accessibility.

ICS accessibility config, with the Large Text option checked

In general, it shows in a couple of places that my phone is somewhat too small and the screen resolution too low. Icons are too small, while text in some applications is too large. Other than that, web browsing and email is usable, and I don’t really need anything else from a phone. It also makes calls and messages, I tested it myself.

AppMenu is here

Qt 4.8 was released last week. The announcement lists Lighthouse integration (aka. QPA) and some optimizations, but nobody mentioned my most-wanted feature yet: Aurelien Gateau’s patch that enables exporting the application menus. On a laptop, especially one of those wide-screen ones like mine, vertical space is very important, so removing one toolbar while keeping all of its actions easily accessible is a killer feature. Ubuntu users had this for some time, but now it has made it way to other distributions as well.

Currently, there are two things you can do with the menu once you rip it out of the application: put it in a plasma panel, or embed it the window decoration. Arch users can get both of them from the AUR. Because I still like the menu to be visually connected to the application, I am now using the second option, with the Oxygen-appmenu window decorations.

Application menus embedded in the window decoration

Currently, this only works for Qt applications. The equivalent patch to Gtk is still included only in Ubuntu and would require recompiling Gtk, something I am not fond of doing. In any case, I have found that at least on the laptop the only non-Qt app I use is Firefox, which includes a similar button as its menu. It doesn’t look as well, but it does the job and keep things consistent. An extension for LibreOffice is avaliable in the AUR as well, but the only text processor I need is Kile.

Gaming again

My summer job is now done, and I have just adjusted to the school rhythm. My girlfriend is studying physics and pharmacy in parallel, so I’m often alone at home with not much to do. I spend some of this time programming Knights or nTasks, but I find it hard to program when I’m tired, and I need breaks even when I’m not. So I started playing computer games again.

My current addiction is Sins of a Solar Empire. It’s a space-based real time strategy, with its characteristic property being its slowness. The game is very detailed with epic battles, so even with the slow pace it keeps you busy, but it’s quite possible to wage two or more battles at one.

The game is rather old, and it wasn’t one of the big-budget productions, so it’s relatively cheap, but both the graphics and the gameplay are really well done.

CyanogenMod

What’s the point of an open-source mobile OS if you can’t change it?

So, yesterday I finally managed to install CyanogenMod. It’s not yet officially released for my Galaxy Ace, but there is a good unofficial build on the xda-developers forum. The process is really easy, but due to it being unofficial the instructions are somewhat hard to find, and that’s why it took me a long time.

As an operating system, it’s not much different than original Android. However, the appearance is heavily modified, with theming support for colors and icons, a better App list, and some really cool wallpapers. A small but important improvement is the lack of pre-installed apps that come with the phone. Additionally, CM add a plethora of configuration options, so you can really make your phone look unique.

Taking screenshots is easier, too

Getting into Web Development

I’ve always been a desktop guy, preferring to use desktop applications over web ones wherever possible. And since I started programming, I only wrote desktop applications with Qt and KDE. Maybe it was the whole paradigm, or just a lucky choice of language and toolkit, but I liked it immediately and soon learned to feel at home with programming. I got used to C++, it’s strictness and plenty of compile-time error checking.

It was only recently that I decided to broaden my horizons. I got a smartphone, I’m using both a desktop computer and a laptop, so I’m starting to find it easier to store non-sensitive data in the cloud. I’ve had GMail for a long time, WordPress as well, and even used Facebook until I deleted my profile because it was a waste of time. This movement of course reflected on my programming as well. After writing Opeke and Knights (C++/KDE), contributing to Orange (Python/Qt), I have decided it’s time to learn something new again.

Platform Choices

The first problem I encountered was the choice of platform. On the desktop, everything is free (at least as in beer), no matter if you choose Windows, Qt, Gtk, or something else. You  write a program, other people compile and run it. After you’ve written the code, that’s it. On the web, everything is different: you (as the author/developer) have to run the application on your own infrastructure, and more importantly you also have to cover the cost. Fortunately, there are a couple of options that do it easily and cheaply. There’s probably more, but these three are the most used ones:

Amazon Elastic Cloud

Image via CrunchBase

It lets you run any OS, but you have to manage the entire software stack yourself. This options grants you the most freedom, and it’s easy to migrate you application from here to a dedicated host and back. This is in my opinion the best option, if it weren’t for the price: the smallest package (micro instance) costs around 15$ a month, even if you don’t use it very much.

There is a free tier that lets you run the 15$ package for free, but it only lasts for a year. I’ve tried it with my http://noughmad.eu website, and it’s noticeable slow even with a small number of visitors (I had around 100 per day). I understand pricing gets better with volume, so as long as you’re prepared to use your own computer or another dedicated host while learning, writing and testing your app, it could be useful when your app is ready for the public. Yet again, because of it costliness, it’s not really appropriate for non-commercial fun projects.

Google AppEngine

Image via CrunchBase

Unlike the Amazon cloud, Google provides a complete managed software stack. This means less work and worries for you, but it comes at a heavy price: Only Java, Python and Go are supported. The runtime environment (especially the database) is designed with scalability in mind, rather than compatibility with existing systems, so porting an app to and from GAE is not trivial.

Still, it’s possible to develop apps that will be compatible with the App Engine and still work without it. For Python, this can be done with Django and django-appengine. For Java, you can use GWT for presentation and JDO for database objects, both of which can work on GAE and on your own server. However, you always have to keep in mind the restrictions Google’s datastore imposes on your data query to ensure scalability.

Windows Azure

It comes from Microsoft, so I’m not touching it. I don’t even know what it offers, but from what I could tell it’s not free, which means bad news for a hobby programmer.

My Choice

I chose to start with Google App Engine. The main reason was cost: It’s free for small apps permanently. Out of the three supported languages I only know Python, so I used that, combined with the power and comfort of Django. There is a large number of modules for it, including Dajax (asynchronous requests) and a REST framework (to easily create an API).

Appearance

Another important difference between desktop and web programming is application visual appearance. On the desktop, the toolkit will guarantee a consistent look and feel with other programs, so you don’t have to think about how your buttons look. You just create a button, and users will see it will all sorts of decorations and text effects, and it will be just like all the buttons in all other programs.

On the web, the default appearance with most browsers looks like horrible. You have to specify the style of every single element of the page, setting colors, borders and layout. The separation between content and appearance is nice, but it would be even nicer if I didn’t have to worry about the appearance at all.

My First Web Application

For the learning experience, I decided on a task management tool. You can find it at http://ntasks.appspot.com. It’s not much yet, and most certainly it doesn’t look like much, but I’m starting to like Django so maybe it’s going to progress faster in the future.

A very short guide to linux distributions

Choose Arch.

Changes in my life

I didn’t do much for KDE this summer, even after GSoC was over, despite my best intentions. The time was instead spent for major changes I made. I hope they will all pay off in that I’ll be more productive on the future.

The real

Maja and I finally found a place to rent, so we’ll be living together now. The flat is rather small, only 30 square meters, but it’s nice and we managed to decorate it really well. We’ve been together for five years now, so out was about time, and we’re closer to our school this way. This is the first time I’m moving away from home, so it’s a big step for me.

The digital

I decided to take the opportunity of moving to also refresh my computers. I bought an Android phone, and I installed more appropriate distributions on the desktop and laptop. I’m still experimenting with the setup, but I think I’ll go with Arch on the desk and Ubuntu or Fedora on the lap.
I also renamed them all after fictional spacecraft, and added desktop backgrounds matching their names.

And the cloudy

I’m also finishing the experiment of running my blog with Drupal on the Amazon cloud. The free period is running out, and a fuzzy feeling is not worth the 15 bucks a month, so I’m moving back to WordPress. It was fun, but I figured I had better things to do than searching and configuring a plethora of modules to replicate the functionality WP has by default. I found that for me, writing code was much more interesting than administering a server, so I’ll focus on that instead.

Amazon’s free micro instances are also rather slow, and the load was showing whenever I posted an article to PlanetKDE. I tried aggressive caching, CloudFlare and FeedBurner, but the site still wasn’t performing as it should.