[GSoC] Templates in KDevelop – Final Report

This year’s Summer of Code is coming to an end, and so is my project. I’m spending the last days until the “pencils down” date reorganizing the code, looking for crashes and memory leaks, and writing online documentation. There are no groundbreaking features since the last report, but the functionality is becoming really solid, and I have a feeling that everything is coming together.

During the last weeks, I made sure that any kind of file template is supported by KDevelop, but only Class and Test template get special attention with additional assistant pages. I’ve ene put together a small flowchart of the assistant to illustrate what pages each type of template gets. Because templates can expose their own custom options, one can write some complex templates with no changes to KDevPlatform code. To test code generation for other types of files, I added a template for CMake Find modules. This means that writing almost any kind of boilerplate can be avoided with a good template.

I also tried to make creating new templates as easy as possible. Apart from a toolview which gives you a preview of the current files as it would be renderer as a template, it is now also possible to extract any loaded template archive, edit it manually, and then import it back with KDevelop. I believe that most people will try to use an existing template with some minor edits, so this is an important usecase. Additionally, there is a project template that generates a project with a single class template.

I would also like to thank my mentor, Milian Wolff. He really stepped in during the last month, not only giving me advice but even fixing my mistakes. The template selection now looks much better, and is consistent between project and file templates.

Finally, I had to document all this. I wrote specifications for both kinds of templates, added tutorials for creating new templates, as well as instructions for using existing ones. Writing may not be the hardest part of a summer project, but it’s certainly the least enjoyable for me, so I hope that what I did produce is both understandable and sufficient.

[GSoC] Class and Test Templates in KDevelop

 

I have generalized the “Create Class” dialog, so that it starts with a template selection page that is similar to what you see when starting a new project. It offers you a selection of templates of different types (currently only Class and Test) and programming languages. Selecting a templates leads to further assistant pages, which are chosen dynamically depending on the selected template’s type.

The first page of the “Create from Template” dialog

I have added wrapper classes for template rendering with Grantlee (KDevelop::TemplateRenderer) and template archives (KDevelop::SourceFileTemplate), so the assistant only deals with those to keep the code clean and readable. Both class and tests use the same format of template archives (described here), only with a some different variables.

As you can see from the screenshot, there’s already quite a few available templates to choose from. Considering a template only has to be written once, every project could have one or more preferred templates for new code. Existing ones can always be re-used, or maybe just tweaked slightly, so a little work can improve coding style and consistency.

 

 

[GSoC] Templates in KDevelop – Week 5

My last report (here) was full of pictures, but since then I’ve spent more time polishing the functionality, behind-the-scene improvements, and fixing bugs. However, the mid-term evaluation is approaching, so I think let everybody know what’s the state of things. I’m happy with my progress, my schedule was a bit vague, but I think I’m ahead of it. So far I’m still enjoying it, and I bought a more comfortable chair, so I have little problems with working long hours.

I split out some functionality to make code more modular, which also allowed me to write unit tests for much of the newly added classes. The main parts of the code (TemplateRenderer, TemplateClassGenerator and TemplatesModel) are covered with test cases.

Class Templates

My main focus was still on templates for creating new classes. I slowly decided on the variables passed to templates. They are also documented, and I think I can start writing more templates about now. The C++ plugin adds some variables of its own, such as namespaces,

Since KDevelop’s Declarations must always point to a location in a source file, they cannot be created directly, I had to add my own classes for describing code before it is generated. This way, data members of a class can be declared. These code description classes are very simple, with only a couple of members, and are written so that Grantlee templates have access to all their properties.

The state of template class generation is such that it covers all functionality of the existing “Create Class” dialog. I have already written a basic C++ template that produces the same output. Of course, it is possible to create different classes, such as ones with private pointers, QObject’s macros, or in different languages.

Writing the Class Templates

As I said, I already wrote a template for a basic C++ class, as well as one with a d-pointer. Since I figured a lot of the code would be shared between templates for the same language, I added some basic templates that can be included. These are for method declarations, argument lists, namespaces, include guards, and some other small conveniences. There is also a library of custom template filters in kdevplatform. The end result is that the templates themselves can be relatively small. I even reduces the amount of whitespace in the rendered output, so that templates can be more readable while the generated classes are compact enough.

However, I don’t think it’s practical to store templates and filters for all possible languages in kdevplatform. So I intend to add a way for templates to specify dependencies on language plugin. Of course, they could still be written from scratch, or simply ship with the needed includes. It is merely a convenience.

The plan is to have language plugins provide some of their own templates and filters, but so far they are only for C++.

Templates, Templates Everywhere

Of course, KDevelop has other utilities for code generation, and I figured templates would be useful there as well. I started with inserting API documentation. The previous implementation manually constructed a Doxygen C++ style comment. I replaced that with a renderer template, which now supports C++, Php and Python.

Pressing Alt-Shitf-D on a declaration of a C++ functions results in this

API documentation with Doxygen for C++

While doing the same thing on a Python function produces this

API documentation with reST for Python

Not only is it formatted in reST, the most common format for Python documentation, but it also is positioned below the declaration, as a true Python docstring. Obviously, we still need to convert __kdevpythondocumentation_builtin type names to python types, but stripping a prefix can be done within a template thanks to Grantlee’s built-in filters.

[GSoC] Templates in KDevelop – Week 2

So, I’ve been working on KDevelop for two weeks now, and it’s time to show some results. My summer project revolves around templates, not in the sense of C++ angle-bracket templates, but templates for source code files. The goal is to auto-generate the boilerplate code.

Sharing Application Templates

KDevelop already had a basic template support. When you create a new project, it gives you a choice of all templates installed by KAppTemplate and a couple of its own. However, those only come from official sources, and while it’s possible to install new ones, there is no GUI way to do it.

So I started by implementing both downloading with GHNS and loading local files.

The “New Project” assistant, with buttons for loading templates

Loading from file works for .zip or. tar.bz2 compressed archives, in which case it simply copies to the directory where all templates are stored. You can also load .kdevtemplate files, the program automatically archives its containing directory and stores it.

Downloading more templates works pretty much as expected, currently there are only three templates available for download, but I plan to make some more.

Application template download dialog

It is also possible to share template right from KDevelop. I have added a new configuration modules, where all different types of templates are listed. From there, you can import, download or upload template files. So far, there are only project templates, but they will soon be joined by class template and possibly others.

Template manager configuration module

The project creation logic works exactly as it did before, the same variables are accepted, so all old templates work. The only exception is that now folder names can have placeholders as well.

Class Templates

Seeing as the current class generator in KDevelop is not very configurable, I wrote a new system that uses templates. However, contrary to creating a project skeleton, classes usually have variable numbers of members, and function signatures look different in every language. To allow loops and other complex constructs in templates, I used the Grantlee template engine, written in Qt by Stephen Kelly. I will add some custom functions (filters) for it, but even without it basic use cases are covered.

To allow this, I modified the “Create Class” assistant a little. The most user-visible change is allowing an arbitrary number of output files, not just two. The following is for a template with a private class declared in a separate header, useful for shared d-pointers.

Choosing output file locations for a template with three files

The template engine already handles most of the properties class functions can have. It is possible to group class members according to their access, have slots and signals declared together, handle constructors and destructors separately, etc. What’s missing is a simple way to declare members before creating the class. Currently, only functions which can be overloaded are offered, but no data members.

Becoming a KDeveloper

So, I’ve just been accepted into this year’s Summer of Code program. I’ll be working on KDevelop, introducing a system for using and sharing code templates. This includes both project templates (integrating KNewStuff into KAppTemplate) and more specialized templates (for code files,  classes, tests, make and cmake files, etc). The full proposal is online at http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/noughmad/37002.

However, all this is still in the future. I’ve already been doing some work on KDevelop, and so far I’ve enjoyed it very much. Not only do I use it every day, its plugin architecture makes it easy to add small pieces of functionality. Note that because of school getting harder every year, I have little free time and my progress is a bit slow. But anyway, I’ll show you what I wrote the past few moths.

Unit tests

KDevelop currently has no support for unit testing. However, both KDevelop and KDevPlatform have loads of them, so a good system for running tests and reporting their results would find a good use. So I introduced some new interfaces for unit tests, including finding and running them. A toolview was adapted from the old and unmaintained Veritas library, however I took steps to better separate the interfaces and their implementations. Currently supported are tests declared with CMake and those using the PHPUnit framework.

A toolview for listing unit test suites and their test cases

Apart from simply listing tests, the relevant declarations are also extracted and a “show source” action is available. Support for running unit tests and reporting results is also functional, however I have trouble deciding which icons to use. There is a difference between a failed test case and an error, and also between an individual testcase and suite results. As a short-term solution I implemented a mess of different icons.

The toolview showing results of a test run

Finding tests declared in CMake files works reliable, whether they use add_test() directly or some wrapper macro (like kde4_add_test()). Unfortunately, this is not true for php tests cases. The problem with PHPUnit is that a test case must inherit from a certain class, but  doesn’t have to include any special file, so the parser find the right declaration by itself. I managed to fix it by modifying the parser, so that most test cases are found, but unfortunately still not all.

Additionally, when running the test cases, their output is displayed in a separate toolview. For test cases using QtTest or PHPUnit, the output is highlighted appropriately (green for success, red for failure). For Qt-based tests, clicking a line in the output also takes you to the declaration in the code.

An output from a test using the QtTest framework

If you wish to test this, the code is in the “unittest” branches of kdevplatform, kdevelop and kdev-php. Any input is appreciated, especially regarding the presentation to the user.

Code checking

Another valuable resource for KDE developers is the Krazy code checker. Seeing how easy it is to create KDevelop plugins, I wrote another one to run Krazy and report back the results. Any issues found are inserted into the declaration-use chain, so they are visible in the Problems toolview, as well as in the code itself.

An issue found by Krazy displayed in the Problems view and in the code

The code is still in my personal scratch repository at http://quickgit.kde.org/index.php?p=scratch%2Fmihac%2Fkdev-krazy.git&a=summary

When all else fails

I also implemented a different kind of code checking: by humiliation. If you stumple upon a horrible piece of code, either written by you or inherited, you can earn a feeling of superiority by sending it over to The Daily WTF.

A context menu entry for submitting an offending code snippet to The Daily WTF

I haven’t yet submitted any code with it, but I tested the plugin with Alex, the site’s maintainer, and he says it works. So watch out, if you write bad code, even if you fool all the unit tests and code checking tools, someone can still expose you.