Hello!
This time I would like to compare the originally-set timeline in my DUNE proposal to the actual results. We’re just over the first half of the project, and the actual work seems to already deviate from the initial plan. The most important change, as per the mentors’ suggestion, is that I focused more on the local part, and ensuring it really works, than a central remote website.
Here is the original plan, accompanied by my notes.
Week 1 [June 17 – June 23]: During the first week, I plan to finalize a short list of test programs (benchmarks). Additional ones could by added any time, but I would like to have a basic list before starting other work. I will write a basic script that only compiles and run these programs, while measuring their time and memory consumption. — Done
Week 2-3 [June 24 – July 7]: I will spend most of these two weeks in Cambridge attending a summer school on the physics of liquid crystals. Any coding time will be severely limited, but I will at least finish the database specification for storing the results, and actually store them. — Done
Week 4-5 [July 8 – July 21]: Add more detailed measurements to the test running script. Big-O measurements with automatically varying problem sizes, multi-thread vs. single-thread comparison, trying different compilers, etc. It may be necessary to write a special utility program for this in something other than shell scripts (Python or C++) for cross-platform compatibility, in which case I will. — Done. The whole thing is written in Python. The program stores problem size, the compiler and its flags, etc.
Week 6 [July 22 -July 28]: Finalize the testing and measuring part of the project. If any time remains, which I think it will, I will start working on a visualization website, and the client-website communication protocol. — Done, but differently. The visualization happens on the client-side, with graphs and all.
Mid-term evaluation [July 29]: At this time, as a milestone, I will have completed the part of running the test programs, measuring their performance in several ways, and allowing for automatic comparisons of problem sizes and compilers. — Partly done. Automatic changing of compilers is not supported, as this is very program-specific.
Weeks 7-8 [July 30 – August 11]: Backend for the dashboard website. Database structure to match records from testing environment, module for statistics and exporting data to charts, correctly receiving data from testers. — Not done. Visualization is done on the client side, with automatic HTML generation.
Weeks 9-10 [August 12 – August 25]: Frontent for the dashboard website. If matching visual style with the Wiki is desired, I will use that, otherwise either Twitter Bootstrap or ZURB Foundation, depending on preference. More important that website style are visualizations with charts, using one of the widely available libraries such as Highcharts. — Todo, although using Bootstrap and DyGraphs instead of Highcharts.
Weeks 11-12 [August 26 – September 8]: Add more advanced features to the website. Identifying and removing outliers, finding long-term trends, alerts for sudden performance drops, exporting data, etc. There could still be something missing from the last four weeks, which I will complete here. If not, I will start writing documentation.
Week 13 [September 9 – September 16]: Everything should work by now, so I will write extensive documentation for both running tests and the website. For projects with more than one developer I tend to document code as I write, but there will have to be user documentation as well.
Suggested pencil-down date [September 16] milestone: Everything specified in this project works. It is possible to run all benchmarks with a single command, upload the results to a central website, and look at the results there. By repeating the procedure, one can see the averages and deviations, as well as trends and comparisons between different setups.
In essence, I haven’t started writing a website yet, and instead paid more attention to the client side. Both visualizations and basic statistics, although not mentioned in the proposal, are now done locally on the developers’ machines.
The Noughmad's Land 