I just returned from the third annual GNU Radio Conference, which was, as always, a really great experience. The project has grown considerably in the last 5-6 years, and more and more people are using GNU Radio for more and more impressive things. Juha's use of Ettus Research USRPs and GNU Radio to map the surface of the moon and implement radar systems, for example, is really incredible.
One of the points of debate that comes up fairly regularly (indeed, it came up several times at the conference) is that some people have the perception that GNU Radio is an 'amateur' project. To elaborate, it is something you use to prototype something, but not a platform you would ever use in a "real-world deployment". I hear this argument very often in the context of comparing GNU Radio with other SDR frameworks / solutions, such as Redhawk.
This generalization of GNU Radio is both partly correct and partly wrong. GNU Radio is used by a lot of users for very real applications. Applications that are incredibly innovative, and that are relied upon to stay running without maintenance for long periods of time. The sweeping statement that GNU Radio is good only for 'prototyping' or 'early R&D' is not only inaccurate, but a gross dismissal of a framework that has proven its strengths very thoroughly since its inception 12 years ago.
That said, there are a number of areas of GNU Radio that don't seem mature - especially if you are coming from the world of expensive frameworks, hardware, and test equipment that look and feel gorgeous even to people that have no idea what they are looking at.
I think the GNU Radio project has grown to the point, however, that it needs to evolve into whatever its next phase is. Continued growth and improvement of the project, as it is now, will indeed continue to add functionality and improve the existing code base, but I don't know that it will carry the project, as a whole, forward - towards the great things that it can become. The GNU Radio project and developers have reached the point that everything is primed to make the jump - we just need to agree upon a vision of what we are jumping to.
I have a list of things I have been mulling over that I think would be significant improvements for GNU Radio. Some of them are relatively straight-forward, and others are a bit more of a paradigm change. Many of these we discussed at the GNU Radio conference. I want to get them down, in writing, to spur discussion and debate about where to focus efforts (hopefully - otherwise it's just a soapbox post).
The Primary Interface
The GNU Radio Companion
Right now, the GNU Radio Companion (GRC) is the universal front-end UI for working with GNU Radio. GRC was originally designed by Josh Blum while still a student at JHU, and has continued to develop it since his graduation. GRC lets you graphically draw the 'flowgraphs' that GNU Radio uses to represent data flow, and then quickly run your application. Here's an example from the GNU Radio Website:
GRC revolutionized the use of GNU Radio, and truly was a paradigm shift for the project. It made GNU Radio immediately more approachable and useful, and has played a huge part in making GNU Radio the success it is today.
Seven years later, GRC is starting to show it's age. It is still incredibly useful as a development tool, but it really isn't a development environment, and doesn't present an easy-to-use project-level workflow. A proper Development Environment (DE) is crucial to improving the project's approachability, ease-of-use, and feature integration - all of which are critical to the continued growth and long-term success of the project.
I'll liken it to CVS, Subversion, and Git. CVS revolutionized software engineering. Subversion was an attempt to do CVS better. Git was the paradigm change and follow-up revolution software engineers needed.
In short, GRC was an evolution of GNU Radio and a paradigm shift. I think it's time for another one - this time in the form of a mature development environment.
When you think of Development Environments, most people jump to Microsoft Visual Studio and IBM's Eclipse. First, though, I want to clear something up. By a GNU Radio DE, I am not talking about writing an editor.
Things to Avoid: The Editor
Eclipse, MSVS, and others all include a very full-featured editor. Attempting to implement an editor for a GNU Radio DE makes little sense, and I don't know that anyone would actually use it, anyway. If making an editor a part of the DE is critical, I think the solution is finding a way to embed the user's Editor-of-Choice into part of the UI (Vim, Emacs, Geany, Codeblocks, etc.,).
Things I Would Like to See
So, what would I like to see? Here is a brief list of the obvious ones:
- Improved flowgraph organization
- Built-in Documentation for block & block settings
- Project-level management (files, blocks, settings)
- Easy custom block creation
None of these are particularly ground-breaking, but they are important improvements. That said, that list ignores many of the major new development efforts that could have a significant impact on GNU Radio. Let's look at some other items that would make great new features in a DE:
- ControlPort - By integrating gr-ctrlport functionality, we could enable debugging, monitoring, and remote deployment of flowgraphs.
- PyBOMBS - Currently under
development (and something I have previously blogged about)
PyBOMBS integration would enable easy management of out-of-tree modules.
- Hardware Management - Right now, there is no real workflow for managing connected hardware. As more and more people use GNU Radio with physical hardware to hack on things, a management UI could be really useful.
- Debugging - There really isn't any debug capability in GNU Radio right now, outside of the usual software debug tools (e.g., gdb) and some blocks that can be useful in the debugging process. As GNU Radio applications become more complex, this will be critical.
- Inclusion of Advanced Design Tools - There have been a number of great design tools made for GNU Radio which are, right now, separate and thus not as heavily used as they might be. A great example is the very impressive GNU Radio Filter Design Tool.
- Graphical Sinks - These need a tremendous face-lift. More on this in the next section.
- RF NoC Integration - First presented at NEWSDR 2013, RF NoC is an architecture currently under development at Ettus Research that will enable dynamic routing between blocks deployed to an FPGA. It really is ground-breaking research, and could have a huge effect on how SDRs are prototyped, developed, and deployed.
These are just some of ideas and features that I've spent time mulling over in the last few weeks, and I heard a number of others at the GNU Radio Conference last week. Creating an application that makes the recent improvements and utilities of GNU Radio user-friendly will really carry the project forward.
Path to a DE
So, how do we get there? An easy, and fairly obvious, answer is to just make an Eclipse plugin. I have a few issues with this route:
- I really dislike Eclipse. I have never had a positive experience with it. My opinion of it is that it is bloated, slow, and overly complex.
- A major feature of Eclipse is the editor, which as I've said above, I don't think should be a focus of the GNU Radio DE.
- Eclipse is written in Java. There are no parts of GNU Radio that currently use Java, and I'm not convinced that introducing it is better than other options.
- We don't control Eclipse, which means we will have to develop and maintain relative to a separate project's changes. You could make the argument that we will have to do the same thing for library dependencies anyway, but there is a big difference between lib dependencies and being a plugin to an entire other application.
- I don't think we would want to use most of what Eclipse comes with. This means a huge install for users, little of which they actually want.
I think an Eclipse plugin might be a good stepping stone to figuring out what it is we actually want, but right now I don't think it is the right long-term solution.
What graphical framework we use to implement the DE is a big decision. GRC is currently implemented with wx in GTK - a combination for which I have little affinity. That said, Sebastian Koslowski, a GNU Radio hacker from the university with the most contributions to GNU Radio, put together a mock-up of GRC in QT which became an active project at the GNU Radio Conference last week. Already, it has some impressive advantages compared to it's wx/GTK sister program. Also, migrating to Qt would enable the removal of the wx & GTK dependencies in GNU Radio. Less deps, less problems.
I'm not absolutely positive that this fresh Qt version of GRC is a good starting point, but it might also be a stepping stone.
Test, Measurement, and Display Sinks
How users can view, analyze, monitor, and interact with data is a critical feature of a software distribution like GNU Radio. There is a reason that so many engineering frameworks advertise themselves with pictures of their data visualization capabilities - they are important. Right now, data visualization in GNU Radio looks like this:
It covers the basics, but it isn't anything to write home about, and it certainly doesn't look amazing.
Sylvain Munaut came to a similar conclusion, and presented work on what he calls 'fosphor' at the GNU Radio Conference. During his presentation, he said something that I think is particularly apt - basically, "This is how GNU Radio graphical sinks look now. What I want is a graphical sink that looks like a real-time spectrum analyzer from Rohde."
When you look at a data visualization, you want it to look sexy. Yes, aesthetics matter. They are exciting, draw users, and are impressive (and this is particularly important to people trying to get funding for something) to people who have no idea what they are looking at.
Also, they have the added benefit of being incredibly useful. Multi-color plots, for example, are very, very useful. I want to be able to look at something like this, but from within GNU Radio:
GNU Radio needs visualization sinks that are as useful, impressive, and crucial to development as a good piece of bench test & measurement equipment. Here are some examples of features the current visualization sinks lack that would be really great to have:
- Multi-color plotting
- Useful signal persistence
- Plot alignment (e.g., FFT + Waterfall)
- 3D plotting
- More interactivity (e.g., click to tune)
- Built-in measurements (e.g., measurement bars, measurement text in-graph,
- Additional types of sinks (e.g., cyclostationarity)
Balint Seeber has actually gone after the last three I've listed up there, and has written blocks that implement some portions of this functionality. I believe they are part of his gr-baz out-of-tree block library.
Incorporating features like these in a modern, GPU-accelerated display, would be a tremendous boon to GNU Radio.
The GNU Radio project has grown significantly, and users' interests, in terms of where people will spend their individual development effort, is more varied now than ever before. There are some people that are really interested in using GNU Radio with FPGAs, and others on ARM-based devices. Some people want to investigate building complex MAC-layer protocols in GNU Radio, and others want to work on the core user interface. There is a lot of development happening amongst a lot of different people, and everyone will be more productive if it is organized in some way. It will also have a much better chance of getting merged at the end of all the work
The GNU Radio maintainers, Tom and Johnathon, did a great job of starting to tackle the above problem at the conference last week. They organized working groups, with recognized GNU Radio contributors leading each one. I think this is absolutely the right approach. Each working group was given a significant level of autonomy in terms of how they wished to organize themselves and keep their work going.
I wouldn't change anything about what Tom and Johnathon started, and I think, very strongly, that these working groups should become a very formal part of the GNU Radio organization. Include each GNU Radio Working Group's (GRWG) progress as part of the dev calls. In the GNU Radio Newsletter that Martin Braun
is going to start sending out, include a section for each GRWG, and have the leaders submit regular updates. We should have regular (not necessarily really often, but definitely very regular) meetings between the core developers and the GRWG leaders to make sure that any needs they have in terms of core framework development is understood and addressed by the right people.
In short, I think that these GRWGs should become first-class citizens of the project organization. GNU Radio has the potential to grow massively into a lot of new domains that are currently only addressed by very expensive and highly proprietary systems. Well-organized sub-teams of developers, tackling specific development challenges and goals, will be key as the GNU Radio user and developer base continues to grow.
Documentation & Theory
Documentation is an acknowledged weakness of GNU Radio. Martin Braun, as the GNU Radio Community Manager, has done a terrific job of organizing new documentation for the GNURadio website and refactoring how it is presented. Even so, though, there is a lot left to be done.
Documentation is a tricky beast. It is perhaps the easiest way for someone new to project to start contributing. Simply documenting what you learn as you first start out with GNU Radio, for example, would be very useful information to the next person that comes around. Unfortunately, writing documentation isn't sexy, and for most people, it isn't fun. It is a bit of chore, which is why GNU Radio has grown to as large of a project as it has with so little of it.
Documentation is a key indicator of a mature and professional project, and is important to attracting high-impact users that have the capability to make really great contributions to the project. In addition, in order to grow GNU Radio outside of its current demographic and reach users that are less tolerant of poorly documented code (and perhaps are new to the concept of Free & Open Source Software (FOSS)), documentation of the code is key.
For a project like GNU Radio, however, documentation goes beyond general code comments. Due to the nature of the project, a tremendous amount of GNU Radio's codebase is implementation of complex DSP, and thus mathematical, algorithms. Many of these implementations were built around academic textbooks & papers on mathematical theory, and experimentation with actual running code.
This means that in order to open up a file and debug code, fix bugs, make feature additions, or even just understand what code you are using, you need to grok not only the software engineering concepts involved, but the DSP theory as well. Unfortunately, much of the theory implemented within GNU Radio isn't documented much better than the code itself, which can make debugging issues very difficult and intimidating to new users (and even experienced developers).
The code comments should include explanations of what the code is doing, not just in terms of software engineering, but in terms of the theory being implemented. It should include references to where the theory came from, and explanations for where something is specifically being done for computational reasons (e.g., optimizing an algorithm for memory caching).
Without this, I think finding people willing to debug and contribute to existing code within GNU Radio will continue to be difficult. In addition, documentation to this degree of thoroughness would really enable GNU Radio to offer something that very few other platforms can.
So, what's the fix?
The problem has grown to where it is today because code is accepted & merged into GNU Radio without good documentation. Writing documentation isn't sexy, but writing code is. If we establish documentation guidelines and requirements for new code being accepted to GNU Radio, and mandate that all new code adhere to these requirements before getting merged, I think we will see a natural uptick in the amount and quality of documentation provided.
The counter-argument to the above is that by doing this, we will deter people from committing to GNU Radio. I disagree with this for a couple of reasons.
Firstly, most people writing code for GNU Radio are actually writing it for out-of-tree projects, which will never be merged into GNU Radio. Thus, they can maintain whatever standard of documentation they want. If they do a poor job, then someone else can come along, re-implement the functionality better and with more thorough documentation, and users will adopt the new implementation instead. FOSS provides for an excellent free-market (lol see what I did there?).
Secondly, good developers value documentation. Having good documentation will draw the sorts of developers that can make really tremendous contributions to the project. I call these high-impact developers. Developers that don't care about documentation, at all, might generate a lot of code, but how much of it will really be usable, accessible, or maintainable in the long run? We want to attract the sort of developers that will make important contributions, and to do that, we need developers that care about documenting.
The GNU Radio project has made tremendous strides in the last few years, and the user base is enthusiastic, diverse, and growing. I said at the beginning of this post that I think GNU Radio is ready to evolve into 'whatever its next phase is'. I don't claim to know categorically what this 'next phase' might be (although I have some thoughts
;)); the above are just things that I think are important, and that I argue should play a role. I'm willing to be convinced otherwise, if someone has a strong argument.
We have an opportunity to really redefine the SDR and DSP domain with GNU Radio. If the project continues to grow and evolve over the next decade as it has in its last, I think we will achieve it. It's just a matter, in my opinion, of identifying and organizing the evolution in the best way possible.
Other Minor Items
This isn't really related to the major notes above. These are just some items that I would like to do (or see done) for GNU Radio that aren't necessarily paradigm changes, but that I think are good hacking opportunities in the project. I'm putting them here so that I don't lose them.
- Run Valgrind against GNU Radio (lots of challenges, here).
- Squash Coverity Bugs
- PyBOMBS code refactoring & interface overhaul
- [Removed item about
liquid-dspbecause I changed my mind]
- Framework improvements (e.g., 'hier' blocks, 'block2' class nomenclature,
- Buffer management necessary for embedded & co-processor work.