Guerilla UX Testing, and Other Experiences From Akademy

Guerilla UX Testing, and Other Experiences From Akademy

It’s about a month now since the end of Akademy 2018 and I’ve finally found the time to write up some of my impressions from my favorite event of every year, and to encourage all of you to embrace both your inner User Experience (UX) Researcher and your inner guerilla.

My Talk: Guerilla UX Testing

I usually try to give at least one talk at each Akademy I attend, and this year I wanted to shake up the way KDE approaches UX design and research a bit again. The idea to give a talk about Guerilla UX came when we had one of those design discussions on Phabricator where different sides of the discussion had different ideas in their minds of what The User™ would want or need, and we had no way to tell which side knew our users better, or even if either side knew them well at all.

Usually, a UX person’s reaction (at least if they don’t suffer from delusions of grandeur) to such situations is “Why don’t we just test it with actual users?”. As a distributed community of mostly volunteers with limited personnel and monetary resources, however, this is much easier said than done. This is where Guerilla UX Research comes in: According to UX Magazine “Guerrilla research is a fast and low-cost way to gain sufficient insights to make informed decisions.”

In short: Guerilla Usability / User Experience Testing has you walk up to someone resembling your target audience (can be a stranger, can be someone you know as long as they are not aware of what you’re currently discussing), show them the product / UI / design you want to find something out about, give them a task and watch them use what you’ve made, recording any user experience issues you notice.
This is often all we need: A relatively quick and cheap way to gain some insight into how our users think and behave, even if the insight is only limited. It’s not as good as detailed user research with specifically selected participants, but as long as the test participants are at least similar enough to the actual target audience, it’s definitely better than generalizing from the project team’s own experience to that of the target audience.
I’d therefore heartily encourage everyone who needs to design a user interface or decide on features and isn’t 100% sure what their users need to take maybe a day (or maybe an hour each if you split the task among the team) to perform some guerilla user testing. I can promise you that you will be surprised by the results and that you will have fun!
For more details see the slides from my talk:
[slideshare id=113955180&doc=guerillausabilitytesting-180911204937]
(in case the embedded viewer doesn’t work for you, here are the slides on Slideshare)

Akademy from the Perspective of a Board Member

This year’s Akademy was especially interesting for the board of directors of KDE e.V.. in several ways.

Sponsors Dinner

On Saturday, we could witness first-hand how KDE is able to connect different organizations: At the sponsors dinner, where representatives from all of Akademy’s sponsors spend an evening together with the board to share our ideas and projects.
I had the opportunity to sit at a table with representatives from four companies that are active in the hardware business and all are connected to KDE now: Blue Systems, Mycroft, Pine64, Purism and Slimbook. It was really cool to see the people from the different companies discuss their plans and their experiences with trying to make hardware in a FOSS-friendly, user-freedom-respecting and ethical way. All of them had already heard of the other companies before, but most of them had never met in person before, so we really had an effect there.

Annual General Assembly

On Monday, we had KDE e.V.‘s Annual General Assembly (the official minutes in German can already be found on our website, the English translation is still in preparation). For me personally, the biggest event of the AGM was the election of a new board member to replace Sandro Andrade, whose term had ended (thank you again for your great work on the board, Sandro!).

Before the AGM, Andy Betts was the only candidate for the position. Not really a problem since Andy was a great candidate, but an election with only one candidate where all one can do is vote for or against that candidate always feels a bit weird to me. That’s why I was happy when Tomaz Canabrava spontaneously stepped up as a second candidate during the AGM. Now we were in the comfortable position of having two excellent candidates to choose from.

After a great Q&A session with both candidates, Andy won the election. I’m very happy to have Andy on the board, though I would have been equally happy with Tomaz.

Meeting with The Qt Company

Right after the AGM, the board met with a delegation from The Qt Company (which included their CEO and CTO) to discuss how we can collaborate to make sure that Qt continues to serve both the needs of The Qt Company and those of KDE (and the ecosystem of Qt users as a whole).
We learned a lot about The Qt Company’s plans and the challenges they’ve overcome and are still facing, and were able to share our experiences as one of the key players in the Qt ecosystem. Overall it was a very informative and constructive meeting, and we’re looking forward to continue collaborating with The Qt Company toward our common goal: To keep Qt as great as it is!

My Highlights from the BoF Days

Unfortunately the meeting with The Qt Company coincided with two BoFs that I had really wanted to attend: One was about the VVAVE project, a very promising KDE music player and music discovery platform created by Camilo Higuita, the other was about the newly-rebooted KDE Human Interface Guidelines. I had contributed a lot to the original content of the HIG but haven’t had much time to work on it recently, so I’m eternally grateful that Fabian Riethmayer took over maintainership and is putting countless hours of work into them.
I spent the first half of Tuesday in two sessions about the current state and goals of the VDG (and Plasma design topics in particular), led by Andy, and the second half discussing mobile and convergence topics in the sessions about our convergent UI toolkit Kirigami UI (led by Maco Martin), and about the MauiKit (led by Camilo) which extends Kirigami further. We discussed what we consider to be the defining attributes of Kirigami, what the commonalities and differences between MauiKit and Kirigami are, and how to make sure that both users and developers have the best possible experience across both.


Thursday, my last day at Akademy, I spent in the Online Fundraising and Campaigning training, led by Florian Engels from more onion (via NPO academy). All participants agreed that we learned a lot of practical strategies and tips for fundraising and campaigning, which we are sure to apply in the coming campaigns for KDE, Krita and Nitrux!

I also heard a lot of positive feedback on our other trainings: The training on Nonviolent Communication by freelance communication trainer Tilman Krakau, the Technical Documentation Training by Stefan Knorr from SUSE’s documentation team, and the Public Speaking training by Marta Rybczynska, technical public speaker and trainer, and active member KDE.

Since I was in charge of setting up the trainings and this was (as far as I know) the first time KDE offered professional training to our contributors at Akademy, I’m glad that they seem to have worked out very well!


As in every year, this was again a very successful Akademy, with lots of productive discussions, interesting talks, great conversations with friends over dinner and beer, and overall a wonderful time!


Comments: 4

  1. Zalbarath says:

    Just as I posted on G+ I’m not convinced to this Guerilla UX test. A random person has no baggage that we – users have. We know what we like, what we are used to. A user who never used an old version of a UI cannot be representative of actual KDE users.
    Or maybe I’m not getting the idea correctly?
    Better, but a bit more hard to implement method would be to create a platform for alpha/GUI testing. Or maybe simply an alpha-dev platform for users who want to be part of the development by sharing their feedback?
    So for example:
    1. Forum initial idea + design schema – first impression, ideas, brainstorming.
    2. Refined design schema – impressions. If the feedback is mostly positive
    3. Create a package to test that and inform about a way to download it to the subscribers of those previous feeds?
    I admit I have no idea how the actual development cycle looks like in details so the idea above may be completely off and not practical.
    I’m just saying, that there are KDE users who will be interested to be part of such process (maybe some additional ppa on Neon for those who want to participate). Even a small sample of KDE hardcore users feels better than a random person trying something without the past background and experience.

    • Thank you for your comment!
      In the slides I go into a little more detail regarding the participant selection. There I say that depending on what you want to find out, an experienced user of the application might be the better choice.
      Often the goal of usability work is to create UIs that don’t need any prior experience to be understood well. That is where complete novices are the best test participants for. If they understand it, experienced users will, too.
      If you are deigning a UI or a features to help experienced users of an application or a desktop environment to be more productive, however, you of course need those users to test with.
      That is where a pool of experienced testers would be really useful for. Just asking them for feedback is not the same as doing a user test, however. If you ask for feedback, you only get people’s subjective opinion, which is often very different from what you observe when watching them use a UI.
      When asked for their opinion, people tend to prefer things that are close to what they’re used to. However, if you watch them use something, you can sometimes observe that in fact they can use something that’s different from what they’re used to more efficiently. They might still tell you that they don’t like it, but if it works better, they will eventually like it.
      So, long story short, it really depends on what your research question is.

Add your comment