This summer I’ve been accepted into the Google Summer of Code programme to work with the GIMP community on a project to “Enhance the plug-in/filter development experience”. I chose this particular project from a list of project ideas that GIMP developers maintain on their developer website.

This is not the first time I’ve worked with the GIMP community. Several years ago I pushed forward the state of the nightly Flatpak builds to allow users to easily install and test the current state of the (at that time) version of GIMP that was to become the 3.0 release.

Overall, I’ve been the occasional user of GIMP for roughly the past decade and getting to contribute to it now feels to me like a nice way to pay back for what it has allowed me to do.

The project at hand

Getting back to the project, the immediate idea is that with the release of GIMP 3.0 a new API for creating filters for NDE (non-destructive editing). This is a fantastic feature that I think many artists have been very much waiting for. One way to use the new API is:

gimp_drawable_append_new_filter (GIMP_DRAWABLE (layer), "gegl:inner-glow",
								 "Inner Glow",
								 GIMP_LAYER_MODE_NORMAL,
								 1.0,
								 "x",           x,
								 "y",           y,
								 "grow-radius", 1.0,
								 "radius",      radius,
								 "value",       value,
								 "opacity",     opacity,
								 NULL);

With this a new filter is created using the gegl:inner-glow GEGL operation, applied with the “Normal” layer mode, opacity set to fully opaque and the properties of the operation set appropriately.

The main beauty lies in the flexibility of this API because one can use any GEGL operation. This raises the question of what operations are available and how can they be configured. There are some ways like the GEGL website, gegl command-line utility or exploring more manually. None of these options covers all use-cases because the website only shows operations ships by GEGL, the utility also shows operations registered through plug-ins but does not show GIMP operations which are registered at runtime. So, there’s a need for a solution that builds this list during GIMP’s runtime.

Together with this, I wanted to explore the state of GIMP’s developer resources, how they are accessible and whether more information and guidance could be given to the individual plug-in/filter developer.

Getting started

Before applying to GSoC I spent some time working on porting the email plugin from using xdg-email to using the Email Portal. It’s a very small plug-in that does only one thing - opens an email client with the currently selected project as an attachment. After putting the implementation together, both me and the maintainers have learned, to our dismay, that this portal does not support the main needed feature - making the attachment available across sandbox boundaries. We put the MR on the back-burner until the portal gains this support.

See https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2200.

After being accepted to GSoC to work on the proposed project, I spent some time trying to collect existing reported issues regarding the developer browsers. I put together a post for GIMP’s UX repository where during the project I got several useful responses. I also posted about my project on two forums related to GIMP: pixls.us and gimp-forum.net. The community on https://gimp-forum.net seems particularly active and I got several responses early after posting.

The main project

From the beginning of the coding period I focused most of my efforts on putting together the new GEGL Filter Browser. It would be akin to the existing Procedure and Plug-in Browsers. The core idea is quite simple and I expected the implementation to be smooth. Now I’d say it was a little bumpy.

Because we agreed to make the new browser as a plug-in, certain restrictions were placed on how it can be implemented. Plug-ins are independent binaries that communicate with GIMP over “the wire” using the PDB (Programmers DataBase). Plug-ins and GIMP do share some code but the vast majority of GIMP code is off-limits for plug-ins. Because of this, collecting the full list of GEGL operations and their metadata needs to happen inside of GIMP and needs to be made available via the PDB.

It took some time and lot of guidance from the fellow maintainers to figure out how to get the full list of GEGL operations and then some extra time to find a nice and scalable way to implement the respective PDB procedures. In the end I settled on the following ones:

gimp_drawable_filter_operation_get_available
gimp_drawable_filter_operation_get_details
gimp_drawable_filter_operation_get_pspecs

We also needed to deal with support for types that are not explicitly supported by the protocol used to communicate between GIMP and it’s plug-ins. The protocol needs to both serialize and deserialize all information that’s to be passed between the two processes. For that, it needs specific instructions for each data-type. And because this API can deal with types introduced only in plug-ins, there’s no way it can support all of them. Jehan has been a great help in handling this behaviour and I took it one step further by mining information about the types and passing it along in a shim.

With a working API available, building the actual browser was quite straightforward because I could (and from the beginning planned to) implement it basically in the same way as the existing browsers are implemented - using pure GTK widgetry. There was one small exception to this - the use of GtkTreeView.

See https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2255.

A screenshot of the new GEGL Filter Browser

During the development of the browser I opened and resized it in all the different ways many times. It was quite necessary to do so, because the default size of the browser window was too small. So, I made a patch that changes how the default size is set and raises the default sizes. Also, now the patch proposes to use GTK’s GtkSearchEntry instead of rolling our own label + entry widget combo.

See https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2334.

With the initial version of GEGL Filter Browser done, the next steps are to make it even better and start looking into the feedback I’ve gotten from the community so far. Personally, I find the next two steps the most interesting:

  • Adding code examples into the browsers,
  • combining, the now 3 browsers, into a single one, and
  • exploring the use of WebKitGTK for the content of the browsers.

I find the first step particularly interesting, because one can develop plug-ins for GIMP in multiple languages. This also means that there are slight differences in how the different types of procedures can be used across the different supported programming languages. Showing examples for every single procedure could help developers to get started with plug-in development. This is especially true for the GEGL Filter Browser because the NDE-related API is still quite fresh. I have an in-development branch for this feature.

A screenshot of the new GEGL Filter Browser that contains a section with examples of use in the C language

See https://gitlab.gnome.org/martymichal/gimp/-/commits/wip/martymichal/filter-browser-code-examples.

The second step requires some more thinking about what the end use is supposed to look like. In version 3.1.2 of GIMP the Procedure Browser contains 960 procedures! This is a fairly extensive list that benefits from being purely about procedures. I have an in-development branch that combines this browser with the new GEGL Filter Browser. This branch uses the exact same layout, but makes use of GtkListBox’s feature to show headers that visually separate sections of the list. Now the combined list contains xxx items which are visually distinct using icons and appropriate headers them separate them into GEGL operations and the different types of procedures. It is the first idea of how this browser could look like.

A screenshot of an in-development browser combining GIMP’s PDB, Plug-in and GEGL Filter browsers

See https://gitlab.gnome.org/martymichal/gimp/-/tree/wip/martymichal/combined-dev-resource-browsers.

The last step is so far only an idea. GIMP already makes use of WebKitGTK in its general “Help” tool, so why not use it, too. Some of the benefits are

  • the possibility to show other documentation like libgimp and libgimpui,
  • having more freedom regarding the form of the content (instead of rendering with pure GTK),
  • getting niceties for navigation (back, forward, search, tabs) almost for free, or
  • full-page search just like in a general browser.

To get these benefits, some background work needs to happen first. At the core is that all the web content for the browser needs to be built first (in the case of procedures and GEGL operations) or included (in the case of libgimp and libgimpui APIs). For libgimp and libgimpui this is already done using gi-docgen at build-time. For the other two use-cases, GIMP would need to build this content at run-time and possibly (that needs to be explored) manually. I think this is an interesting project that could be taken on by either me or other student in the future.

For reference, see https://gitlab.gnome.org/GNOME/gimp/-/blob/master/plug-ins/help to learn more about the “Help” tool.

Side project one: Use of GTK widgets

While inspecting code of the existing browsers I noticed, they use the GtkTreeView widget for showing a list of items. This widget is a quiet complex widget that can primarily render tree structures but also lists. But in my opinion, the API for the purposes of using it with a list is unnecessarily complex. Using an alternative widget, like GtkListBox, sounds more appropriate to me.

I looked a little more around and saw that GtkTreeView is used very generously throughout the code-base and in many cases it could be good to replace it, as well. So, I compiled a whole list and scoured the code-base for the use of GtkTreeView, took screenshots of all widgets. All of this to help in the, now open, UX discussion on where is it good to change the use of GtkTreeView and how exactly.

See https://gitlab.gnome.org/Teams/GIMP/Design/gimp-ux/-/issues/258.

A fellow contributor has noticed that input sliders in a certain plug-in are very difficult to use because they use the available space in a very inefficient manner. I filled an issue for this and expanded it to track the use of the same widget across the whole code-base and fixed the initial case to use a more appropriate widget.

See

There were a few other cases when I touched the UI code in some way. See

Wilber Week

Because of this project, I happened to be invited by the GIMP team to Wilber Week! It is an event where for several days several people from the GIMP community have the opportunity to meet, get to know each other and hack on GIMP together. This time the event has happened at Can Serrat, an international art residency, in Spain. Thanks to the invitation I got to meet many of the people that have been developing GIMP, engaging with the community and simply keeping the project running this whole time. All of these people are very passionate about the project and have dedicated countless hours to it through their volunteer efforts. It is people that keep the project running! Thank you again for the invitation!

During WW we spent a lot of time hacking on different features, bugs and refactorings, and also a lot of time discussing many topics. This also was the time for the GIMP Committee to discuss and make decisions in several matters.

I took this time as an opportunity to discuss with (not only) pippin a particular part of GEGL -> hardware-acceleration.

Side project two: OpenCL

I am fascinated by GPU-computing and hardware-acceleration overall. Getting to discuss about this topic with developers in-person was a rare opportunity. There are many topics to be discussed here, so let’s do it one-by-one. The first would be acceleration of individual GEGL operations.

Many GEGL operations already have, next to their CPU implementation, an OpenCL implementation. No GEGL operation registered in GIMP does. When you use a modular system like GEGL, you need to have as complete a coverage, as possible, to minimize the required data movement between the GPU and the CPU. As an exercise, I chose to implement the very simple gimp:set-alpha operation to OpenCL. This turned a little more involved because the parent operation class did not have the OpenCL machinery wired in. So, I did just that.

See https://gitlab.gnome.org/GNOME/gegl/-/merge_requests/226.

The next step was to implement the actual operation. This still took a little while as I tried to remember how to write GPU code, but I ended up with a working implementation.

See https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2414.

Being a little more equipped, I looked a bit further. All of GIMP’s layer modes (e.g., Normal, Diffuse,…) are implemented as GEGL operations and none of them is accelerated. So, I took a stab at implementing the “Normal” layer mode. This, again, was preceded by me also adding the necessary OpenCL scaffolding to GimpOperationLayerMode. Again, after some trial and error, I ended up with a initial implementation of the “Normal” blending mode in OpenCL.

See https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2415

While working on all this, I learned more about the fundamentals of how OpenCL is implemented in GEGL. I will leave the exact details for the future, but I will say that the current implementation has serious performance issues and will require further work, to get the performance to be on par with the CPU side of GEGL. About that I also learned, just how well optimized it already is.

I took a stab at doing a rewrite one part of the fundamentals during WW but I only ended with an implementation that was even slower and less stable. I plan to come back to it, though, and see if I can do better during my second try.

See https://gitlab.gnome.org/martymichal/gegl/-/commits/wip/martymichal/opencl-refactor.

Everything else

During all the work described above I’ve spent time discussing with people on IRC, doing the occasional review of MRs and filing the occasional issue ticket. A list or

The end?

For the project, yes. For my contributions to GIMP, definitely no! GIMP is a part of my personal toolkit and thus I always have a reason to contribute to it, to make it a better tool for myself and also for all the other users.

I’m very happy that the GEGL Filter Browser has been merged into GIMP and will be coming in the upcoming 3.1.4 release. Despite the end of the project, I plan to finish the in-development branches regarding code examples in the GEGL Filter and Procedure browsers and to continue exploring how these browsers could be merged together.

A very big thank you goes to the whole GIMP community for always being ready to answer my questions, provide me with ideas and in general being a friendly bunch! Also, I’m grateful to Google for its continuous organization of GSoC and that I got to be accepted into the programme.