darktable 2.0 released with printing support
Darktable, free RAW processing software for Linux and Mac, got a major update just in time for your festive season.
The most visible new feature is the print module that uses CUPS. Printing is completely color-managed, you can tweak positions of images on paper etc. All the basics are in place.
The nice “perk” of this new feature is exporting to PDF in the export module.
The other important change is improved color management support. The darkroom mode now features handy toggles for softproofing and gamut check below the viewport (darktable uses a cyan color to fill out of gamut areas). Additionally, thumbnails are properly color-managed now.
Something I personally consider a major improvement in terms of getting darktable to work out of box nicely is that the viewport is finally automatically sized. No longer you need to go through the trial-and-error routine to set it up in the preferences dialog. It just works. Moreover, mipmap cache has been replaced with thumbnail cache which makes a huge difference. Everything is really a lot faster.
If you care about losing your data (of course you do), darktable 2.0 finally supports deleting images to system trash (where available).
The port to Gtk+3 widget set is yet another major change that you might or might not care about much. It’s mostly to bring darktable up to date with recent changes in Gtk+ and simplify support for HiDPI displays (think Retina, 4K, 5K etc.)
The new version features just two additional image processing modules:
- Color reconstruction attempts to restore useful data from overexposed areas in your photos.
- Raw black/white point module is pretty much an internal feature that the team hopes you never ever touch (of course you will). It was a prerequisite step towards dual-ISO support and better denoising.
Other existing modules got all sort of tweaks and updates. Most notably, deflicker from Magic Lantern was added to the exposure module.
Additionally, the watermark module features a simple-text.svg template now, so that you could apply a configurable text line to your photos. Which means that with a frame plugin and two instances of watermark you can use darktable for the most despicable reason ever:
The most important changes in Lua scripting is that script can add buttons, sliders, and other user interface widgets to the lighttable view. To, the team started a new repository for scripts on Github.
Finally, the usual part of every release: updates in the camera support:
- Base curves for 8 more cameras by Canon, Olympus, Panasonic, and Sony.
- White balance presets for 30 new cameras by Canons, Panasonic, Pentax, and Sony.
- Noise profiles for 16 more cameras by Canon, Fujifilm, Nikon, Olympus, Panasonic, Pentax, and Sony.
For a more complete list of changes please refer to the release announcement. Robert Hutton also shot a nice video covering most important changes in this version of darktable:
LGW spoke to Johannes Hanika, Tobias Ellinghaus, Roman Lebedev, and Jeremy Rosen.
Changes in v2.0 could be summarized as one major new feature (printing) and lots of both under-the-hood and user interaction changes (Gtk+3 port, keyboard shortcuts etc.). All in all, it’s more of a gradual improvement of the existing features. Is this mostly because of the time and efforts that the Gtk+3 port took? Or would you say that you are now at the stage where the feature set is pretty much settled?
Tobias: That’s a tough question. The main reason was surely that the Gtk+3 port took some time. Secondly, the main motivation for most of us is scratching our itches, and I guess that most of the major ones are scratched by now. That doesn’t mean that we have no more ideas what we’d like to see changed or added, but at least most low-hanging fruits are picked, so everything new takes more time and effort than big changes done in the past.
Roman: The Gtk+3 port, as it seems, was the thing that got me initially involved with the project. On its own, just the port (i.e. rewriting all the necessary things, and making it compile and mostly be functional) did not took too long, no more than a week, and was finished even before previous release happened (v1.6 that is). But it was the stabilization work, i.e. fixing all those small things that are hard to notice, but are irritating and make bad user experience that took a while.
Johannes: As far as I’m concerned, yes, darktable is feature complete. The under-the-hood changes are also pretty far-reaching and another reason why we call it 2.0.0. The Gtk+3/GUI part is of course the most visible and the one you can most easily summarize.
Jeremy: I’d like to emphasis the “under the hood” part. We did rewrite all our cache management, and that’s a pretty complicated part of our infrastructure. I don’t think this cycle was slow, it’s just that most of it is infrastructure work needed if we want darktable’s visible feature set to grow in the future…
Darktable seems to be following the general industry trend where software for processing RAW images becomes self-sustained, with non-destructive local editing features such as a clone tool, as well as sophisticated selection and masking features. In the past, I’ve seen you talking about not trying to make a general-purpose image editor out of darktable, but these features just seem to crawl in no matter what, you are even considering adding a Liquify-like tool made by a contributor. Would you say that your project vision has substantially changed in the past? How would you define it now?
Tobias: I don’t see too many general image manipulation features creeping in. We have masks since a while, and the liquify/warping thing would be another one, but besides that I don’t see anything. There is also the question where to draw the line. Is everything besides global filters (exposure, levels, …) already a step towards a general purpose editor? Are masks the line being crossed? I don’t know for sure, but for me it’s mostly pushing individual pixels, working with layers, merging several images. We do none of those and I hope we never will.
Johannes: I think this is caused by how darktable is governed. It’s very much driven by the needs of individual developers, and we’re very open when it comes to accepting the work of motivated contributors. we have a large dev basis, so I guess it was just a matter of time until someone felt the need for this or that and just went ahead and implemented it. I guess you could say we weren’t consequent enough in rejecting patches, but so far I don’t think this strategy has hurt us much. To the contrary, it helps to foster a large community of motivated developers.
HDR merging does exist though, and there’s even a feature request to add manual/automatic alignment. And both duplication and configurable blending of processing modules are a lot like working with layers, even though the processing pipeline is fixed.
Tobias: Yes, but that doesn’t counter my point: Editing single pixels is out of context, general calculations like that fit.
Johannes: To give a very specific answer to this very specific question: the HDR merging works on pre-demosaic raw data (which is why we have it, it’s substantially simpler than/different to other tools except Wenzel’s hdrmerge which came after IIRC). So automatic alignment is not possible (or even manual for that matter).
Have you already defined any major milestones for future development?
Tobias: No. Version 2.0 had the predefined milestone “Gtk+3 port”, but that was an exception. Normally we start working on things we like, new features pile up and at some point we say “hey, that looks cool already, and we didn’t have a release for a while, let’s stabilize and get this to the users”. There is a lot less planning involved than many might think.
Roman: As Tobias said, there are rarely pre-defined milestones. It is more like, someone has some cool idea, or needs some functionality that is not there yet, and he has time to implement it.
Personally, I have been working on image operation for highlight reconstruction via inpainting. There are several of them already in darktable, but frankly, currently that is the one of important features that are still not completely handled by darktable.
There has been a lot of preparatory work under-the-hood over the last two releases, which now opened possibility for some interesting things, say native support for Magic Lantern’s Dual ISO, or new version of our profiled denoise image operation.
I’m also looking into adding yet another process() function to image operations, that would not use any intrinsic instructions, but OpenMP SIMD only, and thus, making darktable to not have any hard dependency on x86 processors, i.e. it could work on ARM64 too.
Jeremy: I would like to add the manipulation of actual image parameters to Lua, that is a big chunk of work. Apart from that it will mainly depend on what people do/want to do.
What kind of impact on users’ workflows do you think the adding of Lua scripting has done so far? What are the most interesting things you’ve seen people do with Lua scripting in darktable?
Tobias: Good question. We slowly added Lua support since 1.4, but only now we start to get to a point where more advanced features can be done. In the future I can see quite some fancy scripts being written that people can just use instead of everyone coding the same helpers over and over again. That’s also the motivation for our Lua scripts repository on GitHub. While there are some official scripts, i.e., mostly written and maintained by Jeremy and me, we want them to be seen as an extension to the Lua documentation, so that others can get ideas how to use our Lua API.
The results of that can be seen in the ‘contrib’ directory. The examples there range from background music for darktable’s slideshows to a hook that uses ‘mencoder’ to assemble timelapses. We hope to see many more contributions in the future.
Jeremy: Lua was added mainly for users that have a specific workflow that goes against the most common workflow. Darktable will follow the most common workflow, but Lua will allow other users to adapt DT to their specific need.
That being said, I agree with Tobias that Lua in 1.6 was still missing some bricks to make it really useful. Without the possibility to add widgets (buttons, sliders etc.) to darktable, it was impossible to make a script that was really useable without technical knowledge.
With the Lua repository and the possibility to find widgets, things should go crazy really fast. Did you know that you can remote-control darktable via d-bus by sending Lua commands?
In early days of darktable quite a few features (e.g. wavelet-based) came directly from papers published at SIGGRAPH etc. What’s your relationship with the academic world these days?
Tobias: We didn’t add many new image operations recently, and those that got added were mostly not that sophisticated that we had to take the ideas from papers. That doesn’t mean that our link to the academic world was dropped, Johannes is still working as a researcher in university, and when new papers come out we might think about implementing something new, too.
Johannes: Yes, as Tobias says. But then again graphics research is my profession, and darktable is for fun. No, seriously, the last few siggraphs didn’t have any papers that seemed a good fit for implementation in darktable to me.
Several years ago you switched to rawspeed library by Klaus Post from the Rawstudio project. Now it looks like darktable is the primary “user” of rawspeed, and your own Pedro Côrte-Real is 2nd most active contributor to the library. Doesn’t it feel at least a tiny bit weird? ;)
Tobias: I think it’s a great example of how open source software can benefit from each other. I’m not sure if that’s weird or just a bit funny.
How has your relationship with the Magic Lantern project been evolving, given the deflicker feature etc.?
Tobias: The deflicker code wasn’t so much contributed by the Magic Lantern folks but written by Roman with inspiration from how magic lantern does it. I don’t know if he used any code from them, maybe he can clarify. Apart from deflicker there are also plans to support their dual-iso feature natively.
Roman: The only direct contribution from Magic Lantern project was the highlight reconstruction algorithm that made it into v1.6. The deflicker was implemented by me, as it usually happens, after I needed a way to auto-expose lots of images, and found no way to do it. That being said, it uses exactly the same math as deflick.mo does.
Tobias: Even that was not taking code from them. Jo wrote it after talking with Alex at LGM.
Johannes: But it was most inspiring meeting those folks in person. And yes, I was a lazy ass implementing this dual-iso support natively in darktable ever since LGM.
Darktable seems to be doing pretty well without any kind of community funding which is all the rage these days. What do you think are the causes to that effect?
Tobias: Well, we’d need some legal entity that takes care of taxes. And to be honest, we don’t need that much money. Our server is sponsored by a nice guy and there are no other expenses. Instead we have been asking our users to donate to LGM for several years now and from what we can see that helped a lot.
As for why we have been doing so well, no idea. Maybe because we are doing what we want without caring if anyone would like it. To the best of our knowledge darktable has exactly 17 users (that number is measured with the scientific method of pulling it out of thin air), so whatever we do, we can lose at most those few. Nothing to worry about.
The new version of darktable is available as source code and a .dmg for Mac OS X. Builds for various Linux distributions have either already landed or are pending.