So, with Krita 5.0 nearing completion. There’s been some discussion about what we’ll do next.
On of the proposed topics has been to replace our calligraphy tool with something that can produce nice variable width editable lines.
There’s a number of use-cases for this:
- Inking is exhausting, so making it possible to edit lines after the fact can help in cleaning up a piece after doing the majority of the lines while requiring little motor-skill precision.
- Inking requires a lot of practice, so in a studio setting, you’ll likely have a mix of artists, most of which aren’t that skilled at drawing yet. Editable vector lines can help create some consistency in the line work produced by a variety of artists.
- Variable width lines have some computer uses as well. We could think of erasing lines until an intersection, but also of simplifying the search for path boundaries, for fills.
- Scaling and the like can be done much better with vectors than with raster images, but I personally haven’t seen this used much. Animation, similarly, seems to still per-frame line work as the interpolation tech for vector lines does not solve all use cases.
Usually beginners also really like the idea of editable vector lines because then they can ‘always fix their mistakes’, and I am wary of this as it is a perfectionism pitfall. Strokes that always taper at the beginning and end are another example of something beginners ask for (because they do not have a tablet or the motor skills to taper), but I feel equally uncomfortable about this, as inking without a tablet is a recipe for RSI.
There’s many different ways to approach such lines, and I’m writing this blog post as a study of the different types, problems and what kind of needs we’d have to fulfill if we want it to be on par with raster inking.
To do this, I have used the following sketch and inked it with Krita’s raster tools, Inkscape’s Power stroke and Blender’s Grease Pencil. I will note the good and bad qualities of each. Furthermore, I’ll talk a bit about other solutions I’ve seen as well as Krita’s existing calligraphy tool.
The sketch was made by taking the basic-opacity brush, drawing a basic pose, then using the adjustment curves to quarter the alpha, then halving the brush size and refining the sketch, rinse and repeat. This is a technique I tend to use if I am unsure about what I am drawing. The result tends towards the somewhat stiff but technically sound side of drawing things.
Krita’s raster tools
Our ‘control group’? (Though, iirc, control group is when you explicitly do nothing to the base :D, maybe not really applicable…)
I’m using my own inking brush here that I made like 5, 10 years ago. It’s nothing special. I am also avoiding the stablizer, to give an accurate representation of the kinds of strokes I am making. Stablizer does makes things easier as you grow more tired over the duration of the inking session.
There’s a number of different kinds of strokes.
Long uninterrupted strokes. To make these, I need to use most of my arm. I also make liberal use of the ability to rotate and zoom the canvas so that I don’t have to do strange arm movements.
Smaller sketchier strokes. These tend to form a bigger line visually, and are made when it’s difficult to tell how the shape should go precisely. They result in messier lines than the uninterrupted ones.
Dotted strokes, this is when I am ‘feathering’, which looks a little like hatching, but where hatching is to shade, feathering as an inking term is to create a suggested line.
Erasing is happening in two kinds of situations: 1. Reducing a stroke. 2. removing a stroke that is otherwise overlapped.
To thicken a stroke, we go over it again with a brush.
There are certain situations in which the variable width vector stroke isn’t going to be very useful.
The first situation is when the stroke is textured or semi-transparent. These strokes tend to be made up from very small strokes. While you could imagine a computer doing that, it’s going to be odd.
Similarly, any inking technique that makes liberal use of the eraser is just doing to result in a very difficult to render stroke. Hatching too is easier to deal with in rasters than in vectors, as the number of strokes adds up very quickly.
Inkscape’s Power stroke
Inkscape’s power stroke is a live path effect, which is a feature of Inkscape’s that applies a modifier onto a given vector object. Indeed, if you have used Blender, it’s not dissimilar to it’s mesh modifier stack.
The SVG of the path effect looks as follows:
<inkscape:path-effect effect="powerstroke" id="path-effect17974" is_visible="true" lpeversion="1" offset_points="0.86244098,0.24492331 | 2.4187158,0.29378686" not_jump="true" sort_points="true" interpolator_type="CentripetalCatmullRom" interpolator_beta="0.75" start_linecap_type="round" linejoin_type="spiro" miter_limit="4" scale_width="1" end_linecap_type="round" /> <path d="m 141.35792,216.07057 c -0.0276,0.11281 -0.0716,0.34971 -0.12294,0.59413 -0.0292,0.13899 -0.061,0.27119 -0.1023,0.39613 -0.0148,0.045 -0.0319,0.0918 -0.0515,0.13927 -0.17908,0.45322 -0.40001,0.8479 -0.674,1.35131 -0.25497,0.46846 -0.49545,0.92864 -0.66586,1.44256 -0.0642,0.19583 -0.12884,0.42029 -0.1924,0.65448 -0.0674,0.24842 -0.13667,0.51918 -0.20297,0.78197 -0.11636,0.46121 -0.18741,0.75083 -0.29558,1.17151 -0.0498,0.19358 -0.10042,0.38704 -0.1522,0.57864 -0.1726,0.6388 -0.33964,1.18944 -0.51217,1.6492 -0.10012,0.26679 -0.19306,0.47861 -0.27912,0.64813 -0.12005,0.23648 -0.21672,0.37056 -0.28054,0.44715 a 0.29378686,0.29378686 90 0 0 0.4514,0.37614 c 0.0946,-0.11357 0.21536,-0.28464 0.35398,-0.55589 0.0985,-0.19276 0.20058,-0.42516 0.30701,-0.70769 0.18253,-0.4845 0.35573,-1.05698 0.52881,-1.70397 0.0521,-0.19477 0.10281,-0.39105 0.15221,-0.5866 0.10038,-0.3974 0.18356,-0.74376 0.28915,-1.17699 0.064,-0.26251 0.12997,-0.53067 0.19335,-0.77458 0.06,-0.23085 0.11914,-0.44579 0.17639,-0.62956 0.14323,-0.46633 0.35151,-0.8942 0.59486,-1.3678 0.25852,-0.50314 0.48454,-0.92799 0.66996,-1.41969 0.0217,-0.0596 0.0404,-0.1171 0.0565,-0.17063 0.0448,-0.14874 0.079,-0.29701 0.10986,-0.44306 0.0544,-0.25732 0.0991,-0.47638 0.12396,-0.57792 a 0.24492331,0.24492331 90 0 0 -0.47586,-0.11624 z" id="path-1-34" inkscape:path-effect="#path-effect17974" inkscape:original-d="m 141.59585,216.12869 c -0.0945,0.38686 -0.14019,0.78902 -0.28351,1.16057 -0.375,0.97221 -0.98283,1.79222 -1.30234,2.79068 -0.47491,1.48407 -1.0781,4.97731 -1.95794,6.03318" style="fill:#000000;fill-rule:nonzero;stroke:none;stroke-width:1.99937" />
We have a section in the defs that describe the effect, and then a path which references that. The power stroke itself consist of a base path, to which the live path effect adds nodes that represent the size at that point. There’s a number of additional options that allow controlling the behaviour at the ends of the curve, as well as the interpolation mode and other small details.
Inkscape 1.1 currently has the ability to create such strokes easily, by using the pencil tool and enabling the pressure sensitivity.
The usability is wanting, as this tool over-simplifies the input, resulting in lines that often only have one or two points to indicate the size, making many of them indistinguishable from non-power stroked paths. The bezier part of the paths is fine, but the few size-nodes per stroke is not.
When doing small strokes, this tool bugs out, creating strokes of the wrong size.
Beyond that, the pencil tool has no features to set the pressure curve, nor can the canvas be rotated with finger input (it can however be zoomed with finger input!). It also seems to be view-relative in how big the curves can be, which is really annoying when you’re the type to zoom out for larger curves.
Editing the final curves is quite easy, however. The size handles can be easily moved, and the bezier part reuses the bezier editing functionality from the rest of the program, allowing on-canvas dragging of the curves.
There are a number of editing operations which do not work. The power stroke LPE has no concept of joining paths, or having multiple sub paths in a path. The ‘tweak’ tool is also not possible on these paths, as it seems not to work on non-closed paths. This is unfortunate, as it means a whole subsection of vector editing functionality is not possible. The eraser in inkscape is also implemented using a completely different method, it just produces a variable path, then takes the existing paths, ensures the LPE’s are ‘merged’ by converting them to regular paths, and then subtracts the ‘erasing path’ (if the eraser isn’t set up to remove the whole path).
Overall, I get the impression that Inkscape over-simplifies in order to make the final stroke easy to edit. This however means that the tools cannot track my subtle motions, which makes using it rather frustrating, if not indistinguishable from drawing regular paths. Furthermore, the power stroke path effect isn’t treated as a first class citizen, and much of the vector editing functionality is something that cannot be reused. Perhaps the disconnect between the size and the nodes make it very hard to reason about what a join between two power strokes looks like.
Inkscape’s implementation seems to be inspired by that of Synfig, but I did not test it out.
Blender’s Grease Pencil
Blender’s grease pencil was originally designed to allow artists to leave annotations on each other’s 3d models and animations (hence ‘grease pencil’: an oily pencil that can be rubbed off a smooth surface easily). In recent years, the project has upgraded this tool to handle variable-width strokes and other functionality.
As per usual, Blender’s usability is… idiosyncratic. I used Blender a lot 8~ years ago, and can model in it decently, but this inking session took several tries to set up because I was flailing around trying to navigate it. I will try to avoid commenting too much on Blender being Blender in favour of evaluating the tool itself.
Blender’s implementation works by taking vertices, and giving them extra properties like size, opacity, color, and proly some other things. It then connects these vertices into a polyline. The polyline is then rendered in some way or another, though I don’t dare to say exactly how, the colors seem to be vertex-colors, and it’s capable of a soft edge, but no clue whether said edge is calculate over the line, or that the line is actually stroked with a soft dab.
The usability of creating the strokes is much better with Grease Pencil. The tool has the ability to set a pressure curve, as well as some smoothing options. Rotation of the view is not possible, and zooming and panning cannot be done by using finger gestures. Even worse, there’s no palm-rejection at all, leading to blotches.
The paths produced are very dense. This is noticeable when drawing, as the lines tend to pick up the details of my strokes very nicely. However, this in turn also means they are tricky to edit. The tools given to edit are largely the same ones used to edit the vertices in meshes (like sculpt and rotate/scale/translate of selected nodes), and I suspect this is the primary reason why the grease pencil is a polyline: Blender can then reason about these vertices like it reasons about all other vertices. On top of all this, erasing seems strangely difficult.
Solutions in CSP and Sai
So, I didn’t make an official test of these, but here’s some things I know about the solutions of these.
Like Blender, their implementation seems to consist of nodes that have certain properties, like size, opacity. Unlike Blender, it seems these implementations use a type of curve that does not use control points, but rather goes through each node smoothly (This might be a K-curve, or something like that, see links at the end for more info). The exact nature of these curves still elude me, but the lack of control points as is usual in bspline and bezier curves must be by choice. Another difference with blender is that the size per node seems to be in percentage to a full stroke size, with tools to adjust the stroke size all together.
These implementations are most clear about using a dab-based sort of brush engine to stroke the resulting path. All vectors are treated this way, so circles and rectangles are too always ‘stroked’. Of note is that this kind of stroking does not use any canvas sampling effects like we use in our color smudge, deform and filter brushes.
That said, like Blender’s implementation, editing the vectors is hard. This is partially because of the amount of extra tools that get added (and are in CSP’s case, nearly impossible to find), but also because there’s far more nodes, and the more nodes and properties there are, the harder things become to edit.
Variable width lines are a first-class citizen in these programs, because they are the primary vector implementation, so all the editing tools can edit them, and it’s possible to split, join and use the eraser to erase only what you need.
So, Krita already has editable variable width curves. But they are buggy and you can only edit the positioning. I tried upgrading these curves, but failed.
These curves store size and angle information per-node, and then tries to generate a path from that information. The path generation is bugged, because it does not understand how to handle sharp angled curves.
Furthermore, the parametric path class is not sophisticated enough to edit the size and angle inside the node, as well, we do not store this kind of specialized data at all.
I tried solving the save/load problem by refactoring the whole thing to use KisPaintInformation, like our brush engine uses. This sort of worked, but the reason I had to stop is that PaintInfo and friends tends to change too much, meaning my branch bitrotted in a matter of weeks.
Similarly, getting path rendering to work was a headache as well, and I in fact, never got it to work as it should. I am just not good enough with vector paths for this.
Finally, besides the saving, loading, path generation, and general inability to edit the path beyond positioning, the UX of the calligraphy tool is wanting. Most of the options are very obscure, and while I managed to make some headway in this in my branch, I suspect we might be best off to have the freehand brush tool generate these kind of paths when drawing on a vector layer (though this also has draw backs).
Right now, neither of the open source solutions I checked were really comfortable to use. Studying all examples, we can tell there’s a number of core questions we need to solve:
What base path to use?
The three things I looked at, all used different methods: Bezier curves, straight lines, and some kind of auto-smoothed curves that may be something akin to ‘k-curves’ or auto-smoothed curves.
Krita can already generate bezier paths, and more importantly, we have generalized UX classes for editing these bezier paths by dragging the paths. This can be useful if we feel that showing these nodes is too complex (mesh transform for example allows turning control handles off as the direct curve editing is enough in most cases). What this means is that we do not directly need to worry about the UI becoming too complicated when choosing bezier curves.
Straight lines are easier to reason about, and the weird autosmoothed curves have less UI trouble, but our bezier curve implementation is really thorough so I am not sure why we’d try anything else. Animation, perhaps? Did not research this.
How to render the final stroke?
Here the difference between the various versions is most prominent: Either there’s a vector path being generated from the original path data, or the original path data is stroked by a sort of brush engine.
The stroked version is easier to render, but harder to store to SVG: We’d proly have to store PNG data into the SVG, and then store the path data as a sub-elements, especially if we’d like to avoid having to re-stroke all paths on load. Stroked paths can have more cool effects like color and opacity changes, but are less portable, because only Krita would know how to scale them.
The vector paths version is in contrast, easier to store, it’d be a path, with path data as sub-elements, but harder to generate. Vector paths are more portable, but have less capabilities.
Both is in theory an option, but we’d still, beyond anything, need to choose the primary method at risk of scope-creep.
What data to store?
Inkscape only stores size, and the size nodes are separate beasts from the nodes that make up the bezier paths. Blender and other solutions (as well as Krita’s current solution), store the data into the path nodes. Synfig implements both, as noted in the extra links section.
I think we should go for storing the data into the path nodes, as it makes it a lot easier to reason about when deleting nodes, adding nodes and other path operations that flow from this (I recommend looking at the advanced outline link for reasons against this).
The kind of data we store will depend on how we render the result. Stroked paths will be able to do much more than vector paths. I do not think we should store paint-information, rather we should store end-result size/rotation/opacity/whatever we choose. These will be much easier to edit by the user.
Editing itself will also require a potentially new vector object to be written, as the parametric path shapes are designed for circles and rectangles, and not for bezier paths with extra information per-node.
Trade-off between too many nodes and too few nodes?
Generate too many nodes, and the path becomes harder to edit, generate too few nodes, and the path is harder to create (because it does not sample the user-input correctly).
Generally, I feel like programs seem to go too hard into either direction, and am not sure why we cannot choose how many nodes are placed.
My instinct is that we should generate too many nodes at the start, and then try to simplify afterwards. Perhaps even use ‘stablizers’ to let the user choose the simplification method.
The creation tool and other integration.
Like I said, I believe using the freehand brush tool on a vector layer might be sufficient. We could then imagine a model where we have a class for these kinds of strokes, which stores per-node size/angle/opacity/whatever. These then get generated by the freehand path tool, using the current brush preset to process the PaintInformation into size/angle/opacity/whatever for each node.
There’s one big problem here: Not all of our brush engines are dab-based. Furthermore, we need to communicate to users that the canvas-sampling features of o.a. the Color Smudge, Deform and Filter brushes do not work for these vector objects. Another problem is that we might introduce unique features where it comes to erasing and stablizing, and we’d need to figure out where we would keep track of such options.
Then we need to wonder: are we going to allow the straight line tool and geometric tools to generate such paths? This is particularly important if we go with the stroked render method. Similarly, we should keep track whether the fill and selection tools can use this information and how.
Finally, I personally think it’d be cool if we could use some heuristics to make the tool easier to use. For example, looking at my own test, if I am using small overlapping strokes, I clearly want to have a large final line, so why not allow this kind of gesture to extend an existing line? Other examples include: increasing the size by drawing over it, decreasing the size by erasing, removing overlaps when erasing. This is of course very pie-in-the-sky, and will need a lot of testing.
Who will use it?
We will need to find people who are planning on using these features. Not all of the folks that we usually consult care about this feature, even if they do a lot of line art, because of the reasons I lined out in the ‘situational use’ section.
I don’t think I’ll work on this tool, as I have been trying in the past and failed. Furthermore, there’s a number of other post-Krita 5 projects that I would be more successful in (color management improvements, QML port). None the less, I hope anyone who wants to work on this now has a good overview of the design decisions that need to be made.
Some links on vector lines
Some of these links discuss things that I was unable to cover, such as Synfig’s implementation.
- Synfig Outline Layers
- Synfig Advanced Outline Layer
- SVG proposal variable width lines
- Ralph Leviens Spline Work, this has some background info on k-curves.
- OpenToonz Manual page describing the tools to edit line thickness (Thanks to Jeff for pointing this out).