I’ve been slowly building up an OPDS feed, because I added support to KNewStuff, and I wanted to make sure I was able to make an OPDS feed myself.
Of course, the biggest problem is content. I have some fan comics lying around, and I have some Krita Resource Bundles that are all over the place… But there was a thing missing. Back when I was fifteen, I started work on a comic named Streets of Ganjet, an urban fantasy comic about the people living in the city of Ganjet. I have tried to rewrite it, but never got far enough for it to publish new chapters.
So, last week I spend time on figuring out different animations in terms of raster animation that is then baked into the final tile. This week, I investigated a different approach to game graphics: Shaders.
Shaders are formally tiny programs that are specially written for the graphics card. In game engines, shaders are most prominently used to calculate how a 3d object should be rendered, or shaded. But they’re also used in 2d graphics a lot. Krita for example uses shaders to make the canvas perform better, handling cursor and to display HDR images. Godot also allows applying shaders to 2d graphics, even specific tile graphics. So I spent some time playing with that.
In the case of water, there’s a ton of stuff out there on water shaders in 3d. But I am interested in water shaders in 2d, to spruce up the water effect I had going. So I set up a tileset in Godot, with my water tiles as an ‘animated texture’ as one of the spritesheets/tileatlasses used for the tileset. Godot allows individual textures that are part of a tileset to have a shader assigned to them, so the following visual shader was set up:
Image displaying the full visual shader graph.
This may seem like a very overcomplicated thing, but visual programming always looks overcomplicated, as each block in visual programming is a single operation. I chose to use visual programming here because I imagine there’s people out there for whom regular programming is a bit intimidating, so here’s a confirmation that what I am about to do can be done in the visual programming interface. Let’s break down what is done here!
Blending
Starting out, these are the water tiles we have:
We overlay these over terrain on a seperate tile layer:
This looks a bit dull, so the first order is to blend them with the underlying tiles. We do this in two passes: once with multiply, and once with overlay, so that the contrast gets increased nicely, as is typical of wet things in the real world. So we go from the most basic ‘canvas item’ visual shader:
To this:
In the above, you can see two things not mentioned. First is that the preview is useless here. The second is that we also use vector interpolation to blend the multiply and the overlay 50% with the underlying texture. This is to avoid oversaturated colors.
Next up, we’ll add some clouds to give the effect of an overhead sky. I want to show the overlaid area only when there’s no cloud shadow, so you get the effect of sun reflecting on the water. For this we’ll use a simple black and white texture.
We’re getting to the point where the visual shader graph is a bit too big to really fit, so subsequent images are going to only show a small part, and you’ll have to imagine to integrate them into the whole.
So, the next step is getting the clouds to move. This is as simple as taking the ‘time’ input and using a vector op to add it to the uv input of the clouds. (The uv being a coordinate system that the texture is mapped onto) However, we’ll want to control in which direction the clouds move. We can just use sine and cosine to calculate x and y modification from a radian angle, but most people prefer degrees for input. So we’ll need to convert those degrees to radians. Furthermore, we will also need to make sure the input is limited to 360 degrees.
To do this, we start with a ScalarUniform we’ll name ‘Angle’. Then, whenever you use this shader, Godot will present you with a little input labeled ‘Angle’ that you can set to anything. Another one is ‘speed’, to indicate how fast the clouds go. The visual shader graph is the following:
In the above image, we take the angle, use the modulus/remainder operator to limit the value to 360, then multiply by 3.1427(visual shader editor doesn’t have constants) and divide by 180 to get the corresponding radians. The radian value then has sine and cosine functions applied to it to get the x and y coordinates. These are normalized values, so we can just multiply them with the time vector. The time vector is the time interpolated with 0 using the ‘speed’ uniform as the interpolation factor. That is, if the speed is high, the resulting value will be higher, if the speed is low the resulting value will be low. The resulting value is our time vector, which is then multiplied with the angle vector and that is then added to the screen uv, which is what causes the movement animation.
Water flow
For water flow, we can use a flicker animation and overlay that. Then, similar to the clouds we can move them in a given direction. But first, the flicker animation is 1/8th in size compared to the animated texture. We’ll need to apply a transformation matrix to the uvs of the flicker texture here.
In the above graph there’s still the time stuff, but we’ve hidden it offscreen for now. Once the UVs are aligned properly, you can use the alpha of the flickers to interpolate/blend them onto the main texture.
So, one of the things that I really wanted to figure out was whether I could use flowmaps to control the water direction. There’s been several papers on this, with the most notable one being the valve paper, where flow maps weren’t just a simple but cool result, but also helped players find their way through a map. I tried to glean most of my info from this useful tutorial on the subject.
So, in the above we have a flowmap texture(that is aligned to the original water texture, more on that later), this is then decomposed into xyz values. The xyz values are decoded from their 0 to 1 range to the proper -1 to 1 range so they’re proper vectors. Finally, the result is multiplied with the time vector, and then added to the other UV modifications(offscreen). This is following the initial part of that tutorial, giving the simple ‘distortion’ effect (the tutorial also recommends adding a fraction function so that the time doesn’t repeat into infinity doing weird things in the shader).
Above we replace the vector op ‘multiply’ node with an attempt to rotate the time vector with the vector from the flow map, following the second part of the flowmap tutorial. I was unsure how to do this, as I kept thinking to myself ‘surely, they’re not expecting us to manually multiply the matrices?’, but after some search this youtube video demonstrated that indeed, in the visual shader graph the matrices of the two vectors do need to be multiplied manually. Note that the first vector compose, and two decomposes are unnecessary here, but are just there to help visualize.
In the end, I would’ve liked it if I could have created a second tilemap where I would create the flow using tiles, and then used that to inform the distortion, but that doesn’t seem to be possible. So while it is possible to create a flowmap effect in Godot’s visual shader, I am not sure how to effectively create and align a flowmap to objects in the game engine.
Other
If you add in masks to emphasize the edges, and a mask to make the sparkles in the center less strong, you end up with something like this:
Meanwhile, on the OGA forums, BlueCarrot was able to get a much simpler and easier to use flow effect going by just animating center tiles, so next stop is to experiment with that instead. I just figured I’d document everything I’ve learned up till now. 🙂
Overall, it was a little surprising that the Godot visual shader graph, which is for people not experienced with coding didn’t really have easy ways to generate a transformation matrix from vectors or radians from degrees, or even have access to common maths constants like pi. This makes the shader graph surprisingly barebones. The above shader could probably also be optimized, but right now this was more of a ‘how would we go about it’ rather than worrying about speed.
I’ve only recently become anywhere decent at crochet. That is, making fabric stuff from yarn with a crochet needle. My stepmother taught me how to do the basic slipknot and chain stitch in finger crochet when I was a little girl, but my attempts at getting any better at it failed for the longest time.
The issue was that whenever I would try to learn how to do crochet from a book, the instructions were always incredibly vague. To the point that when I finally learned a stitch beyond the chain stitch, it was not the single/double stitch I was trying to learn, but instead ended up being a sort of weird slip stitch that lead to a very stretchy fabric resembling knitted work rather than crochet.
Videos on the other hand tend to have a little bit too much information going on for me. Like the tutor’s voice, anything wrong with the video lighting wise, any blurriness.
So I really wanted to do some animations that show how a given stitch is done, without too much noise surrounding it.
First, I made a sketch in Krita at 1600×900 pixels. The sketch is super rough as you can see. The idea is to get the motion down first.
I then cleaned it up a little and started playing with aspect ratios.
Here’s one that’s more zoomed in, giving more attention to the actual important part of the image. I do feel that this decreased the sense of how the hands move, which is why I decided against using this one for the final animation.
Another option was a square ratio that was super-zoomed in. This was somewhat inspired by the gifsets on tumblr and cooking videos.
And then shared that on mastodon.art, as I had done when I made a turntable of Kiki:
The general feedback was that the square images were nicest for actually getting the stitch. Because of my worries about the hands, I ended up deciding to continue with the full view version, and make smaller square ones for social media. The idea being that the full version can be viewed full screen and the stitch should be easy to read then.
The Second Day
I spend the second day firstly, building Krita with Address Sanitizer enabled, so that I could catch memory related bugs. The Address Sanitizer allows me to find bugs, in particular we’ve been implementing a lockless hashmap for Krita’s canvas tiles, and this could have some scary bugs that the address sanitizer might be able to find. Basically, the Address Sanitizer will crash as soon as it finds a memory-related bug.
I didn’t find hashmap bugs, but I did find the following bugs:
As well as several other bugs. The big downside to the Address Sanitizer(and GDB as well, really) is that Krita will take up twice as much ram, which was a little bit of an issue, as discussed later.
When animating, I spend more time looking at my hands and the motion I did, and I noticed that I was missing the almost natural first step of making a loop with my fingers. So I added that. I also tried to make the hand motion at the end a little more natural, letting the yarn hand pull at the knot as well.
There’s also pauses now, letting people identify the separate steps, and I colored the thread red.
The feedback I got on mastodon was that the pauses helped a surprising lot. The other feedback I got was that it was still hard to tell what was going on with twisting the loop or the pulling of the yarn into a knot at the end. I decided then that I should add text, as well as try to make the animation smoother so the motion is easier to follow.
Third day.
So, then I started inking the image. I first decided to increase the size from 1600 × 900 to 3840 × 2160, which is the 4k definition. The idea being that if I got it that high res, then I wouldn’t have to worry about the inking lines looking awful. You see, video codecs tend to be optimized for gradients and smooth areas, so images with a lot of contrast, such as traditional raster animation tend to be disadvantaged when converted to video. Having a high resolution offsets this issue as well as other compression errors.
I also doubled the frame count to smoothen it out. The thumbs and rest of the hand are separated. For the crochet needle I made one basic frame and then copied and rotated it all over the place. This part, as well as the hook hand made me wish for tweening on transformation masks, but that’s something that requires a smarter person than me to finish it.
At the end, I tried colouring the hands so it wouldn’t be a mess of lines. I ended up with too much ram usage, so I tried to reduce my ram usage by reducing the layer type or color space used. After all, the line art didn’t need to have 4 channels when all it needed was alpha, so I converted those to greyscale. This had the side effect of turning the onion skins grey too… I also tried using animated fill layers for the hands, but it seemed that upon playback there was a single frame lag for the fill layers but nothing else, which was a bit of a letdown.
Eventually I scaled the canvas down to 3200 × 1600, which is still quite huge, but at these sizes it made a full gig of difference. Anyway, at the end of the day, I had this:
The turntable of Kiki, the Krita mascot was a previous animation project I did which allowed me to practice my knowledge of orthographic projection, which I then could reuse for the hands here.
And the next day, I also animated the yarn, coloured everything, and used transparency masks to ensure that the needle and yarn were masked at the correct moments. There’s a little bit of stretch and squash happening on the yarn in the pulling moment that I hope really sells the motion.
The final animation as screenshotted in Krita. The file is 2.2 gib, my ram and swap(8gib and 4gib respectively) are almost full. The blue frames are the inbetweens and frames that only contain color. The yellow, purple, green and orange frames represent the different steps. The red frames are frames that are the most extreme of a motion. I can’t tell left from right without thinking, so I just named the two hands the yarn and hook hand 🙂
Editing the video
Because Krita was having so much trouble, and I still wanted to add text, I decided to finish this in KDENLive. I couldn’t get anything rendered from Krita’s side even, because I had earlier that week installed earlyoom to prevent my whole desktop locking up indefinitely as it is wont to do due to me doing a little bit too much with my computer. So I basically had to render the frames, but then Krita refused to render the frames and gave no feedback why, but I suspect it was because of running out of memory. So what I did was I just tried to render out frames, and then when Krita errored out, rendering again, but then starting from the last frame Krita rendered. This is sort of what that feature was designed for.
I brought the animation into KDENLive. I Have KDENLive 18.12.3 on this device.
KDENLive, while it can import frame sequences doesn’t seem to like them much. It didn’t allow smooth playback for them, filter effects didn’t seem to work(I wanted to get the animation slowed down to have it playback at 16 fps instead of 24), nor did it have a menu option to transcode. So I had to go into the commandline and convert the different steps into mp4s. When using the log docker or a terminal, Krita will spit out the ffmpeg commandline entry into it before rendering, so I could copy-paste that and use it as a base.
KDENLive also had some other issues, like an issue where it would just kinda, ‘error’ when resizing clips or moving them around, complete with error noise. Afterwards it would refuse to do anything until I restarted it. Another issue was that there was this ‘ghost’ clip somewhere that didn’t show up in the timeline, but it was affecting the render, making it a 19sec animation instead of 11. I had to work around it by defining a zone and only rendering that. Then, later, KDENLive upon startup would just randomly have said clip appear. The final issue is that sometimes when playing back, KDENLive will just have a memory spike or will slowly consume all memory meaning that I couldn’t preview the animation correctly, as earlyoom would kick in and kill KDENLive.
Adding the text was painless, however, and KDENLive’s list of render settings are a blessing. So I rendered the file to webm, and uploaded it.
Of course, 10 hours afterwards, someone goes: ‘well, that last step is too short’. The reason the last step feels too short is because the text for it is too long. If I hadn’t put in any text, this wouldn’t have been an issue. I did this again this morning(that’s a full day afterwards) because someone else had the same issue, and I suspect that each time a line ends there’s a significant lag while our eyes do a ‘carriage return’, so that’s something that ought to be taken into account.
Webvtt
Because I am dyslexic, and also Dutch(we’re a little language obsessed in the Netherlands), I value captions and subtitles a lot, and try to always make them for my videos.
Webvtt is the official webfriendly format, but the typical subtitle creation software like Aegissub don’t support it. So to test this, I made a very simple html file with a reference to a webvtt file:
I then made that file, using KDENLive as a reference to determine the timings. KDENLive’s timestamps go mm:ss:frames instead of hh:mm:ss:miliss, this was a little bit of a surprise.
WEBVTT
REGION
id:title
width:24%
lines:1
regionanchor:0%,0%
viewportanchor:6%,9%
REGION
id:instructions
width:24%
lines:5
regionanchor:0%,0%
viewportanchor:6%,57%
STYLE
::cue {
background-color:#ddd49f;
color:black;
}
STYLE
::cue-region(instructions) {
align:left;
color:black;
}
STYLE
::cue-region(title) {
font-size:200%;
}
00:00:00.000 --> 00:00:03.125 region:title
Crochet
Slipknot
00:00:03.125 --> 00:00:05.333 region:instructions
Step 1:
Make a loop between your fingers.
00:00:05.333 --> 00:00:06.625 region:instructions
Step 2:
Put your hook through the loop.
00:00:06.625 --> 00:08.625 region:instructions
Step 3:
Turn your hook so the loop will twist.
00:00:08.625 --> 00:00:12.375 region:instructions
Step 4:
Now push the hook down and pull up the yarn through the twisted loop.
00:00:12.375 --> 00:00:15.000 region:instructions
Done!
You're ready to start your project.
You can then open the html file in Firefox, and it’ll allow you to select the subtitle track for preview. Firefox despite everything still doesn’t support styling in webvtt files, which is kinda annoying. Chromium also didn’t. VideoLan does support styling, but video lan’s then doesn’t support alignment of the text, which is a little weird.
All of them, however, support regions, which is more than I expected, to be honest… Sadly, peertube, where I uploaded the video to, has a player which does not support regions.
But none the less, here’s the final result:
The final video.
And the square ones of each step:
I made these in Krita eventually, because I couldn’t figure out how to get KDENLive make me a 1:1 project. It was just a case of importing the frames and cropping them and then copy-pasting the layers containing the title.
While doing this I noticed I had masked out the crochet needle in the last step incorrectly so it showed the needle always in front of the yarn instead of briefly behind it to indicate a turn…
Step 1: Make a loop between your fingers.Step 2: Put your hook through the loop.Step 3: Turn your hook so the loop will twist.Step 4: Now push the hook down and pull up the yarn through the twisted loop.
Afterthoughts
Generally, while animating was fun, I always just kinda… lose all motivation when having to deal with the video editing part.
Video editing mistakes tend to haunt me more than anything, and I am not sure why. Maybe it is because my videos actually get comments unlike my writing and my art. Maybe it is because having to fix a mistake in a video, unlike writing, always leads to having to open the video editor, deal with potential bugs, and then you always have to delete the previous video, copy over all your comments, reupload the video, and then have to anticipate the next set of people going ‘oh, hey there’s this mistake over here’ and I have to delete the video, open up the editor, deal with the editors bugginess as I fix the issue, rerender the video, reupload it, just so I can wait for the next set of comment that…
And I also just kind of have the feeling that because I am an experienced artist and have a good sense of rhythm, I am cursed with the ability to see all the ways in which the video is wrong, but not the experience to fix it with confidence.
Other than that, I would like to share the source files for this one, but the issue is that git isn’t very binary-files friendly(which all video and image files are, as far as git cares), so I am not sure how to go about sharing the source files. In total, I think I ended up about 15~ hours on this, of which 10 were the actual animation.
I do kind of want to continue animating these stitches, but there will be a little bit of a pause in between, I think. Hopefully the reported bugs and address sanitizer backtraces mean that others who animate in Krita will have a smoother experience, but I think people will always have to watch their ram usage.
A Comic page in peruse creator with frame and text area definitions. Blue is frames, red is text areas.
So, I got my thesis done, updated the Comic Project Management Tools, and had a lot of time left till I got my thesis results(I did not pass >_> sadly). One thing that was sort of bugging me was that after all the work I did on the CPMT, there just wasn’t much movement happening in Peruse, the KDE comic book reader that can read comic books with ACBF files and make use of the extra functionality.
The middle one being a watercoloury style(probably needs better colour picking to be really useful)
The last being traditional ink-then-colour, softshading style, and the first being a variation where I painted the subject first, and then added the lines. Painting is a little more comfortable for me regarding details like hair and folds(which are nice to sculpt this way), so it’s a little more natural to me to try this manner.
