Rigging

Trails and tribulations of the Advanced Skeleton Plugin

https://www.animationstudios.com.au/advanced-skeleton

In an attempt to lessen the load and skip what I'd heard to be a long process of rigging a character by hand through Maya's standard controls, I thought I'd used an automated plugin programmed to build it for me. I had heard of this plugin through Lipin Murali and Isaac Boardman as they had used this specifically for Rocket Dog. Isaac sent me the link and I got it set up on my PC nice and quickly.

There were some good tutorials on the Advanced Skeleton website and at https://www.youtube.com/user/AdvancedSkeleton but I found a more detailed step-by-step one on YouTube from https://www.youtube.com/c/AnimatorArtistLife which I followed.

The majority of the process was very straightforward, lining up half a standard human joint skeleton to the model, and adjusting the length and position of joints when needed. Once everything was lined up, including arms, fingers and root positioning, the advanced skeleton builds up the other half mirroring the joints onto the other side of the model.

After this stage the Skin Cage was calculated; this creates an area around the character that will be what moves when the joints are moved. This was followed by adjusting the skins so that there was nothing overlapping, articulating focusing on the finger area as them being so close together can cause problems.

An important step was adjusting the joint controls. As the model rig was generated so were the controls and some had generated inside the model making it hard to select and use them easily. Luckily Advance Skeleton has a tool to scale up the controls and adjust this when needed.

To help give extra control of the rig and how the movement will look, there is a step called Painting Skin weights. This is the process of adding weight to the correct areas, for example, the shoulder area is one that is likely to look warped when it is moved in the rig. Doing this helps control how this warp will look.

With that, the body was complete and bound to the model.

Moving on to the face there were many more steps for the build to be completed. However, I noticed there were issues with this setup from the beginning.

Above is what the set-up is supposed to look like. And below is what I was experiencing.

With one of the first steps being to select edges for where the eyelid would be going, I noticed it was not becoming a red blocked-out shape upon finishing it, as shown in the model above. I redid this step a few times just in case there was something I missed but I kept seeing the same result. Thinking it could just be a visual glitch and the system could still work for the overall build, I continued on to the mouth area where I also did not see a red area appear. With the same logic, I continued on and placed the dots around the eyebrows, jaw, cheek, chin and nose as the tutorials instructed. Another worrying sign happened when I did the steps for the forehead and instead of creating that green blocked shape between the vertices I selected, brightly visible on the model above, it created confused-looking green lines. It was definitely looking unhopeful for this step in the rigging process.

Once this was complete I should, in theory, be able to press Build Face Skeleton and it would automate the build, taking some time to go through some expressions and create the rig. Unsurprisingly this did not work. What happened after I clicked for the build to start was a very quick loading time, unlike the tutorials, and then a control appeared similar to the tutorials but only two controls within, one for each eyeball. This eyeball control worked which was great, but unfortunately, nothing else on the face had a control.

I tried a few things to get this fixed. Going over the whole process multiple times didn't get me anywhere so there was nothing I missed. I tried going through it with the unsymmetrical face settings, so doing the selections for the whole face rather than just the one side. This did change the result. I also tried going through the build one part at a time, I feature of the Advance Skeleton specifically there for if the automatic face build didn't work. This again got me as far as the eyeballs before not letting me get any further.

The Advanced Skeleton is specific in saying the model would work best if entirely symmetrical. My model had a slightly different typology on the face due to the hair blocking a larger area of the forehead. In my many attempts to fix the issue, I experimented by mirroring the face topology and this still didn't work. The plugin itself was completely up to date so I really struggled for a while to understand the issue.

I had decided to use the Advanced Skeleton Plugin as a way to overcome the large and long task of building the rig by hand, as I had limited time to complete many steps this term, this seemed like the best solution. Unfortunately, this actually ended up costing me a few days of trial and error and I definitely should have moved on to an alternative method sooner.

Blend Shapes

My solution to the face rig issue was Blend Shapes. It is something I was somewhat aware of from my research but I had dismissed them as something that could potentially take too long due to the idea that you would have to hand sculpt each expression. Looking at it in comparison to rigging the face by hand, however, I could see that this was the better option. It was a choice between trying something I had never done before, going through a 5-hour tutorial and hoping it would pay off, or sculpting in a way I was familiar with and simply changing out the expressions shown clearly in a few 15-minute tutorials. It also helped that I knew the exact expressions I would be needing from what I'd already planned out in the animatic, so I didn't need to make more than what I needed.

Another good thing about using Blend Shapes is that you can mix up your expressions and use a little bit of one to change another slightly. For example, I could use 0.5 of the surprised face and 0.5 of the scared/sad face to show a more shocked expression.

With expressions done that just left the eyeball movement to be rigged and I already knew a method that without a doubt worked for that. So the Advanced Skeleton didn't completely fail me with the face.

Closing Thoughts on the Rigging Step in the Pipeline

Whilst my experience didn't turn out to be the most efficient I think this pipeline of the automatic plugins and blend shapes is an interesting one.

Overall the Advanced Skeleton plugin was extremely effective at giving me a perfectly working body rig. The body has some great natural movement and has the option to use both FK and IK forms of movement. IK legs help the knees and leg joints move naturally and with the options to bend the arms with FK moves or an IK blend mode it really gives you the freedom to animate however you'd like.

For me, with my short film animatic all worked out with the knowledge of exactly what expressions I needed, the blend shapes were a perfect option to save the face rigging issue. If this were a rig for a longer feature or a TV show, however, I believe there would have needed to be a lot more options within the expression library. Even then, the blend shape seems to be a very convenient feature that could be applied to more items than just expressions so a very exciting thing to learn about.

Texturing The Character

The texturing process of the pipeline definitely had its up and downs. It was mostly downs until I finally got to painting on the stylised texture I wanted. To get there, however, there were many setbacks in the process.

UV Mapping

Firstly, I had to start with UV Mapping. This was a process of looking at the unwrapped map of the model and cutting seams for where each part's texture would change. So the skin is on one level, the hair on another, clothing, eyes etc. This was a relatively easy task, I had some great guidance from Kelly Clayton, advising me on things such as how the seams on the clothing parts of the model can just match how they would do on a normal item of clothing. This would then make things look more natural when the final textures were on there.

Baking the Model

After the UV mapping was complete and saved to different texture shaders according to what was going where it was then time to attempt to get the model into Adobe Substance Painter. This is where I had some trouble. You see in order to get the most out of the higher poly sculpted model you have to perform a step known as baking. This bakes the details of the High Poly onto the Low Poly model you have UV Mapped, therefore allowing you to paint your texture onto the right details and have a Normal Map of this you later plug into Maya. A Normal Map is a way of showing the bumps and dents in a 3D object with just a 2d texture.

This is where the setbacks started to raise as I had to spend a good long while figuring out why my model wouldn’t bake correctly. There were many possibilities I went through with the help of Kelly. The main idea was to separate all the main parts of the High Poly model so that it was on clearer geometry, so I got to work doing that. Luckily it wasn’t too difficult as I had a ZBrush file which still contained the parts of the model on separate sub-tools.

There was also another option to solve the baking issue by simply baking it in ZBrush, which did not turn out as simply as first thought though as it required the long repetition of baking each item separately rather than the quick way Substance Painter would get it to work. So I continued work on correcting the model in Maya with the separate High Poly parts from ZBrush. There was an issue here as there had been alternations correcting some positioning on the arms and a number of other corrections. After things had been lined up again and more work done on connecting inconsistencies between the High Poly and Low Poly model, I continued to try and bake the model again in Substance Painter and with the help of Kelly Clayton, it was finally able to be worked on!

We discovered the best way for the back to work was for both the hair and the eyeballs to be moved off the character to be in the air beside it. This made for a difficult time when working on the eyes as I wouldn’t quite be sure that the irises were the right size until I put them back on the model after texturing but at least it was able to work at all!

There were still a few issues that seemed to not line up however I had spent so much time on this stage that I made the decision to move on rather than waste more time fixing small details.

Texturing in Substance Painter

Finally getting to paint the model was so satisfying and it felt so good to get back to something I was a bit more experienced at with digital painting, however, I had never painted onto a 3D object before so getting to grips with that and the new software was definitely the most fun I have had so far in the modelling pipeline!

After a bit of experimenting to understand the tools, layers and masks with the help of multiple YouTube tutorials, I started with one fill layer to get the base colour down. I turn used a paint layer to start adding the variations of colours on the face. As I am going for a style similar to Arcane, stylistic with painterly shading based on real features, I wanted to start with a good base to keep the face looking natural enough through the painterly style.

I used these references to paint the base colour:

Adding these colours then adjusting the layer opacity and a blur filter to blend more with the general skin tone was a great point to start adding more colour to.

For the extra painterly shading I wanted to do I used these reference images as inspiration for shading positions and how far to push the stylisation.

I held back a little with this as it’s hard to say from these references what is texture on the model is and what its cleverly created lighting. I realised after completing this there may have been a way to find what the unlit models from Arcane looked like if I searched deep enough into Art Station, but I am still happy with the outcome and working from these references was working from something closer to what initially inspired me in the first place, so this felt more enjoyable.

The main areas I focused on were around the eyes. I knew this was an area I wanted to add a lot of stylistic texture shading as I feel it adds depth to the character, showing age and exhaustion. I wanted these bags under the eyes to show the strain this character goes through, it indicates his exhaustion before even starting the project. With this design, there’s already this sympathy and empathy some might have for a character who might work full time, be exhausted when they come home, too run down to do self-care acts of keeping their space clear and clean. This was an important aspect I wanted to get across in my character design, using the eye bags and hunched-over frame to indicate perhaps an office job or just a general depressed look.

As well as making use of the various brushes Substance Painter had available, I also used some of the filters to give more painterly looks. Amongst the brushes and textures they had available, there were specific materials that helped me give the clothing the right look. There was a knit texture that I used as a filter to draw on the jumper. There was also a brush that was able to create a seam line which is what I used for the jeans.

Other than the visual development process, this Substance Painter experience was by far my favourite part of the 3D pipeline so far. Despite the struggle with the baking initially, the gratification of finally seeing the model with the colour and style I have been imagining this whole time come together and work was worth it all.

Here is the model before a few changes:

Here is the final model texture with the Toon Shading effect:

Tutorials used:

https://www.youtube.com/watch?v=RQ-hRk0WHJ8&t=10s

Stylisation:

https://www.youtube.com/watch?v=ewJUHANsiFI

https://www.youtube.com/watch?v=h8llGEKIQT0

Face:

https://www.youtube.com/watch?v=VcH47dl1xWg

https://www.youtube.com/watch?v=V_E1ARR_AAg

Figure Drawing at the Botanical Gardens

As the summer term meant there were only postgrad people around there weren't any Life Drawing sessions to go to. However, Lipin and Luke were delighted to find that there was a free session at the Botanical Gardens down the road.

This was a lovely break from the heavy computer work I had recently done for texturing and Animatic Blocking. The session was about 2 and a half hours in the greenhouse and it went quickly with many different poses. The model seemed to love getting as close to nature as possible which was an excellent opportunity to practice the figure drawing in conjunction with the plants.

I tried to draw the environment with each pose to help line the proportions, something I'm always wanting practice with!

I definitely would like more experience working on my lines with different plants and leaves, a stronger connection between some of these figures and plants would have greatly improved these drawings.

There was a pool of materials to choose from in this session so I had fun playing with the colours available, particularly enjoying the bright on black combination.

As well as the line work I was also able to experiment with lighting my favourite being this final one I sketched between the reads.

This was a perfect break from 3D work and I believed loosened up my abilities to finish animatic work and character design.