Facial Expressions for Crew Portraits

Hey Folks! I did tech art today! I know that probably sounds unexciting to most, but it's sort of a significant coincidence for me. Tech art is basically the thing that got me into the games industry, and my first job at BioWare. It covers a lot of things, but is basically like a dual-class artist/programmer. Someone who can create art content and tech features to amplify them (and/or tools to create them).

In today's case, it was an experiment to see if I could generate facial expressions as a UI for current crew status. Since this game is going to focus a lot more on emotional and mental state of AI, and not just survival stats, it makes sense to find useful and engaging ways to communicate that info to the player. I could do a million stat bars, sure, but nobody would read them (let alone understand them). :)

In yesterday's screenshot, you can see one potential portrait style I'm considering. I kinda like how it looks, but it's static. I'd have to draw dozens of sprites to have even a few emotional states for each face type, and if I want any amount of face variety, that gets crazy fast.

However, with some clever slicing and mesh rigging, it's possible to combine face parts on-screen with animation to get a wide range of expressions simply by setting up animation poses for some basic model, and swapping in different sprites later for different face appearances. All of them can use the same animations to show various states.

What's more, I can hook-into the face parts with code to control more than just 100% happy or sad faces. I can do things like blending a little bit from each, or setup partial animations that can combine, or even procedural things like random eye blinks or twitches. There's some real potential here for some cool/immersive UI elements!

Of course, this assumes I can get everything working. And there's a lot of setup. But the good news is this is a mostly front-loaded cost. Once setup, I can theoretically add as many variations of face styles (and new face expressions or decorations) as I want.

In today's screenshot, we see my first draft of a face rig hooked up to some test assets. Some of you old timers might recognize that face. A buddy of mine from way back when I was in web dev showed me Ken's face demo. This tech has come a long way since then, as you see in AAA games with facial acting. However, this demo came to mind as a really simple implementation of a face. Indeed, that was one of Ken's goals for this demo: as few points as possible to successfully emulate certain emotions.

Seemed like a good place to start for simple 2D face sprite deformations!

It also works pretty well with the art style I was trying on yesterday's faces. Flat-shaded means we don't have to worry about seams or shadows much. We can just focus on shapes changing over time. (Ken's demo is fully 3D, so the face can also turn and move. I'm experimenting with layered 2D sprites for now, but it's possible 3D might actually be easier if the model is simple enough.)

I've just got the scene setup, and have started adding a few facial expressions. Tomorrow, I'll try to hook those up to code to see if I can get them to change at will, and maybe look a bit more lively.


Marc13Bautista's picture

Love this take on things! Much truth can be said that facial expressions > only status descriptions. Can't help but think about the live character portraits in the game called "This War of Mine."

Free Elusive Skill = Athletic x4 in ATN Enclave encounter

dcfedor's picture

Did the live portraits in TWoM animate at all? Or do you just mean the faces were photos of real people, and that made it more immersive?

I'm trying to think of other games that have done something like this. A lot of AAA games have full facial animation/acting, of course, but I mean more like a UI element to show status.

Doom is a pretty famous one, with the face showing damage states.
Fallout 1/2 both had facial expressions on named NPCs when in dialogue.

Most RPGs just had color portraits that turned black and white when the character died, and otherwise used buff icons. I can't think of many that did facial poses for various states in the UI.

Did Starfox? I remember there being wingmen in that game.

Dan Fedor - Founder, Blue Bottle Games

Rovlad's picture

The photos in This War of Mine only blinked sometimes (to look more "alive", I suppose), but weren't animated beyond that.

Actually, nothing like you describe comes to mind except maybe Wing Commander 1/2, which used sprite animations for in-flight communications between pilots. In WC3 and beyond they already used FMV for almost everything.

Oh, and some (very) old school RPGs like Realms of Arkania and Lands of Lore actually had separate portraits for "wounded" and "dead" characters. For some reason it never caught on though.

dcfedor's picture

That's too bad (that it died off). As I tinker with it now, it seems like a really cool way to add life to a game, particularly with party members.

I'm already seeing how it could get quite complex, though. I'm basically approaching Dragon Age face complexity asymptotically here. Morph targets, blending trees, swappable face parts...I guess if you're going this far, you might as well have an FPS :)

On the other hand, I won't be doing full VO, cinematic performances, full 3D plus lighting, or even very sophisticated art. So it might end up in some middle ground where it's a powerful and interesting widget. Something that catches players' eyes as being quite different from both old-style CRPG and modern AAA.

Dan Fedor - Founder, Blue Bottle Games