During a panel at GDC in San Francisco, that DualShockers attended, Final Fantasy XV Lead Game Designer Prasert Prasertvithyakarn gave a lot of information on one of the game’s most original features, Prompto’s automatic screenshots and selfies.
While you play Final Fantasy XV, the algorithm behind Prompto doesn’t just take random screenshots, but actually generates “stylish smart-photos” representing his point of view. Since the game’s release, Prompto’s pictures have been “overwhelming” Facebook and Twitter, helping Square Enix in promoting the game.
Prasertvithyakarn, who goes by the nickname “Sun” at Square Enix Business Division 2, (and so we’ll call him here henceforth, because those are seriously too many letters) is specialized in emotional AI, which basically means that he tries to provide artificial intelligence with the illusion of emotion.
In Final Fantasy XV he is in charge of the Buddy system, which includes Prompto, Gladiolus and Ignis. Sun created their thoughts, the way they move, act and talk. He also took care of systems like camping and the photo system itself.
All the systems in the game are designed to maximize the road trip experience, and when you talk about a road trip, you talk about photos. However, it’s difficult to compel the player to take photos in a game, not everyone likes it.
Sun was inspired by a road trip he went to with his friends. Since some of them are camera geeks, they took a lot of pictures on their own. This sparked the idea of having a party member taking pictures for the player during the game.
Prompto’s pictures have a few goals, the first of which is that they must serve as an achievement for the experiences of the player. When you visit a new location, you get photos. If you meet a new character, you get photos. Anything relevant gets you new photos.
This works for both memorable moments and for “unmemorable moments” like when Noctis is hit by a chocobo, gets sent flying through the air, or is turned into stone. Basically, the pictures must be able to showcase each player’s gameplay.
On top of that, pictures need to be able to show more than what the game itself has shown, displaying interesting off-screen moments to expand on the story and characters. This kind of picture also stirs the player’s imagination, and gives them a sensation of fun during the road trip.
When you see two characters posing together, you’ll also get the illusion of life. You’ll imagine that those characters have a private life during which they interact with each other. They might be chatting or flirting with each other just off the screen when the player is not watching.
Sun believes that despite the widespread use of social media, gameplay sharing is still limited to a relatively small group of players. Taking cool screenshots takes effort. However, Prompto’s pictures are all automated. The only question the player is left with is “to share or not to share.” This means that the feature could be a new benchmark for the next generation of gameplay sharing.
To ensure that, photos must be unique and fun. In order to achieve that, you need luck, but luck can be designed. For instance, Ignis normally wears glasses, but removing them for one of the pictures, creates a designed lucky moment that players may want to share.
Players are also allowed to create their own luck with the active snapshot that Prompto can take during battle.
You don’t even need a picture to be good for people to want to share it. For instance, having an NPC pass in front of the camera while the picture is taken doesn’t make for a good quality picture, but it’s fun, so it’ll possibly be shared.
Interestingly, accidents can be turned into entertainment, and this works even better with Prompto, because he’s such a goof.
Yet, this isn’t easily achieved, and Sun explained the processing flow behind the system.
First of all, we have triggering. In order to achieve variety, several kinds of triggering are necessary. After all, there are several reasons why humans take pictures.
The coincidence trigger is based on the concept that if you have cool actors doing cool moves, then a cool picture might be the result. The game’s animators gave a score to their own animations in terms of whether they’re interesting or not. Each character is also evaluated on the same parameter, then the score is combined and calculated during every battle frame. The system tries to snap picture when the score is maximized, while fitting as many characters as possible into the frame.
In addition to that, the event trigger is bound to story moments in cinematics, or interactive events, or when a new destination is reached.
There is even a whim trigger, because, after all, you don’t really need a reason to take a picture of your friend. Yet, there is a pattern behind this. Whim pictures are taken only when there are no other triggering events, helping to control the distribution of pictures during the day.
Last, but not least, there are manual triggers, like souvenir pictures or those activated by the player during battle.
No trigger is perfect when compared to the design policies mentioned before, but when combined together, they can satisfy all the necessary conditions and simulate people’s many reasons to take picture.
Once the team determined when to take a picture, they needed to determined how to take them, and the subject, which is governed by themes. Each theme has its own style, subject choices and specific algorithm.
There are about twenty different themes in the game, but three were brought as examples. “Joyful” pictures focus on characters and in making them look appealing. “Wonderful” pictures focus on the world. “Exciting” focus on combat.
There are also “preset” themes, for instance in cutscenes. AI is disabled during cinematics, because they’re not designed to show different angles. In other cases, it’s worth sacrificing uniqueness in favor of a perfect background which has more value.
Afterwards, details have to be enhanced, since at times “reality can be boring.” It’s important to add variety and life to pictures. This is achieved by separating rendering in the “photo world” from rendering in the game’s world. Things are considerably changed in the photo world without the player noticing.
For instance Noctis can be given a specific expression, improving the believably of the photo. Poses can be exaggerated for effect, and other element can be added, like an additional character and interaction, including Gentiana for her photo-bombing moments.
The system uses the bone information of the characters and environmental collision to create a good frame for each picture. That said, Prompto’s real position takes priority and is used as much as possible.
Information tags are also embedded into the photo, and used for further features, like post-filtering. Some filters don’t match well with some photos. Information tags are used to minimize the probability of getting those combinations.
The auto-album feature reduces the number of pictures to between eight and twelve, but the system actually takes more than twenty. Some have to be discarded, and that is quite tricky. Information tags help here, removing the pictures that are redundant or less unique.
When pictures are displayed, the game creates conversations that match them, giving the player the impression that the characters are actually around the campfire looking at the photos and commenting.
According to Sun, the result is one of the most powerful storytelling tools in Final Fantasy XV. Some believe that RPGs are all about the story, but Sun believes that this is only half correct: RPGs are also about the storytelling. Story and storytelling come together.
When Sun played Final Fantasy VII, and (spoiler alert) Aerith dies, he cried a lot. Since then, he always wanted to create RPGs to make people cry, because he believes that crying through entertainment is one of the best possible things.
That said, it’s not possible to rely only on storytelling based on cinematics and linear level script in an open world RPG, where players spend 99% of the time running around the map. That’s why storytelling needs to be done via the gameplay, using the AI and tools like photos. Photos have their own magic, as they can create an emotional pulse.
Photos are also like “time machines.” They bring you back to the moment they were taken, and remind you that that moment will never come again. This might even be powerful enough to make a player cry.
Sun took away a few lessons from creating this very challenging feature: the first is that you can’t achieve everything, and at times you have to sacrifice something. When the team started working on the feature, no one knew how far they could bring it.
There are lots of temptations when working on something like this, and if you don’t keep your goal firm in your mind, you will go astray, and waste time and resources on things you don’t really need or want.
For instance, a first temptation was to have Prompto gradually improve his photo skill. This was implemented only partially by giving Prompto new filters and unlocking selfies and self-shots, without actually improving the framing algorithm, as players wouldn’t really notice that.
On the other hand, the ability to manually take pictures was added in battle, as there already was an interface to tell your buddies to do something at a certain time. It was cheap to implement.
Another temptation was to have buddies actually pose during gameplay. That didn’t really help any of the goals of the photo feature, so it was not implemented at all.
The team also tempted to tackle 100% AI-driven framing, but it was impossible to achieve that goal, so they switched to something more doable, with the use of a semi-preset camera. The position and direction of subjects is preset, but camera position is calculated by the AI, so that variation is ensured.
Sun concluded by mentioning that now that the game is completed, people tell him that this feature is great, but during development many argued that it wasn’t worth it, and added nothing to the fun of the game. Every day, part of his job was to try to convince those people. Yet it was difficult, because Prompto’s snapshots aren’t just about creating photos, but creating photo experiences, and experience can’t be shared until you finish creating it.
So, the final message is “Be a researcher, but be a producer. Sacrifice something, but never give up.” This is the key to make something new.
In addition to this panel, Final Fantasy XV Director Hajime Tabata also hosted another explaining how the team made Final Fantasy a “challenger” again. He also showcased a very interesting video displaying some impressive technical experiments being conducted using the game as a test environment.
Below you can see all the slides of the presentation. Incidentally if you are a game developer (to which this panel was mainly addressed) but didn’t have a chance to attend, you can send me a mail requesting the full audio recording.
[On-location reporting: Azario Lopez]