How dreams come to life: technology for capturing movements
Admiring the grace of the heroes Assassin’s Creed or looking at the slightly awkward gait of Nathan Drake from Uncharted, Few people think that every movement of heroes on the screen is the result of the work of hundreds of people. Without a technology that seemed a couple of decades ago, it was difficult to imagine any large -budget game today. Remember only last year The Last of Us or Grand Theft Auto 5, Where the performers of the main roles did not just read their remarks into the microphone, but fully played: they rushed along the stage, shook their hands and turned the chairs. Well, when developing Beyond not only body movements were captured, but also the facial facial expressions.
Today, Motion Capture is used almost everywhere – from medicine to sports. However, we are primarily interested in what this technology can give art.
What is it and what it is eaten with?
First, let's figure out how Motion Capture is produced. As a rule, on a specially equipped set (you need very good light, specific “scenery” and, preferably, one-color background), cameras are installed that read round markers (“balls for ping-pong”, as they are often called) from actors' costumes. The result is applied to three -dimensional models, and they "come to life".
► Many mistakenly believe that when using Motion Capture, animators are not needed at all. This is wrong. Make the “captured movements” look as it should – work is not easy.
It would seem that the matter has been done. As if not so – this is only the first stage. After taking movements, the animators will have a long and painstaking work on the smoothness and credibility of the hero’s movements-the technology is still not perfect. In addition, sometimes during the post-production, the authors decide to correct something, remove, replace-the bend of the elbow, turning the head, details in the gait, but anything! And if the actor played not a person, but a creature-let's say, a dragon or a gorilla-only the basis remains from the original animation. The result of all these works is a magical result: fantasies that have spun in the head of the director (screenwriter, game designer) are embodied in reality on the computer monitor.
Varieties of systems for capturing movements today mass. The most popular are marker. Among them there are passive ones, when the “balls” transmit information about their position, reflecting the infrared light sent by him (after recording, the brightness is reduced to such an extent that there is nothing left on the screen except brilliant points). There are active ones where, instead of reflective markers, LEDs are used, equipped with their own processors and radiosynchronization. They also use magnetic (the location of markers is calculated according to the distortions of the magnetic flux) and gyroscopic (the sensors are transmitted directly to the computer) markers and even whole mechanical costumes.
Each system has its own advantages and disadvantages. Passive, for example, is accurate, but if the marker is overlapped with the camera lens, the signal is not counted, but as to magnetic, there will be any interference on the capacity of the capture.
► Yes, this handsome actor also played this handsome man from the Planet of Monkeys.
Finally, there are completely non -macker systems that use expensive and difficult to organize cameras, and, according to their creators, they are “cheaper, more precisely and more convenient” by the classic Motion Capture. The catch is that in most cases between the cheapness, quality and convenience you have to set “or”, and not “and”.
But the equipment evolves, and if earlier, even in special studios, animators faced many problems, then the systems gradually become more accurate, cameras are capable of working at all over the most difficult, and the capture of facial expressions has ceased to be so difficult (although it remains the most difficult).
How did it all begin
The beginning of the history of Motion Capture with a certain stretch can be considered 1915 when the famous innovator of animation Max Fleisher invented the rotoscopy technique: first, a shooting was carried out with a completely living person, and then the “film” was projected onto the glass and the frame of the frame behind the scenes outlined everything. To create the first cartoon using a rotoscope, it took a year and 2600 frames – and this is for a timing per minute!
But the technology was improved, the animated series Fleisher Out of Inkwell Almost everything began to be used about the adventures of the clown, and in one form or another, Rotoscoping began to be used, including Walt Disney Studio. However, now the technology, if used, is for other purposes: artificially increase the mass, add some glows during post production, and so on.
► Before starting shooting, the costumes on costumes must be calibrated so that the accuracy and location of all objects are perfect.
The next impetus was made in the 1970s, and, oddly enough, not filmmakers, but … kinesiologists, whose responsibilities include the study and treatment of our musculoskeletal system. It was they who were the first to think of attaching to the body of patients (and actually subjects) potentiometers and, reading the data from them, digitize movements using a computer.
The method began to borrow everyone who is not lazy, simultaneously introducing improvement. Over time, systems based on LED markers and various reflective elements appeared (the result was not very accurate and required the subsequent “cleaning”), and in 1983 a unique Graphical MarioTte system was developed, which with grief in the rendering of the characters in thus animized in this way. But to the stage when Motion Capture began to be used in cinema and games, more time has passed. Only in the mid -nineties did it become clear that the technology is ready for commercial use.
About capturing moving in the cinema
In the movie Motion Capture in the usual form, they first used in 1990 in the film " Remember everything " With Arnold Schwarzenegger starring. Despite the fact that in terms of special effects, the picture was largely a breakthrough (animatronics, complex makeup and the effect of miniatures were adjacent to the rare CGI at that time), the characters animated using the technology of capturing the movements looked, that sin to hide, not very believable.
► Over time, interest in Performance Capture is shown by increasingly well -known actors. And, only by starting work, they understand how difficult it is to play such roles.
As for the truly loud roles of Performance Capture (this is when not only movements are transmitted, but also facial expressions), they had to wait until 2002, when visitors to the cinemas saw Gollum in " The lord of the rings "Peter Jackson. Yes, there were successful examples before – for example, in the film " Star Wars: Ghost "(1999). However, only Gollum was able to force everyone to believe that the future had come, and many critics vying shouted that the performer of this role Andy Serkis Be sure to award the Academy Award. Unfortunately, representatives of the American Academy of Cinematographic Arts and Sciences did not listen – in their opinion, the actor should attend the screen in the flesh, and all this digital heresy is not at all a game.
► in Gollum, the features of Serkis, his signature plastic and facial expressions are easily recognized. It’s even a pity that “Oscars” still do not give for such roles.
At first, on the set, even Serkis himself seemed to be doing something ridiculous. However, it is thanks to the wealth of his plasticity, voice, as well as the incredible work of artists and animators, even now, after a dozen years, Gollum makes a strong impression. According to Peter Jackson , He was so amazed at Andy’s talent that he gave the animators to the installation: to make facial expressions and appearance of the character as close to facial expressions and appearance as an actor.
As we know today, the decision was correct, but the impressive result was not easy: all scenes were shot several times, with Serkis in a white suit and other actors, without Serkis, but with the rest, and, finally, with only Andy Serkis in the studio, hung a bunch of reflective balls. What is funny, according to the actor himself, the shooting was conducted in some abandoned warehouse, there were few cameras, and computers were hanging almost every half an hour.
In the next film Jackson, " King Kong "(2005), where the role of a huge gorilla also went to Serkis, there were already 52 cameras in a special pavilion reading the actor’s movements, and twenty more were responsible for the face. But the process itself has changed little: part of the time Serkis worked with colleagues in the film in a special “muscular” costume of the gorilla, and some were alone hung with various markers.
► Play gorilla is an incredibly difficult task. But Serkis in King Kong turned out very convincing.
The stage of preparation for filming also deserves separate mention. Having agreed to the role, Serkis read a bunch of books and revised many documentaries about primates, but most importantly, he spent two months in the London Zoo in the company of several gorillas to study their behavior and understand how to play the role of such an unusual role for him.
And how strange the process of filming itself! Serkis, relying his hands on special runners or moving with the help of prostheses that lengthen his hands, imitated the movements of the gorilla, interacted with various constructions depicting trees and houses, and even gave King Kong his facial expressions (for this a special program was written that translated changes in the human faces in grimaces on the King Kong face)!
► Without Motion Capture, such scenes would be almost impossible: to manually make a bunch of animations of the wounded hero for the sake of a few minutes on the screen would be unreasonably expensive and for a long time.
The only scenes in which the filmmakers had to abandon Motion Capture are fights with dinosaurs. Peter Jackson had a “gorilla”, but to find a person who could be plausible to portray a tyrannosaurus, is that task.
A year before King Kong, the screens was released "Polar express" Robert Zemeckis , in which Performance Capture was used for all characters without exception. This provoked a sea of conversations about the death of traditional cinema and the transition from it to digital (but rhetoric, which was funny, was about the same as after the film was released Final Fantasy in 2001).
► Now the heroes of the “polar express” Robert Zemkykis look a little sloppy, but at the time of the release the quality of the animation was struck by the imagination.
Were and "Pirates of the Caribbean: a dead man's chest" (2006) C, without exaggeration, gorgeous Davy Jones. Well, after some time, everyone began to be surprised at the shot in 3D " Avatar ", After which the world of cinema, as it seemed to some, will no longer be the same (and in fact we saw only a few very beautiful films like" Gravity " And " Pi life »Yes, a wave of consumer goods, quickly converted to 3D).
As for the day of today, the most relevant example of using Mocap in the cinema is a blockbuster in the near future "Planet of monkeys: revolution". It is the “monkeys” that fully reflect how the technology has evolved over these years, because most of the filming was not in closed pavilions, but in the real forests, cities and so on. Given that before, even shooting in a specially equipped hall was given with great difficulty, to observe how they are in nature today is amazing.
► "Avatar" at one time became a convincing demonstration of the possibilities of modern technology.
But participation in the project of Andy Serkis – no. For more than ten years have passed, but in the world it would not have appeared about the lounger professional in this field. It was Serkis who got the role of Caesar – a monkey who has to become a key figure in a brewing conflict between people and primates. And this role is the next proof of how hard the actors of Performance Capture do and how amazing results it brings.
Serkis, who already had the experience of playing distant ancestors of a person ( "The uprising of the planet of monkeys" , " King Kong "), It was not enough" only "to simulate the behavior of the monkey – in the film, primates evolved into a highly developed society with their own way of life and language. The need to show in the character both intelligence characteristic of Homo sapiens and animal features is the main difficulty of this role. Did Serkis manage to cope with his task, we will find out very soon.
About capturing movement in games
Andy Serkis made a considerable contribution to the development of the video game industry. Cooperation began almost by accident. The fact is that the mortgage broker, whose services used by the actor, turned out to be the brother of one of the developers Heavenly Sword (2007), which is being prepared at that time exclusive for PS3. It was he who showed Serkis Trailer, who made such a strong impression that the meeting with Ninja Theory , the authors of the game, became only a matter of time.
► Serkis's character in Heavenly Sword.
As a result, he not only brilliantly portrayed King Bohan, the main villain of the game, but also put his hand to almost all key scenes as a director. And then played the memorable role of the monkey (yes, yes!) V Enslaved: Odyssey to the West. In the second project, the influence of Serkis was felt even stronger, and in many ways thanks to him the game turned out to be so good.
But before the transition to the complex Performance Capture, our favorite industry has gone a long way. At first, the best animation was achieved by rotoscoping. The most striking example is Prince of Persia (1989). The author of the game Jordan Mehner outlined the movements of his younger brother, as well as shots from some films – for example, “ Adventures of Robin Hood "(1938).
► on the set of ENSLAVED. Over the years of work, Andy Serkis became a real master of this direction. This is probably why more and more teams invites him not only as an actor, but also as a consultant and even a director.
In 1993, Mehner went even further: he founded a new company, knocked out an unthinkable budget of six million dollars, and for a whole month he was shooting with living actors. Four years later, the material obtained became the basis for one of the most unusual and underestimated games in history – The Last Express. She amazed her script and concept (everything happened in real time – and this is seventeen years ago!), as well as a very unusual style that resembled a revived coloring (the nearest analogue from the world of the cinema – “ Flaw »Richard Linkleter).
However, all the time that the mehner needed on The Last Express, the industry did not stand still. Good Motion Capture demonstrated Mortal Kombat (1992), Rise of the Robots (1994) and three -dimensional Highlander: The Last of the Macleods (1995). But until the moment when the technology for seizing movements was able to save the developers from tiring manual labor (which also did not provide believable), again many years passed.
► ENSLAVED turned out to be far from the perfect game, but the characters, directing and acting there were beyond praise.
By and large, only at the end of the zero animation in the games became really high -quality, and the realistic emotions of virtual actors we saw recently. The best example is the detective simulator in Los Angeles of the 1940s L.A. Noire. The developers lit up the idea of making interrogations by truly emotional and naturalistic, so that according to the facial facial expressions it is possible to understand whether he is lying or not, and therefore do not stint for the new-fangled technology of the Motionscan, created by Australian company Analysis. With her, to capture movements, neither markers, nor a funny costume, nor any “cleaning” of data are needed for her. Only one brightly lit room and thirty -two cameras of very high resolution, shooting at a frequency of 30 frames per second. Well, sitting motionless in the center of this impromptu scene, the actor, of course. The cameras cover his face from all possible angles, and the output is a very lively animation that repeats any, even the most insignificant movement.
► facial animation in l.A. Noire without exaggeration, outstanding. However, the head and body of the heroes still sometimes "live" separately from each other.
True, despite the highest quality of the animation obtained in this way, Motionscan also has enough minuses. Firstly, the actors need to make up before shooting, it is quite difficult to make some fantasy https://bubblesbet-casino.co.uk/ hero. Secondly, the “capture” is in two stages: first the classic Motion Capture for body movements, and then boltology in the chair. After that, the "talking head" is put on the shoulders of the computer model. As David Cage correctly noted, this makes the upper and lower parts of the character somewhat separate. In everyday life, we constantly tilt our heads in different directions during a conversation, and when you sit in a chair, you cannot change the situation. And the role of the “body” often “plays” a completely different person, an understudy or stuntman.
► Meet, Nathan Drake with his own person. Although … who does not know him now? Actor Nolan North voiced several dozen games.
The gamemeracker's opinion is worth listening, he is familiar with the technology of capturing movements firsthand. In him Beyond: Two Souls The largest volume of Motion Capture in the entire history of game construction was performed. During the filming, work was carried out on more than three hundred characters, 23,000 unique animations were created, and the process itself stretched for a whole year. However, no wonder: game Quantic Dream It does not belong to the shooters or the militants in whom many actions are repeated – this is an interactive movie, where every battle, jump and opening of the door should be unique.
Moreover, Quantic Dream is one of the few developing companies in the world where they can do high-quality Performance Capture. Simultaneous recording of body movements, facial animation and voice allows you to greatly accelerate production and fully concentrate on the main thing – working with actors and plot. This is especially important when Titans like Willem Defoe and Ellen Page participate in the project. By the way, according to David Cage, he wrote the script, holding only Ellen Page in his head as a leading role, and no one else.
► Naughty Dog have their own approach to everything, including Motion Capture. This is probably why almost every game collects a whole pile of awards.
Californians from Naughty Dog profess a slightly different approach. In their games, actors do not give the characters facial expressions and appearance (artists draw concept art long before the casting), although they are completely laid out on the set in the presence of a demanding director. Let's say the same tears of the actor on the site will not digest, however, in order to believable to convey the mood, he needs to cry.
But if the facial expressions of roles performers are not fixed with cameras, then why in the same The Last of Us Such high -quality facial animation? The answer is amazing – everything is done manually, in the old fashioned way. Every time Joel wrinkles his forehead or purses his lips, this is the result of the work of the animators' team.
* * *
Being an actor Performance Capture is not easy. You must imagine that the ridiculous structure of metal is a machine, a piece of cardboard that you need to chew with an appetite is pizza, and a partner standing in a stupid “costume of a diver” is a real beauty. At the same time, the requirements for the quality of execution are the same: the hero must be endowed with character, emotions and soul, as in any film or theatrical production, but now it is much more difficult to catch the essence of the role when there is no normal scenery and props around.
► The sensational Move and Kinect work on the same principle as Performance Capture. In the first case, the device uses a marker, in the second – no. However, until the moment when it will be possible to make high -quality animation without leaving at home, it will pass again a lot of time.
As Andy Serkis says, the actor’s game will always depend on the well -coordinated work of many people – animators, costumers, make -up artists, specialists responsible for lighting, installation and special effects. In some scenes, actors are replaced by three-dimensional models, some tricks work out understudies, complex makeup hides half of the face, but all this does not interfere with the American Academy of Cinematographic Arts to nominate performers of such roles on Oscar. And for the specialists of Performance Capture, like the same Serkis, for some reason, are denied rivalry with colleagues or at least a separate nomination. I want to believe that everything will change soon.