Sci fi and UX (Part 2)

by Daria Lanz on 11 April 2016

In my last blog, I made the bold statement that UX was the secret ingredient for sci fi stories to stand the test of time. I began exploring the relationship between user experience and science fiction, and compared the films 'Minority Report' and 'The Island' specifically, and how affective their use of user experience is.

The 'Minority Report' UI feels like it is a middle step between modern computers and holograms. Like a hologram, the interface is emitted by light rays, expanding the UI boundary beyond pixels and screen. However it is still dependant on a screen surface—like computers today.

I would say it was a precursor to the hologram, except we all know holograms have been around long before 'Minority Report' was made (2002). 'Star Wars' (1977) and 'Back to the Future 2' (1989) are good examples of this. 40 years after the first 'Star Wars' movie was released, films are still using holograms to portray the future, because we still haven't managed to crack the code. And until we find a way to stop light rays in mid air, we won't crack it.

But more importantly to the point, it is the user's experience of interacting with a hologram that makes it so ideal for sci fi, and emphasises my point that UX is the key ingredient to timeless science fiction.

Wouldn't it be amazing if interacting with our data could be fun?


Let's explore a more evolved hologram: Jarvis from 'Iron Man'. Jarvis, Tony Stark's operating system, is the next level up. Robert Downy Jr.'s character doesn't need any wearables or screens to interact with his OS, he simple uses his bare hands. He can pull data from seemingly thin air, move it around in 3D space, walk through it, under it, on top of it, collapse it, expand it, all with a few simple hand gestures.

He has the ultimate personalised user experience. He can bring the interface to him, and match it to his needs whether he's standing, sitting, running, dancing, or flying. It's like he's playing with it. He's having a conversation with Jarvis, building off the responses he receives.

Wouldn't it be amazing if interacting with our data could be so fun? Instead we are stuck (for now) sitting in an uncomfortable chair that is most likely damaging our backs, hunched over a tiny screen that is slowly deteriorating our eyesight.

Let me take a moment here to clarify that most UI design in movies would actually make for crap experiences in the real world. They are packed with flashy gimmickry, added for entertainment value. Jarvis is no exception, and would most likely fail any usability testing.

But that's not what I'm arguing. What I'm arguing is the simple fact that Tony Stark can use his bare hands to interact with a holographic interface in 3D space, using voice commands in simple conversational language – this experience is futuristic and beyond what we're capable of today.

In another scene, Tony Stark seems to be using a more traditional computer. He's interacting with a keyboard, 3 monitors, and a stylus. However, the UX has still evolved from current interfaces.

The keyboard is interactive, loading a foreign-looking set of keys and layout with the swipe of his hand. We could assume that this keyboard could have many layouts, depending on the type of interface needed. The stylus controls objects on the monitors through motion rather than contact. This allows for a more natural gesture than using a mouse, trackpad or pen tablet.

Here the familiar meets the futuristic. Tony Stark drags something off his triple-monitor setup onto a flat holographic table board. A hologram emerges, he ditches the stylus and continues to interact with it with his bare hands – similar to the way he interacts with Jarvis.

 Iron man1.png

Skeuomorphism is introduced with the trashcan to delete objects. This combined with his more familiar computer set up lends us to believe the tech could be relatively new. Designers are using skeuomorphism to help users learn the new interfaces.

As the new technology is developed (holograms), users adopt these systems and integrate them with their old systems (basic computers). There is a gradual shift as opposed to an abrupt change; we can expect to see older and newer systems being used simultaneously.

Iron man 2.png

Interestingly, if you look closely at his 3 monitors, the interface on the screen looks a bit like the familiar Windows platform, coated in a more sci-fi colour scheme. Many individual windows with lots of code are scattered across the screens, with the odd random colour wheel and organisational folder window.

You have to ask, would operating systems and programs in the future still look like this? Or could this design element potentially date this movie in the near future?

Humanising interfaces

'Her' is another great example of a story that explores the relationship between human and operating system.

The movie takes this to the next level, when Joaquin Phoenix's character, Theo, falls in love with his OS and begins a romantic relationship with her. It begs the question – is the idea of a human falling in love with an OS wrong? Are humans capable of falling in love with a 'human mind' only?

Considering the explosion of online dating where people fall in love with strangers across the globe, their only correspondence often being text messages, emails, phone calls or sometimes video calls, is there a massive difference?

The key in all this is the humanisation of the voice command system. In 'Her', users are first asked a few fundamental questions about themselves in order to be paired with an appropriate OS – who's name, in Theo's case, is Samantha. Similarly, Tony Stark's OS is named too – Jarvis.

Most importantly, both protagonists interact with their OS like they're another human being. They have formed meaningful relationships. Tony Stark converses with Jarvis, bouncing ideas back and forth like he is a genius lab partner-in-crime. Theo falls in love with Samantha.

Her 1.png

Humanisation of interfaces isn't something new; these movies have simply expanded it far beyond what we're capable of today.

In modern time, GPS systems have provided nationality (accent) and gender options for their navigators. Apple's Siri introduces a human element to your iOS. She has a name, gender, and a sense of humour (anyone who's asked her to beatbox can attest to this). We are constantly seeking to create and imitate human behaviour – something I'll deep-dive into in the third and final episode of this blog.

Where current audio interfaces fall short is artificial intelligence. There's a sizeable difference between the way Samantha and Jarvis operate to the way Siri operates. At best, Siri is more successful at providing entertainment than being a helpful personal assistance.

Learning to express ourselves

But perhaps there is even something to this. Perhaps we are seeking this emotional fulfillment rather than daily admin assistance. Humans have become so disconnected with our emotions. We rely on our phones to communicate, and we shy away from actual human connection. How many people have forgotten or haven't learned the art of having a real conversation? How many people sit on their phones instead of speaking to another person/meeting someone new when at the bar? Or on nights out? At a social event, parties, networking events? How many people break up over texts? Emails? Or even hire external companies who do the break-up for you (yes, this really exists).

How can we learn to be present in the world? This theme of emotions and expressing oneself runs through 'Her''s plot also. It is one of the brilliant things about the movie.

Theo's job is to express emotion for other people. Strangers hire him to write emotional letters expressing their feelings to their lovers, friends, family, and community. It brings to light this concept that we are so bad at expressing ourselves – so locked up – that we need to outsource it to a computer, an app, a company.

Theo himself is locked up, broken from his failed marriage. It takes his relationship with his OS to become more comfortable with his feelings again. He feels liberated and free. And his writing at work improves; he gets recognised for his 'emotionally beautiful work'.

So what is the meaning of all this? In the next episode of this blog I'll explore more thoroughly the concept of human connection, artificial intelligence and robots.


Thank you for your comment. It has been submitted for approval.

Comment Again?

Leave a comment

Please enter your name.
Sorry, this email address is not valid.
Please enter your comment.

Course basket x


Price per place

Add another courseCheckout

Pay now by credit card or later by invoice (invoice payments only possible if all courses are 3+ weeks in advance)