Google Pixel 2 and Lightbulb Theory

A Short Tale Of Retail

I have been following technology for years, having spent many of those working in retail environments with high end equipment. Customers ranged from film production companies to members of the public with no knowledge of the current state of technology. This could be cinema cameras to smartphones. The most recent company I have worked with is Google, hence this article. I am also a professional photographer so I can observe this subject from both sides.

From the customer service side, every new encounter is a new puzzle to solve. How do you explain this technology / theory to someone with no knowledge of it or its sub technologies? If they are at a professional level, can you switch to using as much technical info as necessary? How many different ways can you explain the same thing? These translations of the features and benefits of a product require a genuine understanding of the technologies involved. There is no waffling your way through it. (That doesn’t mean other people working in my field don’t try to but that’s another story).

Oh, and I’ll come back to the translation model later.

It has been frustrating to witness this endless cycle of companies over the years trying to impress consumers with buzz words about their new gadget. They mention all of the options you are going to get and all of the customisability. For the more technical among us this style of product design and communication tends to work. We want more power, more options, more freedom. More options to play with in the settings, a thicker manual to read… Okay maybe scrap that last one. The problem is the vast majority of consumers have no interest at all in such things. They just want it to work.

Smartphone

Smartphones are one such sub-industry. They may have smaller sensors and fewer options (and a fixed lens, generally) but the growing marketplace for them makes the shrinking DSLR and mirrorless [digital cameras] market look pathetic in comparison. In 2016, global shipments of digital cameras were in the region of 25 million units. Source. In contrast, global smartphone shipments approached 1.5 billion units. Source. With this in mind it is no surprise that in 2017, over 85% of all images taken and shared are done so with smartphones. Source. So much for a ‘sub-industry!’

This massive consumer market (60x larger infact) naturally invites more investment due to greater competition. Smartphone manufacturers may be pressed to want to add more and more options. But is that what the consumer wants? Most of them have no idea what different settings actually do. In fact it kind of scares them. I think Google may have recognised this.

Google Wants More For Less

So it can be argued that this ‘legacy’ way of developing products (or marketing) can be based on an incorrect assertion. It isn’t that consumers don’t want more out of their devices. They just don’t want to put more work in to get it. In my experience, people are inspired to capture any given moment because of its emotional impact. Google has actually trained its neural network by feeding it images of people so it can recognise humans in photographs. A posh camera or one with fifteen pages of camera options does not inspire moments.

Most consumers don’t know or care how to use cameras and most of them didn’t want to have to read a book to use their device. They just wanted a camera to take photos of their newborn / family / holiday. They would purchase an £1000+ interchangeable lens full frame camera and leave it in ‘Auto’. Why? Because they think it will get them the best out of the camera without having to put the work in. That option may be fine for taking standard looking photos of a city scene in the middle of the day, but later in the evening, those same settings to photograph the moment when your child is doing a Tasmanian Devil impression may not work. The user won’t know why it is blurry, they’ll just be unhappy with their device. Not the best experience for a consumer to have. Sub-optimal experiences increase usage resistance.

Smarterphone

We have established that the smartphone market is seeing the most sales and the greatest fight to produce the most technologically advanced products. In this race (as in the digital camera space) you have hardware and software. The user interacts with the software and the software tells the hardware what to do. The hardware then collects data and sends it back to the software which processes it and presents it back to the user. In larger dedicated cameras the emphasis has traditionally been on hardware. ‘Better’ sensors, faster shutters, faster burst modes, battery life etc. This is because that market is populated with tech heads who constantly go on about that stuff and generally have underlying knowledge of the subject. The software takes a back seat.

Lightbulb Moment

But like I keep saying, the mass market consumer just wants it to be easy to use. Enter the humble lightbulb. Everyone knows how to use one, you plug it in and turn it on. Done. I bet most people have no idea who invented it or how many failed attempts came before it. They probably don’t really know how they work either. But that isn’t why people buy them. They buy lightbulbs because they produce light and consumers use the light to do other things. You could call the bulb itself a conduit technology in the sense that it merely facilitates another process and its success came because the usage resistance dropped so low it become ubiquitous. It gives you the result you want with minimum effort. Plug it in, turn it on. Done.

In my opinion, the same can be argued for smartphones. Phones began by providing the means to communicate, then they expanded to give you the power to consume content, the next step is to give you the power to create. But with all of these advancements in processing power and display quality, ergonomics etc, the weakest link is no longer the device, but its user.

I suppose it was only a matter of time until smartphones added that to their repertoire.

Google Pixel Visual Core

Let’s keep it simple. The above mentioned processor in the Pixel 2 replaces the role the user traditionally had in choosing the correct settings to get the best capture of a moment (based on the capabilities of the hardware). It is a custom built machine-learning processor that excels at crunching lots of numbers fast and efficiently. No it probably wouldn’t make the best general-use processor but you have the Snapdragon 835 for that. When it comes to processing image data the Visual Core is a whole new beast. When your average consumer puts the Pixel 2 into portrait mode and takes a photo and goes ‘oooh that’s nice’, this is what happens (pretty much).

Press Shutter Button:

  • Take burst of images y
  • Process y*12 million pixels of RAW data from the sensor
  • Lines up pixels from different shots in the burst, discarding ones that aren’t good enough
  • Averages the data in a larger luma/chroma volume
  • Goes through each pixel and chooses the one from the range that suits what the final image should have
  • Receives phase detection data from the sensor
  • AI uses it to produce a depth map from phase data
  • Refer to machine-learning analysis (trained on a million images)
  • Recognises continuity in contrast, lines, shapes, faces
  • Selectively blurs areas of the photo that should be out of focus

How long would it take you to manually edit an image like that? The Pixel Visual Core does it in four seconds. Also, processing is deferred until you have finished taking photos. How considerate.

Who Cares?

Does the consumer need to know all of this? Nope. They don’t need to either as this all happens automatically. They can always Google it if they want to know. The HDR+ system in the Pixel 2 produces images that exceed the limits that the hardware can normally provide. Consider this. We capture, edit, share and view images on smartphones more than anything else. The average smartphone has a HD (2MP) screen, a 4K UHD TV has an (8MP) screen. The Pixel 2 produces images at 12MP.

Most people don’t pixel peep. Mobile phone screens are a few inches across. They only spend a few seconds looking at it. The decision on whether they like it is predicated on the image being well exposed and of a subject that inspires emotion. These things don’t need to be ‘correct’, they just need to be interesting to look at. The person viewing it rarely cares about the technical side either. That is what the Pixel 2 does. Minimal usage resistance! Maybe only thing to add would be artificial intelligence that can also choose when to take the picture.

Google’s AI Is Just Waking Up

Artificial Intelligence applications are growing exponentially. They leverage hardware in ways that augment its value beyond what the hardware can do on its own. If you were into using buzzwords you could say it is synergistic. Coincidentally, the attack on digital camera sales by smartphones will hopefully pressure the digital camera market to innovate. Futhermore, they cannot efficiently expand their market share without reducing usage resistance and Artificial Intelligence integration is one way to do that.

In my next article I will be presenting a list of ways AI / machine-learning can continue to add value to imaging hardware. This will improve consumer experience and my hopes for a renaissance in the digital market by integrating AI into their imaging pipelines. Finally, I would have provided more examples of Pixel 2 photography but I was time-constrained due to other projects I was working on concurrently. Google are welcome to send me a Pixel 2 to continue using however 🙂


Journey In Street Photography

Shake It Off

A heatwave in London. Record temperatures and humidity to put you in a coma. Combine those two things with a lack of motivation to go out into the world and you have the perfect set up for staying at home in the dark and leaving street photography for another day.

Depression - Street Photography - Simon Goldsmith

Regent’s Park – Resting Man – Simon Goldsmith

But… as it had been a while I decided to drag myself out and see what street photography brings. During this process I discovered an interesting psychological aspect to street photography (or exploratory photography). When I usually considered the task beforehand it would lead me to talk myself out of it, especially if it features an area or theme I’ve shot before. This invariably led to the following chain of thought:

‘What’s the point of going to location x, I know the parks, the roads, I won’t see any angle I haven’t seen before (etc).’

I do this all the time. This epic procrastination even extends to new locations. Consider the following self-argument:

‘I don’t know what it will be like in location y, maybe it’ll be a waste of time. Boring buildings, ugly pigeons, traffic. Crap light today. How BORING.’

Definitely a state of mind I needed to overcome.

I guess what it comes down to is having a reason to go and do it. For me not only is that reason always the same it is coincidentally the one thing I can’t think of beforehand to motivate me in the first place.

Moments.

Those things you can’t imagine or predict beforehand. They just happen in front of you (or behind) and you react to it and then it’s gone. I know it may not be ‘correct thinking’ and I know there are those that specialise in landscapes, architecture and the like. I shoot all of that but not because ‘it’s what I do‘, I shoot it because I needed that moment. The camera is merely a tool to capture it. The more comfortable you are with the camera, the more confidence you have in its ability to get what you want and the more fun you’ll have. That’s the theory anyway right? Not that I don’t have my gear nerd thing happening with technology. But that’s more to do with my love of science and engineering.

I like to think there is a difference.

So I got my butt out and went for a walk. I went through two of London’s Royal Parks, Regent’s Park and Primrose Hill. It’s actually a nice walk, peaceful, lots of green space, improved dramatically by the baking heat and a gentle breeze.

A pleasing cocktail.

Depression - Street Photography - Simon Goldsmith

Regent’s Canal – Baby Looks Back – Simon Goldsmith

Now, each time I go out I try and do something different. I usually shoot raw but on this day I kept it jpeg. I also usually take a camera bag with a bunch of lenses, variable ND filters, external microphone blah blah, enough gear to shoot a freaking film! This is bad. Experience has taught me two things:

  • Carrying lots of gear is just added stress; more stuff to carry, to look after, to worry about.
  • Having lots of options with lenses / filters etc, can lead to indecision. Too much choice leads you to spend more time thinking about lenses and other technical crap and less about actually taking in your environment.

In response to this I had begun taking out a minimum amount of equipment. It appears to be the same thing that happened with my music composing and production. I realised I had become obsessed with plug-ins, virtual instruments and technical points of production and ended up not actually making any music.

Depression - Street Photography - Simon Goldsmith

What’s The Point?

To restrict myself I decided not to do any filming. Knowing just how wonderful the video quality of the NX1 is, even if only using it for frame grabbing can lead to you shooting lots of (technically) sweet looking video when you should be taking photos. BAD BOY!

Focus on photos only. Benefits are you have less content to deal with (or get distracted by) when bringing it in for editing and that all means increased productivity. Not only ‘getting the shot’ but ‘getting the shot out there into the world’. They’re both sides of the same coin. The same can be applied to music production of course (I think I’m more telling myself this).

I took out my camera with the 50-150mm S, an excellent telephoto lens. Sharp, fast, focuses instantly and the balance with the battery grip is excellent. The effect is to move the center of gravity away from your wrist and into your lower forearm, much better. Probably the best in fact. It’s so comfortable that I’ve taken the grip out with me just because I like the feel of it, the balance more than compensating for the extra mass.

Depression - Street Photography - Simon Goldsmith

Regent’s Park – Resting Woman – Simon Goldsmith

Days Made For Street Photography

I like these two parks for different reasons. For street photography, Regent’s Park is much larger (lots of animals and separate areas, flower gardens, lakes, protected areas). Whereas Primrose Hill is smaller and closer to built up areas.

I visit Regent’s Park first, it’s further away so I can hit the other park on the wander back. I walk along Regent’s Canal (where the shot with the baby looking into the camera was taken).

These walks never seem to go the way I expect. Like chaos theory, the same location on a different day can yield completely different results due to variables. I don’t just mean lighting, I mean people and other animals interacting. All of those things I actually want to capture, the reason I go out there in the first place, the one thing you cannot predict beforehand.

As I mentioned previously it was a hot day, people weren’t doing much apart from melting into the ground. The camera held up perfectly well in the heat, it even got almost hot to the touch when I was filming on the way home. (Yes I know I said I wouldn’t, but it was worth it, you’ll see why later).

I wandered through the park, instinctively snapping away while at the same time making a conscious effort to shoot less. Who needs two hundred shots of roses anyway. The birds were staying close to the water and keeping their heads down, except if there’s food.

Depression - Street Photography - Simon Goldsmith

Regent’s Park – Water Bird – Simon Goldsmith

I do tend to have exposure compensation down about a third of a stop as I know lifting shadows is better than pulling down highlights with the NX1. This goes back to my conversation about editing.

In The Shade

Depression - Street Photography - Simon Goldsmith

Regent’s Park – Man Bird Bird – Simon Goldsmith

Take the bird in the shade in the shot above. I could have exposed for that area, which would risk blowing out details in the grass in the foreground. This would also blow the highlights on the arm of the man laying there and most probably the leaves too, but because for this shot I wanted all three subjects lined up in that way (geometry is your friend) they had to all ‘fit’ exposure-wise.

Depression - Street Photography - Simon Goldsmith

Regent’s Park – Water Bird – Simon Goldsmith

Standard bird in the water shot there. I don’t actually like this shot for the bird. I like it because of the pattern the water makes as it is breaking around its body. The anatomy of a shock wave. On that same lake humans were floating about on boats…

Depression - Street Photography - Simon Goldsmith

Regent’s Park – Boats – Simon Goldsmith

Eventually I got fed up and wandered back. To be honest I wasn’t feeling very good, hadn’t done for a while (mentally). I decided to walk back topside and head down a road called Parkway for a short cut but I wasn’t paying attention. Wrapped up in my thoughts and feelings of self-loathing I ended up walking the wrong way!

I went back down to the canal. As I turn the corner I saw this guy just chilling out by the canal side. A woman’s voice echoes across the water. This is exactly what I needed. Time for some street photography!

At first I just walked past (as I said I wouldn’t shoot any video) but I found myself turning around and just having to capture it. I was still using spot metering, really helping to bring out the subject with the challenging movement and dynamic range of light in that area.

Depression - Street Photography - Simon Goldsmith

Sam Garrett, Singer Songwriter

I’ll just leave it here, shot in 4K UHD. I didn’t have any time to set anything and I had no intention of interfering with the performance in any way. So I just crouched down and shot everything hand held. The odd aperture adjustment here and there with audio recorded using the NX1’s on-board stereo mic.

I spent quite some time there, listening to the performances, filming and photographing the artists. Street photography is random. You can have a totally uninspiring day walking and looking and have it all rewarded in a moment by a wrong turn. There is a lesson in there somewhere!

I recorded more but I’ll put the videos all together in a separate post in the future. For now, that’s it! Hope you enjoyed the pictures or the rambling, or the music. Until next time 🙂

Depression - Street Photography - Simon Goldsmith

Ny Oh, Singer Songwriter performing in Camden.


The Importance Of Forgetting

Photography - Birds - London -Simon Goldsmith

Forgetting – Feeding Frenzy – Simon Goldsmith

This is a mental lesson for anyone who practices in the creative industries. This could be musicians, writers, filmmakers or whomever. Here I will be dealing with the context of photography and our potential photographic pile ups, of which there are many. But one at a time shall we?

Photography Ingredients

A lot goes on behind the scenes before the viewer finally gets to see a shot online or in print. You have colour correction, editing, filtering, cropping, even the naming of images being valid stops along the track which the Photography Express travels. You also need to capture the moment in the first place and that means being at the right place at the right time, knowing that kind of image you want to capture, settings to use, how to frame the shot and your state of mind as the photographer. Like ingredients in a meal these all factor in to the end result.

To keep the analogy going, these determine the carriages of the train. Either way, there is certainly much going on and it can clutter up the mind if you’re the self-analytical type. And who is better at self doubt than creative people? Maybe it is easier to just dump the images on a hard drive and come back to them later? They can store a quadrillion images and they’ll probably come in useful one day.

Enter The Hide Drive

So they drop out of sight and out of mind. You then go off with your trusty camera to capture more mediocre moments. You are out and about again taking more photos than there are interesting subjects mainly to keep your hands busy. And not feel useless.

I obviously began to think this, hence these comments. For years I did this, adding folder after folder of crap to the HD. Occasionally I would look through them in the evenings, looking for some gem I didn’t already extract but mostly they were just boring. The kind of images you can capture at any time in that location. Still, I’d leave them there and go off and do my thing.

Easy Way Out

Time passed, files collected and I found myself going out for street photography less and less. I wouldn’t say this was because I didn’t enjoy it, but because I had already visited all those areas. This internal monologue had a part of my brain saying ‘what’s the point, you have countless photos saved, go through them again and see if you’ve missed anything‘. This happens to be very unhealthy for the creative brain as you repeatedly retrace your steps and wallow in your own  mental mess. These are the times when you need to shake things up a bit. Considering the way this is written you could be forgiven for thinking I was aware of this beforehand. If only!

Sort It Out

One evening after trawling through my images (again) as I was setting up my website at www.simon-goldsmith.co.uk I had decided that my back-ups had become an almighty mess and now was the time to clean house. I told myself no more putting it off, just get stuck in.

It took two days. I went through every folder, every shot, delete here delete there, thousands upon thousands of images removed and I felt a weight lifting with every click. This was not something I was expecting as photo opportunities started reappearing in my mind, I can go back there and shoot! The clutter from that area was gone, I couldn’t use those images as a crutch for procrastination. I was removing ballast from the Photography Express, a smoother ride you see. I can shoot as an experienced photographer knowing exactly what he wants, and not as a hobbyist just pressing the shutter button in order to justify me being there in the first place. Keeping bad shots clouds your psychology, I’m sure I can go back to the drive and delete more but I sent over 4000 shots into the ether, that’s a good start.

Fresh Photography!

So for me the moral of the story is simple. If you feel yourself in a slump sometimes it’s a good idea to go through your old ideas and destroy the ones that have not helped you. You can also use them to create new ideas or new motivation. Once you have de-cluttered your archive (and your brain) pick up your camera, pen, guitar, and create something new and beautiful. Something without the baggage. Something wonderful!

At least that’s the idea!