🔮 AI beyond deep learning; self-driving cars; thinking about attention; European startups; green ships, cars and planes; subcultures, cephalopods & free houses++ #195
Limits of deep learning
Exponential View
Azeem Azhar’s Weekly Wondermissive: Future, Tech & Society
This issue has been supported by Hanover Communications: Protecting your brand in the age of AI.
Dept of podcasts 🎧
The basic assumption on our side is that technology transcends everything. The distinction between what is local and what is global makes no sense anymore, as platforms have no respect for national borders.
The world's first Ambassador appointed to represent a nation-state to the technology industries around the world, Casper Klynge, joins me in London to discuss the state of relations between leading technology companies and governments. We acknowledge a unique moment in history we're in: trust in government in the West is declining, the effects of emerging technologies are combinatorial, and a shift from West to East is happening.
Listen wherever you get your podcasts:
iTunes | Spotify | Stitcher | Soundcloud | Breaker | Overcast
Dept of the near future
🥊 The limits of deep learning. Lovely Gary Marcus argument against "the notion that deep learning is without demonstrable limits and might, all by itself, get us to general intelligence, if we just give it a little more time and a little more data". Moderately technical but an important argument as we are in the midst of an AI investment boom driven by the broad applicability of deep learning as a technology. True, as Marcus himself points out, we don't yet know its limits. (See a longer discussion below.)
📳 Attention is not a resource but a way of being alive to the world, argues Dan Nixon. "The attention-economy narrative, for all its usefulness, reinforces a conception of attention-as-a-resource, rather than attention-as-experience." Would it help if we "return to an embodied, exploratory mode of attention"?
🛴💨 Micromobility startups Lime and Bird are both worth more than $10bn, and following growth trajectories similar to Uber and Lyft in their early days. Economies of scale and network effects are helping. (Elsewhere, Waymo launched its self-driving taxi service in Phoenix, good write-up here. Reporters followed Waymo's minivans for hundreds of miles and found impressive, yet clunky, results often in the face of aggressive driving. And Kaveh Wadell points out that piloting autonomous cars may cost city governments hundreds of thousands of dollars per year.)
🧐 The state of the European startup ecosystem was released by Atomico. European VC funding hit $23bn in 2018, up from $5bn in 2012 (globally VC funding was $177bn in 2017, so Europe is still underweight on a GDP basis). Nearly half of women entrepreneurs say they experienced discrimination, and 93% of funds went to all-male teams. Founders are sanguine about the role of government and of the GDPR.
🔬 Ed Yong: The CRISPR babies experiment was much worse than we thought. (Also, Paul Knoepfler: He Jiankiu didn't really gene edit the babies, he mutated them. And to what extent was He's work tacitly or explicitly supported by the Chinese government?)
💣 Britain has its own information warfare capability, the Joint Threat Research Intelligence Group. Many governments seem willing to weaponise information. This poses a risk to international stability and trust on the internet, and societies should prepare for the age of information warfare. (Related, see this analysis on how the Gilets Jaunes protests in France were amplified as Facebook promoted local news on its platform. And also, Charles Arthur's good analysis on Huawei, and its imminent removal from the UK telecoms network.)
Dept of expertise 📚
I am building a small directory of experts who are also EV readers. It is just a way for me to understand better who I can turn to if i need to understand a particular area better.
We'll keep this directory private, but if you'd like to be included, please add your name to the form here. It'll help when I'm looking for expert comment, interviews, podcasts, etc. Please fill this out even if you know me really well. I don't want to overlook your expertise because of our familiarity!
To add yourself to the (private) directory, click here.
Not adding to the directory? Then please take a moment to share EV with your friends on Twitter, LinkedIn or email.
Dept of AI & automation
I find Gary Marcus's argument, linked above, intuitive: that it seems likely that we'll need other approaches to cracking the AI nut than deep learning.
We are starting to see hybrid models, combining deep learning with symbolic methods: there are several funded startups doing this. And reinforcement learning is another promising avenue. One great example of hybrid approaches is probably Deepmind's AlphaZero which has now mastered certain games, such as Go and Chess. AlphaZero requires more than deep neural networks: it is primed with significant prior knowledge (the rules of the game), reinforcement learning and a Monte Carlo tree search algorithm. (Deepmind's recent post on AlphaZero is a lovely read as is Gary Kasparov's commentary: "AlphaZero shows us that machines can be the experts, not merely expert tools.")
Nevertheless, deep learning will remain fecund.
For example, this recent paper shows how deep-learning workloads in Facebook have nearly quadrupled in a less than two years. And deep learning has triggered a wave of practical applications resulting in:
many more researchers going into it and competitive fields (more than 4,000 academic papers were presented at the NeurIPS conference this week. One of most highly regarded was a novel approach to deep learning, from David Duvenaud),
investment in the tools of deep learning and complementary products,
the creation of many downstream businesses that use the technology,
as I wrote in EV#7 in May 2015, becoming "as essential a part of our daily business toolkit as Microsoft Excel."
Whether it is a silver bullet to AGI or not, its impact is legendary.
Elsewhere:
The rise of consumer AI startups. Connie Chan on how Chinese apps like TikTok & Soul are a new breed of consumer AI product. "TikTok, for example, never presents a list of recommendations to the user, and never asks the user to explicitly express intent — the platform infers and decides entirely what the user should watch" while Soul, a dating app, uses "AI [to help] restore trust in anonymous chats "
AlphaFold, from Google DeepMind, wiped the floor in a protein-folding prediction contest. Protein folding is an important part of the drug discovery process. Important to note this dispassionate assessment of the results echoes much of Gary Marcus' thinking above. To wit:
progress is not going to be the discovery of some general solution. It’s going to be a mixture [...] of better understanding of the physical processes involved, larger databases of reliable experimental data covering more structural classes, and more efficient ways for searching through all these [...] and generalizing rules to tell us when we’re closing in on something accurate.
ML startup,Diffbot, reckons there are as many as 720k machine learning professional globally, a little over a third in the US. This compares to earlier estimates of only 90k from Element.AI. Either they used different definitions or different methodologies. There are about 20m software developers in the world, so ML as a skill-set is rapidly taking hold. My sense would be that the top handful of firms, the GAFAs, employ a very large percentage of the global ML talent pool. Does the number 700k really make sense? ML techniques are diffusing rapidly in a number of ways—via open source, into more traditional internet products, and via cloud providers, like AWS and Azure. The incentives to be an ML engineer are high. The average ML engineer in the UK is paid £66k per annum, 55% higher than a PHP developer. (Although bizarrely blockchain developers are paid even more than ML developers.)
It wouldn't be surprising if the number of such trained engineers is already as high as 700k. Of course, not every one of them will be Geoff Hinton, and in a few years the number of ML devs will probably be 10-fold what it is today.
🤖 The robotic process automation market is booming, says EV reader, Phil Fersht. It hit $1.7bn in 2018 and will more than double in the next three years. Enterprise adoption is faster than expected and it is likely to directly impact the ways in which many organisations and employees work in the relatively near future. RPA is a route into white-collar automation, often sold as a way of reducing headcount. I'm starting to wonder if it isn't more than a sticking plaster across a combination of technology and management debt. And the problem is that systems like RPA are very brittle. Much more so than human workforces. We are rather good at adapting. So I'm musing whether enterprises who have gone heavy on RPA, especially justified by headcount costs, will be paying the price in lower adaptability in years to come. Any insight or experience on RPA? Ping me.
Walmart is rolling out 360 autonomous floor-cleaning robots from Brain Corp. According to a Walmart VP "BrainOS is a powerful tool in helping our associates complete repetitive tasks so they can focus on other tasks within role and spend more time serving customers." We'll see how that works out.
🐻 Twenty-four Amazon workers were hospitalized after a warehouse robot tore open a can of bear repellent.
👀 Just how bad are traditional credit scoring algorithms? Pretty awful is my hunch. This fantastic research from Germany corroborates: "The scoring procedure is broken... not trustworthy...and discriminate[s]."
Dept of energy transition & transport
Next week's podcast is an absolute cracker with Michael Liebreich, a world-expert on the transport and energy transition. Prep for it by 1) subscribing to the podcast, if you haven't already (iTunes, Spotify, Stitcher, Soundcloud, Overcast, Breaker), 2) reading these:
🏭 Global emissions tick up again.
Simon Kuper: A brilliant essay on how to sell climate change. (Hint number one: focus on the benefits of green growth.)
🙄 Maersk, the world's largest shipping liner, will target zero carbon emissions for 2050. Laudable, but not fast enough. We need to get to net zero by 2035-2040 to avoid 2° warming.
🙄 VW's next generation of ICE vehicles, rolling out in 2026, will be its last. (Laudable, but slow.)
🙄 Audi will spend $15bn over the next five years on electrification, autonomous and other digital technologies. By 2025, half of its range will be all electric (Laudable, but slow.)
Luxembourg is the first city in the world to abolish public transit fees
A Tesla drove its drunk, sleeping driver safely down California's Highway 101.
The Eviation Alice is an all-electric airplane available now.
Short morsels to appear smart at dinner parties
💯 EV subscriber, Fred Wilson, on understanding centralization vs decentralization.
🇮🇳 What business models will emerge to grow the internet into lower-income segments of the massive Indian market?
Sexual subcultures are the collateral damage of Tumblr's ban on pornographic content.
Super interesting: the funnel of human experience. (15% of all human experience has occurred amongst people alive today & women have spent about 1bn hours in labour.)
Japan is giving away houses for free.
🧠 Fibrous doppelgänger of the human brain.
🆒 A new metamaterial which can change its rigidity via magnetism.
Appreciating the non-human intelligence of the octopus.
Men underestimate the level of harassment against women. 😧
🌨️ A third of rivers and streams in the US has gotten saltier in the past 25 years.
End note
Crumbs. What a week. So much exponential.
The world's biggest shindig of ML researchers, NeurIPS, kicked off. Combined with end of the year research reports, and the massive investment attention in these technologies, there was almost too much to comprehend this week. I've done my best to characterise themes and give you things to think about without getting pulled into the weeds of the next announcement. Tell me if I'm getting it wrong. And tell your friends if I'm getting it right.
We'll need a bit more help with EV, so I'm looking for a researcher/editor to help around a day a week. Ideally, for language and timezone reasons, you'll be based in the UK, but happy to look at brilliant and inspiring folk elsewhere, too. Happy to look at former analysts, journalists, researchers or founders... curiosity, intellectual chops, analytical thinking, quantitative savvy, great writing & regular availability are all helpful. Please fill this out.
Have a great week,
Azeem
P.S. Scroll down for news about your fellow readers! And don’t forget to share EV with your network: share this on Linkedin, Twitter, forward this email.
What you are up to
Congrats to Hetan Shah for joining the board at Ada Lovelace Institute!
Congrats to Obi Felten as one of FT Change Makers of the year!
Chris Locke shares Caribou Digital & DFS Lab's research into how the long-tail of languages might leave behind Africa and Asia when it comes to AI and voice interfaces.
Jason Richardson is one of hundreds of artists participating in this global musical project with the goals to compose with sounds of nature from 55 countries.
Eliot Peper talks about his work and the power of science fiction on Whose Century is It? podcast.
Keep on sharing your news and projects, email Marija.