Toggle light / dark theme

Deep Learning systems can achieve remarkable, even super-human performance through supervised learning on large, labeled datasets. However, there are two problems: First, collecting ever more labeled data is expensive in both time and money. Second, these deep neural networks will be high performers on their task, but cannot easily generalize to other, related tasks, or they need large amounts of data to do so. In this blog post, Yann LeCun and Ishan Misra of Facebook AI Research (FAIR) describe the current state of Self-Supervised Learning (SSL) and argue that it is the next step in the development of AI that uses fewer labels and can transfer knowledge faster than current systems. They suggest as a promising direction to build non-contrastive latent-variable predictive models, like VAEs, but ones that also provide high-quality latent representations for downstream tasks.

OUTLINE:
0:00 — Intro & Overview.
1:15 — Supervised Learning, Self-Supervised Learning, and Common Sense.
7:35 — Predicting Hidden Parts from Observed Parts.
17:50 — Self-Supervised Learning for Language vs Vision.
26:50 — Energy-Based Models.
30:15 — Joint-Embedding Models.
35:45 — Contrastive Methods.
43:45 — Latent-Variable Predictive Models and GANs.
55:00 — Summary & Conclusion.

Paper (Blog Post): https://ai.facebook.com/blog/self-supervised-learning-the-da…telligence.
My Video on BYOL: https://www.youtube.com/watch?v=YPfUiOMYOEE

ERRATA:
- The difference between loss and energy: Energy is for inference, loss is for training.
- The R(z) term is a regularizer that restricts the capacity of the latent variable. I think I said both of those things, but never together.
- The way I explain why BERT is contrastive is wrong. I haven’t figured out why just yet, though smile

Video approved by Antonio.

Abstract:

Women constitute a mere 22 per cent or less than a quarter of professionals in the field of AI and Data Science.

There is a troubling and persistent absence of women when it comes to the field of artificial intelligence and data science. Women constitute a mere 22 per cent or less than a quarter of professionals in this field, as says the report “Where are the women? Mapping the gender job gap in AI,” from The Turing Institute. Yet, despite low participation and obstacles, women are breaking the silos and setting an example for players out in the field of AI.

To honour their commitment and work done, we have listed some of the women innovators and researchers who have worked tirelessly and contributed significantly to the field of AI and data science. The list below is provided in no particular order.

The brainchild behind and the founder of The Algorithmic Justice League (AJL), Joy Buolamwini, has started the organisation that combines art and research to illuminate the social implications and harms of artificial intelligence. With her pioneering work on algorithmic bias, Joy opened the eyes of the world and brought out the gender bias and racial prejudices embedded in facial recognition systems. As a result, Amazon, Microsoft, and IBM all halted their facial recognition services, admitting that the technology was not yet ready for widespread usage. One can watch the famous documentary ‘Coded Bias’ to understand her work. Her contributions will surely pave the way for a more inclusive and diversified AI community in the near future.

The CEO of Tesla and SpaceX is not only the world’s richest person, but he’s also worth more than Warren Buffet and Bill Gates combined! Stay tuned to find out what other billionaires think of Elon Musk and subscribe to Futurity.

#elonMusk #jeffBezos #tesla.

Here at Futurity, we scour the globe for all the latest tech releases, news and info just so you don’t have to! Covering everything from cryptocurrency to robotics, small startups to multinational corporations like Tesla and Jeff Bezos to Elon Musk and everything in between!

“We are proud to be able to showcase the world’s first fully electric and self-propelled container ship,” said Svein Holsether, CEO of Norwegian chemical company Yara International. “It will cut 1,000 tonnes of CO2 and replace 40,000 trips by diesel-powered trucks a year.”

Yara has collaborated since 2017 with maritime technology company Kongsberg to develop the ship, which sailed from Horten to Oslo, a distance of approximately 35 nautical miles (65 km). Powered by 7 MWh batteries, it uses an automatic identification system (AIS), cameras (including infrared), a lidar, and radar system. It will begin commercial operations in 2022, transporting mineral fertiliser between ports in southern Norway at up to 15 knots (28 km/h).

“Norway is a big ocean and maritime nation, and other nations look to Norway for green solutions at sea. Yara Birkeland is the result of the strong knowledge and experience we have in the Norwegian maritime cluster and industry,” said Geir Håøy, CEO of the Kongsberg Group. “The project demonstrates how we have developed a world-leading innovation that contributes to the green transition and provides great export opportunities for Norwegian technology and industry.”

There are synergies between the two kinds of intelligence. The brain serves the genes by improving the organism’s capability to survive and reproduce. In exchange, evolution favors genetic mutations that improve the brain’s innate and learning capacities for each species (this is why some animals are born with the ability to walk while others learn it weeks or months later).

At the same time, the brain comes with tradeoffs. Genes lose some of their control over the behavior of the organism when they relegate their duties to the brain. Sometimes, the brain can go chasing rewards that do not serve the self-replication of the genes (e.g., addiction, suicide). Also, the behavior learned by the brain does not pass on through genes (this is why you didn’t inherit your parents’ knowledge and had to learn language, math, and sports from scratch).

As Lee writes in Birth of Intelligence, “The fact that brain functions can be modified by experience implies that genes do not fully control the brain. However, this does not mean that the brain is completely free from genes, either. If the behaviors selected by the brain prevent the self-replication of its own genes, such brains would be eliminated during evolution. Thus, the brain interacts with the genes bidirectionally.”

A team of researchers from TU Delft managed to design one of the world’s most precise microchip sensors. The device can function at room temperature—a ‘holy grail’ for quantum technologies and sensing. Combining nanotechnology and machine learning inspired by nature’s spiderwebs, they were able to make a nanomechanical sensor vibrate in extreme isolation from everyday noise. This breakthrough, published in the Advanced Materials Rising Stars Issue, has implications for the study of gravity and dark matter as well as the fields of quantum internet, navigation and sensing.

One of the biggest challenges for studying vibrating objects at the smallest scale, like those used in sensors or quantum hardware, is how to keep ambient thermal noise from interacting with their fragile states. Quantum hardware for example is usually kept at near absolute zero (−273.15°C) temperatures, and refrigerators cost half a million euros apiece. Researchers from TU Delft created a web-shaped microchip sensor that resonates extremely well in isolation from room temperature noise. Among other applications, their discovery will make building quantum devices much more affordable.

It looks like a normal car but the white taxi by the kerb has nobody driving it, and communicates with customers digitally to obtain directions and take payment.

Beijing this week approved its first autonomous taxis for commercial use, bringing dozens of the so-called “robotaxis” to the streets of the Chinese capital.

The vehicles can only carry two passengers at a time and are confined to the city’s southern Yizhuang area.

In the future, soft robotic hands with advanced sensors could help diagnose and care for patients or act as more lifelike prostheses.

But one roadblock to encoding soft robotic hands with human-like sensing capabilities and dexterity has been the stretchability of sensors. Although pressure sensors—needed for a robotic hand to grasp and pick up an object, or even take a pulse from a wrist—have been able to bend or stretch, their performance has been significantly affected by such movement.

Researchers at the Pritzker School of Molecular Engineering (PME) at the University of Chicago have found a way to address this issue and have designed a new pressure sensor that can be stretched up to 50 percent while maintaining almost the same sensing performance. It is also sensitive enough to sense the pressure of a small piece of paper, and it can respond to pressures almost instantaneously.

Artificial intelligence has reached a point where it can compose text that sounds so human that it dupes most people into thinking it was written by another person. These AI programs—based on what are called autoregressive models—are being successfully used to create and deliberately spread everything from fake political news to AI-written blog posts that seem authentic to the average person and are published under human-sounding byline.

However, though autoregressive models can successfully fool most humans, their capabilities are always going to be limited, according to research by Chu-Cheng Lin, a Ph.D. candidate in the Whiting School of Engineering’s Department of Computer Science.

“Our work reveals that some desired qualities of intelligence—for example, the ability to form consistent arguments without errors—will never emerge with any reasonably sized, reasonably fast autoregressive model,” said Lin, a member of the Center for Language and Speech Processing.