Category: News

Daily news from the world of science and research!

Categories
News

Amputees with prosthetic arms may get their natural sense of touch restored

Originally written for the University of Chicago Medicine & Biological Sciences by Matt Wood.

Research provides blueprint for building neuroprosthetic devices that recreate the sense of touch by directly stimulating the nervous system.

Scientists at the University of Chicago and Case Western Reserve University have found a way to produce realistic sensations of touch in two human amputees by directly stimulating the nervous system.
The study confirms earlier research on how the nervous system encodes the intensity, or magnitude, of sensations. It is the continuation of work by University of Chicago neuroscientist Sliman Bensmaia, PhD, using neuroprosthetic devices to recreate the sense of touch for amputee or quadriplegic patients with a “biomimetic” approach that approximates the natural, intact nervous system.
Bensmaia has previously been a part of a team led by Robert Gaunt, PhD, from the University of Pittsburgh, who announced that for the first time, a paralyzed human patient was able to experience the sense of touch through a robotic arm that he controls with his brain. In that study, researchers interfaced directly with the patient’s brain, through an electrode array implanted in the areas of the brain responsible for hand movements and for touch, which allowed the man to both move the robotic arm and feel objects through it.

Electrical stimulation was delivered by an external stimulator (top left) through percutaneous leads to FINEs implanted on the median, ulnar, and radial nerves of an upper-limb amputee (bottom left). Each electrode contact evokes sensory percepts on small regions of the missing hand of the subject. (Image: Graczyk et al, Sci. Transl. Med.)

The new study takes a similar approach in amputees, working with two male subjects who each lost an arm after traumatic injuries. In this case, both subjects were implanted with neural interfaces, devices embedded with electrodes that were attached to the median, ulnar and radial nerves of the arm. Those are the same nerves that would carry signals from the hand were it still intact.
“If you want to create a dexterous hand for use in an amputee or a quadriplegic patient, you need to not only be able to move it, but have sensory feedback from it,” said Bensmaia, who is an associate professor of organismal biology and anatomy at the University of Chicago. “To do this, we first need to look at how the intact hand and the intact nervous system encodes this information, and then, to the extent that we can, try to mimic that in a neuroprosthesis.”
The latest research is a joint effort by Bensmaia and Dustin Tyler, PhD, the Kent H. Smith Professor of Biomedical Engineering at Case Western Reserve University, who works with a large team trying to make bionic hands clinically viable. Tyler’s team, led by doctoral student Emily Graczyk, systematically tested the subjects’ ability to distinguish the magnitude of the sensations evoked when their nerves were stimulated through the interface. They varied aspects of the signals, such as frequency and intensity of each electrical pulse. The goal was to understand if there was a systematic way to manipulate the sensory magnitude.
Earlier research from Bensmaia’s lab predicted how the nervous system discerns intensity of touch, for example, how hard an object is pressing against the skin. That work suggested that the number of times certain nerve fibers fire in response to a given stimulus, known as the population spike rate, determines the perceived intensity of a given stimulus.
Results from the new study verify this hypothesis: A single feature of electrical stimulation—dubbed the activation charge rate—was found to determine the strength of the sensation. By changing the activation charge rate, the team could change sensory magnitude in a highly predictable way. The team then showed that the activation charge rate was also closely related to the evoked population spike rate.
While the new study furthers the development of neural interfaces for neuroprosthetics, artificial touch will only be as good as the devices providing input. Earlier this year, Bensmaia and his team tested the sensory abilities of a robotic fingertip equipped with touch sensors.
Using the same behavioral techniques that are used to test human sensory abilities, Bensmaia’s team, led by Benoit Delhaye and Erik Schluter, tested the finger’s ability to distinguish different touch locations, different pressure levels, the direction and speed of surfaces moving across it and the identity of textures scanned across it. The robotic finger (with the help of machine learning algorithms) proved to be almost as good as a human at most of these sensory tasks. By combining such high-quality input with the algorithms and data Bensmaia and Tyler produced in the other study, researchers can begin building neuroprosthetics that approximate natural sensations of touch.
Without realistic, natural-feeling sensations, neuroprosthetics will never come close to achieving the dexterity of our native hands. To illustrate the importance of touch, Bensmaia referred to a piano. Playing the piano requires a delicate touch, and an accomplished pianist knows how softly or forcefully to strike the keys based on sensory signals from the fingertips. Without these signals, the sounds the piano would make would not be very musical.
“The idea is that if we can reproduce those signals exactly, the amputee won’t have to think about it, he can just interact with objects naturally and automatically. Results from this study constitute a first step towards conveying finely graded information about contact pressure,” Bensmaia said.

Categories
News

You may think that you browse the web anonymously, but you really don't

You may think you are anonymous on the web, but you really aren’t. Stanford and Princeton researchers have shown how anonymous browsing history and Twitter usage is enough to reveal a user’s real identity.

The research team launched what they call the “Footprints Project” over the summer, inviting people to participate in an online experiment, disclosing their browser history, including information about active Twitter usage. Based on that information alone, they have managed to correctly identify 11 out of 13 people on their first day of operation.
The Footprints experiment ended in October, studying almost 300 users, and accurately identifying 80% of them.
“This is kind of scary,” says Stanford undergraduate Ansh Shukla, a senior studying mathematics, who is working on the project with Stanford Engineering assistant professor Sharad Goel and Stanford computer science PhD student Jessica Su.

“You should kind of go into the internet assuming that everything you go to someone might learn about someday,” Shukla says.

How did it work? Volunteers who participated in Footprints gave the researchers permission to gather the names of any websites that a participant clicked on through Twitter while using Google Chrome. This unique set of links is a fingerprint. To find that user, the researchers crawled through millions of Twitter profiles to see who everyone is following.
So imagine that Jane Doe, John Smith and Susie Q all participated anonymously, and that each of these three volunteers follow 100 Twitter accounts. All three might follow the official Stanford Engineering Twitter account. But Jane and John also follow the New York Times’ Twitter account for their news, while Susie instead follows the Los Angeles Times as her newspaper of choice. Researchers can then deduce that the person who visited links tweeted from Stanford Engineering and the New York Times is more likely to be Jane or John, not Susie.
And you can bet that many advertisers and internet companies are already using similar techniques to learn everything they can about individual internet users. Even without linking an online user with their real name, companies can cross-reference databases to learn very interesting – and lucrative – information. You may have heard of cookies, but it just doesn’t end there.
It goes to show that our digital footprints are considerably more vulnerable and exploitable than we often think, and clearing cookies and browsing history does not help. It also takes more than to check “do not track” settings in a browser. An average user on an average machine is quite frankly an open book, and it’s only up to us whether or we mind.

Categories
News

Quantifying street safety opens avenues for AI-assisted urban planning

Urban revitalization is getting an update that combines crowdsourcing and machine learning.

Theories in urban planning that infer a neighborhood’s safety based on certain visual characteristics have just received support from a research team from MIT, University of Trento and the Bruno Kessler Foundation, who have developed a system that assigns safety scores to images of city streets.
The work stems from a database of images that MIT Media Lab was gathering for years around several major cities. These images have now been scored based on how safe they look, how affluent, how lively, and so on.
Adjusted for factors such as population density and distance from city centers, the correlation between perceived safety and visitation rates was strong, but it was particularly strong for women and people over 50. The correlation was negative for people under 30, which means that males in their 20s were actually more likely to visit neighborhoods generally perceived to be unsafe than to visit neighborhoods perceived to be safe.
César Hidalgo, one of the senior authors of the paper, has noted that their work is connected to two urban planning theories – the defensible-space theory of Oscar Newman, and the eyes-on-the-street theory of Jane Jacobs.
Jacobs’ theory, Hidalgo says, is that neighborhoods in which residents can continuously keep track of street activity tend to be safer; a corollary is that buildings with street-facing windows tend to create a sense of safety, since they imply the possibility of surveillance. Newman’s theory is an elaboration on Jacobs’, suggesting that architectural features that demarcate public and private spaces, such as flights of stairs leading up to apartment entryways or archways separating plazas from the surrounding streets, foster the sense that crossing a threshold will bring on closer scrutiny.
Researchers have identified features that align with these theories, confirming that buildings with street-facing windows appear to increase people’s sense of safety, and that in general, upkeep seems to matter more than distinctive architectural features.
Hidalgo’s group launched its project to quantify the emotional effects of urban images in 2011, with a website that presents volunteers with pairs of images and asks them to select the one that ranks higher according to some criterion, such as safety or liveliness. On the basis of these comparisons, the researchers’ system assigns each image a score on each criterion.
So far, volunteers have performed more than 1.4 million comparisons, but that’s still not nearly enough to provide scores for all the images in the researchers’ database. For instance, the images in the data sets for Rome and Milan were captured every 100 meters or so. And the database includes images from 53 cities.
So three years ago, the researchers began using the scores generated by human comparisons to train a machine-learning system that would assign scores to the remaining images. “That’s ultimately how you’re able to take this type of research to scale,” Hidalgo says. “You can never scale by crowdsourcing, simply because you’d have to have all of the Internet clicking on images for you.”
To determine which features of visual scenes correlated with perceptions of safety, the researchers designed an algorithm that selectively blocked out apparently continuous sections of images — sections that appear to have clear boundaries. The algorithm then recorded the changes to the scores assigned the images by the machine-learning system.

Categories
News

Tesla is equipping their new cars with fully fledged self-driving tech (video)

If there was ever any doubt that Elon Musk’s Tesla was all in on autonomous cars, today’s announcements settle it for good. What is reportedly a fully capable autopilot is already in all Tesla cars currently in production.

Tesla Motors announced earlier today that “All Teslas in production now have Full Self-Driving hardware”, in a monumental step towards making autonomous vehicles available to the general public. They are talking about fully autonomous self-driving cars that navigate through daily traffic, plan optimal routes, park independently after dropping you off at your destination, and then pick you up again at a tap of a button, Batman style.

We are excited to announce that, as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.

Tesla is equipping their newest system with hardware that reportedly ensure safety two times greater than any human driver ever could. That includes eight camera that provide 360 degrees of visibility around the car at up to 250 meters of range, twelve ultrasonic sensors that allow for detection of objects at nearly twice the distance of the prior system, and a forward-facing radar able to see through heavy rain, fog, dust, and even the car ahead:

Source: Tesla
Source: Tesla

An autopiloted Tesla should be indistinguishable from other cars on the road in terms of behavior, according to what they have promised. These cars should be able to automatically adjust their speed to traffic conditions, keep within a lane, and even switch lanes when needed, or when a faster lane is available. We recommend that you take a look at the video below that Tesla published a few hours ago, detailing the capabilities of the new self-driving system.
However, if you happen to already own a Tesla, you are fresh out of luck. Elon Musk later took to Twitter and clarified that retrofitting a car with the new self-driving hardware would actually be more expensive than buying a new car.
Nevertheless, the ideal of simply getting inside a car, telling it where to go, and reaching that destination as quickly and safely as possible, is within reach. We will be holding our breaths to hear what the first customers have to say.

Categories
News

IAM Innovator magazine Issue #4 is out!

The latest issue of the IAM Innovator Magazine, dedicated to highlights from the EAI Community in 2016 so far, is now available for download!
Transportation is changing before our very eyes with electric cars, autonomous driving systems and the revolutionary Hyperloop. 5G is about to usher us in the new age of internet, while AI, machine learning and robotics are pulling the strings of progress. And it all converges in a market where some worry about their job security, while others are already knee-deep in the sharing economy. Winds of change are blowing and the world is waiting for the next Henry Ford to
announce a three-day weekend. But it is important to remember the incremental nature of evolution, and the individuals who are at the helm of this age of innovation. Neither smart and secure networking, nor intuitive human-computer interaction, or a successful business venture happen overnight. We are happy to honor these individuals and their crucial incremental contributions in this issue of IAM Innovator magazine.
We hope you enjoy it!

Categories
News

Stretchy optical fibers are ready to be implanted

Biocompatible fibers could use light to stimulate cells or sense signs of disease.

Originally published by MIT News Office, written by Jennifer Chu.
Researchers from MIT and Harvard Medical School have developed a biocompatible and highly stretchable optical fiber made from hydrogel — an elastic, rubbery material composed mostly of water. The fiber, which is as bendable as a rope of licorice, may one day be implanted in the body to deliver therapeutic pulses of light or light up at the first sign of disease. The researchers say the fiber may serve as a long-lasting implant that would bend and twist with the body without breaking down. The team has published its results online in the journal Advanced Materials.
Using light to activate cells, and particularly neurons in the brain, is a highly active field known as optogenetics, in which researchers deliver short pulses of light to targeted tissues using needle-like fibers, through which they shine light from an LED source.
“But the brain is like a bowl of Jell-O, whereas these fibers are like glass — very rigid, which can possibly damage brain tissues,” says Xuanhe Zhao, the Robert N. Noyce Career Development Associate Professor in MIT’s Department of Mechanical Engineering. “If these fibers could match the flexibility and softness of the brain, they could provide long-term more effective stimulation and therapy.”
The researchers tested the optical fibers’ ability to propagate light by shining a laser through fibers of various lengths. Each fiber transmitted light without significant  attenuation, or fading. They also found that fibers could be stretched over seven times their original length without breaking.
Now that they had developed a highly flexible and robust optical fiber, made from a hydrogel material that was also biocompatible, the researchers began to play with the fiber’s optical properties, to see if they could design a fiber that could sense when and where it was being stretched.
They first loaded a fiber with red, green, and blue organic dyes, placed at specific spots along the fiber’s length. Next, they shone a laser through the fiber and stretched, for instance, the red region. They measured the spectrum of light that made it all the way through the fiber, and noted the intensity of the red light. They reasoned that this intensity relates directly to the amount of light absorbed by the red dye, as a result of that region being stretched.
In other words, by measuring the amount of light at the far end of the fiber, the researchers can quantitatively determine where and by how much a fiber was stretched.
“When you stretch a certain portion of the fiber, the dimensions of that part of the fiber changes, along with the amount of light that region absorbs and scatters, so in this way, the fiber can serve as a sensor of strain,” Liu explains.
“This is like a multistrain sensor through a single fiber,” Yuk adds. “So it can be an implantable or wearable strain gauge.”
The researchers imagine that such stretchable, strain-sensing optical fibers could be implanted or fitted along the length of a patient’s arm or leg, to monitor for signs of improving mobility.
Zhao envisions the fibers may also serve as sensors, lighting up in response to signs of disease.
“We may be able to use optical fibers for long-term diagnostics, to optically monitor tumors or inflammation,” he says. “The applications can be impactful.”
“Hydrogel fibers are very interesting and provide a compelling direction for embedding light within the human body,” says Fiorenzo Omenetto, a professor of biological engineering at Tufts University, who was not involved in the work.  “These efforts in optimizing and managing the physical and mechanical properties of fibers are necessary and important next steps that will enable practical applications of medical relevance.”

Categories
News

This lego-like wall creates 3D soundscapes that put your surround sound to shame

Researchers are promising intricate 3D soundscapes that are to audio what holograms are to images – all coming from a single lego-like source. You probably never felt like it should be possible to get more out of your home stereo, but it is starting to look like we have been missing out.

It is a relatively simple concept – Research Triangle engineers are moulding the sound waves to make it seem like the audio source is much more intricate than it really is. Just imagine what the difference is between listening to a live orchestra, and a recording coming from the speakers.  The fact of the matter is that sound doesn’t only carry notes and volume – it also carries spatial information – and anyone could hear and feel the difference.

“We show the exact same control over a sound wave as people have previously achieved with light waves,” said Steve Cummer, professor of electrical and computer engineering at Duke University. “It’s like an acoustic virtual reality display. It gives you a more realistic sense of the spatial pattern of the sound field.”

The comparison to visual holograms is quite apt, since they manipulate light to make it appear as though a 3D object is sitting in empty space. These optical tricks work by shaping the electromagnetic field so that it mimics light bouncing off an actual object.
Sound also travels in waves. But rather than electromagnetic energy traveling through space, sound propagates as pressure waves that momentarily compress the molecules they are traveling through. And just like visible light, these waves can be manipulated into three-dimensional patterns.

A computer rendering of a sound wave that traveled through an array of acoustic metamaterial and was shaped into a pattern like the letter A one foot past the array. This pattern could not be seen, only heard. (Source: Duke University)
A computer rendering of a sound wave that traveled through an array of acoustic metamaterial and was shaped into a pattern like the letter A one foot past the array. This pattern could not be seen, only heard. (Source: Duke University)

Duke and North Carolina State University researchers have demonstrated that they can produce any three-dimensional soundscape they want with sound waves with metamaterials – synthetic materials composed of many individual, engineered cells that together produce unnatural properties. Each individual block is made of plastic by a 3-D printer and contains a spiral within. The tightness of the spiral affects the way sound travels through it — the tighter the coil, the slower sound waves travel through it.
While the individual blocks can’t influence the sound wave’s direction, the entire device effectively can. For example, if one side of the sound wave is slowed down but not the other, the resulting wave fronts will be redirected so that the sound is bent toward the slow side. By calculating how 12 different types of acoustic metamaterial building blocks will affect the sound wave, researchers can arrange them in a wall to form any wave pattern on the other side that they want. With enough care, the sound waves can produce a specific hologram at a specific distance away.
Possible uses for an acoustic hologram extend beyond home entertainment – to advanced aerial sensing and imaging technologies – but there is a good chance that sound companies and speaker manufacturers will be very interested. Not to mention virtual reality enthusiasts. You may want to keep an eye on this one if immersive virtual escapism sounds like something that floats your boat.

Categories
News

MIT keeps the dream of unlimited clean energy alive with record-breaking fusion reactor

The pursuit for clean energy has achieved a major leap on the last day of MIT’s 23-year old nuclear fusion reactor.

Nuclear fusion is the holy grail of energy sources, and for a good reason. The same process that is powering our sun would provide us with a nearly limitless clean, safe, and carbon-free energy resource that produces more power than it needs to keep itself running.
On Earth, it can be realized in reactors that simulate the conditions of ultrahot miniature “stars” of plasma — superheated gas — that are contained within a magnetic field. And though good things usually don’t come easily – nuclear fusion being no wild exception – any progress made in this field is great news.
This progress in particular is a straight up world record for plasma pressure, a key ingredient to producing energy from nuclear fusion.
MIT’s Plasma Science and Fusion Center have achieved over 2 atmospheres of pressure for the first time in the Institute’s Alcator C-Mod tokamak nuclear fusion reactor, with the temperature inside reaching over 35 million degrees Celsius, which is approximately twice as hot as the center of the sun.

But we still have ways to go. For over 50 years it has been known that to make fusion viable on the Earth’s surface, the plasma must be very hot (more than 50 million degrees), it must be stable under intense pressure, and it must be contained in a fixed volume. Successful fusion also requires that the product of three factors — a plasma’s particle density, its confinement time, and its temperature — reaches a certain value. Above this value (the so-called “triple product”), the energy released in a reactor exceeds the energy required to keep the reaction going.
Pressure, which is the product of density and temperature, accounts for about two-thirds of the challenge. The amount of power produced increases with the square of the pressure — so doubling the pressure leads to a fourfold increase in energy production.
During the 23 years that Alcator C-Mod has been in operation at MIT, it has repeatedly advanced the record for plasma pressure in a magnetic confinement device. The previous record of 1.77 atmospheres was set in 2005 (also at Alcator C-Mod), while the new record represents 2.05 atmospheres, a 15 percent improvement. The plasma produced 300 trillion fusion reactions per second and had a central magnetic field strength of 5.7 tesla. It carried 1.4 million amps of electrical current and was heated with over 4 million watts of power. The reaction occurred in a volume of approximately 1 cubic meter (not much larger than a coat closet) and the plasma lasted for two full seconds.
Other fusion experiments conducted in reactors similar to Alcator have reached these temperatures, but at pressures closer to 1 atmosphere; MIT’s results exceeded the next highest pressure achieved in non-Alcator devices by approximately 70 percent.
Although the Alcator C-Mod reactor’s funding has been ceased due to budgetary pressures, it went out on a high note. “This is a remarkable achievement that highlights the highly successful Alcator C-Mod program at MIT,” says Dale Meade, former deputy director at the Princeton Plasma Physics Laboratory, who was not directly involved in the experiments. “The record plasma pressure validates the high-magnetic-field approach as an attractive path to practical fusion energy.”
Alcator C-Mod is the world’s only compact, high-magnetic-field fusion reactor with advanced shaping in a design called a tokamak (a transliteration of a Russian word for “toroidal chamber”), which confines the superheated plasma in a donut-shaped chamber. C-Mod’s high-intensity magnetic field — up to 8 tesla, or 160,000 times the Earth’s magnetic field — allows the device to create the dense, hot plasmas and keep them stable at more than 80 million degrees. Its magnetic field is more than double what is typically used in other designs, which quadruples its ability to contain the plasma pressure.
Unless a new device is announced and constructed, the pressure record just set in C-Mod will likely stand for the next 15 years. ITER, a tokamak currently under construction in France, will be approximately 800 times larger in volume than Alcator C-Mod, but it will operate at a lower magnetic field. ITER is expected to reach 2.6 atmospheres when in full operation by 2032, according to a recent Department of Energy report.
To understand how Alcator C-Mod’s design principles could be applied to power generation, MIT’s fusion group is now working on adapting newly available high-field, high-temperature superconductors that will be capable of producing magnetic fields of even greater strength without consuming electricity or generating heat. These superconductors are a central ingredient of a conceptual pilot plant called the Affordable Robust Compact (ARC) reactor, which could generate up to 250 million watts of electricity.

Categories
News

Long exposure to cosmic rays may cause dementia and cognitive impairments, study reveals

UCI study reveals that galactic cosmic rays may cause brain damage after prolonged exposure, raising concerns regarding future interplanetary spaceflights.

It has only been two weeks since Elon Musk unveiled his ambitious plan to colonize Mars, and the list of practical issues to be solved before we officially set out on an interplanetary journey has already grown. We are talking about galactic cosmic rays, a phenomenon that we have been aware of for a long time, but one that we have never addressed. Musk has even acknowledged that they may increase the risk of cancer, but a definite answer to the problem has yet to arrive.
Now, researchers from the University of California, Irvine have demonstrated that prolonged exposure to highly energetic charged particles – much like the cosmic rays that astronauts will be bombarded by on their journey to Mars and beyond – causes significant long-term brain damage in test rodents, resulting in cognitive impairments and dementia.
In other words, astronauts may not even remember much of their journey, and their cognitive abilities and mental health will suffer greatly, and potentially permanently.

“The space environment poses unique hazards to astronauts. Exposure to these particles can lead to a range of potential central nervous system complications that can occur during and persist long after actual space travel – such as various performance decrements, memory deficits, anxiety, depression and impaired decision-making. Many of these adverse consequences to cognition may continue and progress throughout life,” said Charles Limoli, the professor of radiation oncology in UCI’s School of Medicine.

For the study, rodents were subjected to charged particle irradiation (fully ionized oxygen and titanium) at the NASA Space Radiation Laboratory at New York’s Brookhaven National Laboratory and then sent to Limoli’s UCI lab. Limoli’s work is part of NASA’s Human Research Program. Investigating how space radiation affects astronauts and learning ways to mitigate those effects are critical to further human exploration of space.
Six months after exposure, the researchers still found significant levels of brain inflammation and damage to neurons. Imaging revealed that the brain’s neural network was impaired through the reduction of dendrites and spines on these neurons, which disrupts the transmission of signals among brain cells. These deficiencies were parallel to poor performance on behavioral tasks designed to test learning and memory.
But it doesn’t end there. Limoli and his colleagues have revealed that the radiation also affected “fear extinction”, an active brain process that continuously supresses prior negative or stressful associations, as when someone who nearly drowned learns to enjoy water again. A deficit in such a major mental defense mechanism could make a person prone to anxiety and depression, i.e. the last thing you want to see in interplanetary pioneers, who may already be staring death in the face every step of the way.
Though dementia-like deficits in astronauts would take months to manifest, he said, the time required for a mission to Mars is sufficient for such impairments to develop. People working for extended periods on the International Space Station, however, do not face the same level of bombardment with galactic cosmic rays because they are still within the Earth’s protective magnetosphere.
Partial solutions are being explored, Limoli noted. Spacecraft could be designed to include areas of increased shielding, such as those used for rest and sleep. However, these highly energetic charged particles will traverse the ship nonetheless, he added, “and there is really no escaping them.”
Preventive treatments offer some hope. Limoli’s group is working on pharmacological strategies involving compounds that scavenge free radicals and protect neurotransmission.
Break out those tinfoil hats.

Categories
News

Google makes robots learn new skills faster by sharing their experience

Google Brain Team and their subsidiaries DeepMind and [X] have revealed the learning method for robots to end all learning methods.

Humanity has an entire list of activities that we cannot wait to delegate to robots. Some are dangerous, others are unpleasant, and some are simply better suited for a mechanical mind. But even a task as simple as assisting the elderly with chores and daily activities is incredibly complex for a robot. If we think about it, we will find that that the most mundane task we can think of actually relies on a lot of decision-making and previous experience.
In other words, it takes a lot of work for a robot to learn a skill that we wouldn’t normally describe as robotic. Reason being that a robot relies on its own experience to hone its skill, which takes an impractical amount of time, no matter how sophisticated its learning algorithm is. This is especially true if motor skills are involved. We are naturally good at integrating our senses, reflexes, and muscles in a closely coordinated feedback loop. “Naturally”, because our behaviors are well-honed for the variability and complexity of the environment. Not the case for robots.
Thankfully, Sergey Levine (Google Brain Team), Timothy Lilicrap (DeepMind) and Mrinal Kalahrishnan (X) have developed and demonstrated a method that allows robots to learn from each other’s experiences. By enabling them to learn collectively, they gain more experience quicker.
These robots instantaneously transmit their experience to other robots over the network – sometimes known as “cloud robotics” – and it is this ability that can let them learn from each other to perform motion skills in close coordination with sensing in realistic environments.
Researchers have performed three experiments designed to investigate three possible approaches for general-purpose skill learning across multiple robots: learning motion skills directly from experience, learning internal models of physics, and learning skills with human assistance. In all three cases, multiple robots shared their experiences to build a common model of the skill.
Learning from raw experience with model-free reinforcement learning
Trial-and-error learning is very popular among humans and animals, and can actually be extended well to robots. This kind of learning is called “model-free” because there is no explicit model of environment formed – they explore variations on their existing behavior and then reinforce and exploite the variations that give bigger rewards. In combination with deep neural networks, model-free algorithms have been key to sucess in the past, most notably in the game of Go. Having multiple robots learn this way speeds up the process significantly.
In these experiments, robots were tasked with trying to move their arms to goal locations, or reaching to and opening a door. Each robot has a copy of a neural network that allows it to estimate the value of taking a given action in a given state. By querying this network, the robot can quickly decide what actions might be worth taking in the world. When a robot acts, noise is added to the actions it selects, so the resulting behavior is sometimes a bit better than previously observed, and sometimes a bit worse. This allows each robot to explore different ways of approaching a task. Records of the actions taken by the robots, their behaviors, and the final outcomes are sent back to a central server. The server collects the experiences from all of the robots and uses them to iteratively improve the neural network that estimates value for different states and actions. The model-free algorithms we employed look across both good and bad experiences and distill these into a new network that is better at understanding how action and success are related. Then, at regular intervals, each robot takes a copy of the updated network from the server and begins to act using the information in its new network. Given that this updated network is a bit better at estimating the true value of actions in the world, the robots will produce better behavior. This cycle can then be repeated to continue improving on the task. In the video below, a robot explores the door opening task.

With a few hours of practice, robots sharing their raw experience learn to make reaches to targets, and to open a door by making contact with the handle and pulling. In the case of door opening, the robots learn to deal with the complex physics of the contacts between the hook and the door handle without building an explicit model of the world, as can be seen in the example below:

Learning how the world works by interacting with objects
Direct trial-and-error reinforcement learning is a great way to learn individual skills. However, humans and animals don’t learn exclusively by trial and error. We also build mental models about our environment and imagine how the world might change in response to our actions.
We can start with the simplest of physical interactions, and have our robots learn the basics of cause and effect from reflecting on their own experiences. In this experiment, researchers had the robots play with a wide variety of common household objects by randomly prodding and pushing them inside a tabletop bin. The robots again shared their experiences with each other and together built a single predictive model that attempted to forecast what the world might look like in response to their actions. This predictive model can make simple, if slightly blurry, forecasts about future camera images when provided with the current image and a possible sequence of actions that the robot might execute:

Top row: robotic arms interacting with common household items. Bottom row: Predicted future camera images given an initial image and a sequence of actions.

Once this model is trained, the robots can use it to perform purposeful manipulations, for example based on user commands. In this prototype, a user can command the robot to move a particular object simply by clicking on that object, and then clicking on the point where the object should go:

The robots in this experiment were not told anything about objects or physics: they only see that the command requires a particular pixel to be moved to a particular place. However, because they have seen so many object interactions in their shared past experiences, they can forecast how particular actions will affect particular pixels. In order for such an implicit understanding of physics to emerge, the robots must be provided with a sufficient breadth of experience. This requires either a lot of time, or sharing the combined experiences of many robots.


Learning with the help of humans
Robots can learn entirely on their own, but human guidance is important, not just for telling the robot what to do, but also for helping the robots along. We have a lot of intuition about how various manipulation skills can be performed, and it only seems natural that transferring this intuition to robots can help them learn these skills a lot faster. In the next experiment, each robot was provided with a different door, and guided each of them by hand to show how these doors can be opened. These demonstrations are encoded into a single combined strategy for all robots, called a policy. The policy is a deep neural network which converts camera images to robot actions, and is maintained on a central server. The following video shows the instructor demonstrating the door-opening skill to a robot:

Next, the robots collectively improve this policy through a trial-and-error learning process. Each robot attempts to open its own door using the latest available policy, with some added noise for exploration. These attempts allow each robot to plan a better strategy for opening the door the next time around, and improve the policy accordingly:

Not surprisingly, robots learn more effectively if they are trained on a curriculum of tasks that are gradually increasing in difficulty. In the experiment, each robot starts off by practicing the door-opening skill on a specific position and orientation of the door that the instructor had previously shown it. As it gets better at performing the task, the instructor starts to alter the position and orientation of the door to be just a bit beyond the current capabilities of the policy, but not so difficult that it fails entirely. This allows the robots to gradually increase their skill level over time, and expands the range of situations they can handle. The combination of human-guidance with trial-and-error learning allowed the robots to collectively learn the skill of door-opening in just a couple of hours. Since the robots were trained on doors that look different from each other, the final policy succeeds on a door with a handle that none of the robots had seen before:

These are relatively simple tasks, involving relatively simple skills, but the method is essential, and possibly a key to enabling robots to assist us in our daily lives. Maybe start thinking about which chores you would like to get rid of first. Maybe consider how easily a robot could replace you at your job. Learn a new skill or two.
All media and experiment descriptions courtesy of Sergey Levine, Timothy Lillicrap, and Mrinal Kalakrishnan, Google Research Blog.