MIT News - Machine learning MIT News is dedicated to communicating to the media and the public the news and achievements of the students, faculty, staff and the greater MIT community. en Mon, 09 Mar 2020 00:00:00 -0400 The elephant in the server room Catherine D’Ignazio’s new book, “Data Feminism,” examines problems of bias and power that beset modern information. Mon, 09 Mar 2020 00:00:00 -0400 Peter Dizikes | MIT News Office <p>Suppose you would like to know mortality rates for women during childbirth, by country, around the world. Where would you look? One option is the <a href="" target="_blank">WomanStats</a> Project, the website of an academic research effort investigating the links between the security and activities of nation-states, and the security of the women who live in them.</p> <p>The project, founded in 2001, meets a need by patching together data from around the world. Many countries are indifferent to collecting statistics about women’s lives. But even where countries try harder to gather data, there are clear challenges to arriving at useful numbers — whether it comes to women’s physical security, property rights, and government participation, among many other issues. &nbsp;</p> <p>For instance: In some countries, violations of women’s rights may be reported more regularly than in other places. That means a more responsive legal system may create the appearance of greater problems, when it provides relatively more support for women. The WomanStats Project notes many such complications.</p> <p>Thus the WomanStats Project offers some answers — for example, Australia, Canada, and much of Western Europe have low childbirth mortality rates — while also showing what the challenges are to taking numbers at face value. This, according to MIT professor Catherine D’Ignazio, makes the site unusual, and valuable.</p> <p>“The data never speak for themselves,” says D’Ignazio, referring to the general problem of finding reliable numbers about women’s lives. “There are always humans and institutions speaking for the data, and different people have their own agendas. The data are never innocent.”</p> <p>Now D’Ignazio, an assistant professor in MIT’s Department of Urban Studies and Planning, has taken a deeper look at this issue in a new book, co-authored with Lauren Klein, an associate professor of English and quantitative theory and methods at Emory University. In the book, “<a href="" target="_blank">Data Feminism</a>,” published this month by the MIT Press, the authors use the lens of intersectional feminism to scrutinize how data science reflects the social structures it emerges from.</p> <p>“Intersectional feminism examines unequal power,” write D’Ignazio and Klein, in the book’s introduction. “And in our contemporary world, data is power too. Because the power of data is wielded unjustly, it must be challenged and changed.”</p> <p><strong>The 4 percent problem</strong></p> <p>To see a clear case of power relations generating biased data, D’Ignazio and Klein note, consider research led by MIT’s own Joy Buolamwini, who as a graduate student in a class studying facial-recognition programs, observed that the software in question could not “see” her face. Buolamwini found that for the facial-recognition system in question, the software was based on a set of faces which were 78 percent male and 84 percent white; only 4 percent were female and dark-skinned, like herself.&nbsp;</p> <p>Subsequent media coverage of Buolamwini’s work, D’Ignazio and Klein write, contained “a hint of shock.” But the results were probably less surprising to those who are not white males, they think.&nbsp;&nbsp;</p> <p>“If the past is racist, oppressive, sexist, and biased, and that’s your training data, that is what you are tuning for,” D’Ignazio says.</p> <p>Or consider another example, from tech giant Amazon, which tested an automated system that used AI to sort through promising CVs sent in by job applicants. One problem: Because a high percentage of company employees were men, the algorithm favored men’s names, other things being equal.&nbsp;</p> <p>“They thought this would help [the] process, but of course what it does is train the AI [system] to be biased toward women, because they themselves have not hired that many women,” D’Ignazio observes.</p> <p>To Amazon’s credit, it did recognize the problem. Moreover, D’Ignazio notes, this kind of issue is a problem that can be addressed. “Some of the technologies can be reformed with a more participatory process, or better training data. … If we agree that’s a good goal, one path forward is to adjust your training set and include more people of color, more women.”</p> <p><strong>“Who’s on the team? Who had the idea? Who’s benefiting?” </strong></p> <p>Still, the question of who participates in data science is, as the authors write, “the elephant in the server room.” As of 2011, only 26 percent of all undergraduates receiving computer science degrees in the U.S. were women. That is not only a low figure, but actually a decline from past levels: In 1985, 37 percent of computer science graduates were women, the highest mark on record.</p> <p>As a result of the lack of diversity in the field, D’Ignazio and Klein believe, many data projects are radically limited in their ability to see all facets of the complex social situations they purport to measure.&nbsp;</p> <p>“We want to try to tune people in to these kinds of power relationships and why they matter deeply,” D’Ignazio says. “Who’s on the team? Who had the idea? Who’s benefiting from the project? Who’s potentially harmed by the project?”</p> <p>In all, D’Ignazio and Klein outline seven principles of data feminism, from examining and challenging power, to rethinking binary systems and hierarchies, and embracing pluralism. (Those statistics about gender and computer science graduates are limited, they note, by only using the “male” and “female” categories, thus excluding people who identify in different terms.)</p> <p>People interested in data feminism, the authors state, should also “value multiple forms of knowledge,” including firsthand knowledge that may lead us to question seemingly official data. Also, they should always consider the context in which data are generated, and “make labor visible” when it comes to data science. This last principle, the researchers note, speaks to the problem that even when women and other excluded people contribute to data projects, they often receive less credit for their work.</p> <p>For all the book’s critique of existing systems, programs, and practices, D’Ignazio and Klein are also careful to include examples of positive, successful efforts, such as the WomanStats project, which has grown and thrived over two decades.</p> <p>“For people who are data people but are new to feminism, we want to provide them with a very accessible introduction, and give them concepts and tools they can use in their practice,” D’Ignazio says. “We’re not imagining that people already have feminism in their toolkit. On the other hand, we are trying to speak to folks who are very tuned in to feminism or social justice principles, and highlight for them the ways data science is both problematic, but can be marshalled in the service of justice.”</p> Catherine D’Ignazio is the co-author of a new book, “Data Feminism,” published by MIT Press in March 2020. Image: Diana Levine and MIT PressData, Women, Faculty, Research, Books and authors, MIT Press, Diversity and inclusion, Ethics, Technology and society, Artificial intelligence, Machine learning, Computer science and technology, Urban studies and planning, School of Architecture and Planning “Doing machine learning the right way” Professor Aleksander Madry strives to build machine-learning models that are more reliable, understandable, and robust. Sat, 07 Mar 2020 23:59:59 -0500 Rob Matheson | MIT News Office <p>The work of MIT computer scientist Aleksander Madry is fueled by one core mission: “doing machine learning the right way.”</p> <p>Madry’s research centers largely on making machine learning — a type of artificial intelligence — more accurate, efficient, and robust against errors. In his classroom and beyond, he also worries about questions of ethical computing, as we approach an age where artificial intelligence will have great impact on many sectors of society.</p> <p>“I want society to truly embrace machine learning,” says Madry, a recently tenured professor in the Department of Electrical Engineering and Computer Science. “To do that, we need to figure out how to train models that people can use safely, reliably, and in a way that they understand.”</p> <p>Interestingly, his work with machine learning dates back only a couple of years, to shortly after he joined MIT in 2015. In that time, his research group has published several critical papers demonstrating that certain models can be easily tricked to produce inaccurate results — and showing how to make them more robust.</p> <p>In the end, he aims to make each model’s decisions more interpretable by humans, so researchers can peer inside to see where things went awry. At the same time, he wants to enable nonexperts to deploy the improved models in the real world for, say, helping diagnose disease or control driverless cars.</p> <p>“It’s not just about trying to crack open the machine-learning black box. I want to open it up, see how it works, and pack it back up, so people can use it without needing to understand what’s going on inside,” he says.</p> <p><strong>For the love of algorithms</strong></p> <p>Madry was born in Wroclaw, Poland, where he attended the University of Wroclaw as an undergraduate in the mid-2000s. While he harbored interest in computer science and physics, “I actually never thought I’d become a scientist,” he says.</p> <p>An avid video gamer, Madry initially enrolled in the computer science program with intentions of programming his own games. But in joining friends in a few classes in theoretical computer science and, in particular, theory of algorithms, he fell in love with the material. Algorithm theory aims to find efficient optimization procedures for solving computational problems, which requires tackling difficult mathematical questions. “I realized I enjoy thinking deeply about something and trying to figure it out,” says Madry, who wound up double-majoring in physics and computer science.</p> <p>When it came to delving deeper into algorithms in graduate school, he went to his first choice: MIT. Here, he worked under both Michel X. Goemans, who was a major figure in applied math and algorithm optimization, and Jonathan A. Kelner, who had just arrived to MIT as a junior faculty working in that field. For his PhD dissertation, Madry developed algorithms that solved a number of longstanding problems in graph algorithms, earning the 2011 George M. Sprowls Doctoral Dissertation Award for the best MIT doctoral thesis in computer science.</p> <p>After his PhD, Madry spent a year as a postdoc at Microsoft Research New England, before teaching for three years at the Swiss Federal Institute of Technology Lausanne — which Madry calls “the Swiss version of MIT.” But his alma mater kept calling him back: “MIT has the thrilling energy I was missing. It’s in my DNA.”</p> <p><strong>Getting adversarial</strong></p> <p>Shortly after joining MIT, Madry found himself swept up in a novel science: machine learning. In particular, he focused on understanding the re-emerging paradigm of deep learning. That’s an artificial-intelligence application that uses multiple computing layers to extract high-level features from raw input — such as using pixel-level data to classify images. MIT’s campus was, at the time, buzzing with new innovations in the domain.</p> <p>But that begged the question: Was machine learning all hype or solid science? “It seemed to work, but no one actually understood how and why,” Madry says.</p> <p>Answering that question set his group on a long journey, running experiment after experiment on deep-learning models to understand the underlying principles. A major milestone in this journey was an influential paper they published in 2018, developing a methodology for making machine-learning models more resistant to “adversarial examples.” Adversarial examples are slight perturbations to input data that are imperceptible to humans — such as changing the color of one pixel in an image — but cause a model to make inaccurate predictions. They illuminate a major shortcoming of existing machine-learning tools.</p> <p>Continuing this line of work, Madry’s group showed that the existence of these mysterious adversarial examples may contribute to how machine-learning models make decisions. In particular, models designed to differentiate images of, say, cats and dogs, make decisions based on features that do not align with how humans make classifications. Simply changing these features can make the model consistently misclassify cats as dogs, without changing anything in the image that’s really meaningful to humans.</p> <p>Results indicated some models — which may be used to, say, identify abnormalities in medical images or help autonomous cars identify objects in the road —&nbsp;aren’t exactly up to snuff. “People often think these models are superhuman, but they didn’t actually solve the classification problem we intend them to solve,” Madry says. “And their complete vulnerability to adversarial examples was a manifestation of that fact. That was an eye-opening finding.”</p> <p>That’s why Madry seeks to make machine-learning models more interpretable to humans. New models he’s developed show how much certain pixels in images the system is trained on can influence the system’s predictions. Researchers can then tweak the models to focus on pixels clusters more closely correlated with identifiable features — such as detecting an animal’s snout, ears, and tail. In the end, that will help make the models more humanlike —&nbsp;or “superhumanlike” —&nbsp;in their decisions. To further this work, Madry and his colleagues recently founded the <a href="">MIT Center for Deployable Machine Learning</a>, a collaborative research effort within the <a href="" target="_blank">MIT Quest for Intelligence</a> that is working toward building machine-learning tools ready for real-world deployment.&nbsp;</p> <p>“We want machine learning not just as a toy, but as something you can use in, say, an autonomous car, or health care. Right now, we don’t understand enough to have sufficient confidence in it for those critical applications,” Madry says.</p> <p><strong>Shaping education and policy</strong></p> <p>Madry views artificial intelligence and decision making (“AI+D” is one of the three <a href="">new academic units</a> in the Department of Electrical Engineering and Computer Science) as “the interface of computing that’s going to have the biggest impact on society.”</p> <p>In that regard, he makes sure to expose his students to the human aspect of computing. In part, that means considering consequences of what they’re building. Often, he says, students will be overly ambitious in creating new technologies, but they haven’t thought through potential ramifications on individuals and society. “Building something cool isn’t a good enough reason to build something,” Madry says. “It’s about thinking about not if we can build something, but if we should build something.”</p> <p>Madry has also been engaging in conversations about laws and policies to help regulate machine learning. A point of these discussions, he says, is to better understand the costs and benefits of unleashing machine-learning technologies on society.</p> <p>“Sometimes we overestimate the power of machine learning, thinking it will be our salvation. Sometimes we underestimate the cost it may have on society,” Madry says. “To do machine learning right, there’s still a lot still left to figure out.”</p> Alexander MadryImage: Ian MacLellanComputer science and technology, Algorithms, Artificial intelligence, Machine learning, Computer vision, Technology and society, Faculty, Profile, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering, MIT Schwarzman College of Computing, Quest for Intelligence Showing robots how to do your chores By observing humans, robots learn to perform complex tasks, such as setting a table. Thu, 05 Mar 2020 23:59:59 -0500 Rob Matheson | MIT News Office <p>Training interactive robots may one day be an easy job for everyone, even those without programming expertise. Roboticists are developing automated robots that can learn new tasks solely by observing humans. At home, you might someday show a domestic robot how to do routine chores. In the workplace, you could train robots like new employees, showing them how to perform many duties.</p> <p>Making progress on that vision, MIT researchers have designed a system that lets these types of robots learn complicated tasks that would otherwise stymie them with too many confusing rules. One such task is setting a dinner table under certain conditions. &nbsp;</p> <p>At its core, the researchers’ “Planning with Uncertain Specifications” (PUnS) system gives robots the humanlike planning ability to simultaneously weigh many ambiguous —&nbsp;and potentially contradictory —&nbsp;requirements to reach an end goal. In doing so, the system always chooses the most likely action to take, based on a “belief” about some probable specifications for the task it is supposed to perform.</p> <p>In their work, the researchers compiled a dataset with information about how eight objects — a mug, glass, spoon, fork, knife, dinner plate, small plate, and bowl — could be placed on a table in various configurations. A robotic arm first observed randomly selected human demonstrations of setting the table with the objects. Then, the researchers tasked the arm with automatically setting a table in a specific configuration, in real-world experiments and in simulation, based on what it had seen.</p> <p>To succeed, the robot had to weigh many possible placement orderings, even when items were purposely removed, stacked, or hidden. Normally, all of that would confuse robots too much. But the researchers’ robot made no mistakes over several real-world experiments, and only a handful of mistakes over tens of thousands of simulated test runs. &nbsp;</p> <p>“The vision is to put programming in the hands of domain experts, who can program robots through intuitive ways, rather than describing orders to an engineer to add to their code,” says first author Ankit Shah, a graduate student in the Department of Aeronautics and Astronautics (AeroAstro) and the Interactive Robotics Group, who emphasizes that their work is just one step in fulfilling that vision. “That way, robots won’t have to perform preprogrammed tasks anymore. Factory workers can teach a robot to do multiple complex assembly tasks. Domestic robots can learn how to stack cabinets, load the dishwasher, or set the table from people at home.”</p> <p>Joining Shah on the paper are AeroAstro and Interactive Robotics Group graduate student Shen Li and Interactive Robotics Group leader Julie Shah, an associate professor in AeroAstro and the Computer Science and Artificial Intelligence Laboratory.</p> <div class="cms-placeholder-content-video"></div> <p><strong>Bots hedging bets</strong></p> <p>Robots are fine planners in tasks with clear “specifications,” which help describe the task the robot needs to fulfill, considering its actions, environment, and end goal. Learning to set a table by observing demonstrations, is full of uncertain specifications. Items must be placed in certain spots, depending on the menu and where guests are seated, and in certain orders, depending on an item’s immediate availability or social conventions. Present approaches to planning are not capable of dealing with such uncertain specifications.</p> <p>A popular approach to planning is “reinforcement learning,” a trial-and-error machine-learning technique that rewards and penalizes them for actions as they work to complete a task. But for tasks with uncertain specifications, it’s difficult to define clear rewards and penalties. In short, robots never fully learn right from wrong.</p> <p>The researchers’ system, called PUnS (for Planning with Uncertain Specifications), enables a robot to hold a “belief” over a range of possible specifications. The belief itself can then be used to dish out rewards and penalties. “The robot is essentially hedging its bets in terms of what’s intended in a task, and takes actions that satisfy its belief, instead of us giving it a clear specification,” Ankit Shah says.</p> <p>The system is built on “linear temporal logic” (LTL), an expressive language that enables robotic reasoning about current and future outcomes. The researchers defined templates in LTL that model various time-based conditions, such as what must happen now, must eventually happen, and must happen until something else occurs. The robot’s observations of 30 human demonstrations for setting the table yielded a probability distribution over 25 different LTL formulas. Each formula encoded a slightly different preference — or specification — for setting the table. That probability distribution becomes its belief.</p> <p>“Each formula encodes something different, but when the robot considers various combinations of all the templates, and tries to satisfy everything together, it ends up doing the right thing eventually,” Ankit Shah says.</p> <p><strong>Following criteria</strong></p> <p>The researchers also developed several criteria that guide the robot toward satisfying the entire belief over those candidate formulas. One, for instance, satisfies the most likely formula, which discards everything else apart from the template with the highest probability. Others satisfy the largest number of unique formulas, without considering their overall probability, or they satisfy several formulas that represent highest total probability. Another simply minimizes error, so the system ignores formulas with high probability of failure.</p> <p>Designers can choose any one of the four criteria to preset before training and testing. Each has its own tradeoff between flexibility and risk aversion. The choice of criteria depends entirely on the task. In safety critical situations, for instance, a designer may choose to limit possibility of failure. But where consequences of failure are not as severe, designers can choose to give robots greater flexibility to try different approaches.</p> <p>With the criteria in place, the researchers developed an algorithm to convert the robot’s belief — the probability distribution pointing to the desired formula — into an equivalent reinforcement learning problem. This model will ping the robot with a reward or penalty for an action it takes, based on the specification it’s decided to follow.</p> <p>In simulations asking the robot to set the table in different configurations, it only made six mistakes out of 20,000 tries. In real-world demonstrations, it showed behavior similar to how a human would perform the task. If an item wasn’t initially visible, for instance, the robot would finish setting the rest of the table without the item. Then, when the fork was revealed, it would set the fork in the proper place. “That’s where flexibility is very important,” Ankit Shah says. “Otherwise it would get stuck when it expects to place a fork and not finish the rest of table setup.”</p> <p>Next, the researchers hope to modify the system to help robots change their behavior based on verbal instructions, corrections, or a user’s assessment of the robot’s performance. “Say a person demonstrates to a robot how to set a table at only one spot. The person may say, ‘do the same thing for all other spots,’ or, ‘place the knife before the fork here instead,’” Ankit Shah says. “We want to develop methods for the system to naturally adapt to handle those verbal commands, without needing additional demonstrations.”&nbsp;&nbsp;</p> Roboticists are developing automated robots that can learn new tasks solely by observing humans. At home, you might someday show a domestic robot how to do routine chores.Image: Christine Daniloff, MITResearch, Computer science and technology, Algorithms, Artificial intelligence, Machine learning, Robots, Robotics, Assistive technology, Aeronautical and astronautical engineering, Computer Science and Artificial Intelligence Laboratory (CSAIL), School of Engineering A new model of vision Computer model of face processing could reveal how the brain produces richly detailed visual representations so quickly. Wed, 04 Mar 2020 14:00:00 -0500 Anne Trafton | MIT News Office <p>When we open our eyes, we immediately see our surroundings in great detail. How the brain is able to form these richly detailed representations of the world so quickly is one of the biggest unsolved puzzles in the study of vision.</p> <p>Scientists who study the brain have tried to replicate this phenomenon using computer models of vision, but so far, leading models only perform much simpler tasks such as picking out an object or a face against a cluttered background. Now, a team led by MIT cognitive scientists has produced a computer model that captures the human visual system’s ability to quickly generate a detailed scene description from an image, and offers some insight into how the brain achieves this.</p> <p>“What we were trying to do in this work is to explain how perception can be so much richer than just attaching semantic labels on parts of an image, and to explore the question of how do we see all of the physical world,” says Josh Tenenbaum, a professor of computational cognitive science and a member of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Center for Brains, Minds, and Machines (CBMM).</p> <p>The new model posits that when the brain receives visual input, it quickly performs a series of computations that reverse the steps that a computer graphics program would use to generate a 2D representation of a face or other object. This type of model, known as efficient inverse graphics (EIG), also correlates well with electrical recordings from face-selective regions in the brains of nonhuman primates, suggesting that the primate visual system may be organized in much the same way as the computer model, the researchers say.</p> <p>Ilker Yildirim, a former MIT postdoc who is now an assistant professor of psychology at Yale University, is the lead author of the paper, which appears today in <em>Science Advances</em>. Tenenbaum and Winrich Freiwald, a professor of neurosciences and behavior at Rockefeller University, are the senior authors of the study. Mario Belledonne, a graduate student at Yale, is also an author.</p> <p><strong>Inverse graphics</strong></p> <p>Decades of research on the brain’s visual system has studied, in great detail, how light input onto the retina is transformed into cohesive scenes. This understanding has helped artificial intelligence researchers develop computer models that can replicate aspects of this system, such as recognizing faces or other objects.</p> <p>“Vision is the functional aspect of the brain that we understand the best, in humans and other animals,” Tenenbaum says. “And computer vision is one of the most successful areas of AI at this point. We take for granted that machines can now look at pictures and recognize faces very well, and detect other kinds of objects.”</p> <p>However, even these sophisticated artificial intelligence systems don’t come close to what the human visual system can do, Yildirim says.</p> <p>“Our brains don’t just detect that there’s an object over there, or recognize and put a label on something,” he says. “We see all of the shapes, the geometry, the surfaces, the textures. We see a very rich world.”</p> <p>More than a century ago, the physician, physicist, and philosopher Hermann von Helmholtz theorized that the brain creates these rich representations by reversing the process of image formation. He hypothesized that the visual system includes an image generator that would be used, for example, to produce the faces that we see during dreams. Running this generator in reverse would allow the brain to work backward from the image and infer what kind of face or other object would produce that image, the researchers say.</p> <p>However, the question remained: How could the brain perform this process, known as inverse graphics, so quickly? Computer scientists have tried to create algorithms that could perform this feat, but the best previous systems require many cycles of iterative processing, taking much longer than the 100 to 200 milliseconds the brain requires to create a detailed visual representation of what you’re seeing. Neuroscientists believe perception in the brain can proceed so quickly because it is implemented in a mostly feedforward pass through several hierarchically organized layers of neural processing.</p> <p>The MIT-led team set out to build a special kind of deep neural network model to show how a neural hierarchy can quickly infer the underlying features of a scene — in this case, a specific face. In contrast to the standard deep neural networks used in computer vision, which are trained from labeled data indicating the class of an object in the image, the researchers’ network is trained from a model that reflects the brain’s internal representations of what scenes with faces can look like.</p> <p>Their model thus learns to reverse the steps performed by a computer graphics program for generating faces. These graphics programs begin with a three-dimensional representation of an individual face and then convert it into a two-dimensional image, as seen from a particular viewpoint. These images can be placed on an arbitrary background image. The researchers theorize that the brain’s visual system may do something similar when you dream or conjure a mental image of someone’s face.</p> <p>The researchers trained their deep neural network to perform these steps in reverse — that is, it begins with the 2D image and then adds features such as texture, curvature, and lighting, to create what the researchers call a “2.5D” representation. These 2.5D images specify the shape and color of the face from a particular viewpoint. Those are then converted into 3D representations, which don’t depend on the viewpoint.</p> <p>“The model gives a systems-level account of the processing of faces in the brain, allowing it to see an image and ultimately arrive at a 3D object, which includes representations of shape and texture, through this important intermediate stage of a 2.5D image,” Yildirim says.</p> <p><strong>Model performance</strong></p> <p>The researchers found that their model is consistent with data obtained by studying certain regions in the brains of macaque monkeys. In a study published in 2010, Freiwald and Doris Tsao of Caltech recorded the activity of neurons in those regions and analyzed how they responded to 25 different faces, seen from seven different viewpoints. That study revealed three stages of higher-level face processing, which the MIT team now hypothesizes correspond to three stages of their inverse graphics model: roughly, a 2.5D viewpoint-dependent stage; a stage that bridges from 2.5 to 3D; and a 3D, viewpoint-invariant stage of face representation.</p> <p>“What we show is that both the quantitative and qualitative response properties of those three levels of the brain seem to fit remarkably well with the top three levels of the network that we’ve built,” Tenenbaum says.</p> <p>The researchers also compared the model’s performance to that of humans in a task that involves recognizing faces from different viewpoints. This task becomes harder when researchers alter the faces by removing the face’s texture while preserving its shape, or distorting the shape while preserving relative texture. The new model’s performance was much more similar to that of humans than computer models used in state-of-the-art face-recognition software, additional evidence that this model may be closer to mimicking what happens in the human visual system.</p> <p>“This work is exciting because it introduces interpretable stages of intermediate representation into a feedforward neural network model of face recognition,” says Nikolaus Kriegeskorte, a professor of psychology and neuroscience at Columbia University, who was not involved in the research. “Their approach merges the classical idea that vision inverts a model of how the image was generated, with modern deep feedforward networks. It’s very interesting that this model better explains neural representations and behavioral responses.”</p> <p>The researchers now plan to continue testing the modeling approach on additional images, including objects that aren’t faces, to investigate whether inverse graphics might also explain how the brain perceives other kinds of scenes. In addition, they believe that adapting this approach to computer vision could lead to better-performing AI systems.</p> <p>“If we can show evidence that these models might correspond to how the brain works, this work could lead computer vision researchers to take more seriously and invest more engineering resources in this inverse graphics approach to perception,” Tenenbaum says. “The brain is still the gold standard for any kind of machine that sees the world richly and quickly.”</p> <p>The research was funded by the Center for Brains, Minds, and Machines at MIT, the National Science Foundation, the National Eye Institute, the Office of Naval Research, the New York Stem Cell Foundation, the Toyota Research Institute, and Mitsubishi Electric.</p> MIT cognitive scientists have developed a computer model of face recognition that performs a series of computations that reverse the steps that a computer graphics program would use to generate a 2D representation of a face.Image: courtesy of the researchersResearch, Computer vision, Brain and cognitive sciences, Center for Brains Minds and Machines, Computer Science and Artificial Intelligence Laboratory (CSAIL), School of Science, School of Engineering, National Science Foundation (NSF), Artificial intelligence, Machine learning, Neuroscience Demystifying the world of deep networks Researchers discover that no magic is required to explain why deep networks generalize despite going against statistical intuition. Fri, 28 Feb 2020 14:40:01 -0500 Kris Brewer | Center for Brains, Minds and Machines <p>Introductory statistics courses teach us that, when fitting a model to some data, we should have more data than free parameters to avoid the danger of overfitting — fitting noisy data too closely, and thereby failing to fit new data. It is surprising, then, that in modern deep learning the practice is to have orders of magnitude more parameters than data. Despite this, deep networks show good predictive performance, and in fact do better the more parameters they have. Why would that be?</p> <p>It has been known for some time that good performance in machine learning comes from controlling the complexity of networks, which is not just a simple function of the number of free parameters. The complexity of a classifier, such as a neural network, depends on measuring the “size” of the space of functions that this network represents, with multiple technical measures previously suggested: Vapnik–Chervonenkis dimension, covering numbers, or Rademacher complexity, to name a few. Complexity, as measured by these notions, can be controlled during the learning process by imposing a constraint on the norm of the parameters — in short, on how “big” they can get. The surprising fact is that no such explicit constraint seems to be needed in training deep networks. Does deep learning lie outside of the classical learning theory? Do we need to rethink the foundations?</p> <p>In a new <em>Nature Communications</em> paper, “Complexity Control by Gradient Descent in Deep Networks,” a team from the Center for Brains, Minds, and Machines led by Director Tomaso Poggio, the Eugene McDermott Professor in the MIT Department of Brain and Cognitive Sciences, has shed some light on this puzzle by addressing the most practical and successful applications of modern deep learning: classification problems.</p> <p>“For classification problems, we observe that in fact the parameters of the model do not seem to converge, but rather grow in size indefinitely during gradient descent. However, in classification problems only the normalized parameters matter — i.e., the direction they define, not their size,” says co-author and MIT PhD candidate Qianli Liao. “The not-so-obvious thing we showed is that the commonly used gradient descent on the unnormalized parameters induces the desired complexity control on the normalized ones.”</p> <p>“We have known for some time in the case of regression for shallow linear networks, such as kernel machines, that iterations of gradient descent provide an implicit, vanishing regularization effect,” Poggio says. “In fact, in this simple case we probably know that we get the best-behaving maximum-margin, minimum-norm solution. The question we asked ourselves, then, was: Can something similar happen for deep networks?”</p> <p>The researchers found that it does. As co-author and MIT postdoc Andrzej Banburski explains, “Understanding convergence in deep networks shows that there are clear directions for improving our algorithms. In fact, we have already seen hints that controlling the rate at which these unnormalized parameters diverge allows us to find better performing solutions and find them faster.”</p> <p>What does this mean for machine learning? There is no magic behind deep networks. The same theory behind all linear models is at play here as well. This work suggests ways to improve deep networks, making them more accurate and faster to train.</p> MIT researchers (left to right) Qianli Liao, Tomaso Poggio, and Andrzej Banburski stand with their equations. Image: Kris BrewerCenter for Brains Minds and Machines, Brain and cognitive sciences, Electrical engineering and computer science (EECS), Machine learning, Artificial intelligence, Research Machine learning picks out hidden vibrations from earthquake data Technique may help scientists more accurately map vast underground geologic structures. Fri, 28 Feb 2020 13:00:46 -0500 Jennifer Chu | MIT News Office <p>Over the last century, scientists have developed methods to map the structures within the Earth’s crust, in order to identify resources such as oil reserves, geothermal sources, and, more recently, reservoirs where excess carbon dioxide could potentially be sequestered. They do so by tracking seismic waves that are produced naturally by earthquakes or artificially via explosives or underwater air guns. The way these waves bounce and scatter through the Earth can give scientists an idea of the type of structures that lie beneath the surface.</p> <p>There is a narrow range of seismic waves — those that occur at low frequencies of around 1 hertz — that could give scientists the clearest picture of underground structures spanning wide distances. But these waves are often drowned out by Earth’s noisy seismic hum, and are therefore difficult to pick up with current detectors. Specifically generating low-frequency waves would require pumping in enormous amounts of energy. For these reasons, low-frequency seismic waves have largely gone missing in human-generated seismic data.</p> <p>Now MIT researchers have come up with a machine learning workaround to fill in this gap.</p> <p>In a paper appearing in the journal <em>Geophysics</em>, they describe a method in which they trained a neural network on hundreds of different simulated earthquakes. When the researchers presented the trained network with only the high-frequency seismic waves produced from a new simulated earthquake, the neural network was able to imitate the physics of wave propagation and accurately estimate the quake’s missing low-frequency waves.</p> <p>The new method could allow researchers to artificially synthesize the low-frequency waves that are hidden in seismic data, which can then be used to more accurately map the Earth’s internal structures.</p> <p>“The ultimate dream is to be able to map the whole subsurface, and be able to say, for instance, ‘this is exactly what it looks like underneath Iceland, so now you know where to explore for geothermal sources,’” says co-author Laurent Demanet, professor of applied mathematics at MIT. “Now we’ve shown that deep learning offers a solution to be able to fill in these missing frequencies.”</p> <p>Demanet’s co-author is lead author Hongyu Sun, a graduate student in MIT’s Department of Earth, Atmospheric and Planetary Sciences.</p> <p><strong>Speaking another frequency</strong></p> <p>A neural network is a set of algorithms modeled loosely after the neural workings of the human brain. The algorithms are designed to recognize patterns in data that are fed into the network, and to cluster these data into categories, or labels. A common example of a neural network involves visual processing; the model is trained to classify an image as either a cat or a dog, based on the patterns it recognizes between thousands of images that are specifically labeled as cats, dogs, and other objects.</p> <p>Sun and Demanet adapted a neural network for signal processing, specifically, to recognize patterns in seismic data. They reasoned that if a neural network was fed enough examples of earthquakes, and the ways in which the resulting high- and low-frequency seismic waves travel through a particular composition of the Earth, the network should be able to, as they write in their paper, “mine the hidden correlations among different frequency components” and extrapolate any missing frequencies if the network were only given an earthquake’s partial seismic profile.</p> <p>The researchers looked to train a convolutional neural network, or CNN, a class of deep neural networks that is often used to analyze visual information. A CNN very generally consists of an input and output layer, and multiple hidden layers between, that process inputs to identify correlations between them.</p> <p>Among their many applications, CNNs have been used as a means of generating visual or auditory “deepfakes” — content that has been extrapolated or manipulated through deep-learning and neural networks, to make it seem, for example, as if a woman were talking with a man’s voice.</p> <p>“If a network has seen enough examples of how to take a male voice and transform it into a female voice or vice versa, you can create a sophisticated box to do that,” Demanet says. “Whereas here we make the Earth speak another frequency — one that didn’t originally go through it.”</p> <p><strong>Tracking waves</strong></p> <p>The researchers trained their neural network with inputs that they generated using the Marmousi model, a complex two-dimensional geophysical model that simulates the way seismic waves travel through geological structures of varying density and composition. &nbsp;</p> <p>In their study, the team used the model to simulate nine “virtual Earths,” each with a different subsurface composition. For each Earth model, they simulated 30 different earthquakes, all with the same strength, but different starting locations. In total, the researchers generated hundreds of different seismic scenarios. They fed the information from almost all of these simulations into their neural network and let the network find correlations between seismic signals.</p> <p>After the training session, the team introduced to the neural network a new earthquake that they simulated in the Earth model but did not include in the original training data. They only included the high-frequency part of the earthquake’s seismic activity, in hopes that the neural network learned enough from the training data to be able to infer the missing low-frequency signals from the new input.</p> <p>They found that the neural network produced the same low-frequency values that the Marmousi model originally simulated.</p> <p>“The results are fairly good,” Demanet says. “It’s impressive to see how far the network can extrapolate to the missing frequencies.”</p> <p>As with all neural networks, the method has its limitations. Specifically, the neural network is only as good as the data that are fed into it. If a new input is wildly different from the bulk of a network’s training data, there’s no guarantee that the output will be accurate. To contend with this limitation, the researchers say they plan to introduce a wider variety of data to the neural network, such as earthquakes of different strengths, as well as subsurfaces of more varied composition.</p> <p>As they improve the neural network’s predictions, the team hopes to be able to use the method to extrapolate low-frequency signals from actual seismic data, which can then be plugged into seismic models to more accurately map the geological structures below the Earth’s surface. The low frequencies, in particular, are a key ingredient for solving the big puzzle of finding the correct physical model.</p> <p>“Using this neural network will help us find the missing frequencies to ultimately improve the subsurface image and find the composition of the Earth,” Demanet says.</p> <p>This research was supported, in part, by Total SA and the U.S. Air Force Office of Scientific Research.</p> MIT researchers have used a neural network to identify low-frequency seismic waves hidden in earthquake data. The technique may help scientists more accurately map the Earth’s interior.Image: Christine Daniloff, MITEAPS, Earthquakes, Environment, Geology, Mathematics, Research, School of Science, Machine learning, Artificial intelligence, Earth and atmospheric sciences To self-drive in the snow, look under the road Weather’s a problem for autonomous cars. MIT’s new system shows promise by using “ground-penetrating radar” instead of cameras or lasers. Wed, 26 Feb 2020 14:50:00 -0500 Adam Conner-Simons | MIT CSAIL <p>Car companies have been feverishly working to improve the technologies behind self-driving cars. But so far even the most high-tech vehicles still fail when it comes to safely navigating in rain and snow.&nbsp;</p> <p>This is because these weather conditions wreak havoc on the most common approaches for sensing, which usually involve either lidar sensors or cameras. In the snow, for example, cameras can no longer recognize lane markings and traffic signs, while the lasers of lidar sensors malfunction when there’s, say, stuff flying down from the sky.</p> <p>MIT researchers have recently been wondering whether an entirely different approach might work. Specifically, what if we instead looked under the road?&nbsp;</p> <div class="cms-placeholder-content-video"></div> <p>A team from MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) has developed a new system that uses an existing technology called ground-penetrating radar (GPR) to send electromagnetic pulses underground that measure the area’s specific combination of soil, rocks, and roots. Specifically, the CSAIL team used a particular form of GPR instrumentation developed at <a href="" target="_blank">MIT Lincoln Laboratory</a> called <a href="" target="_self">localizing ground-penetrating radar</a>, or LGPR. The mapping process creates a unique fingerprint of sorts that the car can later use to localize itself when it returns to that particular plot of land.</p> <p>“If you or I grabbed a shovel and dug it into the ground, all we’re going to see is a bunch of dirt,” says CSAIL PhD student Teddy Ort, lead author on a new paper about the project that will be published in the <em>IEEE Robotics and Automation Letters</em> journal later this month. “But LGPR can quantify the specific elements there and compare that to the map it’s already created, so that it knows exactly where it is, without needing cameras or lasers.”</p> <p>In tests, the team found that in snowy conditions the navigation system’s average margin of error was on the order of only about an inch compared to clear weather. The researchers were surprised to find that it had a bit more trouble with rainy conditions, but was still only off by an average of 5.5 inches. (This is because rain leads to more water soaking into the ground, leading to a larger disparity between the original mapped LGPR reading and the current condition of the soil.)</p> <p>The researchers said the system’s robustness was further validated by the fact that, over a period of six months of testing, they never had to unexpectedly step in to take the wheel.&nbsp;</p> <p>“Our work demonstrates that this approach is actually a practical way to help self-driving cars navigate poor weather without actually having to be able to ‘see’ in the traditional sense using laser scanners or cameras,” says MIT Professor Daniela Rus, director of CSAIL and senior author on the new paper, which will also be presented in May at the International Conference on Robotics and Automation in Paris.</p> <p>While the team has only tested the system at low speeds on a closed country road, Ort said that existing work from Lincoln Laboratory suggests that the system could easily be extended to highways and other high-speed areas.&nbsp;</p> <p>This is the first time that developers of self-driving systems have employed ground-penetrating radar, which has previously been used in fields like construction planning, landmine detection, and even <a href="" target="_blank">lunar exploration</a>. The approach wouldn’t be able to work completely on its own, since it can’t detect things above ground. But its ability to localize in bad weather means that it would couple nicely with lidar and vision approaches.</p> <p>“Before releasing autonomous vehicles on public streets, localization and navigation have to be totally reliable at all times,” says Roland Siegwart, a professor of autonomous systems at ETH Zurich who was not involved in the project. “The CSAIL team’s innovative and novel concept has the potential to push autonomous vehicles much closer to real-world deployment.”&nbsp;</p> <p>One major benefit of mapping out an area with LGPR is that underground maps tend to hold up better over time than maps created using vision or lidar, since features of an above-ground map are much more likely to change. LGPR maps also take up only about 80 percent of the space used by traditional 2D sensor maps that many companies use for their cars.&nbsp;</p> <p>While the system represents an important advance, Ort notes that it’s far from road-ready. Future work will need to focus on designing mapping techniques that allow LGPR datasets to be stitched together to be able to deal with multi-lane roads and intersections. In addition, the current hardware is bulky and 6 feet wide, so major design advances need to be made before it’s small and light enough to fit into commercial vehicles.</p> <p>Ort and Rus co-wrote the paper with CSAIL postdoc Igor Gilitschenski. The project was supported, in part, by MIT Lincoln Laboratory.</p> MIT's new system allows a self-driving car to situate itself in snowy conditions.Photo courtesy of the researchersComputer Science and Artificial Intelligence Laboratory (CSAIL), Lincoln Laboratory, Robotics, Electrical engineering and computer science (EECS), School of Engineering, Research, Distributed Robotics Laboratory, Artificial intelligence, Automobiles, Autonomous vehicles, Faculty, Computer vision, Machine learning, Transportation, MIT Schwarzman College of Computing Dreaming big in a small country MIT students teach machine learning and entrepreneurship in Uruguay through MIT Global Startup Labs. Mon, 24 Feb 2020 15:15:01 -0500 MISTI <p>When Miguel Brechner started planning a new ambitious plan to foster a new generation of data scientists in Uruguay and Latin America, he immediately thought of MIT. “There is no question that MIT is a world leader in science and technology. In Uruguay we are a small country, but we dream big.” Brechner is president of Plan Ceibal, an internationally awarded public initiative that has as main goals to distribute technology, promote knowledge, and generate social equity by widening access to digital technologies.</p> <p>In 2019, Uruguayan public institutions like Plan Ceibal, ANII (Agencia Nacional de Investigaci<span class="st">ó</span>n e Innovaci<span class="st">ó</span>n), and UTEC (<span class="st">Universidad Tecnológica del Uruguay)</span> began collaborating with MIT International Science and Technology Initiatives&nbsp;(<a href="">MISTI</a>) and the Abdul Latif Jameel World Education Lab (<a href="">J-WEL</a>). The partnership supports 60 Latin American students that are part of <a href="">the Program in Data Science</a>, a program which includes online courses from <em>MITx</em> and on-site workshops run by J-WEL and MISTI. Local students include CEOs, entrepreneurs, engineers, economists, medical professionals, and senior administrators.</p> <p>The MISTI Global Startup Labs (GSL) program, now in its 20th&nbsp;year, has expanded its partnerships to include Uruguayan institutions to promote entrepreneurship and data science across Latin America. GSL is a unique program designed to offer the opportunity to blend digital technologies and entrepreneurship in emerging regions in the world. Since 1998, hundreds of MIT students have traveled to more than 15 countries to be part of the program that has benefited thousands of technology entrepreneurs around the world. GSL instructors are MIT graduate and undergraduate students, selected among many applicants from all over the institute. GSL programs in different countries are uniquely crafted based on the needs of the local partners, and MIT student instructors take the lead teaching app and web development, coding, data science, entrepreneurship, and intrapreneurship.</p> <p>The new GSL, one of the first to be run over Independent Acitivities Period, took place during January in Montevideo. The Uruguay program focused specifically on machine learning and the business opportunities of the technology. The local student participants had previously taken courses from the <em>MITx</em> MicroMasters in Data Science, and the GSL workshop gave them the opportunity to experience project-based learning in data science. This hands-on experiential immersion in the subject matter is the core methodology of the GSL program.</p> <p>More than 30 graduate and undergraduate students applied to be part of GSL in Uruguay this year, and 13 were selected to be part of the workshop in Montevideo. Eduardo Rivera, managing director for Uruguay, explained the process: “Recruiting students for GSL is always a challenge. We look for expertise and experience teaching, but also for team players and risk-takers. The team is composed of students from different disciplines and levels of studies, which makes the experience a unique opportunity for our students to learn from their MIT peers in new and challenging contexts.” Rivera adds, “At MIT, we are fortunate to have plenty of talented and passionate students, willing to cross borders and oceans to teach and learn.”</p> <p>Over the course of a month, the local students were taught how to build prototypes, create business models, and pitch presentations. The class pursued projects ranging from predictive maintenance to autism detection to logistics optimization. The final results were presented in a pitch event hosted in Zonamerica, Uruguay's premier hub for technology and innovation.</p> <p>"Working with our local students was a truly unique and unforgettable opportunity," says electrical engineering and computer science (EECS) senior Ryan Sander. "I'm certain I learned just as much from the students as they learned from us. What really left an impression on me was observing not only how bright our students are, but also how passionate these people are about solving real-world problems with high impact."</p> <p>For MBA student Kenny Li, the opportunity to interact with the local students was broadening. “In today’s world, you need to be able to understand people’s cultures, how do they approach business, how they interact at work …GSL gave me a great learning opportunity to understand the global context of entrepreneurs.”</p> <p>When not teaching classes, the MIT students were able to visit various places around Montevideo, including the beautiful beaches of Punta del Este, the neighboring city of Buenos Aires, and relaxing getaways to Colonia. After classes, the teaching team was steps away from the beach and could wind each day down with a beautiful sunset, soaking up the warm summer weather in January.</p> <p>Rivera finds these cultural connections to be one of the major benefits of the program. “At MISTI, we are certain that international teaching activities contribute not only to the academic formation of the students but also give them valuable tools to interact in multicultural environments and confront new challenges in different locations. For future global leaders, this is a unique opportunity. We often hear from our students that MISTI experiences are life-changing, not only in professional life but also in their personal life.”</p> <p>"The weekends and weekday evenings were a great way for us to bond with each other and our students," says Victoria Pisini, a senior in the MIT Sloan School of Management. “We went to beaches together, traveled to different cities, and shared a lot of unforgettable moments."</p> <p>The MIT students participating in this year’s GSL were Amauche Emenari (EECS PhD student), Devin Zhang (MBA student), Evan Mu (EECS PhD student), Geeticka Chauhan (EECS PhD student), Hans Nowak (MBA student), Julian Alverio (EECS MEng student), Kenny Li (MBA student), Madhav Kumar (MIT Sloan PhD candidate), Maya Murad (Leaders for Global Operations master's and PhD student), Ryan Sander (EECS student), Taylor Elise Baum (EECS PhD student), Tiffany Fung (MBA student), and Victoria Pisini (MBA student).</p> <p>GSL programs are planned in multiple countries for summer 2020 and Independent Activities Period 2021, and there are still a small number of available opportunities for instructors this summer.</p> (Left to right:) MIT students Kenny Li, Maya Murad, Devin Zhang, Tiffany Fung, Eduardo Rivera, Evan Pu, Taylor Baum, Julian Alverio, and Madhav Kumar participated in this year's MISTI Global Startup Labs program.MISTI, Abdul Latif Jameel World Education Lab (J-WEL), MITx, Machine learning, Electrical engineering and computer science (EECS), Sloan School of Management, Independent Activities Period, School of Humanities Arts and Social Sciences, Innovation and Entrepreneurship (I&E), International initiatives, Global, Center for International Studies, Latin America Bringing deep learning to life MIT duo uses music, videos, and real-world examples to teach students the foundations of artificial intelligence. Mon, 24 Feb 2020 14:10:01 -0500 Kim Martineau | MIT Quest for Intelligence <p>Gaby Ecanow loves listening to music, but never considered writing her own until taking <a href="" target="_blank">6.S191</a> (Introduction to Deep Learning). By her second class, the second-year MIT student had composed an original Irish folk song with the help of a recurrent neural network, and was considering how to adapt the model to create her own Louis the Child-inspired dance beats.</p> <p>“It was cool,” she says. “It didn’t sound at all like a machine had made it.”&nbsp;</p> <p>This year, 6.S191 kicked off as usual, with students spilling into the aisles of Stata Center’s Kirsch Auditorium during Independent Activities Period (IAP). But the opening lecture featured a twist: a&nbsp;<a href=";" target="_blank">recorded welcome</a>&nbsp;from former President Barack Obama. The video was quickly revealed to be an AI-generated fabrication, one of many twists that&nbsp;<a href="" target="_blank">Alexander Amini</a>&nbsp;’17 and&nbsp;<a href="" target="_blank">Ava Soleimany</a>&nbsp;’16 introduce throughout their for-credit course to make the equations and code come alive.&nbsp;</p> <div class="cms-placeholder-content-video"></div> <p>As&nbsp;hundreds of their peers look on, Amini and Soleimany take turns at the podium. If they appear at ease, it’s because they know the material cold; they designed the curriculum themselves, and have taught it for the past three years.&nbsp;The course covers the technical foundations of deep learning and its societal implications through lectures and software labs focused on real-world applications. On the final day, students compete for prizes by pitching their own ideas for research projects. In the weeks leading up to class, Amini and Soleimany spend hours updating the labs, refreshing their lectures, and honing their presentations.</p> <p>A branch of&nbsp;machine learning,&nbsp;deep learning harnesses massive data and algorithms&nbsp;modeled loosely on how the brain processes information to make predictions. The class has been credited with helping to spread machine-learning tools into research labs across MIT. That’s by design, says Amini, a graduate student in MIT’s&nbsp;<a href="">Department of Electrical Engineering and Computer Science</a>&nbsp;(EECS), and Soleimany, a graduate student at MIT and Harvard University.</p> <p>Both are using machine learning in their own research — Amini in engineering robots, and Soleimany in developing diagnostic tools for cancer — and they wanted to make sure the curriculum would prepare students to do the same. In addition to the lab on developing a music-generating AI,&nbsp;they offer labs on building a face-recognition model with convolutional neural networks and a bot that uses reinforcement learning to play the vintage Atari video game, Pong.&nbsp;After students master the basics, those taking the class for credit go on to create applications of their own.&nbsp;</p> <p>This year, 23 teams&nbsp;presented projects. Among the prize winners was Carmen&nbsp;Martin, a graduate student in the&nbsp;<a href="" target="_blank">Harvard-MIT Program in Health Sciences and Technology</a>&nbsp;(HST), who&nbsp;proposed using a type of neural net called a graph convolutional network to predict the spread of coronavirus. She combined several data streams: airline ticketing data to measure population fluxes, real-time confirmation of new infections, and a ranking of how well countries are equipped to prevent and respond to&nbsp;a pandemic.&nbsp;</p> <p>“The goal is to train the model to predict cases to guide national governments and the World Health Organization in their recommendations to limit new cases and save lives,” she says.</p> <p>A second prize winner, EECS graduate student Samuel Sledzieski, proposed building a model to predict protein interactions using only their amino acid sequences. Predicting protein behavior is key to designing drug targets, among other clinical applications, and Sledzieski wondered if deep learning could speed up the search for viable protein pairs.&nbsp;</p> <p>“There’s still work to be done, but I’m excited by how far I was able to get in three days,” he says. “Having easy-to-follow examples in TensorFlow and Keras helped me understand how to actually build and train these models myself.”&nbsp;He plans to continue the work in his current lab rotation with&nbsp;<a href="">Bonnie Berger</a>, the Simons Professor of Mathematics in EECS and the&nbsp;<a href="" target="_blank">Computer Science and Artificial Intelligence Laboratory</a>&nbsp;(CSAIL).&nbsp;</p> <p>Each year, students also hear about emerging deep-learning applications from companies sponsoring the course. David Cox, co-director of the&nbsp;<a href="">MIT-IBM Watson AI Lab</a>, covered neuro-symbolic AI, a hybrid approach that combines symbolic programs with deep learning’s expert pattern-matching ability. Alex Wiltschko, a senior researcher at Google Brain, spoke about using a network analysis tool to predict the scent of small molecules. Chuan Li, chief scientific officer at Lambda Labs, discussed neural rendering, a tool for reconstructing and generating graphics scenes. Animesh Garg, a senior researcher at NVIDIA, covered strategies for developing robots that perceive and act more human-like.</p> <p>With 350 students taking the&nbsp;live course each year, and&nbsp;more than a million people who have watched the lectures online, Amini and Soleimany have become prominent ambassadors for deep learning. Yet, it was tennis that first brought them together.&nbsp;</p> <p>Amini competed nationally as a high school student in Ireland&nbsp;and built an award-winning&nbsp;AI model to help amateur and pro tennis players improve their strokes; Soleimany was a two-time captain of the MIT women’s tennis team. They met on the court as undergraduates&nbsp;and discovered they shared a passion for machine learning.&nbsp;</p> <p>After finishing their undergraduate degrees, they decided to challenge themselves and fill what they saw as an increasing need at MIT for a foundational course in deep learning.&nbsp;6.S191 was launched in 2017 by two grad students, Nick Locascio and Harini Suresh, and Amini and Soleimany had a vision for transforming the course into something more. They created a series of software labs, introduced new cutting-edge topics like robust and ethical AI, and added content to appeal to a broad range of students, from computer scientists to aerospace engineers and MBAs.</p> <p>“Alexander and I are constantly brainstorming, and those discussions are key to how 6.S191 and some of our own collaborative research projects have developed,” says Soleimany.&nbsp;</p> <p>They cover one of those research collaborations&nbsp;in class. During the computer vision lab, students learn about algorithmic bias and how to test for and address racial and gender bias in face-recognition tools. The lab is based on an algorithm that Amini and Soleimany developed with their respective advisors,&nbsp;<a href="" target="_blank">Daniela Rus</a>, director of CSAIL, and&nbsp;<a href="" target="_blank">Sangeeta Bhatia</a>, the John J. and Dorothy Wilson Professor of HST and EECS. This year they also covered hot topics in robotics, including recent work of Amini’s on driverless cars.&nbsp;</p> <p>But they don’t plan to stop there.&nbsp;“We’re committed to making 6.S191 the best that it can be, each year we teach it,” says Amini “and that means moving the course forward as deep learning continues to evolve.”&nbsp;</p> A crash course in deep learning organized and taught by grad students Alexander Amini (right) and Ava Soleimany reaches more than 350 MIT students each year; more than a million other people have watched their lectures online over the past three years.Photo: Gretchen ErtlQuest for Intelligence, Independent Activities Period, Electrical engineering and computer science (EECS), Harvard-MIT Health Sciences and Technology, Computer Science and Artificial Intelligence Laboratory (CSAIL), MIT-IBM Watson AI Lab, School of Engineering, Artificial intelligence, Algorithms, Computer science and technology, Machine learning, Classes and programs, Faculty A human-machine collaboration to defend against cyberattacks PatternEx merges human and machine expertise to spot and respond to hacks. Fri, 21 Feb 2020 14:12:18 -0500 Zach Winn | MIT News Office <p>Being a cybersecurity analyst at a large company today is a bit like looking for a needle in a haystack — if that haystack were hurtling toward you at fiber optic speed.</p> <p>Every day, employees and customers generate loads of data that establish a normal set of behaviors. An attacker will also generate data while using any number of techniques to infiltrate the system; the goal is to find that “needle” and stop it before it does any damage.</p> <p>The data-heavy nature of that task lends itself well to the number-crunching prowess of machine learning, and an influx of AI-powered systems have indeed flooded the cybersecurity market over the years. But such systems can come with their own problems, namely a never-ending stream of false positives that can make them more of a time suck than a time saver for security analysts.</p> <p>MIT startup PatternEx starts with the assumption that algorithms can’t protect a system on their own. The company has developed a closed loop approach whereby machine-learning models flag possible attacks and human experts provide feedback. The feedback is then incorporated into the models, improving their ability to flag only the activity analysts care about in the future.</p> <p>“Most machine learning systems in cybersecurity have been doing anomaly detection,” says Kalyan Veeramachaneni, a co-founder of PatternEx and a principal research scientist at MIT. “The problem with that, first, is you need a baseline [of normal activity]. Also, the model is usually unsupervised, so it ends up showing a lot of alerts, and people end up shutting it down. The big difference is that PatternEx allows the analyst to inform the system and then it uses that feedback to filter out false positives.”</p> <p>The result is an increase in analyst productivity. When compared to a generic anomaly detection software program, PatternEx’s Virtual Analyst Platform successfully identified 10 times more threats through the same number of daily alerts, and its advantage persisted even when the generic system gave analysts five times more alerts per day.</p> <p>First deployed in 2016, today the company’s system is being used by security analysts at large companies in a variety of industries along with firms that offer cybersecurity as a service.</p> <p><strong>Merging human and machine approaches to cybersecurity</strong></p> <p>Veeramachaneni came to MIT in 2009 as a postdoc and now directs a research group in the Laboratory for Information and Decision Systems. His work at MIT primarily deals with big data science and machine learning, but he didn’t think deeply about applying those tools to cybersecurity until a brainstorming session with PatternEx co-founders Costas Bassias, Uday Veeramachaneni, and Vamsi Korrapati in 2013.</p> <p>Ignacio Arnaldo, who worked with Veeramachaneni as a postdoc at MIT between 2013 and 2015, joined the company shortly after. Veeramachaneni and Arnaldo knew from their time building tools for machine-learning researchers at MIT that a successful solution would need to seamlessly integrate machine learning with human expertise.</p> <p>“A lot of the problems people have with machine learning arise because the machine has to work side by side with the analyst,” Veeramachaneni says, noting that detected attacks still must be presented to humans in an understandable way for further investigation. “It can’t do everything by itself. Most systems, even for something as simple as giving out a loan, is augmentation, not machine learning just taking decisions away from humans.”</p> <p>The company’s first partnership was with a large online retailer, which allowed the founders to train their models to identify potentially malicious behavior using real-world data. One by one, they trained their algorithms to flag different types of attacks using sources like Wi-Fi access logs, authentication logs, and other user behavior in the network.</p> <p>The early models worked best in retail, but Veeramachaneni knew how much businesses in other industries were struggling to apply machine learning in their operations from his many conversations with company executives at MIT (a subject PatternEx recently published <a href="">a paper</a> on).</p> <p>“MIT has done an incredible job since I got here 10 years ago bringing industry through the doors,” Veeramachaneni says. He estimates that in the past six years as a member of MIT’s Industrial Liaison Program he’s had 200 meetings with members of the private sector to talk about the problems they’re facing. He has also used those conversations to make sure his lab’s research is addressing relevant problems.</p> <p>In addition to enterprise customers, the company began offering its platform to security service providers and teams that specialize in hunting for undetected cyberattacks in networks.</p> <p>Today analysts can build machine learning models through PatternEx’s platform without writing a line of code, lowering the bar for people to use machine learning as part of a larger trend in the industry toward what Veeramachaneni calls the democratization of AI.</p> <p>“There’s not enough time in cybersecurity; it can’t take hours or even days to understand why an attack is happening,” Veeramachaneni says. “That’s why getting the analyst the ability to build and tweak machine learning models &nbsp;is the most critical aspect of our system.”</p> <p><strong>Giving security analysts an army</strong></p> <p>PatternEx’s Virtual Analyst Platform is designed to make security analysts feel like they have an army of assistants combing through data logs and presenting them with the most suspicious behavior on their network.</p> <p>The platform uses machine learning models to go through more than 50 streams of data and identify suspicious behavior. It then presents that information to the analyst for feedback, along with charts and other data visualizations that help the analyst decide how to proceed. After the analyst determines whether or not the behavior is an attack, that feedback is incorporated back into the models, which are updated across PatternEx’s entire customer base.</p> <p>“Before machine learning, someone would catch an attack, probably a little late, they might name it, and then they’ll announce it, and all the other companies will call and find out about it and go in and check their data,” Veeramachaneni says. “For us, if there’s an attack, we take that data, and because we have multiple customers, we have to transfer that in real time to other customer’s data to see if it’s happening with them too. We do that very efficiently on a daily basis.”</p> <p>The moment the system is up and running with new customers, it is able to identify 40 different types of cyberattacks using 170 different prepackaged machine learning models. Arnaldo notes that as the company works to grow those figures, customers are also adding to PatternEx’s model base by building solutions on the platform that address specific threats they’re facing.</p> <p>Even if customers aren’t building their own models on the platform, they can deploy PatternEx’s system out of the box, without any machine learning expertise, and watch it get smarter automatically.</p> <p>By providing that flexibility, PatternEx is bringing the latest tools in artificial intelligence to the people who understand their industries most intimately. It all goes back to the company’s founding principle of empowering humans with artificial intelligence instead of replacing them.</p> <p>“The target users of the system are not skilled data scientists or machine learning experts — profiles that are hard for cybersecurity teams to hire — but rather domain experts already on their payroll that have the deepest understanding of their data and uses cases,” Arnaldo says.</p> PatternEx’s Virtual Analyst Platform uses machine learning models to detect suspicious activity on a network. That activity is then presented to human analysts for feedback that improves the systems’ ability to flag activity analysts care about.Innovation and Entrepreneurship (I&E), Startups, Computer Science and Artificial Intelligence Laboratory (CSAIL), Machine learning, Artificial intelligence, Computer science and technology, Data, Cyber security, MIT Schwarzman College of Computing, Laboratory for Information and Decision Systems (LIDS) Artificial intelligence yields new antibiotic A deep-learning model identifies a powerful new drug that can kill many species of antibiotic-resistant bacteria. Thu, 20 Feb 2020 10:59:59 -0500 Anne Trafton | MIT News Office <p>Using a machine-learning algorithm, MIT researchers have identified a powerful new antibiotic compound. In laboratory tests, the drug killed many of the world’s most problematic disease-causing bacteria, including some strains that are resistant to all known antibiotics. It also cleared infections in two different mouse models.</p> <p>The computer model, which can screen more than a hundred million chemical compounds in a matter of days, is designed to pick out potential antibiotics that kill bacteria using different mechanisms than those of existing drugs.</p> <p>“We wanted to develop a platform that would allow us to harness the power of artificial intelligence to usher in a new age of antibiotic drug discovery,” says James Collins, the Termeer Professor of Medical Engineering and Science in MIT’s Institute for Medical Engineering and Science (IMES) and Department of Biological Engineering. “Our approach revealed this amazing molecule which is arguably one of the more powerful antibiotics that has been discovered.”</p> <p>In their new study, the researchers also identified several other promising antibiotic candidates, which they plan to test further. They believe the model could also be used to design new drugs, based on what it has learned about chemical structures that enable drugs to kill bacteria.</p> <p>“The machine learning model can explore, in silico, large chemical spaces that can be prohibitively expensive for traditional experimental approaches,” says Regina Barzilay, the Delta Electronics Professor of Electrical Engineering and Computer Science in MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).</p> <p>Barzilay and Collins, who are faculty co-leads for MIT’s Abdul Latif Jameel Clinic for Machine Learning in Health (J-Clinic), are the senior authors of the study, which appears today in <em>Cell</em>. The first author of the paper is Jonathan Stokes, a postdoc at MIT and the Broad Institute of MIT and Harvard.</p> <p><strong>A new pipeline</strong></p> <p>Over the past few decades, very few new antibiotics have been developed, and most of those newly approved antibiotics are slightly different variants of existing drugs. Current methods for screening new antibiotics are often prohibitively costly, require a significant time investment, and are usually limited to a narrow spectrum of chemical diversity.</p> <p>“We’re facing a growing crisis around antibiotic resistance, and this situation is being generated by both an increasing number of pathogens becoming resistant to existing antibiotics, and an anemic pipeline in the biotech and pharmaceutical industries for new antibiotics,” Collins says.</p> <p>To try to find completely novel compounds, he teamed up with Barzilay, Professor Tommi Jaakkola, and their students Kevin Yang, Kyle Swanson, and Wengong Jin, who have previously developed machine-learning computer models that can be trained to analyze the molecular structures of compounds and correlate them with particular traits, such as the ability to kill bacteria.</p> <p>The idea of using predictive computer models for “in silico” screening is not new, but until now, these models were not sufficiently accurate to transform drug discovery. Previously, molecules were represented as vectors reflecting the presence or absence of certain chemical groups. However, the new neural networks can learn these representations automatically, mapping molecules into continuous vectors which are subsequently used to predict their properties.</p> <p>In this case, the researchers designed their model to look for chemical features that make molecules effective at killing <em>E. coli</em>. To do so, they trained the model on about 2,500 molecules, including about 1,700 FDA-approved drugs and a set of 800 natural products with diverse structures and a wide range of bioactivities.</p> <p>Once the model was trained, the researchers tested it on the Broad Institute’s Drug Repurposing Hub, a library of about 6,000 compounds. The model picked out one molecule that was predicted to have strong antibacterial activity and had a chemical structure different from any existing antibiotics. Using a different machine-learning model, the researchers also showed that this molecule would likely have low toxicity to human cells.</p> <p>This molecule, which the researchers decided to call halicin, after the fictional artificial intelligence system from “2001: A Space Odyssey,” has been previously investigated as possible diabetes drug. The researchers tested it against dozens of bacterial strains isolated from patients and grown in lab dishes, and found that it was able to kill many that are resistant to treatment, including <em>Clostridium difficile</em>, <em>Acinetobacter baumannii</em>, and <em>Mycobacterium tuberculosis</em>. The drug worked against every species that they tested, with the exception of <em>Pseudomonas aeruginosa</em>, a difficult-to-treat lung pathogen.</p> <p>To test halicin’s effectiveness in living animals, the researchers used it to treat mice infected with <em>A. baumannii</em>, a bacterium that has infected many U.S. soldiers stationed in Iraq and Afghanistan. The strain of <em>A. baumannii</em> that they used is resistant to all known antibiotics, but application of a halicin-containing ointment completely cleared the infections within 24 hours.</p> <p>Preliminary studies suggest that halicin kills bacteria by disrupting their ability to maintain an electrochemical gradient across their cell membranes. This gradient is necessary, among other functions, to produce ATP (molecules that cells use to store energy), so if the gradient breaks down, the cells die. This type of killing mechanism could be difficult for bacteria to develop resistance to, the researchers say.</p> <p>“When you’re dealing with a molecule that likely associates with membrane components, a cell can’t necessarily acquire a single mutation or a couple of mutations to change the chemistry of the outer membrane. Mutations like that tend to be far more complex to acquire evolutionarily,” Stokes says.</p> <p>In this study, the researchers found that <em>E. coli</em> did not develop any resistance to halicin during a 30-day treatment period. In contrast, the bacteria started to develop resistance to the antibiotic ciprofloxacin within one to three days, and after 30 days, the bacteria were about 200 times more resistant to ciprofloxacin than they were at the beginning of the experiment.</p> <p>The researchers plan to pursue further studies of halicin, working with a pharmaceutical company or nonprofit organization, in hopes of developing it for use in humans.</p> <p><strong>Optimized molecules</strong></p> <p>After identifying halicin, the researchers also used their model to screen more than 100 million molecules selected from the ZINC15 database, an online collection of about 1.5 billion chemical compounds. This screen, which took only three days, identified 23 candidates that were structurally dissimilar from existing antibiotics and predicted to be nontoxic to human cells.</p> <p>In laboratory tests against five species of bacteria, the researchers found that eight of the molecules showed antibacterial activity, and two were particularly powerful. The researchers now plan to test these molecules further, and also to screen more of the ZINC15 database.</p> <p>The researchers also plan to use their model to design new antibiotics and to optimize existing molecules. For example, they could train the model to add features that would make a particular antibiotic target only certain bacteria, preventing it from killing beneficial bacteria in a patient’s digestive tract.</p> <p>“This groundbreaking work signifies a paradigm shift in antibiotic discovery and indeed in drug discovery more generally,” says Roy Kishony, a professor of biology and computer science at Technion (the Israel Institute of Technology), who was not involved in the study. “Beyond in silica screens, this approach will allow using deep learning at all stages of antibiotic development, from discovery to improved efficacy and toxicity through drug modifications and medicinal chemistry.”</p> <p>The research was funded by the Abdul Latif Jameel Clinic for Machine Learning in Health, the Defense Threat Reduction Agency, the Broad Institute, the DARPA Make-It Program, the Canadian Institutes of Health Research, the Canadian Foundation for Innovation, the Canada Research Chairs Program, the Banting Fellowships Program, the Human Frontier Science Program, the Pershing Square Foundation, the Swiss National Science Foundation, a National Institutes of Health Early Investigator Award, the National Science Foundation Graduate Research Fellowship Program, and a gift from Anita and Josh Bekenstein.</p> MIT researchers used a machine-learning algorithm to identify a drug called halicin that kills many strains of bacteria. Halicin (top row) prevented the development of antibiotic resistance in E. coli, while ciprofloxacin (bottom row) did not.Image: courtesy of the Collins Lab at MITResearch, Biological engineering, Electrical Engineering & Computer Science (eecs), Institute for Medical Engineering and Science (IMES), Computer Science and Artificial Intelligence Laboratory (CSAIL), Broad Institute, School of Engineering, Bacteria, Microbes, Medicine, Health, Machine learning, Artificial intelligence, Algorithms, J-Clinic, National Institutes of Health (NIH), National Science Foundation (NSF) Richard Dudley, professor emeritus of mathematics, dies at 81 Longtime MIT professor strongly influenced the fields of probability, statistics, and machine learning. Tue, 18 Feb 2020 15:25:01 -0500 Department of Mathematics <p>Richard Mansfield Dudley, MIT professor emeritus of mathematics, died on Jan. 19 following a long illness. He was 81. Dudley served on the MIT mathematics faculty from 1967 until 2015, when he officially retired. Over the course of those 48 years, during which he published over 100 articles as well as numerous books and monographs, he made fundamental breakthroughs in the theory of stochastic process and the general theory of weak convergence.</p> <p>Dudley’s work, starting in the 1960s, shaped the fields of probability, mathematical statistics, and machine learning, with highly influential contributions to the theory of Gaussian processes and empirical processes. What is now widely known as “Dudley’s entropy bound” has become a standard tool of modern research in probability, statistics, and machine learning. Dudley’s work also had a transformative impact on the theory of empirical processes initiated by Vladimir Vapnik and Alexey Chervonenkis in the context of machine learning. Over a series of papers, starting with his landmark paper “Central limit theorems for empirical processes” (<em>Annals of Probability</em>, 1978) and culminating with his influential Saint-Flour lecture notes (1984) and later, his book “Uniform Central Limit Theorems”<em> </em>(Cambridge University Press, 1999), Dudley distilled and developed these ideas into an actionable theory that still today is the reference framework in mathematical statistics and statistical learning theory. The larger communities of probability and statistics remember his excellent taste for mathematically rich and impactful subjects, as well as his highest standard of rigor.</p> <p>Dudley gave a number of distinguished research talks. He was an invited speaker at the 1974 International Congress of Mathematicians as well as at meetings of the American Mathematical Society, the Institute of Mathematical Statistics, and the Bernoulli society. He was also an invited lecturer at Saint-Flour probability summer school in probability in 1982 and several of the Vilnius Conferences on Probability Theory and Mathematical Statistics. He was a regular participant and organizer of several conferences and meetings, including Probability in Banach Spaces.</p> <p>In 1976, Dudley visited the University of Aarhus, and there produced a set of graduate lecture notes, “Probabilities and Metrics.” These were to become a part of his graduate text, “Real Analysis and Probability,”<em> </em>published by Wadsworth, Inc. in 1989. An early review of this work in the <em>London Mathematical Society Bulletin</em> (July 1990) found that it “could be compared to the appearance of Breiman or Loève's classic probability texts.” The text has since become a standard, and in 2002 was reissued by Cambridge University Press and continues to be in print.</p> <p>Dudley was always highly regarded as a graduate mentor throughout his career. He advised 33 PhD candidates (32<em> </em>at MIT), yielding some 105 academic “descendants.”</p> <p>Dudley served the scholarly community as associate editor (1972-78) and then chief editor (1979-81) of <em>Annals of Probability</em>. He was a member of the editorial board of the <em>Wadsworth/Brooks/Cole Advanced Series in Statistics/Probability </em>from 1982 to 1992. For many years while on the MIT faculty, Dudley worked with the MIT Science Library in overseeing their collection of mathematical journals. He sought to explain to the faculty how the library's budget decisions were reached, to help them effectively express their research needs.</p> <p>Among his honors, Dudley was an Alfred P. Sloan Research fellow from 1966-68 and Guggenheim Foundation Fellow in 1991. He was selected to serve on the honorary Advisory Board of Stochastic Processes and their Applications from 1987-2001. In 1993, Dudley was elected a fellow of the American Statistical Association, “for world-recognized contributions to probability theory with far-reaching consequences for statistics, for founding the modern theory of empirical processes, and for dedication to many successful PhD students.” He was also elected fellow of the Institute of Mathematical Statistics, the American Association for the<em> </em>Advancement of Science, and the American Mathematical Society and was selected to be a member of the International Statistical Institute.</p> <p>Born on July 28, 1938, in Cleveland, Ohio, Dudley completed a BA from Harvard University, summa cum laude, in 1959. He wrote a doctoral dissertation under two advisors at Princeton University, Gilbert A. Hunt and Edward Nelson, completing his PhD in mathematics in 1962. He was an instructor at the University of California at Berkley in 1962-63, and an assistant professor from 1963 to 1967, before moving to MIT.</p> <p>Dudley is survived by his wife, Elizabeth (Liza) Martin; his sisters, Edith D. Sylla and Alice D. Carmel; brother-in-law Richard E. Sylla; and nieces Anne Sylla, Margaret S. Padua, and Genevieve Carmel.&nbsp;</p> <p>Memorial contributions may be made in Dudley's name to the <a href="" target="_blank">Environmental Defense Fund</a> or to <a href="" target="_blank">Partners in Health</a>.</p> Richard M. Dudley taught mathematics at MIT for 48 years, until he retired in 2015.Image courtesy of the Department of Mathematics.Mathematics, Obituaries, Faculty, School of Science, Machine learning, Education, teaching, academics Bringing artificial intelligence into the classroom, research lab, and beyond Through the Undergraduate Research Opportunities Program, students work to build AI tools with impact. Thu, 13 Feb 2020 16:50:01 -0500 Kim Martineau | MIT Quest for Intelligence <p>Artificial intelligence is reshaping how we live, learn, and work, and this past fall, MIT undergraduates got to explore and build on some of the tools coming out of research labs at MIT. Through the&nbsp;<a href="" target="_blank">Undergraduate Research Opportunities Program</a>&nbsp;(UROP), students worked with researchers at the MIT Quest for Intelligence and elsewhere on projects to improve AI literacy and K-12 education, understand face recognition and how the brain forms new memories, and speed up tedious tasks like cataloging new library material. Six projects are featured below.</p> <p><strong>Programming Jibo to forge an emotional bond with kids</strong></p> <p>Nicole Thumma met her first robot when she was 5, at a museum.&nbsp;“It was incredible that I could have a conversation, even a simple conversation, with this machine,” she says. “It made me think&nbsp;robots&nbsp;are&nbsp;the most complicated manmade thing, which made me want to learn more about them.”</p> <p>Now a senior at MIT, Thumma spent last fall writing dialogue for the social robot Jibo, the brainchild of&nbsp;<a href="">MIT Media Lab</a> Associate Professor&nbsp;<a href="">Cynthia Breazeal</a>. In a UROP project co-advised by Breazeal and researcher&nbsp;<a href="">Hae Won Park</a>, Thumma scripted mood-appropriate dialogue to help Jibo bond with students while playing learning exercises together.</p> <p>Because emotions are complicated, Thumma riffed on a set of basic feelings in her dialogue — happy/sad, energized/tired, curious/bored. If Jibo was feeling sad, but energetic and curious, she might program it to say, “I'm feeling blue today, but something that always cheers me up is talking with my friends, so I'm glad I'm playing with you.​” A tired, sad, and bored Jibo might say, with a tilt of its head, “I don't feel very good. It's like my wires are all mixed up today. I think this activity will help me feel better.”&nbsp;</p> <p>In these brief interactions, Jibo models its vulnerable side and teaches kids how to express their emotions. At the end of an interaction, kids can give Jibo a virtual token to pick up its mood or energy level. “They can see what impact they have on others,” says Thumma. In all, she wrote 80 lines of dialogue, an experience that led to her to stay on at MIT for an MEng in robotics. The Jibos she helped build are now in kindergarten classrooms in Georgia, offering emotional and intellectual support as they read stories and play word games with their human companions.</p> <p><strong>Understanding why familiar faces stand out</strong></p> <p>With a quick glance, the faces of friends and acquaintances jump out from those of strangers. How does the brain do it?&nbsp;<a href="">Nancy Kanwisher</a>’s lab in the&nbsp;<a href="">Department of Brain and Cognitive Sciences</a> (BCS) is building computational models to understand the face-recognition process.&nbsp;<a href="">Two key findings</a>: the brain starts to register the gender and age of a face before recognizing its identity, and that face perception is more robust for familiar faces.</p> <p>This fall, second-year student Joanne Yuan worked with postdoc&nbsp;<a href="">Katharina Dobs</a>&nbsp;to understand&nbsp;why this is so.&nbsp;In earlier experiments, subjects were shown multiple photographs of familiar faces of American celebrities and unfamiliar faces of German celebrities while their brain activity was measured with magnetoencephalography. Dobs found that subjects processed age and gender before the celebrities’ identity regardless of whether the face was familiar. But they were much better at unpacking the gender and identity of faces they knew, like Scarlett Johansson, for example. Dobs suggests that the improved gender and identity recognition for familiar faces is due to a feed-forward mechanism rather than top-down retrieval of information from memory.&nbsp;</p> <p>Yuan has explored both hypotheses with a type of model, convolutional neural networks (CNNs), now widely used in face-recognition tools. She trained a CNN on the face images and studied its layers to understand its processing steps. She found that the model, like Dobs’ human subjects, appeared to process gender and age before identity, suggesting that both CNNs and the brain are primed for face recognition in similar ways. In another experiment, Yuan trained two CNNs on familiar and unfamiliar faces and found that the CNNs, again like humans, were better at identifying the familiar faces.</p> <p>Yuan says she enjoyed exploring two fields — machine learning and neuroscience — while gaining an appreciation for the simple act of recognizing faces. “It’s pretty complicated and there’s so much more to learn,” she says.</p> <p><strong>Exploring memory formation</strong></p> <p>Protruding from the branching dendrites of brain cells are microscopic nubs that grow and change shape as memories form. Improved imaging techniques have allowed researchers to move closer to these nubs, or spines, deep in the brain to learn more about their role in creating and consolidating memories.</p> <p><a href="">Susumu Tonegawa</a>, the Picower Professor of Biology and Neuroscience, has&nbsp;pioneered a technique for labeling clusters of brain cells, called “engram cells,” that are linked to specific memories in mice. Through conditioning, researchers train a mouse, for example, to recognize an environment. By tracking the evolution of dendritic spines in cells linked to a single memory trace, before and after the learning episode, researchers can estimate where memories may be physically stored.&nbsp;</p> <p>But it takes time. Hand-labeling spines in a stack of 100 images can take hours — more, if the researcher needs to consult images from previous days to verify that a spine-like nub really is one, says&nbsp;Timothy O’Connor, a software engineer in BCS helping with the project.&nbsp;With 400 images taken in a typical session, annotating the images can take longer than collecting them, he adds.</p> <p>O’Connor&nbsp;contacted the Quest&nbsp;<a href="">Bridge</a>&nbsp;to see if the process could be automated. Last fall, undergraduates Julian Viera and Peter Hart began work with Bridge AI engineer Katherine Gallagher to train a neural network to automatically pick out the spines. Because spines vary widely in shape and size, teaching the computer what to look for is one big challenge facing the team as the work continues. If successful, the tool could be useful to a hundred other labs across the country.</p> <p>“It’s exciting to work on a project that could have a huge amount of impact,” says Viera. “It’s also cool to be learning something new in computer science and neuroscience.”</p> <p><strong>Speeding up the archival process</strong></p> <p>Each year, Distinctive Collections at the MIT Libraries receives&nbsp;a large volume of personal letters, lecture notes, and other materials from donors inside and outside of MIT&nbsp;that tell MIT’s story and document the history of science and technology.&nbsp;Each of these unique items must be organized and described, with a typical box of material taking up to 20 hours to process and make available to users.</p> <p>To make the work go faster, Andrei Dumitrescu and Efua Akonor, undergraduates at MIT and Wellesley College respectively, are working with Quest Bridge’s Katherine Gallagher to develop an automated system for processing archival material donated to MIT. Their goal: to&nbsp;develop a machine-learning pipeline that can categorize and extract information from scanned images of the records. To accomplish this task, they turned to the U.S. Library of Congress (LOC), which has digitized much of its extensive holdings.&nbsp;</p> <p>This past fall, the students pulled images of about&nbsp;70,000 documents, including correspondence, speeches, lecture notes, photographs, and books&nbsp;housed at the LOC, and trained a classifier to distinguish a letter from, say, a speech. They are now using optical character recognition and a text-analysis tool&nbsp;to extract key details like&nbsp;the date, author, and recipient of a letter, or the date and topic of a lecture. They will soon incorporate object recognition to describe the content of a&nbsp;photograph,&nbsp;and are looking forward to&nbsp;testing&nbsp;their system on the MIT Libraries’ own digitized data.</p> <p>One&nbsp;highlight of the project was learning to use Google Cloud. “This is the real world, where there are no directions,” says Dumitrescu. “It was fun to figure things out for ourselves.”&nbsp;</p> <p><strong>Inspiring the next generation of robot engineers</strong></p> <p>From smartphones to smart speakers, a growing number of devices live in the background of our daily lives, hoovering up data. What we lose in privacy we gain in time-saving personalized recommendations and services. It’s one of AI’s defining tradeoffs that kids should understand, says third-year student Pablo&nbsp;Alejo-Aguirre.&nbsp;“AI brings us&nbsp;beautiful and&nbsp;elegant solutions, but it also has its limitations and biases,” he says.</p> <p>Last year, Alejo-Aguirre worked on an AI literacy project co-advised by Cynthia Breazeal and graduate student&nbsp;<a href="">Randi Williams</a>. In collaboration with the nonprofit&nbsp;<a href="">i2 Learning</a>, Breazeal’s lab has developed an AI curriculum around a robot named Gizmo that teaches kids how to&nbsp;<a href="">train their own robot</a>&nbsp;with an Arduino micro-controller and a user interface based on Scratch-X, a drag-and-drop programming language for children.&nbsp;</p> <p>To make Gizmo accessible for third-graders, Alejo-Aguirre developed specialized programming blocks that give the robot simple commands like, “turn left for one second,” or “move forward for one second.” He added Bluetooth to control Gizmo remotely and simplified its assembly, replacing screws with acrylic plates that slide and click into place. He also gave kids the choice of rabbit and frog-themed Gizmo faces.&nbsp;“The new design is a lot sleeker and cleaner, and the edges are more kid-friendly,” he says.&nbsp;</p> <p>After building and testing several prototypes, Alejo-Aguirre and Williams demoed their creation last summer at a robotics camp. This past fall, Alejo-Aguirre manufactured 100 robots that are now in two schools in Boston and a third in western Massachusetts.&nbsp;“I’m proud of the technical breakthroughs I made through designing, programming, and building the robot, but I’m equally proud of the knowledge that will be shared through this curriculum,” he says.</p> <p><strong>Predicting stock prices with machine learning</strong></p> <p>In search of a practical machine-learning application to learn more about the field, sophomores Dolapo Adedokun and Daniel Adebi hit on stock picking. “We all know buy, sell, or hold,” says Adedokun. “We wanted to find an easy challenge that anyone could relate to, and develop a guide for how to use machine learning in that context.”</p> <p>The two friends approached the Quest Bridge with their own idea for a UROP project after they were turned away by several labs because of their limited programming experience, says Adedokun. Bridge engineer Katherine Gallagher, however, was willing to take on novices. “We’re building machine-learning tools for non-AI specialists,” she says. “I was curious to see how Daniel and Dolapo would approach the problem and reason through the questions they encountered.”</p> <p>Adebi wanted to learn more about reinforcement learning, the trial-and-error AI technique that has allowed computers to surpass humans at chess, Go, and a growing list of video games. So, he and Adedokun worked with Gallagher to structure an experiment to see how reinforcement learning would fare against another AI technique, supervised learning, in predicting stock prices.</p> <p>In reinforcement learning, an agent is turned loose in an unstructured environment with one objective: to maximize a specific outcome (in this case, profits) without being told explicitly how to do so. Supervised learning, by contrast, uses labeled data to accomplish a goal, much like a problem set with the correct answers included.</p> <p>Adedokun and Adebi trained both models on seven years of stock-price data, from 2010-17, for Amazon, Microsoft, and Google. They then compared profits generated by the reinforcement learning model and a trading algorithm based on the supervised model’s price predictions for the following 18 months; they found that their reinforcement learning model produced higher returns.</p> <p>They developed a Jupyter notebook to share what they learned and explain how they built and tested their models. “It was a valuable exercise for all of us,” says Gallagher. “Daniel and Dolapo got hands-on experience with machine-learning fundamentals, and I got insight into the types of obstacles users with their background might face when trying to use the tools we’re building at the Bridge.”</p> Students participating in MIT Quest for Intelligence-funded UROP projects include: (clockwise from top left) Nicole Thumma, Joanne Yuan, Julian Viera, Andrei Dumitrescu, Pablo Alejo-Aguirre, and Dolapo Adedokun.Photo panel: Samantha SmileyQuest for Intelligence, Brain and cognitive sciences, Media Lab, Libraries, School of Engineering, School of Science, Artifical intelligence, Algorithms, Computer science and technology, Machine learning, Undergraduate Research Opportunities Program (UROP), Students, Undergraduate, Electrical engineering and computer science (EECS) “Sensorized” skin helps soft robots find their bearings Flexible sensors and an artificial intelligence model tell deformable robots how their bodies are positioned in a 3D environment. Wed, 12 Feb 2020 23:59:59 -0500 Rob Matheson | MIT News Office <p>For the first time, MIT researchers have enabled a soft robotic arm to understand its configuration in 3D space, by leveraging only motion and position data from its own “sensorized” skin.</p> <p>Soft robots constructed from highly compliant materials, similar to those found in living organisms, are being championed as safer, and more adaptable, resilient, and bioinspired alternatives to traditional rigid robots. But giving autonomous control to these deformable robots is a monumental task because they can move in a virtually infinite number of directions at any given moment. That makes it difficult to train planning and control models that drive automation.</p> <p>Traditional methods to achieve autonomous control use large systems of multiple motion-capture cameras that provide the robots feedback about 3D movement and positions. But those are impractical for soft robots in real-world applications.</p> <p>In a paper being published in the journal <em>IEEE Robotics and Automation Letters</em>, the researchers describe a system of soft sensors that cover a robot’s body to provide “proprioception” — meaning awareness of motion and position of its body. That feedback runs into a novel deep-learning model that sifts through the noise and captures clear signals to estimate the robot’s 3D configuration. The researchers validated their system on a soft robotic arm resembling an elephant trunk, that can predict its own position as it autonomously swings around and extends.</p> <p>The sensors can be fabricated using off-the-shelf materials, meaning any lab can develop their own systems, says Ryan Truby, a postdoc in the MIT Computer Science and Artificial Laboratory (CSAIL) who is co-first author on the paper along with CSAIL postdoc Cosimo Della Santina.</p> <p>“We’re sensorizing soft robots to get feedback for control from sensors, not vision systems, using a very easy, rapid method for fabrication,” he says. “We want to use these soft robotic trunks, for instance, to orient and control themselves automatically, to pick things up and interact with the world. This is a first step toward that type of more sophisticated automated control.”</p> <p>One future aim is to help make artificial limbs that can more dexterously handle and manipulate objects in the environment. “Think of your own body: You can close your eyes and reconstruct the world based on feedback from your skin,” says co-author Daniela Rus, director of CSAIL and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. “We want to design those same capabilities for soft robots.”</p> <p><strong>Shaping soft sensors</strong></p> <p>A longtime goal in soft robotics has been fully integrated body sensors. Traditional rigid sensors detract from a soft robot body’s natural compliance, complicate its design and fabrication, and can cause various mechanical failures. Soft-material-based sensors are a more suitable alternative, but require specialized materials and methods for their design, making them difficult for many robotics labs to fabricate and integrate in soft robots.</p> <p>While working in his CSAIL lab one day looking for inspiration for sensor materials, Truby made an interesting connection. “I found these sheets of conductive materials used for electromagnetic interference shielding, that you can buy anywhere in rolls,” he says. These materials have “piezoresistive” properties, meaning they change in electrical resistance when strained. Truby realized they could make effective soft sensors if they were placed on certain spots on the trunk. As the sensor deforms in response to the trunk’s stretching and compressing, its electrical resistance is converted to a specific output voltage. The voltage is then used as a signal correlating to that movement.</p> <p>But the material didn’t stretch much, which would limit its use for soft robotics. Inspired by kirigami —&nbsp;a variation of origami that includes making cuts in a material — Truby designed and laser-cut rectangular strips of conductive silicone sheets into various patterns, such as rows of tiny holes or crisscrossing slices like a chain link fence. That made them far more flexible, stretchable, “and beautiful to look at,” Truby says.</p> <p><img alt="" src="/sites/" style="width: 500px; height: 281px;" /></p> <p><img alt="" src="/sites/" style="width: 500px; height: 281px;" /></p> <p><em style="font-size: 10px;">Credit: Ryan L. Truby, MIT&nbsp;CSAIL</em></p> <p>The researchers’ robotic trunk comprises three segments, each with four fluidic actuators (12 total) used to move the arm. They fused one sensor over each segment, with each sensor covering and gathering data from one embedded actuator in the soft robot. They used “plasma bonding,” a technique that energizes a surface of a material to make it bond to another material. It takes roughly a couple hours to shape dozens of sensors that can be bonded to the soft robots using a handheld plasma-bonding device.</p> <p><img alt="" src="/sites/" style="width: 500px; height: 281px;" /></p> <p><span style="font-size:10px;"><em>Credit: Ryan L. Truby, MIT&nbsp;CSAIL</em></span></p> <p><strong>“Learning” configurations</strong></p> <p>As hypothesized, the sensors did capture the trunk’s general movement. But they were really noisy. “Essentially, they’re nonideal sensors in many ways,” Truby says. “But that’s just a common fact of making sensors from soft conductive materials. Higher-performing and more reliable sensors require specialized tools that most robotics labs do not have.”</p> <p>To estimate the soft robot’s configuration using only the sensors, the researchers built a deep neural network to do most of the heavy lifting, by sifting through the noise to capture meaningful feedback signals. The researchers developed a new model to kinematically describe the soft robot’s shape that vastly reduces the number of variables needed for their model to process.</p> <p>In experiments, the researchers had the trunk swing around and extend itself in random configurations over approximately an hour and a half. They used the traditional motion-capture system for ground truth data. In training, the model analyzed data from its sensors to predict a configuration, and compared its predictions to that ground truth data which was being collected simultaneously. In doing so, the model “learns” to map signal patterns from its sensors to real-world configurations. Results indicated, that for certain and steadier configurations, the robot’s estimated shape matched the ground truth.</p> <p>Next, the researchers aim to explore new sensor designs for improved sensitivity and to develop new models and deep-learning methods to reduce the required training for every new soft robot. They also hope to refine the system to better capture the robot’s full dynamic motions.</p> <p>Currently, the neural network and sensor skin are not sensitive to capture subtle motions or dynamic movements. But, for now, this is an important first step for learning-based approaches to soft robotic control, Truby says: “Like our soft robots, living systems don’t have to be totally precise. Humans are not precise machines, compared to our rigid robotic counterparts, and we do just fine.”</p> MIT researchers have created a “sensorized” skin, made with kirigami-inspired sensors, that gives soft robots greater awareness of the motion and position of their bodies.Ryan L. Truby, MIT CSAILResearch, Computer science and technology, Algorithms, Robots, Robotics, Soft robotics, Design, Machine learning, Materials Science and Engineering, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering Automated system can rewrite outdated sentences in Wikipedia articles Text-generating tool pinpoints and replaces specific information in sentences while retaining humanlike grammar and style. Wed, 12 Feb 2020 13:51:56 -0500 Rob Matheson | MIT News Office <p>A system created by MIT researchers could be used to automatically update factual inconsistencies in Wikipedia articles, reducing time and effort spent by human editors who now do the task manually.</p> <p>Wikipedia comprises millions of articles that are in constant need of edits to reflect new information. That can involve article expansions, major rewrites, or more routine modifications such as updating numbers, dates, names, and locations. Currently, humans across the globe volunteer their time to make these edits.&nbsp;&nbsp;</p> <p>In a paper being presented at the AAAI Conference on Artificial Intelligence, the researchers describe a text-generating system that pinpoints and replaces specific information in relevant Wikipedia sentences, while keeping the language similar to how humans write and edit.</p> <p>The idea is that humans would type into an interface an unstructured sentence with updated information, without needing to worry about style or grammar. The system would then search Wikipedia, locate the appropriate page and outdated sentence, and rewrite it in a humanlike fashion. In the future, the researchers say, there’s potential to build a fully automated system that identifies and uses the latest information from around the web to produce rewritten sentences in corresponding Wikipedia articles that reflect updated information.</p> <p>“There are so many updates constantly needed to Wikipedia articles. It would be beneficial to automatically modify exact portions of the articles, with little to no human intervention,” says Darsh Shah, a PhD student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and one of the lead authors. “Instead of hundreds of people working on modifying each Wikipedia article, then you’ll only need a few, because the model is helping or doing it automatically. That offers dramatic improvements in efficiency.”</p> <p>Many other bots exist that make automatic Wikipedia edits. Typically, those work on mitigating vandalism or dropping some narrowly defined information into predefined templates, Shah says. The researchers’ model, he says, solves a harder artificial intelligence problem: Given a new piece of unstructured information, the model automatically modifies the sentence in a humanlike fashion. “The other [bot] tasks are more rule-based, while this is a task requiring reasoning over contradictory parts in two sentences and generating a coherent piece of text,” he says.</p> <p>The system can be used for other text-generating applications as well, says co-lead author and CSAIL graduate student Tal Schuster. In their paper, the researchers also used it to automatically synthesize sentences in a popular fact-checking dataset that helped reduce bias, without manually collecting additional data. “This way, the performance improves for automatic fact-verification models that train on the dataset for, say, fake news detection,” Schuster says.</p> <p>Shah and Schuster worked on the paper with their academic advisor Regina Barzilay, the Delta Electronics Professor of Electrical Engineering and Computer Science and a professor in CSAIL.</p> <p><strong>Neutrality masking and fusing</strong></p> <p>Behind the system is a fair bit of text-generating ingenuity in identifying contradictory information between, and then fusing together, two separate sentences. It takes as input an “outdated” sentence from a Wikipedia article, plus a separate “claim” sentence that contains the updated and conflicting information. The system must automatically delete and keep specific words in the outdated sentence, based on information in the claim, to update facts but maintain style and grammar. That’s an easy task for humans, but a novel one in machine learning.</p> <p>For example, say there’s a required update to this sentence (in bold): “Fund A considers <strong>28 of their 42</strong> minority stakeholdings in operationally active companies to be of particular significance to the group.” The claim sentence with updated information may read: “Fund A considers <strong>23 of 43</strong> minority stakeholdings significant.” The system would locate the relevant Wikipedia text for “Fund A,” based on the claim. It then automatically strips out the outdated numbers (28 and 42) and replaces them with the new numbers (23 and 43), while keeping the sentence exactly the same and grammatically correct. (In their work, the researchers ran the system on a dataset of specific Wikipedia sentences, not on all Wikipedia pages.)</p> <p>The system was trained on a popular dataset that contains pairs of sentences, in which one sentence is a claim and the other is a relevant Wikipedia sentence. Each pair is labeled in one of three ways: “agree,” meaning the sentences contain matching factual information; “disagree,” meaning they contain contradictory information; or “neutral,” where there’s not enough information for either label. The system must make all disagreeing pairs agree, by modifying the outdated sentence to match the claim. That requires using two separate models to produce the desired output.</p> <p>The first model is a fact-checking classifier — pretrained to label each sentence pair as “agree,” “disagree,” or “neutral” — that focuses on disagreeing pairs. Running in conjunction with the classifier is a custom “neutrality masker” module that identifies which words in the outdated sentence contradict the claim. The module removes the minimal number of words required to “maximize neutrality” — meaning the pair can be labeled as neutral. That’s the starting point: While the sentences don’t agree, they no longer contain obviously contradictory information. The module creates a binary “mask” over the outdated sentence, where a 0 gets placed over words that most likely require deleting, while a 1 goes on top of keepers.</p> <p>After masking, a novel two-encoder-decoder framework is used to generate the final output sentence. This model learns compressed representations of the claim and the outdated sentence. Working in conjunction, the two encoder-decoders fuse the dissimilar words from the claim, by sliding them into the spots left vacant by the deleted words (the ones covered with 0s) in the outdated sentence.</p> <p>In one test, the model scored higher than all traditional methods, using a technique called “SARI” that measures how well machines delete, add, and keep words compared to the way humans modify sentences. They used a dataset with manually edited Wikipedia sentences, which the model hadn’t seen before. Compared to several traditional text-generating methods, the new model was more accurate in making factual updates and its output more closely resembled human writing. In another test, crowdsourced humans scored the model (on a scale of 1 to 5) based on how well its output sentences contained factual updates and matched human grammar. The model achieved average scores of 4 in factual updates and 3.85 in matching grammar.</p> <p><strong>Removing bias</strong></p> <p>The study also showed that the system can be used to augment datasets to eliminate bias when training detectors of “fake news,” a form of propaganda containing disinformation created to mislead readers in order to generate website views or steer public opinion. Some of these detectors train on datasets of agree-disagree sentence pairs to “learn” to verify a claim by matching it to given evidence.</p> <p>In these pairs, the claim will either match certain information with a supporting “evidence” sentence from Wikipedia (agree) or it will be modified by humans to include information contradictory to the evidence sentence (disagree). The models are trained to flag claims with refuting evidence as “false,” which can be used to help identify fake news.</p> <p>Unfortunately, such datasets currently come with unintended biases, Shah says: “During training, models use some language of the human written claims as “give-away” phrases to mark them as false, without relying much on the corresponding evidence sentence. This reduces the model’s accuracy when evaluating real-world examples, as it does not perform fact-checking.”</p> <p>The researchers used the same deletion and fusion techniques from their Wikipedia project to balance the disagree-agree pairs in the dataset and help mitigate the bias. For some “disagree” pairs, they used the modified sentence’s false information to regenerate a fake “evidence” supporting sentence. Some of the give-away phrases then exist in both the “agree” and “disagree” sentences, which forces models to analyze more features. Using their augmented dataset, the researchers reduced the error rate of a popular fake-news detector by 13 percent.</p> <p>“If you have a bias in your dataset, and you’re fooling your model into just looking at one sentence in a disagree pair to make predictions, your model will not survive the real world,” Shah says. “We make models look at both sentences in all agree-disagree pairs.”</p> MIT researchers have created an automated text-generating system that pinpoints and replaces specific information in relevant Wikipedia sentences, while keeping the language similar to how humans write and edit.Image: Christine Daniloff, MITResearch, Computer science and technology, Algorithms, Machine learning, Data, Internet, Crowdsourcing, Social media, Technology and society, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering Maintaining the equipment that powers our world By organizing performance data and predicting problems, Tagup helps energy companies keep their equipment running. Wed, 12 Feb 2020 09:39:37 -0500 Zach Winn | MIT News Office <p>Most people only think about the systems that power their cities when something goes wrong. Unfortunately, many people in the San Francisco Bay Area had a lot to think about recently when their utility company began scheduled power outages in an attempt to prevent wildfires. The decision came after devastating fires last year were found to be the result of faulty equipment, including transformers.</p> <p>Transformers are the links between power plants, power transmission lines, and distribution networks. If something goes wrong with a transformer, entire power plants can go dark. To fix the problem, operators work around the clock to assess various components of the plant, consider disparate data sources, and decide what needs to be repaired or replaced.</p> <p>Power equipment maintenance and failure is such a far-reaching problem it’s difficult to attach a dollar sign to. Beyond the lost revenue of the plant, there are businesses that can’t operate, people stuck in elevators and subways, and schools that can’t open.</p> <p>Now the startup Tagup is working to modernize the maintenance of transformers and other industrial equipment. The company’s platform lets operators view all of their data streams in one place and use machine learning to estimate if and when components will fail.</p> <p>Founded by CEO Jon Garrity ’11 and CTO Will Vega-Brown ’11, SM ’13 —&nbsp;who recently completed his PhD program in MIT’s Department of Mechanical Engineering and will be graduating this month — Tagup is currently being used by energy companies to monitor approximately 60,000 pieces of equipment around North America and Europe. That includes transformers, offshore wind turbines, and reverse osmosis systems for water filtration, among other things.</p> <p>“Our mission is to use AI to make the machines that power the world safer, more reliable, and more efficient,” Garrity says.</p> <p><strong>A light bulb goes on</strong></p> <p>Vega-Brown and Garrity crossed paths in a number of ways at MIT over the years. As undergraduates, they took a few of the same courses, with Vega-Brown double majoring in mechanical engineering and physics and Garrity double majoring in economics and physics. They were also fraternity brothers as well as teammates on the football team.</p> <p>Garrity was&nbsp;first exposed&nbsp;to entrepreneurship as an undergraduate in MIT’s Energy Ventures class and in the Martin Trust Center for Entrepreneurship.&nbsp;Later, when Garrity returned to campus while attending Harvard Business School and Vega-Brown was pursuing his doctorate, they were again classmates in MIT’s New Enterprises course.</p> <p>Still, the founders didn’t think about starting a company until 2015, after Garrity had worked at GE Energy and Vega-Brown was well into his PhD work at MIT’s Computer Science and Artificial Intelligence Laboratory.</p> <p>At GE, Garrity discovered an intriguing business model through which critical assets like jet engines were leased by customers — in this case airlines — rather than purchased, and manufacturers held responsibility for remotely monitoring and maintaining them. The arrangement allowed GE and others to leverage their engineering expertise while the customers focused on their own industries.</p> <p>"When I worked at GE, I always wondered: Why isn’t this service available for any equipment type? The answer is economics.” Garrity says. “It is expensive to set up a remote monitoring center, to instrument the equipment in the field, to staff the 50 or more engineering subject matter experts, and to provide the support required to end customers. The cost of equipment failure, both in terms of business interruption and equipment breakdown, must be enormous to justify the high average fixed cost."</p> <p>“We realized two things,” Garrity continues. “With the increasing availability of sensors and cloud infrastructure, we can dramatically reduce the cost [of monitoring critical assets] from the infrastructure and communications side. And, with new machine-learning methods, we can increase the productivity of engineers who review equipment data manually.”</p> <p>That realization led to Tagup, though it would take time to prove the founders’ technology. “The problem with using AI for industrial applications is the lack of high-quality data,” Vega-Brown explains. “Many of our customers have giant datasets, but the information density in industrial data is often quite low. That means we need to be very careful in how we hunt for signal and validate our models, so that we can reliably make accurate forecasts and predictions.”</p> <p>The founders leveraged their MIT ties to get the company off the ground. They received guidance from MIT’s Venture Mentoring Service, and Tagup was in the first cohort of startups accepted into the MIT Industrial Liaison Program’s (ILP) STEX 25 accelerator, which connects high potential startups with members of industry. Tagup has since secured several customers through ILP, and those early partnerships helped the company train and validate some of its machine-learning models.</p> <p><strong>Making power more reliable</strong></p> <p>Tagup’s platform combines all of a customer’s equipment data into one sortable master list that displays the likelihood of each asset causing a disruption. Users can click on specific assets to see charts of historic data and trends that feed into Tagup’s models.</p> <p>The company doesn’t deploy any sensors of its own. Instead, it combines customers’ real-time sensor measurements with other data sources like maintenance records and machine parameters to improve its proprietary machine-learning models.</p> <p>The founders also began with a focused approach to building their system. Transformers were one of the first types of equipment they worked with, and they’ve expanded to other groups of assets gradually.</p> <p>Tagup’s first deployment was in August of 2016 with a power plant that faces the Charles River close to MIT’s campus. Just a few months after it was installed, Garrity was at a meeting overseas when he got a call from the plant manager about a transformer that had just gone offline unexpectedly. From his phone, Garrity was able to inspect real-time data from the transformer&nbsp;and give the manager the information he needed to restart the system. Garrity says it saved the plant about 26 hours of downtime and $150,000 in revenue.</p> <p>“These are really catastrophic events in terms of business outcomes,” Garrity says, noting transformer failures are estimated to cost $23 billion annually.</p> <p>Since then they’ve secured partnerships with several large utility companies, including National Grid and Consolidated Edison Company of New York.</p> <p>Down the line, Garrity and Vega-Brown are excited about using machine learning to control the operation of equipment. For example, a machine could manage itself in the same way an autonomous car can sense an obstacle and steer around it.</p> <p>Those capabilities have major implications for the systems that ensure the lights go on when we flip switches at night.</p> <p>“Where it gets really exciting is moving toward optimization,” Garrity says. Vega-Brown agrees, adding, “Enormous amounts of power and water are wasted because there aren't enough experts to tune the controllers on every industrial machine in the world. If we can use AI to capture some of the expert knowledge in an algorithm, we can cut inefficiency and improve safety at scale.”</p> Tagup's industrial equipment monitoring platform is currently being used by energy companies to monitor approximately 60,000 pieces of equipment around North America and Europe. That includes transformers, offshore wind turbines, and reverse osmosis systems for water filtration.Innovation and Entrepreneurship (I&E), Startups, Alumni/ae, Computer Science and Artificial Intelligence Laboratory (CSAIL), Mechanical engineering, School of Engineering, Machine learning, Energy, Artificial intelligence Bridging the gap between human and machine vision Researchers develop a more robust machine-vision architecture by studying how human vision responds to changing viewpoints of objects. Tue, 11 Feb 2020 16:40:01 -0500 Kris Brewer | Center for Brains, Minds and Machines <p>Suppose you look briefly from a few feet away at a person you have never met before. Step back a few paces and look again. Will you be able to recognize her face? “Yes, of course,” you probably are thinking. If this is true, it would mean that our visual system, having seen a single image of an object such as a specific face, recognizes it robustly despite changes to the object’s position and scale, for example. On the other hand, we know that state-of-the-art classifiers, such as vanilla deep networks, will fail this simple test.</p> <p>In order to recognize a specific face under a range of transformations, neural networks need to be trained with many examples of the face under the different conditions. In other words, they can achieve invariance through memorization, but cannot do it if only one image is available. Thus, understanding how human vision can pull off this remarkable feat is relevant for engineers aiming to improve their existing classifiers. It also is important for neuroscientists modeling the primate visual system with deep networks. In particular, it is possible that the invariance with one-shot learning exhibited by biological vision requires a rather different computational strategy than that of deep networks.&nbsp;</p> <p>A new paper by MIT PhD candidate in electrical engineering and computer science Yena Han and colleagues in <em>Nature Scientific Reports</em> entitled “Scale and translation-invariance for novel objects in human vision” discusses how they study this phenomenon more carefully to create novel biologically inspired networks.</p> <p>"Humans can learn from very few examples, unlike deep networks. This is a huge difference with vast implications for engineering of vision systems and for understanding how human vision really works," states co-author Tomaso Poggio — director of the Center for Brains, Minds and Machines (CBMM) and the Eugene McDermott Professor of Brain and Cognitive Sciences at MIT. "A key reason for this difference is the relative invariance of the primate visual system to scale, shift, and other transformations. Strangely, this has been mostly neglected in the AI community, in part because the psychophysical data were so far less than clear-cut. Han's work has now established solid measurements of basic invariances of human vision.”</p> <p>To differentiate invariance rising from intrinsic computation with that from experience and memorization, the new study measured the range of invariance in one-shot learning. A one-shot learning task was performed by presenting Korean letter stimuli to human subjects who were unfamiliar with the language. These letters were initially presented a single time under one specific condition and tested at different scales or positions than the original condition. The first experimental result is that — just as you guessed — humans showed significant scale-invariant recognition after only a single exposure to these novel objects. The second result is that the range of position-invariance is limited, depending on the size and placement of objects.</p> <p>Next, Han and her colleagues performed a comparable experiment in deep neural networks designed to reproduce this human performance. The results suggest that to explain invariant recognition of objects by humans, neural network models should explicitly incorporate built-in scale-invariance. In addition, limited position-invariance of human vision is better replicated in the network by having the model neurons’ receptive fields increase as they are further from the center of the visual field. This architecture is different from commonly used neural network models, where an image is processed under uniform resolution with the same shared filters.</p> <p>“Our work provides a new understanding of the brain representation of objects under different viewpoints. It also has implications for AI, as the results provide new insights into what is a good architectural design for deep neural networks,” remarks Han, CBMM researcher and lead author of the study.</p> <p>Han and Poggio were joined by Gemma Roig and Gad Geiger in the work.</p> Yena Han (left) and Tomaso Poggio stand with an example of the visual stimuli used in a new psychophysics study.Photo: Kris BrewerCenter for Brains Minds and Machines, Brain and cognitive sciences, Machine learning, Artificial intelligence, Computer vision, Research, School of Science, Computer science and technology, Electrical Engineering & Computer Science (eecs), School of Engineering Brainstorming energy-saving hacks on Satori, MIT’s new supercomputer Three-day hackathon explores methods for making artificial intelligence faster and more sustainable. Tue, 11 Feb 2020 11:50:01 -0500 Kim Martineau | MIT Quest for Intelligence <p>Mohammad Haft-Javaherian planned to spend an hour at the&nbsp;<a href="">Green AI Hackathon</a>&nbsp;— just long enough to get acquainted with MIT’s new supercomputer,&nbsp;<a href="">Satori</a>. Three days later, he walked away with $1,000 for his winning strategy to shrink the carbon footprint of artificial intelligence models trained to detect heart disease.&nbsp;</p> <p>“I never thought about the kilowatt-hours I was using,” he says. “But this hackathon gave me a chance to look at my carbon footprint and find ways to trade a small amount of model accuracy for big energy savings.”&nbsp;</p> <p>Haft-Javaherian was among six teams to earn prizes at a hackathon co-sponsored by the&nbsp;<a href="">MIT Research Computing Project</a>&nbsp;and&nbsp;<a href="">MIT-IBM Watson AI Lab</a> Jan. 28-30. The event was meant to familiarize students with Satori, the computing cluster IBM&nbsp;<a href="">donated</a> to MIT last year, and to inspire new techniques for building energy-efficient AI models that put less planet-warming carbon dioxide into the air.&nbsp;</p> <p>The event was also a celebration of Satori’s green-computing credentials. With an architecture designed to minimize the transfer of data, among other energy-saving features, Satori recently earned&nbsp;<a href="">fourth place</a>&nbsp;on the Green500 list of supercomputers. Its location gives it additional credibility: It sits on a remediated brownfield site in Holyoke, Massachusetts, now the&nbsp;<a href="">Massachusetts Green High Performance Computing Center</a>, which runs largely on low-carbon hydro, wind and nuclear power.</p> <p>A postdoc at MIT and Harvard Medical School, Haft-Javaherian came to the hackathon to learn more about Satori. He stayed for the challenge of trying to cut the energy intensity of his own work, focused on developing AI methods to screen the coronary arteries for disease. A new imaging method, optical coherence tomography, has given cardiologists a new tool for visualizing defects in the artery walls that can slow the flow of oxygenated blood to the heart. But even the experts can miss subtle patterns that computers excel at detecting.</p> <p>At the hackathon, Haft-Javaherian ran a test on his model and saw that he could cut its energy use eight-fold by reducing the time Satori’s graphics processors sat idle. He also experimented with adjusting the model’s number of layers and features, trading varying degrees of accuracy for lower energy use.&nbsp;</p> <p>A second team, Alex Andonian and Camilo Fosco, also won $1,000 by showing they could train a classification model nearly 10 times faster by optimizing their code and losing a small bit of accuracy. Graduate students in the Department of Electrical Engineering and Computer Science (EECS), Andonian and Fosco are currently training a classifier to tell legitimate videos from AI-manipulated fakes, to compete in Facebook’s&nbsp;<a href="">Deepfake Detection Challenge</a>. Facebook launched the contest last fall to crowdsource ideas for stopping the spread of misinformation on its platform ahead of the 2020 presidential election.</p> <p>If a technical solution to deepfakes is found, it will need to run on millions of machines at once, says Andonian. That makes energy efficiency key. “Every optimization we can find to train and run more efficient models will make a huge difference,” he says.</p> <p>To speed up the training process, they tried streamlining their code and lowering the resolution of their 100,000-video training set by eliminating some frames. They didn’t expect a solution in three days, but Satori’s size worked in their favor. “We were able to run 10 to 20 experiments at a time, which let us iterate on potential ideas and get results quickly,” says Andonian.&nbsp;</p> <p>As AI continues to improve at tasks like reading medical scans and interpreting video, models have grown bigger and more calculation-intensive, and thus, energy intensive. By one&nbsp;<a href="">estimate</a>, training a large language-processing model produces nearly as much carbon dioxide as the cradle-to-grave emissions from five American cars. The footprint of the typical model is modest by comparison, but as AI applications proliferate its environmental impact is growing.&nbsp;</p> <p>One way to green AI, and tame the exponential growth in demand for training AI, is to build smaller models. That’s the approach that a third hackathon competitor, EECS graduate student Jonathan Frankle, took. Frankle is looking for signals early in the training process that point to subnetworks within the larger, fully-trained network that can do the same job.&nbsp;The idea builds on his award-winning&nbsp;<a href="">Lottery Ticket Hypothesis</a>&nbsp;paper from last year that found a neural network could perform with 90 percent fewer connections if the right subnetwork was found early in training.</p> <p>The hackathon competitors were judged by John Cohn, chief scientist at the MIT-IBM Watson AI Lab, Christopher Hill, director of MIT’s Research Computing Project, and Lauren Milechin, a research software engineer at MIT.&nbsp;</p> <p>The judges recognized four&nbsp;other teams: Department of Earth, Atmospheric and Planetary Sciences (EAPS) graduate students Ali Ramadhan,&nbsp;Suyash Bire, and James Schloss,&nbsp;for adapting the programming language Julia for Satori; MIT Lincoln Laboratory postdoc Andrew Kirby, for adapting code he wrote as a graduate student to Satori using a library designed for easy programming of computing architectures; and Department of Brain and Cognitive Sciences graduate students Jenelle Feather and Kelsey Allen, for applying a technique that drastically simplifies models by cutting their number of parameters.</p> <p>IBM developers were on hand to answer questions and gather feedback.&nbsp;&nbsp;“We pushed the system — in a good way,” says Cohn. “In the end, we improved the machine, the documentation, and the tools around it.”&nbsp;</p> <p>Going forward, Satori will be joined in Holyoke by&nbsp;<a href="">TX-Gaia</a>, Lincoln Laboratory’s new supercomputer.&nbsp;Together, they will provide feedback on the energy use of their workloads. “We want to raise awareness and encourage users to find innovative ways to green-up all of their computing,” says Hill.&nbsp;</p> Several dozen students participated in the Green AI Hackathon, co-sponsored by the MIT Research Computing Project and MIT-IBM Watson AI Lab. Photo panel: Samantha SmileyQuest for Intelligence, MIT-IBM Watson AI Lab, Electrical engineering and computer science (EECS), EAPS, Lincoln Laboratory, Brain and cognitive sciences, School of Engineering, School of Science, Algorithms, Artificial intelligence, Computer science and technology, Data, Machine learning, Software, Climate change, Awards, honors and fellowships, Hackathon, Special events and guest speakers Hey Alexa! Sorry I fooled you ... MIT’s new system TextFooler can trick the types of natural-language-processing systems that Google uses to help power its search results, including audio for Google Home. Fri, 07 Feb 2020 11:20:01 -0500 Rachel Gordon | MIT CSAIL <p>A human can likely tell the difference between a turtle and a rifle. Two years ago, Google’s AI wasn’t so <a href="">sure</a>. For quite some time, a subset of computer science research has been dedicated to better understanding how machine-learning models handle these “adversarial” attacks, which are inputs deliberately created to trick or fool machine-learning algorithms.&nbsp;</p> <p>While much of this work has focused on <a href="">speech</a> and <a href="">images</a>, recently, a team from MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) tested the boundaries of text. They came up with “TextFooler,” a general framework that can successfully attack natural language processing (NLP) systems — the types of systems that let us interact with our Siri and Alexa voice assistants — and “fool” them into making the wrong predictions.&nbsp;</p> <p>One could imagine using TextFooler for many applications related to internet safety, such as email spam filtering, hate speech flagging, or “sensitive” political speech text detection — which are all based on text classification models.&nbsp;</p> <p>“If those tools are vulnerable to purposeful adversarial attacking, then the consequences may be disastrous,” says Di Jin, MIT PhD student and lead author on a new paper about TextFooler. “These tools need to have effective defense approaches to protect themselves, and in order to make such a safe defense system, we need to first examine the adversarial methods.”&nbsp;</p> <p>TextFooler works in two parts: altering a given text, and then using that text to test two different language tasks to see if the system can successfully trick machine-learning models.&nbsp;&nbsp;</p> <p>The system first identifies the most important words that will influence the target model’s prediction, and then selects the synonyms that fit contextually. This is all while maintaining grammar and the original meaning to look “human” enough, until the prediction is altered.&nbsp;</p> <p>Then, the framework is applied to two different tasks — text classification, and entailment (which is the relationship between text fragments in a sentence), with the goal of changing the classification or invalidating the entailment judgment of the original models.&nbsp;</p> <p>In one example, TextFooler’s input and output were:</p> <p>“The characters, cast in impossibly contrived situations, are totally estranged from reality.”&nbsp;</p> <p>“The characters, cast in impossibly engineered circumstances, are fully estranged from reality.”&nbsp;</p> <p>In this case, when testing on an NLP model, it gets the example input right, but then gets the modified input wrong.&nbsp;</p> <p>In total, TextFooler successfully attacked three target models, including “BERT,” the popular open-source NLP model. It fooled the target models with an accuracy of over 90 percent to under 20 percent, by changing only 10 percent of the words in a given text. The team evaluated success on three criteria: changing the model's prediction for classification or entailment; whether it looked similar in meaning to a human reader, compared with the original example; and whether the text looked natural enough.&nbsp;</p> <p>The researchers note that while attacking existing models is not the end goal, they hope that this work will help more abstract models generalize to new, unseen data.&nbsp;</p> <p>“The system can be used or extended to attack any classification-based NLP models to test their robustness,” says Jin. “On the other hand, the generated adversaries can be used to improve the robustness and generalization of deep-learning models via adversarial training, which is a critical direction of this work.”&nbsp;</p> <p>Jin wrote the paper alongside MIT Professor Peter Szolovits, Zhijing Jin of the University of Hong Kong, and Joey Tianyi Zhou of A*STAR, Singapore. They will present the paper at the AAAI Conference on Artificial Intelligence in New York.&nbsp;</p> CSAIL PhD student Di Jin led the development of the TextFooler system.Photo: Jason Dorfman/MIT CSAILComputer Science and Artificial Intelligence Laboratory (CSAIL), Computer science and technology, Machine learning, Algorithms, Data, Natural language processing, Artificial intelligence, Electrical Engineering & Computer Science (eecs), School of Engineering, Technology and society Researchers develop a roadmap for growth of new solar cells Starting with higher-value niche markets and then expanding could help perovskite-based solar panels become competitive with silicon. Thu, 06 Feb 2020 10:57:11 -0500 David L. Chandler | MIT News Office <p>Materials called perovskites show strong potential for a new generation of solar cells, but they’ve had trouble gaining traction in a market dominated by silicon-based solar cells. Now, a study by researchers at MIT and elsewhere outlines a roadmap for how this promising technology could move from the laboratory to a significant place in the global solar market.</p> <p>The “technoeconomic” analysis shows that by starting with higher-value niche markets and gradually expanding, solar panel manufacturers could avoid the very steep initial capital costs that would be required to make perovskite-based panels directly competitive with silicon for large utility-scale installations at the outset. Rather than making a prohibitively expensive initial investment, of hundreds of millions or even billions of dollars, to build a plant for utility-scale production, the team found that starting with more specialized applications could be accomplished for more realistic initial capital investment on the order of $40 million.</p> <p>The results are described in a paper in the journal <em>Joule</em> by MIT postdoc Ian Mathews, research scientist Marius Peters, professor of mechanical engineering Tonio Buonassisi, and five others at MIT, Wellesley College, and Swift Solar Inc.</p> <p>Solar cells based on perovskites — a broad category of compounds characterized by a certain arrangement of their molecular structure — could provide dramatic improvements in solar installations. Their constituent materials are inexpensive, and they could be manufactured in a roll-to-roll process like printing a newspaper, and printed onto lightweight and flexible backing material. This could greatly reduce costs associated with transportation and installation, although they still require further work to improve their durability. Other promising new solar cell materials are also under development in labs around the world, but none has yet made inroads in the marketplace.</p> <p>“There have been a lot of new solar cell materials and companies launched over the years,” says Mathews, “and yet, despite that, silicon remains the dominant material in the industry and has been for decades.”</p> <p>Why is that the case? “People have always said that one of the things that holds new technologies back is that the expense of constructing large factories to actually produce these systems at scale is just too much,” he says. “It’s difficult for a startup to cross what’s called ‘the valley of death,’ to raise the tens of millions of dollars required to get to the scale where this technology might be profitable in the wider solar energy industry.”</p> <p>But there are a variety of more specialized solar cell applications where the special qualities of perovskite-based solar cells, such as their light weight, flexibility, and potential for transparency, would provide a significant advantage, Mathews says. By focusing on these markets initially, a startup solar company could build up to scale gradually, leveraging the profits from the premium products to expand its production capabilities over time.</p> <p>Describing the literature on perovskite-based solar cells being developed in various labs, he says, “They’re claiming very low costs. But they’re claiming it once your factory reaches a certain scale. And I thought, we’ve seen this before — people claim a new photovoltaic material is going to be cheaper than all the rest and better than all the rest. That’s great, except we need to have a plan as to how we actually get the material and the technology to scale.”</p> <p>As a starting point, he says, “We took the approach that I haven’t really seen anyone else take: Let’s actually model the cost to manufacture these modules as a function of scale. So if you just have 10 people in a small factory, how much do you need to sell your solar panels at in order to be profitable? And once you reach scale, how cheap will your product become?”</p> <p>The analysis confirmed that trying to leap directly into the marketplace for rooftop solar or utility-scale solar installations would require very large upfront capital investment, he says. But “we looked at the prices people might get in the internet of things, or the market in building-integrated photovoltaics. People usually pay a higher price in these markets because they’re more of a specialized product. They’ll pay a little more if your product is flexible or if the module fits into a building envelope.” Other potential niche markets include self-powered microelectronics devices.</p> <p>Such applications would make the entry into the market feasible without needing massive capital investments. “If you do that, the amount you need to invest in your company is much, much less, on the order of a few million dollars instead of tens or hundreds of millions of dollars, and that allows you to more quickly develop a profitable company,” he says.</p> <p>“It’s a way for them to prove their technology, both technically and by actually building and selling a product and making sure it survives in the field,” Mathews says, “and also, just to prove that you can manufacture at a certain price point.”</p> <p>Already, there are a handful of startup companies working to try to bring perovskite solar cells to market, he points out, although none of them yet has an actual product for sale. The companies have taken different approaches, and some seem to be embarking on the kind of step-by-step growth approach outlined by this research, he says. “Probably the company that’s raised the most money is a company called Oxford PV, and they’re looking at tandem cells,” which incorporate both silicon and perovskite cells to improve overall efficiency. Another company is one started by Joel Jean PhD ’17 (who is also a co-author of this paper) and others, called Swift Solar, which is working on flexible perovskites. And there’s a company called Saule Technologies, working on printable perovskites.</p> <p>Mathews says the kind of technoeconomic analysis the team used in its study could be applied to a wide variety of other new energy-related technologies, including rechargeable batteries and other storage systems, or other types of new solar cell materials.</p> <p>“There are many scientific papers and academic studies that look at how much it will cost to manufacture a technology once it’s at scale,” he says. “But very few people actually look at how much does it cost at very small scale, and what are the factors affecting economies of scale? And I think that can be done for many technologies, and it would help us accelerate how we get innovations from lab to market.”</p> <p>The research team also included MIT alumni Sarah Sofia PhD ’19 and Sin Cheng Siah PhD ’15, Wellesley College student Erica Ma, and former MIT postdoc Hannu Laine. The work was supported by the European Union’s Horizon 2020 research and innovation program, the Martin Family Society for Fellows of Sustainability, the U.S. Department of Energy, Shell, through the MIT Energy Initiative, and the Singapore-MIT Alliance for Research and Technology.</p> Perovskites, a family of materials defined by a particular kind of molecular structure as illustrated here, have great potential for new kinds of solar cells. A new study from MIT shows how these materials could gain a foothold in the solar marketplace.Image: Christine Daniloff, MITResearch, School of Engineering, Energy, Solar, Nanoscience and nanotechnology, Materials Science and Engineering, Mechanical engineering, National Science Foundation (NSF), Renewable energy, Alternative energy, Sustainability, Artificial intelligence, Machine learning, MIT Energy Initiative, Singapore-MIT Alliance for Research and Technology (SMART) Technique reveals whether models of patient risk are accurate Computer scientists’ new method could help doctors avoid ineffective or unnecessarily risky treatments. Thu, 23 Jan 2020 05:00:00 -0500 Anne Trafton | MIT News Office <p>After a patient has a heart attack or stroke, doctors often use risk models to help guide their treatment. These models can calculate a patient’s risk of dying based on factors such as the patient’s age, symptoms, and other characteristics.</p> <p>While these models are useful in most cases, they do not make accurate predictions for many patients, which can lead doctors to choose ineffective or unnecessarily risky treatments for some patients.</p> <p>“Every risk model is evaluated on some dataset of patients, and even if it has high accuracy, it is never 100 percent accurate in practice,” says Collin Stultz, a professor of electrical engineering and computer science at MIT and a cardiologist at Massachusetts General Hospital. “There are going to be some patients for which the model will get the wrong answer, and that can be disastrous.”</p> <p>Stultz and his colleagues from MIT, IBM Research, and the University of Massachusetts Medical School have now developed a method that allows them to determine whether a particular model’s results can be trusted for a given patient. This could help guide doctors to choose better treatments for those patients, the researchers say.</p> <p>Stultz, who is also a professor of health sciences and technology, a member of MIT’s Institute for Medical Engineering and Sciences and Research Laboratory of Electronics, and an associate member of the Computer Science and Artificial Intelligence Laboratory, is the senior author of the <a href="" target="_blank">new study</a>. MIT graduate student Paul Myers is the lead author of the paper, which appears today in <em>Digital Medicine</em>.</p> <p><strong>Modeling risk</strong></p> <p>Computer models that can predict a patient’s risk of harmful events, including death, are used widely in medicine. These models are often created by training machine-learning algorithms to analyze patient datasets that include a variety of information about the patients, including their health outcomes.</p> <p>While these models have high overall accuracy, “very little thought has gone into identifying when a model is likely to fail,” Stultz says. “We are trying to create a shift in the way that people think about these machine-learning models. Thinking about when to apply a model is really important because the consequence of being wrong can be fatal.”</p> <p>For instance, a patient at high risk who is misclassified would not receive sufficiently aggressive treatment, while a low-risk patient inaccurately determined to be at high risk could receive unnecessary, potentially harmful interventions.</p> <p>To illustrate how the method works, the researchers chose to focus on a widely used risk model called the GRACE risk score, but the technique can be applied to nearly any type of risk model. GRACE, which stands for Global Registry of Acute Coronary Events, is a large dataset that was used to develop a risk model that evaluates a patient’s risk of death within six months after suffering an acute coronary syndrome (a condition caused by decreased blood flow to the heart). The resulting risk assessment is based on age, blood pressure, heart rate, and other readily available clinical features.</p> <p>The researchers’ new technique generates an “unreliability score” that ranges from 0 to 1. For a given risk-model prediction, the higher the score, the more unreliable that prediction. The unreliability score is based on a comparison of the risk prediction generated by a particular model, such as the GRACE risk-score, with the prediction produced by a different model that was trained on the same dataset. If the models produce different results, then it is likely that the risk-model prediction for that patient is not reliable, Stultz says.</p> <p>“What we show in this paper is, if you look at patients who have the highest unreliability scores — in the top 1 percent — the risk prediction for that patient yields the same information as flipping a coin,” Stultz says. “For those patients, the GRACE score cannot discriminate between those who die and those who don’t. It’s completely useless for those patients.”</p> <p>The researchers’ findings also suggested that the patients for whom the models don’t work well tend to be older and to have a higher incidence of cardiac risk factors.</p> <p>One significant advantage of the method is that the researchers derived a formula that tells how much two predictions would disagree, without having to build a completely new model based on the original dataset.&nbsp;</p> <p>“You don’t need access to the training dataset itself in order to compute this unreliability measurement, and that’s important because there are privacy issues that prevent these clinical datasets from being widely accessible to different people,” Stultz says.</p> <p><strong>Retraining the model</strong></p> <p>The researchers are now designing a user interface that doctors could use to evaluate whether a given patient’s GRACE score is reliable. In the longer term, they also hope to improve the reliability of risk models by making it easier to retrain models on data that include more patients who are similar to the patient being diagnosed.</p> <p>“If the model is simple enough, then retraining a model can be fast. You could imagine a whole suite of software integrated into the electronic health record that would automatically tell you whether a particular risk score is appropriate for a given patient, and then try to do things on the fly, like retrain new models that&nbsp;might be more appropriate,” Stultz says.</p> <p>The research was funded by the MIT-IBM Watson AI Lab. Other authors of the paper include MIT graduate student Wangzhi Dai; Kenney Ng, Kristen Severson, and Uri Kartoun of the Center for Computational Health at IBM Research; and Wei Huang and Frederick Anderson of the Center for Outcomes Research at the University of Massachusetts Medical School.</p> Researchers have developed a method that allows them to determine whether a particular risk model’s results can be trusted for a given patient.Research, Health care, Medicine, Machine learning, Artificial intelligence, Electrical Engineering & Computer Science (eecs), Health sciences and technology, Research Laboratory for Electronics, Computer Science and Artificial Intelligence Laboratory (CSAIL), Institute for Medical Engineering and Science (IMES), School of Engineering Using artificial intelligence to enrich digital maps Model tags road features based on satellite images, to improve GPS navigation in places with limited map data. Thu, 23 Jan 2020 00:00:00 -0500 Rob Matheson | MIT News Office <p>A model invented by researchers at MIT and Qatar Computing Research Institute (QCRI) that uses satellite imagery to tag road features in digital maps could help improve GPS navigation.&nbsp;&nbsp;</p> <p>Showing drivers more details about their routes can often help them navigate in unfamiliar locations. Lane counts, for instance, can enable a GPS system to warn drivers of diverging or merging lanes. Incorporating information about parking spots can help drivers plan ahead, while mapping bicycle lanes can help cyclists negotiate busy city streets. Providing updated information on road conditions can also improve planning for disaster relief.</p> <p>But creating detailed maps is an expensive, time-consuming process done mostly by big companies, such as Google, which send<s>s</s> vehicles around with cameras strapped to their hoods to capture video and images of an area’s roads. Combining that with other data can create accurate, up-to-date maps. Because this process is expensive, however, some parts of the world are ignored.</p> <p>A solution is to unleash machine-learning models on satellite images — which are easier to obtain and updated fairly regularly — to automatically tag road features. But roads can be occluded by, say, trees and buildings, making it a challenging task. In a <a href="">paper</a> being presented at the Association for the Advancement of Artificial Intelligence conference, the MIT and QCRI researchers describe “RoadTagger,” which uses a combination of neural network architectures to automatically predict the number of lanes and road types (residential or highway) behind obstructions.</p> <p>In testing RoadTagger on occluded roads from digital maps of 20 U.S. cities, the model counted lane numbers with 77 percent accuracy and inferred road types with 93 percent accuracy. The researchers are also planning to enable RoadTagger to predict other features, such as parking spots and bike lanes.</p> <p>“Most updated digital maps are from places that big companies care the most about. If you’re in places they don’t care about much, you’re at a disadvantage with respect to the quality of map,” says co-author Sam Madden, a professor in the Department of Electrical Engineering and Computer Science (EECS) and a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL). “Our goal is to automate the process of generating high-quality digital maps, so they can be available in any country.”</p> <p>The paper’s co-authors are CSAIL graduate students Songtao He, Favyen Bastani, and Edward Park; EECS undergraduate student Satvat Jagwani; CSAIL professors Mohammad Alizadeh and Hari Balakrishnan; and QCRI researchers Sanjay Chawla, Sofiane Abbar, and Mohammad Amin Sadeghi.</p> <p><strong>Combining CNN and GNN</strong></p> <p>Qatar, where QCRI is based, is “not a priority for the large companies building digital maps,” Madden says. Yet, it’s constantly building new roads and improving old ones, especially in preparation for hosting the 2022 FIFA World Cup.</p> <p>“While visiting Qatar, we’ve had experiences where our Uber driver can’t figure out how to get where he’s going, because the map is so off,” Madden says. “If navigation apps don’t have the right information, for things such as lane merging, this could be frustrating or worse.”</p> <p>RoadTagger relies on a novel combination of a convolutional neural network (CNN) — commonly used for images-processing tasks — and a graph neural network (GNN). GNNs model relationships between connected nodes in a graph and have become popular for analyzing things like social networks and molecular dynamics. The model is “end-to-end,” meaning it’s fed only raw data and automatically produces output, without human intervention.</p> <p>The CNN takes as input raw satellite images of target roads. The GNN breaks the road into roughly 20-meter segments, or “tiles.” Each tile is a separate graph node, connected by lines along the road. For each node, the CNN extracts road features and shares that information with its immediate neighbors. Road information propagates along the whole graph, with each node receiving some information about road attributes in every other node. If a certain tile is occluded in an image, RoadTagger uses information from all tiles along the road to predict what’s behind the occlusion.</p> <p>This combined architecture represents a more human-like intuition, the researchers say. Say part of a four-lane road is occluded by trees, so certain tiles show only two lanes. Humans can easily surmise that a couple lanes are hidden behind the trees. Traditional machine-learning models — say, just a CNN — extract features only of individual tiles and most likely predict the occluded tile is a two-lane road.</p> <p>“Humans can use information from adjacent tiles to guess the number of lanes in the occluded tiles, but networks can’t do that,” He says. “Our approach tries to mimic the natural behavior of humans, where we capture local information from the CNN and global information from the GNN to make better predictions.”</p> <p><strong>Learning weights&nbsp;&nbsp;&nbsp; </strong></p> <p>To train and test RoadTagger, the researchers used a real-world map dataset, called OpenStreetMap, which lets users edit and curate digital maps around the globe. From that dataset, they collected confirmed road attributes from 688 square kilometers of maps of 20 U.S. cities — including Boston, Chicago, Washington, and Seattle. Then, they gathered the corresponding satellite images from a Google Maps dataset.</p> <p>In training, RoadTagger learns weights — which assign varying degrees of importance to features and node connections —&nbsp;of the CNN and GNN. The CNN extracts features from pixel patterns of tiles and the GNN propagates the learned features along the graph. From randomly selected subgraphs of the road, the system learns to predict the road features at each tile. In doing so, it automatically learns which image features are useful and how to propagate those features along the graph. For instance, if a target tile has unclear lane markings, but its neighbor tile has four lanes with clear lane markings and shares the same road width, then the target tile is likely to also have four lanes. In this case, the model automatically learns that the road width is a useful image feature, so if two adjacent tiles share the same road width, they’re likely to have the same lane count.</p> <p>Given a road not seen in training from OpenStreetMap, the model breaks the road into tiles and uses its learned weights to make predictions. Tasked with predicting a number of lanes in an occluded tile, the model notes that neighboring tiles have matching pixel patterns and, therefore, a high likelihood to share information. So, if those tiles have four lanes, the occluded tile must also have four.</p> <p>In another result, RoadTagger accurately predicted lane numbers in a dataset of synthesized, highly challenging road disruptions. As one example, an overpass with two lanes covered a few tiles of a target road with four lanes. The model detected mismatched pixel patterns of the overpass, so it ignored the two lanes over the covered tiles, accurately predicting four lanes were underneath.</p> <p>The researchers hope to use RoadTagger to help humans rapidly validate and approve continuous modifications to infrastructure in datasets such as OpenStreetMap, where many maps don’t contain lane counts or other details. A specific area of interest is Thailand, Bastani says, where roads are constantly changing, but there are few if any updates in the dataset.</p> <p>“Roads that were once labeled as dirt roads have been paved over so are better to drive on, and some intersections have been completely built over. There are changes every year, but digital maps are out of date,” he says. “We want to constantly update such road attributes based on the most recent imagery.”</p> An AI model developed at MIT and Qatar Computing Research Institute that uses only satellite imagery to automatically tag road features in digital maps could improve GPS navigation, especially in countries with limited map data.Image: Google Maps/MIT NewsResearch, Computer science and technology, Algorithms, Transportation, Cities, Automobiles, Machine learning, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering, Artificial intelligence How to verify that quantum chips are computing correctly A new method determines whether circuits are accurately executing complex operations that classical computers can’t tackle. Mon, 13 Jan 2020 10:59:03 -0500 Rob Matheson | MIT News Office <p>In a step toward practical quantum computing, researchers from MIT, Google, and elsewhere have designed a system that can verify when quantum chips have accurately performed complex computations that classical computers can’t.</p> <p>Quantum chips perform computations using quantum bits, called “qubits,” that can represent the two states corresponding to classic binary bits — a 0 or 1 — or a “quantum superposition” of both states simultaneously. The unique superposition state can enable quantum computers to solve problems that are practically impossible for classical computers, potentially spurring breakthroughs in material design, drug discovery, and machine learning, among other applications.</p> <p>Full-scale quantum computers will require millions of qubits, which isn’t yet feasible. In the past few years, researchers have started developing “Noisy Intermediate Scale Quantum” (NISQ) chips, which contain around 50 to 100 qubits. That’s just enough to demonstrate “quantum advantage,” meaning the NISQ chip can solve certain algorithms that are intractable for classical computers. Verifying that the chips performed operations as expected, however, can be very inefficient. The chip’s outputs can look entirely random, so it takes a long time to simulate steps to determine if everything went according to plan.</p> <p>In a paper published today in <em>Nature Physics</em>, the researchers describe a novel protocol to efficiently verify that an NISQ chip has performed all the right quantum operations. They validated their protocol on a notoriously difficult quantum problem running on custom quantum photonic chip.</p> <p>“As rapid advances in industry and academia bring us to the cusp of quantum machines that can outperform classical machines, the task of quantum verification becomes time critical,” says first author Jacques Carolan, a postdoc in the Department of Electrical Engineering and Computer Science (EECS) and the Research Laboratory of Electronics (RLE). “Our technique provides an important tool for verifying a broad class of quantum systems. Because if I invest billions of dollars to build a quantum chip, it sure better do something interesting.”</p> <p>Joining Carolan on the paper are researchers from EECS and RLE at MIT, as well from the Google Quantum AI Laboratory, Elenion Technologies, Lightmatter, and Zapata Computing. &nbsp;</p> <p><strong>Divide and conquer</strong></p> <p>The researchers’ work essentially traces an output quantum state generated by the quantum circuit back to a known input state. Doing so reveals which circuit operations were performed on the input to produce the output. Those operations should always match what researchers programmed. If not, the researchers can use the information to pinpoint where things went wrong on the chip.</p> <p>At the core of the new protocol, called “Variational Quantum Unsampling,” lies a “divide and conquer” approach, Carolan says, that breaks the output quantum state into chunks. “Instead of doing the whole thing in one shot, which takes a very long time, we do this unscrambling layer by layer. This allows us to break the problem up to tackle it in a more efficient way,” Carolan says.</p> <p>For this, the researchers took inspiration from neural networks — which solve problems through many layers of computation —&nbsp;to build a novel “quantum neural network” (QNN), where each layer represents a set of quantum operations.</p> <p>To run the QNN, they used traditional silicon fabrication techniques to build a 2-by-5-millimeter NISQ chip with more than 170 control parameters — tunable circuit components that make manipulating the photon path easier. Pairs of photons are generated at specific wavelengths from an external component and injected into the chip. The photons travel through the chip’s phase shifters — which change the path of the photons — interfering with each other. This produces a random quantum output state —&nbsp;which represents what would happen during computation. The output is measured by an array of external photodetector sensors.</p> <p>That output is sent to the QNN. The first layer uses complex optimization techniques to dig through the noisy output to pinpoint the signature of a single photon among all those scrambled together. Then, it “unscrambles” that single photon from the group to identify what circuit operations return it to its known input state. Those operations should match exactly the circuit’s specific design for the task. All subsequent layers do the same computation — removing from the equation any previously unscrambled photons — until all photons are unscrambled.</p> <p>As an example, say the input state of qubits fed into the processor was all zeroes. The NISQ chip executes a bunch of operations on the qubits to generate a massive, seemingly randomly changing number as output. (An output number will constantly be changing as it’s in a quantum superposition.) The QNN selects chunks of that massive number. Then, layer by layer, it determines which operations revert each qubit back down to its input state of zero. If any operations are different from the original planned operations, then something has gone awry. Researchers can inspect any mismatches between the expected output to input states, and use that information to tweak the circuit design.</p> <p><strong>Boson “unsampling”</strong></p> <p>In experiments, the team successfully ran a popular computational task used to demonstrate quantum advantage, called “boson sampling,” which is usually performed on photonic chips. In this exercise, phase shifters and other optical components will manipulate and convert a set of input photons into a different quantum superposition of output photons. Ultimately, the task is to calculate the probability that a certain input state will match a certain output state. That will essentially be a sample from some probability distribution.</p> <p>But it’s nearly impossible for classical computers to compute those samples, due to the unpredictable behavior of photons. It’s been theorized that NISQ chips can compute them fairly quickly. Until now, however, there’s been no way to verify that quickly and easily, because of the complexity involved with the NISQ operations and the task itself.</p> <p>“The very same properties which give these chips quantum computational power makes them nearly impossible to verify,” Carolan says.</p> <p>In experiments, the researchers were able to “unsample” two photons that had run through the boson sampling problem on their custom NISQ chip — and in a fraction of time it would take traditional verification approaches.</p> <p>“This is an excellent paper that employs a nonlinear quantum neural network to learn the unknown unitary operation performed by a black box,” says Stefano Pirandola, a professor of computer science who specializes in quantum technologies at the University of York. “It is clear that this scheme could be very useful to verify the actual gates that are performed by a quantum circuit — [for example] by a NISQ processor. From this point of view, the scheme serves as an important benchmarking tool for future quantum engineers. The idea was remarkably implemented on a photonic quantum chip.”</p> <p>While the method was designed for quantum verification purposes, it could also help capture useful physical properties, Carolan says. For instance, certain molecules when excited will vibrate, then emit photons based on these vibrations. By injecting these photons into a photonic chip, Carolan says, the unscrambling technique could be used to discover information about the quantum dynamics of those molecules to aid in bioengineering molecular design. It could also be used to unscramble photons carrying quantum information that have accumulated noise by passing through turbulent spaces or materials. &nbsp;</p> <p>“The dream is to apply this to interesting problems in the physical world,” Carolan says.</p> Researchers from MIT, Google, and elsewhere have designed a novel method for verifying when quantum processors have accurately performed complex computations that classical computers can’t. They validate their method on a custom system (pictured) that’s able to capture how accurately a photonic chip (“PNP”) computed a notoriously difficult quantum problem.Image: Mihika PrabhuResearch, Computer science and technology, Algorithms, Light, Photonics, Nanoscience and nanotechnology, Quantum computing, electronics, Machine learning, Artificial intelligence, Research Laboratory of Electronics, Electrical Engineering & Computer Science (eecs), School of Engineering How well can computers connect symptoms to diseases? Models that map these relationships based on patient data require fine-tuning for certain conditions, study shows. Thu, 09 Jan 2020 17:10:38 -0500 Rob Matheson | MIT News Office <p>A new MIT study finds “health knowledge graphs,” which show relationships between symptoms and diseases and are intended to help with clinical diagnosis, can fall short for certain conditions and patient populations. The results also suggest ways to boost their performance.</p> <p>Health knowledge graphs have typically been compiled manually by expert clinicians, but that can be a laborious process. Recently, researchers have experimented with automatically generating these knowledge graphs from patient data. The MIT team has been studying how well such graphs hold up across different diseases and patient populations.</p> <p>In a paper presented at the Pacific Symposium on Biocomputing 2020, the researchers evaluated automatically generated health knowledge graphs based on real datasets comprising more than 270,000 patients with nearly 200 diseases and more than 770 symptoms.</p> <p>The team analyzed how various models used electronic health record (EHR) data, containing medical and treatment histories of patients, to automatically “learn” patterns of disease-symptom correlations. They found that the models performed particularly poorly for diseases that have high percentages of very old or young patients, or high percentages of male or female patients — but that choosing the right data for the right model, and making other modifications, can improve performance.</p> <p>The idea is to provide guidance to researchers about the relationship between dataset size, model specification, and performance when using electronic health records to build health knowledge graphs. That could lead to better tools to aid physicians and patients with medical decision-making or to search for new relationships between diseases and symptoms.</p> <p>“In the last 10 years, EHR use has skyrocketed in hospitals, so there’s an enormous amount of data that we hope to mine to learn these graphs of disease-symptom relationships,” says first author Irene Y. Chen, a graduate student in the Department of Electrical Engineering and Computer Science (EECS). “It is essential that we closely examine these graphs, so that they can be used as the first steps of a diagnostic tool.”</p> <p>Joining Chen on the paper are Monica Agrawal, a graduate student in MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL); Steven Horng of Beth Israel Deaconess Medical Center (BIDMC); and EECS Professor David Sontag, who is a member of CSAIL and the Institute for Medical Engineering and Science, and head of the Clinical Machine Learning Group.</p> <p><strong>Patients and diseases</strong></p> <p>In health knowledge graphs, there are hundreds of nodes, each representing a different disease and symptom. Edges (lines) connect disease nodes, such as “diabetes,” with correlated symptom nodes, such as “excessive thirst.” Google famously launched its own version in 2015, which was manually curated by several clinicians over hundreds of hours and is considered the gold standard. When you Google a disease now, the system displays associated symptoms.</p> <p>In a 2017 <em>Nature Scientific Reports</em> paper, Sontag, Horng, and other researchers leveraged data from the same 270,00 patients in their current study — which came from the emergency department at BIDMC between 2008 and 2013 — to build health knowledge graphs. They used three model structures to generate the graphs, called logistic regression, naive Bayes, and noisy OR. Using data provided by Google, the researchers compared their automatically generated health knowledge graph with the Google Health Knowledge Graph (GHKG). The researchers’ graph performed very well.</p> <p>In their new work, the researchers did a rigorous error analysis to determine which specific patients and diseases the models performed poorly for. Additionally, they experimented with augmenting the models with more data, from beyond the emergency room.</p> <p>In one test, they broke the data down into subpopulations of diseases and symptoms. For each model, they looked at connecting lines between diseases and all possible symptoms, and compared that with the GHKG. In the paper, they sort the findings into the 50 bottom- and 50 top-performing diseases. Examples of low performers are polycystic ovary syndrome (which affects women), allergic asthma (very rare), and prostate cancer (which predominantly affects older men). High performers are the more&nbsp;common diseases and conditions, such as heart arrhythmia and plantar fasciitis, which is tissue swelling along the feet.</p> <p>They found the noisy OR model was the most robust against error overall for nearly all of the diseases and patients. But accuracy decreased among all models for patients that have many co-occurring diseases and co-occurring symptoms, as well as patients that are very young or above the age of 85. Performance also suffered for patient populations with very high or low percentages of any sex.</p> <p><a name="_gjdgxs"></a>Essentially, the researchers hypothesize, poor performance is caused by patients and diseases that have outlier predictive performance, as well as potential unmeasured confounders. Elderly patients, for instance, tend to enter hospitals with more diseases and related symptoms than younger patients. That means it’s difficult for the models to correlate specific diseases with specific symptoms, Chen says. “Similarly,” she adds, “young patients don’t have many diseases or as many symptoms, and if they have a rare disease or symptom, it doesn’t present in a normal way the models understand.”</p> <p><strong>Splitting data</strong></p> <p>The researchers also collected much more patient data and created three distinct datasets of different granularity to see if that could improve performance. For the 270,000 visits used in the original analysis, the researchers extracted the full EHR history of the 140,804 unique patients, tracking back a decade, with around 7.4 million annotations total from various sources, such as physician notes.</p> <p>Choices in the dataset-creation process impacted the model performance as well. One of the datasets aggregates each of the 140,400 patient histories as one data point each. Another dataset treats each of the 7.4 million annotations as a separate data point. A final one creates “episodes” for each patient, defined as a continuous series of visits without a break of more than 30 days, yielding a total of around 1.4 million episodes.</p> <p>Intuitively, a dataset where the full patient history is aggregated into one data point should lead to greater accuracy since the entire patient history is considered. Counterintuitively, however, it also caused the naive Bayes model&nbsp;to perform more poorly for some diseases. “You assume the more intrapatient information, the better, with machine-learning models. But these models are dependent on the granularity of the data you feed them,” Chen says. “The type of model you use could get overwhelmed.”</p> <p>As expected, feeding the model demographic information can also be effective. For instance, models can use that information to exclude all male patients for, say, predicting cervical cancer. And certain diseases far more common for elderly patients can be eliminated in younger patients.</p> <p>But, in another surprise, the demographic information didn’t boost performance for the most successful model, so collecting that data may be unnecessary. That’s important, Chen says, because compiling data and training models on the data can be expensive and time-consuming. Yet, depending on the model, using scores of data may not actually improve performance.</p> <p>Next, the researchers hope to use their findings to build a robust model to deploy in clinical settings. Currently, the health knowledge graph learns relations between diseases and symptoms but does not give a direct prediction of disease from symptoms. “We hope that any predictive model and any medical knowledge graph would be put under a stress test so that clinicians and machine-learning researchers can confidently say, ‘We trust this as a useful diagnostic tool,’” Chen says.</p> A variety of new diagnostic models can analyze patient data and real-time symptoms to predict if a given patient has a particular disease. Research, Computer science and technology, Algorithms, Machine learning, Health, Health care, Disease, Medicine, Institute for Medical Engineering and Science (IMES), Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering Tool predicts how fast code will run on a chip Machine-learning system should enable developers to improve computing efficiency in a range of applications. Mon, 06 Jan 2020 00:00:00 -0500 Rob Matheson | MIT News Office <p>MIT researchers have invented a machine-learning tool that predicts how fast computer chips will execute code from various applications.&nbsp;&nbsp;</p> <p>To get code to run as fast as possible, developers and compilers — programs that translate programming language into machine-readable code — typically use performance models that run the code through a simulation of given chip architectures.&nbsp;</p> <p>Compilers use that information to automatically optimize code, and developers use it to tackle performance bottlenecks on the microprocessors that will run it. But performance models for machine code are handwritten by a relatively small group of experts and are not properly validated. As a consequence, the simulated performance measurements often deviate from real-life results.&nbsp;</p> <p>In series of conference papers, the researchers describe a novel machine-learning pipeline that automates this process, making it easier, faster, and more accurate. In a&nbsp;<a href="">paper</a>&nbsp;presented at the International Conference on Machine Learning in June, the researchers presented Ithemal, a neural-network model that trains on labeled data in the form of “basic blocks” — fundamental snippets of computing instructions — to automatically predict how long it takes a given chip to execute previously unseen basic blocks. Results suggest Ithemal performs far more accurately than traditional hand-tuned models.&nbsp;</p> <p>Then, at the November IEEE International Symposium on Workload Characterization, the researchers&nbsp;<a href="">presented</a>&nbsp;a benchmark suite of basic blocks from a variety of domains, including machine learning, compilers, cryptography, and graphics that can be used to validate performance models. They pooled more than 300,000 of the profiled blocks into an open-source dataset called BHive.&nbsp;During their evaluations, Ithemal predicted how fast Intel chips would run code even better than a performance model built by Intel itself.&nbsp;</p> <p>Ultimately, developers and compilers can use the tool to generate code that runs faster and more efficiently on an ever-growing number of diverse and “black box” chip designs.&nbsp;“Modern computer processors are opaque, horrendously complicated, and difficult to understand. It is also incredibly challenging to write computer code that executes as fast as possible for these processors,” says co-author on all three papers Michael Carbin, an assistant professor in the Department of Electrical Engineering and Computer Science (EECS) and a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL). “This tool is a big step forward toward fully modeling the performance of these chips for improved efficiency.”</p> <p>Most recently, in a&nbsp;<a href="">paper</a>&nbsp;presented at the NeurIPS conference in December, the team proposed a new technique to automatically generate compiler optimizations.&nbsp;&nbsp;Specifically, they automatically generate an algorithm, called Vemal, that converts certain code into vectors, which can be used for parallel computing. Vemal outperforms hand-crafted vectorization algorithms used in the LLVM compiler — a popular compiler used in the industry.</p> <p><strong>Learning from data</strong></p> <p>Designing performance models by hand can be “a black art,” Carbin says. Intel provides extensive documentation of more than 3,000 pages describing its chips’ architectures. But there currently exists only a small group of experts who will build performance models that simulate the execution of code on those architectures.&nbsp;</p> <p>“Intel’s documents are neither error-free nor complete, and Intel will omit certain things, because it’s proprietary,” says co-author on all three papers Charith Mendis, a graduate student in EECS and CSAIL. “However, when you use data, you don’t need to know the documentation. If there’s something hidden you can learn it directly from the data.”</p> <p>To do so, the researchers clocked the average number of cycles a given microprocessor takes to compute basic block instructions — basically, the sequence of boot-up, execute, and shut down — without human intervention. Automating the process enables rapid profiling of hundreds of thousands or millions of blocks.&nbsp;</p> <p><strong>Domain-specific architectures</strong></p> <p>In training, the Ithemal model analyzes millions of automatically profiled basic blocks to learn exactly how different chip architectures will execute computation. Importantly, Ithemal takes raw text as input and does not require manually adding features to the input data. In testing, Ithemal can be fed previously unseen basic blocks and a given chip, and will generate a single number indicating how fast the chip will execute that code.&nbsp;</p> <p>The researchers found Ithemal cut error rates in accuracy —&nbsp;meaning the difference between the predicted speed versus real-world speed —&nbsp;by 50 percent over traditional hand-crafted models. Further,&nbsp;in their next&nbsp;paper, they showed that&nbsp;Ithemal’s error rate was 10 percent, while the Intel performance-prediction model’s error rate was 20 percent on a variety of basic blocks across multiple different domains.</p> <p>The tool now makes it easier to quickly learn performance speeds for any new chip architectures, Mendis says. For instance, domain-specific architectures, such as Google’s new Tensor Processing Unit used specifically for neural networks, are now being built but aren’t widely understood. “If you want to train a model on some new architecture, you just collect more data from that architecture, run it through our profiler, use that information to train Ithemal, and now you have a model that predicts performance,” Mendis says.</p> <p>Next, the researchers are studying methods to make models interpretable. Much of machine learning is a black box, so it’s not really clear why a particular model made its predictions. “Our model is saying it takes a processor, say, 10 cycles to execute a basic block. Now, we’re trying to figure out why,” Carbin says. “That’s a fine level of granularity that would be amazing for these types of tools.”</p> <p>They also hope to use Ithemal to enhance the performance of Vemal even further and achieve better performance automatically.</p> MIT researchers have built a new benchmark tool that can accurately predict how long it takes given code to execute on a computer chip, which can help programmers tweak the code for better performance.Research, Computer science and technology, Algorithms, Machine learning, Data, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering Finding a good read among billions of choices As natural language processing techniques improve, suggestions are getting speedier and more relevant. Fri, 20 Dec 2019 12:55:01 -0500 Kim Martineau | MIT Quest for Intelligence <p>With billions of books, news stories, and documents online, there’s never been a better time to be reading — if you have time to sift through all the options. “There’s a ton of text on the internet,” says&nbsp;<a href="">Justin Solomon</a>, an assistant professor at MIT. “Anything to help cut through all that material is extremely useful.”</p> <p>With the&nbsp;<a href="">MIT-IBM Watson AI Lab</a>&nbsp;and his&nbsp;<a href="">Geometric Data Processing Group</a>&nbsp;at MIT, Solomon recently presented a new technique for cutting through massive amounts of text at the&nbsp;<a href="">Conference on Neural Information Processing Systems</a> (NeurIPS). Their method combines three popular text-analysis tools — topic modeling, word embeddings, and optimal transport — to deliver better, faster results than competing methods on a popular benchmark for classifying documents.</p> <p>If an algorithm knows what you liked in the past, it can scan the millions of possibilities for something similar. As natural language processing techniques improve, those “you might also like” suggestions are getting speedier and more relevant.&nbsp;</p> <p>In the method presented at NeurIPS, an algorithm summarizes a collection of, say, books, into topics based on commonly-used words in the collection. It then divides each book into its five to 15 most important topics, with an estimate of how much each topic contributes to the book overall.&nbsp;</p> <p>To compare books, the researchers use two other tools: word embeddings, a technique that turns words into lists of numbers to reflect their similarity in popular usage, and optimal transport, a framework for calculating the most efficient way of moving objects — or data points — among multiple destinations.&nbsp;</p> <p>Word embeddings make it possible to leverage optimal transport twice: first to compare topics within the collection as a whole, and then, within any pair of books, to measure how closely common themes overlap.&nbsp;</p> <p>The technique works especially well when scanning large collections of books and lengthy documents. In the study, the researchers offer the example of Frank Stockton’s “The Great War Syndicate,” a 19th&nbsp;century American novel that anticipated the rise of nuclear weapons. If you’re looking for a similar book, a topic model would help to identify the dominant themes shared with other books — in this case, nautical, elemental, and martial.&nbsp;</p> <p>But a topic model alone wouldn’t identify Thomas Huxley’s 1863 lecture,&nbsp;“<a href="">The Past Condition of Organic Nature</a>,” as a good match. The writer was a champion of Charles Darwin’s theory of evolution, and his lecture, peppered with mentions of fossils and sedimentation, reflected emerging ideas about geology. When the themes in Huxley’s lecture are matched with Stockton’s novel via optimal transport, some cross-cutting motifs emerge: Huxley’s geography, flora/fauna, and knowledge themes map closely to Stockton’s nautical, elemental, and martial themes, respectively.</p> <p>Modeling books by their representative topics, rather than individual words, makes high-level comparisons possible. “If you ask someone to compare two books, they break each one into easy-to-understand concepts, and then compare the concepts,” says the study’s lead author&nbsp;<a href="">Mikhail Yurochkin</a>, a researcher at IBM.&nbsp;</p> <p>The result is faster, more accurate comparisons, the study shows. The researchers compared 1,720 pairs of books in the Gutenberg Project dataset in one second — more than 800 times faster than the next-best method.</p> <p>The technique also does a better job of accurately sorting documents than rival methods — for example, grouping books in the Gutenberg dataset by author, product reviews on Amazon by department, and BBC sports stories by sport. In a series of visualizations, the authors show that their method neatly clusters documents by type.</p> <p>In addition to categorizing documents quickly and more accurately, the method offers a window into the model’s decision-making process. Through the list of topics that appear, users can see why the model is recommending a document.</p> <p>The study’s other authors are&nbsp;<a href="">Sebastian Claici</a>&nbsp;and&nbsp;<a href="">Edward Chien</a>, a graduate student and a postdoc, respectively, at MIT’s Department of Electrical Engineering and Computer Science and Computer Science and Artificial Intelligence Laboratory, and&nbsp;<a href="">Farzaneh Mirzazadeh</a>, a researcher at IBM.</p> In a new study, researchers at MIT and IBM combine three popular text-analysis tools — topic modeling, word embeddings, and optimal transport — to compare thousands of documents per second. Here, they show that their method (left) clusters newsgroup posts by category more tightly than a competing method. Image courtesy of the researchers.Quest for Intelligence, MIT-IBM Watson AI Lab, Electrical engineering and computer science (EECS), Computer Science and Artificial Intelligence Laboratory (CSAIL), School of Engineering, Algorithms, Artificial intelligence, Computer science and technology, Data, Machine learning, Natural language processing When machine learning packs an economic punch Study: After eBay improved its translation software, international commerce increased sharply. Fri, 20 Dec 2019 10:04:08 -0500 Peter Dizikes | MIT News Office <p>A new study co-authored by an MIT economist shows that improved translation software can significantly boost international trade online — a notable case of machine learning having a clear impact on economic activity.</p> <p>The research finds that after eBay improved its automatic translation program in 2014, commerce shot up by 10.9 percent among pairs of countries where people could use the new system.&nbsp; &nbsp;</p> <p>“That’s a striking number. To have it be so clear in such a short amount of time really says a lot about the power of this technology,” says Erik Brynjolfsson, an MIT economist and co-author of a new paper detailing the results.</p> <p>To put the results in perspective, he adds, consider that physical distance is, by itself, also a significant barrier to global commerce. The 10.9 percent change generated by eBay’s new translation software increases trade by the same amount as “making the world 26 percent smaller, in terms of its impact on the goods that we studied,” he says.</p> <p>The paper, “Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform,” appears in the December issue of <em>Management Science</em>. The authors are Brynjolfsson, who is the Schussel Family Professor of Management Science at the MIT Sloan School of Management, and Xiang Hui and Meng Liu, who are both assistant professors in the Olin Business School at Washington University in St. Louis.</p> <p><strong>Just cause</strong></p> <p>To conduct the study, the scholars examined what happened after eBay, in 2014, introduced its new eBay Machine Translation (eMT) system — a proprietary machine-learning program that, by several objective measures, significantly improved translation quality on eBay’s site. The new system initially was focused on English-Spanish translations, to facilitate trade between the United States and Latin America</p> <p>Previously, eBay had used Bing Translator to render the titles of objects for sale. By one evaluation measure, called the Human Acceptance Rate (HAR), in which three experts accept or reject translations, the eMT system increased the number of acceptable Spanish-language item titles on eBay from 82 percent to 90 percent.</p> <p>Using administrative data from eBay, the researchers then examined the volume of trade on the platform, within countries, after the eMT system went into use. Other factors being equal, the study showed that the new translation system not only had an effect on sales, but that trade increased by 1.06 percent for each additional word in the titles of items on eBay.</p> <p>That is a substantial change for a commerce platform on which, as the paper notes, items for sale often have long, descriptive titles such as “Diamond-Cut Stackable Thin Wedding Ring New .925 Sterling Silver Band Sizes 4-12,” or “Alpine Swiss Keira Women’s Trench Coast Double Breasted Wool Jacket Belted.” In those cases, making the translation clearer helps potential buyers understand exactly what they might be purchasing.</p> <p>Given the study’s level of specificity, Brynjolfsson calls it “a really fortunate natural experiment, with a before-and-after that sharply distinguished what happened when you had machine translation and when you didn’t.”</p> <p>The structure of the study, he adds, has enabled the researchers to say with confidence that the new eBay program, and not outside factors, directly generated the change in trade volume among affected countries.</p> <p>“In economics, it’s often hard to do causal analyses and prove that A caused B, not just that A was associated with B,” says Brynjolfsson. “But in this case, I feel very comfortable using causal language and saying that improvement in machine translation caused the increase in international trade.”</p> <p><strong>Larger puzzle: The productivity issue</strong></p> <p>The genesis of the paper stems from an ongoing question about new technology and economic productivity. While many forms of artificial intelligence have been developed and expanded in the last couple of decades, the impact of AI, including things like machine-translation systems, has not been obvious in economics statistics.</p> <p>“There’s definitely some amazing progress in the core technologies, including in things like natural language processing and translation,” Brynjolfsson says. “But what’s been lacking has been evidence of an economic impact, or business impact. So that’s a bit of a puzzle.”</p> <p>When looking to see if an economic impact for various forms of AI could be measured, Brynjolfsson, Hui, and Liu thought machine translation “made sense, because it’s a relatively straightforward implementation,” Brynjolfsson adds. That is, better translations could influence economic activity, at least on eBay, without any other changes in technology occurring.</p> <p>In this vein, the findings fit with a larger postulation Brynjolfsson has developed in recent years — that the adoption of AI technologies produces a “J-curve” in productivity. As Brynjolfsson has previously written, broad-ranging AI technologies nonetheless “require significant complementary investments, including business process redesign, co-invention of new products and business models, and investments in human capital” to have a large economic impact.</p> <p>As a result, when AI technologies are introduced, productivity may appear to slow down, and when the complementary technologies are developed, productivity may appear to take off — in the “J-curve” shape.</p> <p>So while Brynjolfsson believes the results of this study are clear, he warns against generalizing too much on the basis of this finding about the impact of machine learning and other forms of AI on economic activity. Every case is different, and AI will not always produce such notable changes by itself.</p> <p>“This was a case where not a lot of other changes had to happen in order for the technology to benefit the company,” Brynjolfsson says. “But in many other cases, much more complicated, complementary changes are needed. That’s why, in most cases with machine learning, it takes longer for the benefits to be delivered.”</p> A study co-authored by an MIT economist shows that an improved, automated language-translation system significantly boosted commerce on eBay’s website.Sloan School of Management, Business and management, Machine learning, Artificial intelligence, Economics, Technology and society, Social sciences, Innovation and Entrepreneurship (I&E) Exploring hip hop history with art and technology With its centerpiece exhibit for the forthcoming Universal Hip Hop Museum, an MIT team uses artificial intelligence to explore the rich history of hip hop music. Fri, 20 Dec 2019 09:00:00 -0500 Suzanne Day | Office of Open Learning <p>A new museum is coming to New York City in 2023, the year of hip-hop’s 50th birthday, and an MIT team has helped to pave the way for the city to celebrate the legacy of this important musical genre — by designing unique creative experiences at the intersection of art, learning, and contemporary technology.</p> <p>With “The [R]evolution of Hip Hop Breakbeat Narratives,” a team led by D. Fox Harrell, professor of digital media and artificial intelligence and director of the MIT Center for Advanced Virtuality, has created an art installation that takes museum-goers on an interactive, personalized journey through hip hop history.</p> <p>The installation served as the centerpiece of an event held this month by leaders of the highly anticipated Universal Hip Hop Museum (UHHM), which will officially open in just a few years in the Bronx — the future home of the UHHM, and where many agree that the genre of hip hop music originated.</p> <p>“Hip hop is much more than a musical genre. It is a global phenomenon, with a rich history and massive social and cultural impact, with local roots in the Bronx,” Harrell says. “As an educational center, the Universal Hip Hop Museum will have the power to connect people to the surrounding community.”</p> <p>Harrell’s immersive art installation takes museum-goers on a journey through hip hop culture and history, from the 1970s to the present. However, not everyone experiences the installation in the same way. Using a computational model of users’ preferences and artificial intelligence technologies to drive interaction, the team of artists and computer scientists from the Center for Advanced Virtuality has created layered, personalized virtual experiences.</p> <p>When approaching the exhibit, museum-goers are greeted by “The Elementals,” or novel characters named after the five elements of hip hop (MC, DJ, Breakdance, Graffiti Art, and Knowledge) that guide users and ask key questions — “What is your favorite hip hop song?” or “Which from this pair of lyrics do you like the most?” Based on those answers, the Elementals take users through their own personalized narrative of hip hop history.</p> <p>Harrell developed the Elementals with professors John Jennings of the University of California at Riverside and Stacey Robinson of the University of Illinois — artists collectively known as Black Kirby. This visual aesthetic ties the work into the rich, imaginative cultures and iconography of the African diaspora.</p> <p>Through these conversations with the Elementals they encounter, people can explore broad social issues surrounding hip hop, such as gender, fashion, and location. At the end of their journey, they can take home a personalized playlist of songs.&nbsp;</p> <p>“We designed the Breakbeat Narratives installation by integrating Microsoft conversational AI technologies, which made our user modeling more personable, with a music visualization platform from the TunesMap Educational Foundation,” Harrell says.</p> <p>The exploration of social issues is about as close to the heart of Harrell’s mission in the Center for Advanced Virtuality as one can get. In the center, Harrell designs virtual technologies to stimulate creative expression, cultural analysis, and positive social change.</p> <p>“We wanted to tell stories that pushed beyond stereotypical representations, digging into the complexities of both empowering and problematic representations that often coexist,” he says. “This work fits into our endeavor called the Narrative, Orality, and Improvisation Research (NOIR) Initiative that uses AI technologies to forward the art forms of diverse global cultures.”</p> <p>Through this art project enabled by contemporary technologies, Harrell hopes that he has helped museum leadership to achieve their goal of celebrating hip-hop’s heritage and legacy.</p> <p>“Now, people internationally can have a stake in this great art.”</p> Designed by an MIT team using artificial intelligence, “The [R]evolution of Hip Hop Breakbeat Narratives” is an immersive art installation designed for the forthcoming Universal Hip Hop Museum in New York City.Photo: MIT Center for Advanced VirtualityOffice of Open Learning, Machine learning, Artificial intelligence, History, Arts, Comparative Media Studies/Writing, Technology and society, Music, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Humanities Arts and Social Sciences, School of Engineering Model beats Wall Street analysts in forecasting business financials Using limited data, this automated system predicts a company’s quarterly sales. Thu, 19 Dec 2019 09:31:03 -0500 Rob Matheson | MIT News Office <p>Knowing a company’s true sales can help determine its value. Investors, for instance, often employ financial analysts to predict a company’s upcoming earnings using various public data, computational tools, and their own intuition. Now MIT researchers have developed an automated model that significantly outperforms humans in predicting business sales using very limited, “noisy” data.</p> <p>In finance, there’s growing interest in using imprecise but frequently generated consumer data — called “alternative data” —&nbsp;to help predict a company’s earnings for trading and investment purposes. Alternative data can comprise credit card purchases, location data from smartphones, or even satellite images showing how many cars are parked in a retailer’s lot. Combining alternative data with more traditional but infrequent ground-truth financial data — such as quarterly earnings, press releases, and stock prices — can paint a clearer picture of a company’s financial health on even a daily or weekly basis.</p> <p>But, so far, it’s been very difficult to get accurate, frequent estimates using alternative data. In a paper published this week in the <em>Proceedings of ACM Sigmetrics Conference</em>, the researchers describe a model for forecasting financials that uses only anonymized weekly credit card transactions and three-month earning reports.</p> <p>Tasked with predicting quarterly earnings of more than 30 companies, the model outperformed the combined estimates of expert Wall Street analysts on 57 percent of predictions. Notably, the analysts had access to any available private or public data and other machine-learning models, while the researchers’ model used a very small dataset of the two data types.</p> <p>“Alternative data are these weird, proxy signals to help track the underlying financials of a company,” says first author Michael Fleder, a postdoc in the Laboratory for Information and Decision Systems (LIDS). “We asked, ‘Can you combine these noisy signals with quarterly numbers to estimate the true financials of a company at high frequencies?’ Turns out the answer is yes.”</p> <p>The model could give an edge to investors, traders, or companies looking to frequently compare their sales with competitors. Beyond finance, the model could help social and political scientists, for example, to study aggregated, anonymous data on public behavior. “It’ll be useful for anyone who wants to figure out what people are doing,” Fleder says.</p> <p>Joining Fleder on the paper is EECS Professor Devavrat Shah, who is the director of MIT’s Statistics and Data Science Center, a member of the Laboratory for Information and Decision Systems, a principal investigator for the MIT Institute for Foundations of Data Science, and an adjunct professor at the Tata Institute of Fundamental Research. &nbsp;</p> <p><strong>Tackling the “small data” problem</strong></p> <p>For better or worse, a lot of consumer data is up for sale. Retailers, for instance, can buy credit card transactions or location data to see how many people are shopping at a competitor. Advertisers can use the data to see how their advertisements are impacting sales. But getting those answers still primarily relies on humans. No machine-learning model has been able to adequately crunch the numbers.</p> <p>Counterintuitively, the problem is actually lack of data. Each financial input, such as a quarterly report or weekly credit card total, is only one number. Quarterly reports over two years total only eight data points. Credit card data for, say, every week over the same period is only roughly another 100 “noisy” data points, meaning they contain potentially uninterpretable information.</p> <p>“We have a ‘small data’ problem,” Fleder says. “You only get a tiny slice of what people are spending and you have to extrapolate and infer what’s really going on from that fraction of data.”</p> <p>For their work, the researchers obtained consumer credit card transactions —&nbsp;at typically weekly and biweekly intervals — and quarterly reports for 34 retailers from 2015 to 2018 from a hedge fund. Across all companies, they gathered 306 quarters-worth of data in total.</p> <p>Computing daily sales is fairly simple in concept. The model assumes a company’s daily sales remain similar, only slightly decreasing or increasing from one day to the next. Mathematically, that means sales values for consecutive days are multiplied by some constant value plus some statistical noise value — which captures some of the inherent randomness in a company’s sales. Tomorrow’s sales, for instance, equal today’s sales multiplied by, say, 0.998 or 1.01, plus the estimated number for noise.</p> <p>If given accurate model parameters for the daily constant&nbsp;and noise level, a standard inference algorithm can calculate that equation to output an accurate forecast of daily sales. But the trick is calculating those parameters.</p> <p><strong>Untangling the numbers</strong></p> <p>That’s where quarterly reports and probability techniques come in handy. In a simple world, a quarterly report could be divided by, say, 90 days to calculate the daily sales (implying sales are roughly constant day-to-day). In reality, sales vary from day to day. Also, including alternative data to help understand how sales vary over a quarter complicates matters: Apart from being noisy, purchased credit card data always consist of some indeterminate fraction of the total sales. All that makes it very difficult to know how exactly the credit card totals factor into the overall sales estimate.</p> <p>“That requires a bit of untangling the numbers,” Fleder says. “If we observe 1 percent of a company’s weekly sales through credit card transactions, how do we know it’s 1 percent? And, if the credit card data is noisy, how do you know how noisy it is? We don’t have access to the ground truth for daily or weekly sales totals. But the quarterly aggregates help us reason about those totals.”</p> <p>To do so, the researchers use a variation of the standard inference algorithm, called Kalman filtering or Belief Propagation, which has been used in various technologies from space shuttles to smartphone GPS. Kalman filtering uses data measurements observed over time, containing noise inaccuracies, to generate a probability distribution for unknown variables over a designated timeframe. In the researchers’ work, that means estimating the possible sales of a single day.</p> <p>To train the model, the technique first breaks down quarterly sales into a set number of measured days, say 90 — allowing sales to vary day-to-day. Then, it matches the observed, noisy credit card data to unknown daily sales. Using the quarterly numbers and some extrapolation, it estimates the fraction of total sales the credit card data likely represents. Then, it calculates each day’s fraction of observed sales, noise level, and an error estimate for how well it made its predictions.</p> <p>The inference algorithm plugs all those values into the formula to predict daily sales totals. Then, it can sum those totals to get weekly, monthly, or quarterly numbers. Across all 34 companies, the model beat a consensus benchmark — which combines estimates of Wall Street analysts —&nbsp;on 57.2 percent of 306 quarterly predictions.</p> <p>Next, the researchers are designing the model to analyze a combination of credit card transactions and other alternative data, such as location information. “This isn’t all we can do. This is just a natural starting point,” Fleder says.</p> An automated machine-learning model developed by MIT researchers significantly outperforms human Wall Street analysts in predicting quarterly business sales.Research, Computer science and technology, Algorithms, Laboratory for Information and Decision Systems (LIDS), IDSS, Data, Machine learning, Finance, Industry, Electrical Engineering & Computer Science (eecs), School of Engineering Differences between deep neural networks and human perception Stimuli that sound or look like gibberish to humans are indistinguishable from naturalistic stimuli to deep networks. Thu, 12 Dec 2019 13:05:01 -0500 Kenneth I. Blum | Center for Brains, Minds and Machines <p>When your mother calls your name, you know it’s her voice — no matter the volume, even over a poor cell phone connection. And when you see her face, you know it’s hers — if she is far away, if the lighting is poor, or if you are on a bad FaceTime call. This robustness to variation is a hallmark of human perception. On the other hand, we are susceptible to illusions: We might fail to distinguish between sounds or images that are, in fact, different. Scientists have explained many of these illusions, but we lack a full understanding of the invariances in our auditory and visual systems.
</p> <p>Deep neural networks also have performed speech recognition and image classification tasks with impressive robustness to variations in the auditory or visual stimuli. But are the invariances learned by these models similar to the invariances learned by human perceptual systems? A group of MIT researchers has discovered that they are different. They presented their findings yesterday at the 2019 <a href="">Conference on Neural Information Processing Systems</a>.
</p> <p>The researchers made a novel generalization of a classical concept: “metamers” — physically distinct stimuli that generate the same perceptual effect. The most famous examples of metamer stimuli arise because most people have three different types of cones in their retinae, which are responsible for color vision. The perceived color of any single wavelength of light can be matched exactly by a particular combination of three lights of different colors — for example, red, green, and blue lights. Nineteenth-century scientists inferred from this observation that humans have three different types of bright-light detectors in our eyes. This is the basis for electronic color displays on all of the screens we stare at every day. Another example in the visual system is that when we fix our gaze on an object, we may perceive surrounding visual scenes that differ at the periphery as identical. In the auditory domain, something analogous can be observed. For example, the “textural” sound of two swarms of insects might be indistinguishable, despite differing in the acoustic details that compose them, because they have similar aggregate statistical properties. In each case, the metamers provide insight into the mechanisms of perception, and constrain models of the human visual or auditory systems.
</p> <p>In the current work, the researchers randomly chose natural images and sound clips of spoken words from standard databases, and then synthesized sounds and images so that deep neural networks would sort them into the same classes as their natural counterparts. That is, they generated physically distinct stimuli that are classified identically by models, rather than by humans. This is a new way to think about metamers, generalizing the concept to swap the role of computer models for human perceivers. They therefore called these synthesized stimuli “model metamers” of the paired natural stimuli. The researchers then tested whether humans could identify the words and images.
</p> <p>“Participants heard a short segment of speech and had to identify from a list of words which word was in the middle of the clip. For the natural audio this task is easy, but for many of the model metamers humans had a hard time recognizing the sound,” explains first-author Jenelle Feather, a graduate student in the <a href="" target="_blank">MIT Department of Brain and Cognitive Sciences</a> (BCS) and a member of the <a href="" target="_blank">Center for Brains, Minds, and Machines</a> (CBMM). That is, humans would not put the synthetic stimuli in the same class as the spoken word “bird” or the image of a bird. In fact, model metamers generated to match the responses of the deepest layers of the model were generally unrecognizable as words or images by human subjects.
</p> <p><a href="">Josh McDermott</a>, associate professor in BCS and investigator in CBMM, makes the following case: “The basic logic is that if we have a good model of human perception, say of speech recognition, then if we pick two sounds that the model says are the same and present these two sounds to a human listener, that human should also say that the two sounds are the same. If the human listener instead perceives the stimuli to be different, this is a clear indication that the representations in our model do not match those of human perception.”
</p> <p>Examples of the model metamer stimuli can be found in the video below.</p> <div class="cms-placeholder-content-video"></div> <p>Joining Feather and McDermott on the paper are Alex Durango, a post-baccalaureate student, and Ray Gonzalez, a research assistant, both in BCS.
</p> <p>There is another type of failure of deep networks that has received a lot of attention in the media: adversarial examples (see, for example, "<a href="">Why did my classifier just mistake a turtle for a rifle?</a>"). These are stimuli that appear similar to humans but are misclassified by a model network (by design — they are constructed to be misclassified). They are complementary to the stimuli generated by Feather's group, which sound or appear different to humans but are designed to be co-classified by the model network. The vulnerabilities of model networks exposed to adversarial attacks are well-known — face-recognition software might mistake identities; automated vehicles might not recognize pedestrians.
</p> <p>The importance of this work lies in improving models of perception beyond deep networks. Although the standard adversarial examples indicate differences between deep networks and human perceptual systems, the new stimuli generated by the McDermott group arguably represent a more fundamental model failure — they show that generic examples of stimuli classified as the same by a deep network produce wildly different percepts for humans.
</p> <p>The team also figured out ways to modify the model networks to yield metamers that were more plausible sounds and images to humans. As McDermott says, “This gives us hope that we may be able to eventually develop models that pass the metamer test and better capture human invariances.”
</p> <p>“Model metamers demonstrate a significant failure of present-day neural networks to match the invariances in the human visual and auditory systems,” says Feather, “We hope that this work will provide a useful behavioral measuring stick to improve model representations and create better models of human sensory systems.”
</p> Associate Professor Josh McDermott (left) and graduate student Jenelle Feather generated physically distinct stimuli that are classified identically by models, rather than by humans.Photos: Justin Knight and Kris BrewerBrain and cognitive sciences, Center for Brains Minds and Machines, McGovern Institute, Research, Machine learning, Artificial intelligence, School of Science, Neuroscience This object-recognition dataset stumped the world’s best computer vision models Objects are posed in varied positions and shot at odd angles to spur new AI techniques. Tue, 10 Dec 2019 11:00:01 -0500 Kim Martineau | MIT Quest for Intelligence <p>Computer vision models have learned to identify objects in photos so accurately that some can outperform humans on some datasets. But when those same object detectors are turned loose in the real world, their performance noticeably drops, creating reliability concerns for self-driving cars and other safety-critical systems that use machine vision.</p> <p>In an effort to close this performance gap, a team of MIT and IBM researchers set out to create a very different kind of object-recognition dataset. It’s called <a href="">ObjectNet,</a> a play on ImageNet, the crowdsourced database of photos responsible for launching much of the modern boom in artificial intelligence.&nbsp;</p> <p>Unlike ImageNet, which features photos taken from Flickr and other social media sites, ObjectNet features photos taken by paid freelancers. Objects are shown tipped on their side, shot at odd angles, and displayed in clutter-strewn rooms. When leading object-detection models were tested on ObjectNet<strong>,</strong> their accuracy rates fell from a high of 97 percent on ImageNet to just 50-55 percent.</p> <p>“We created this dataset to tell people the object-recognition problem continues to be a hard problem,” says <a href="">Boris Katz</a>, a research scientist at MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) and <a href="">Center for Brains, Minds and Machines</a> (CBMM).&nbsp; “We need better, smarter algorithms.” Katz and his colleagues will present ObjectNet and their results at the <a href="">Conference on Neural Information Processing Systems (NeurIPS)</a>.</p> <p>Deep learning, the technique driving much of the recent progress in AI, uses layers of artificial "neurons" to find patterns in vast amounts of raw data. It learns to pick out, say, the chair in a photo after training on hundreds to thousands of examples. But even datasets with millions of images can’t show each object in all of its possible orientations and settings, creating problems when the models encounter these objects in real life.</p> <p>ObjectNet is different from conventional image datasets in another important way: it contains no training images. Most datasets are divided into data for training the models and testing their performance. But the training set often shares subtle similarities with the test set, in effect giving the models a sneak peak at the test.&nbsp;</p> <p>At first glance, <a href="">ImageNet</a>, at 14 million images, seems enormous. But when its training set is excluded, it’s comparable in size to ObjectNet, at 50,000 photos.&nbsp;</p> <p>“If we want to know how well algorithms will perform in the real world, we should test them on images that are unbiased and that they’ve never seen before,” says study co-author <a href="">Andrei Barbu</a>, a research scientist at CSAIL and CBMM.<em>&nbsp;</em></p> <p><strong>A dataset that tries to capture the complexity of real-world objects&nbsp;</strong></p> <p>Few people would think to share the photos from ObjectNet with their friends, and that’s the point. The researchers hired freelancers from Amazon Mechanical Turk to take photographs of hundreds of randomly posed household objects. Workers received photo assignments on an app, with animated instructions telling them how to orient the assigned object, what angle to shoot from, and whether to pose the object in the kitchen, bathroom, bedroom, or living room.&nbsp;</p> <p>They wanted to eliminate three common biases: objects shown head-on, in iconic positions, and in highly correlated settings — for example, plates stacked in the kitchen.&nbsp;</p> <p>It took three years to conceive of the dataset and design an app that would standardize the data-gathering process. “Discovering how to gather data in a way that controls for various biases was incredibly tricky,” says study co-author <a href="">David Mayo</a>, a graduate student at MIT’s <a href="">Department of Electrical Engineering and Computer Science.</a> “We also had to run experiments to make sure our instructions were clear and that the workers knew exactly what was being asked of them.”&nbsp;</p> <p>It took another year to gather the actual data, and in the end, half of all the photos freelancers submitted had to be discarded for failing to meet the researchers’ specifications. In an attempt to be helpful, some workers added labels to their objects, staged them on white backgrounds, or otherwise tried to improve on the aesthetics of the photos they were assigned to shoot.</p> <p>Many of the photos were taken outside of the United States, and thus, some objects may look unfamiliar. Ripe oranges are green, bananas come in different sizes, and clothing appears in a variety of shapes and textures.</p> <p><strong>Object Net vs. ImageNet: how leading object-recognition models compare</strong></p> <p>When the researchers tested state-of-the-art computer vision models on ObjectNet, they found a performance drop of 40-45 percentage points from ImageNet. The results show that object detectors still struggle to understand that objects are three-dimensional and can be rotated and moved into new contexts, the researchers say. “These notions are not built into the architecture of modern object detectors,” says study co-author <a href="">Dan Gutfreund</a>, a researcher at IBM.</p> <p>To show that ObjectNet is difficult precisely because of how objects are viewed and positioned, the researchers allowed the models to train on half of the ObjectNet data before testing them on the remaining half. Training and testing on the same dataset typically improves performance, but here the models improved only slightly, suggesting that object detectors have yet to fully comprehend how objects exist in the real world.</p> <p>Computer vision models have progressively improved since 2012, when an object detector called AlexNet crushed the competition at the annual ImageNet contest. As datasets have gotten bigger, performance has also improved.</p> <p>But designing bigger versions of ObjectNet, with its added viewing angles and orientations, won’t necessarily lead to better results, the researchers warn. The goal of ObjectNet is to motivate researchers to come up with the next wave of revolutionary techniques, much as the initial launch of the ImageNet challenge did.</p> <p>“People feed these detectors huge amounts of data, but there are diminishing returns,” says Katz. “You can’t view an object from every angle and in every context. Our hope is that this new dataset will result in robust computer vision without surprising failures in the real world.”</p> <p>The study’s other authors are Julian Alvero, William Luo, Chris Wang, and Joshua Tenenbaum of MIT. The research was funded by the National Science Foundation, MIT’s Center for Brains, Minds, and Machines, the MIT-IBM Watson AI Lab, Toyota Research Institute, and the SystemsThatLearn@CSAIL initiative.</p> ObjectNet, a dataset of photos created by MIT and IBM researchers, shows objects from odd angles, in multiple orientations, and against varied backgrounds to better represent the complexity of 3D objects. The researchers hope the dataset will lead to new computer vision techniques that perform better in real life. Photo collage courtesy of the researchers.Quest for Intelligence, Center for Brains Minds and Machines, Electrical engineering and computer science (EECS), Computer Science and Artificial Intelligence Laboratory (CSAIL), School of Engineering, Algorithms, Artifical intelligence, Computer science and technology, Computer vision, Data, Machine learning, Software Intelligent Towing Tank propels human-robot-computer research A novel experimental facility integrates automation and active learning, illuminating a path to accelerated scientific discovery. Mon, 09 Dec 2019 16:05:01 -0500 MIT Sea Grant <p>In its first year of operation, the Intelligent Towing Tank (ITT) conducted about 100,000 total experiments, essentially completing the equivalent of a PhD student’s five years’ worth of experiments in a matter of weeks.</p> <p>The automated experimental facility, developed in the MIT Sea Grant Hydrodynamics Laboratory, automatically and adaptively performs, analyzes, and designs experiments exploring vortex-induced vibrations (VIVs). Important for engineering offshore ocean structures like marine drilling risers that connect underwater oil wells to the surface, VIVs remain somewhat of a phenomenon to researchers due to the high number of parameters involved.</p> <p>Guided by active learning, the ITT conducts series of experiments wherein the parameters of each next experiment are selected by a computer. Using an “explore-and-exploit” methodology, the system dramatically reduces the number of experiments required to explore and map the complex forces governing VIVs.</p> <p>What began as then-PhD candidate Dixia Fan’s quest to cut back on conducting a thousand or so laborious experiments — by hand — led to the design of the innovative system and a <a href="">paper</a> recently published in the journal <em>Science Robotics</em>.</p> <p>Fan, now a postdoc, and a team of researchers from the MIT Sea Grant College Program and MIT’s Department of Mechanical Engineering, École Normale Supérieure de Rennes, and Brown University, reveal a potential paradigm shift in experimental research, where humans, computers, and robots can collaborate more effectively to accelerate scientific discovery.</p> <p>The 33-foot whale of a tank comes alive, working without interruption or supervision on the venture at hand — in this case, exploring a canonical problem in the field of fluid-structure interactions. But the researchers envision applications of the active learning and automation approach to experimental research across disciplines, potentially leading to new insights and models in multi-input/multi-output nonlinear systems.</p> <p>VIVs are inherently-nonlinear motions induced on a structure&nbsp;in an oncoming irregular cross-stream, which prove vexing to study. The researchers report that the number of experiments completed by the ITT is already comparable to the total number of experiments done to date worldwide on the subject of VIVs.</p> <p>The reason for this is the large number of independent parameters, from flow velocity to pressure, involved in studying the complex forces at play. According to Fan, a systematic brute-force approach — blindly conducting 10 measurements per parameter in an eight-dimensional parametric space — would require 100 million experiments.</p> <p>With the ITT, Fan and his collaborators have taken the problem into a wider parametric space than previously practicable to explore. “If we performed traditional techniques on the problem we studied,” he explains, “it would take 950 years to finish the experiment.” Clearly infeasible, so Fan and the team integrated a Gaussian process regression learning algorithm into the ITT. In doing so, the researchers reduced the experimental burden by several orders of magnitude, requiring only a few thousand experiments.</p> <p>The robotic system automatically conducts an initial sequence of experiments, periodically towing a submerged structure along the length of the tank at a constant velocity. Then, the ITT takes partial control over the parameters of each next experiment by minimizing suitable acquisition functions of quantified uncertainties and adapting to achieve a range of objectives, like reduced drag.</p> <p>Earlier this year, Fan was awarded an&nbsp;MIT Mechanical Engineering de Florez Award for "Outstanding Ingenuity and Creative Judgment"&nbsp;in the development of the ITT. “Dixia’s design of the Intelligent Towing Tank is an outstanding example of using novel methods to reinvigorate mature fields,” says Michael Triantafyllou, Henry L. and Grace Doherty Professor in Ocean Science and Engineering, who acted as Fan’s doctoral advisor.</p> <p>Triantafyllou, a co-author on this paper and the director of the MIT Sea Grant College Program, says, “MIT Sea Grant has committed resources and funded projects using deep-learning methods in ocean-related problems for several years that are already paying off.” Funded by the National Oceanic and Atmospheric Administration and administered by the National Sea Grant Program, MIT Sea Grant is a federal-Institute partnership that brings the research and engineering core of MIT&nbsp;to bear on ocean-related challenges.</p> <p>Fan’s research points to a number of others utilizing automation and artificial intelligence in science: At Caltech, a robot scientist named “Adam” generates and tests hypotheses; at the Defense Advanced Research Projects Agency, the Big Mechanism program reads tens of thousands of research papers to generate new models.</p> <p>Similarly, the ITT applies human-computer-robot collaboration to accelerate experimental efforts. The system demonstrates a potential paradigm shift in conducting research, where automation and uncertainty quantification can considerably accelerate scientific discovery. The researchers assert that the machine learning methodology described in this paper can be adapted and applied in and beyond fluid mechanics, to other experimental fields.</p> <p>Other contributors to the paper include George Karniadakis from Brown University, who is also affiliated with MIT Sea Grant; Gurvan Jodin from ENS Rennes; MIT PhD candidate in mechanical engineering Yu Ma; and Thomas Consi, Luca Bonfiglio, and Lily Keyes from MIT Sea Grant.</p> <p>This work was supported by DARPA, Fariba Fahroo, and Jan Vandenbrande through an EQUiPS (Enabling Quantification of Uncertainty in Physical Systems) grant, as well as Shell, Subsea 7, and the MIT Sea Grant College Program.</p> Dixia Fan stands in front of the ITT holding some of the structures towed in the tank to study VIVs, which are important for engineering offshore ocean structures like marine drilling risers that connect underwater oil wells to the surface.Photo: Lily Keyes/MIT Sea GrantMIT Sea Grant, Mechanical engineering, Oceanography and ocean engineering, School of Engineering, Fluid dynamics, Robots, Robotics, automation, Artificial intelligence, Machine learning, Algorithms Restructuring the MIT Department of Electrical Engineering and Computer Science The Institute&#039;s largest academic department reorganizes with new leadership as part of the formation of the MIT Schwarzman College of Computing. Thu, 05 Dec 2019 15:00:01 -0500 Terri Park | Lori LoTurco | MIT Schwarzman College of Computing | School of Engineering <p>As part of the founding of the MIT Stephen A. Schwarzman College of Computing, the Department of Electrical Engineering and Computer Science (EECS), the largest academic unit at MIT, has been restructured to provide a stronger base for enhancing existing programs, creating new opportunities, and increasing connections to other parts of the Institute.</p> <p>Jointly part of the School of Engineering and Schwarzman College of Computing, EECS is now composed of three overlapping sub-units in electrical engineering (EE), computer science (CS), and artificial intelligence and decision-making (AI+D), which brings together computer science-heritage AI and machine learning with electrical engineering-heritage information and decision systems to exploit their significant synergies. The department will remain responsible for Course 6.</p> <p>The <a href="" target="_blank">organizational plan for EECS</a> was developed over the summer based on the <a href="" target="_blank">final report</a> of the Organizational Structure Working Group of the Computing Task Force.</p> <p>“It is hard to imagine a School of Engineering without electrical engineering and a College of Computing without computer science. We expect that the creation of this new configuration will lead to a highly collaborative approach not only within EECS, but across campus and across disciplines,” says Dan Huttenlocher, dean of the MIT Schwarzman College of Computing.</p> <p>The plan calls for each of the units, termed a “faculty” to signify its differentiation from a traditional academic structure, each managed by a head of the faculty to lead the respective area and to contribute to the overall leadership of EECS, under the direction of the department head who will continue to oversee cross-cutting matters. The three faculty heads and the EECS head will each report jointly to Huttenlocher and to Anantha Chandrakasan, dean of the MIT School of Engineering.</p> <p>“This restructure will provide more autonomy to each unit,” says Chandrakasan. “The faculties will focus on faculty recruiting, mentoring, promotion, academic programs, and community building.”</p> <p>Asu Ozdaglar, Distinguished Professor of Engineering, a principal investigator at the Laboratory for Information and Decision Systems, and the newly appointed deputy dean of academics for the College of Computing, will remain the head of EECS, a position she has held since 2018. During her tenure, Ozdaglar has championed initiatives such as curriculum innovations to keep up with the ever-growing interest in computing, creation of new joint majors such as 6-14 (Computer Science, Economics, and Data Science), and Rising Stars in EECS, a workshop for female graduate students and postdocs interested in pursuing academic careers in computer engineering and electrical engineering, among many others.</p> <p>Joel Voldman, a professor of electrical engineering and computer science and an associate department head at EECS, will be the head of the faculty of electrical engineering. A principal investigator in the Research Laboratory of Electronics and the Microsystems Technology Laboratories, Voldman’s research focus is on developing microfluidic technology for biology and medicine, with an emphasis on cell sorting and immunology. In addition, he co-developed two introductory EECS courses: 6.03&nbsp;(Introduction to&nbsp;EECS via Medical Technology) and 6.S08/6.08&nbsp;(Interconnected Embedded Systems), and recently helped revise 6.002 (Circuits and Electronics).</p> <p>Arvind, the Charles W. and Jennifer C. Johnson Professor in Computer Science and Engineering, will step into the role of head of the faculty of computer science. Arvind’s research focuses on the specification and synthesis of complex digital systems, including microprocessors and accelerators, using a formalism known as guarded atomic actions. He leads the Computation Structures Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL), and is a fellow of the&nbsp;Institute of Electrical and Electronics Engineers (IEEE) and the Association for Computing Machinery. He was elected into the&nbsp;National Academy of Engineering&nbsp;in 2008 and the Academy for Arts and Sciences in 2012.</p> <p>Antonio Torralba, the Thomas and Gerd Perkins Professor of Electrical Engineering and Computer Science, has been named the head of the faculty of artificial intelligence and decision-making. A principal investigator at CSAIL, a member of the Center for Brains, Minds, and Machines, and director of the MIT Quest for Intelligence and MIT-IBM Watson AI Lab, Torralba is the recipient of the 2008 National Science Foundation Career award, the best student paper award at the IEEE Conference on Computer Vision and Pattern Recognition in 2009, and the 2010 J.K. Aggarwal Prize from the International Association for Pattern Recognition. In 2017, he received the Frank Quick Faculty Research Innovation Fellowship and the Louis D. Smullin (’39) Award for Teaching Excellence.</p> <p>An advisory search committee made up of EECS faculty members — Ozdaglar (chair), Hari Balakrishnan, Marc Baldo, Duane Boning, Tommi Jaakkola, Patrick Jaillet, Dina Katabi, Jing Kong, Tomas Lozano-Perez, Alan Oppenheim, Daniela Rus, Armando Solar-Lezama, Collin Stultz, Ryan Williams, and Daniel Hastings — was formed to identify candidates to lead all three units to help guide the two deans in selecting the heads.</p> <p>Voldman, Arvind, and Torralba will begin their respective appointments on Jan. 1, 2020. Current Associate Department Head Saman Amarasinghe, an EECS professor and lead of the Commit compiler research group in CSAIL, will continue in his role until the new heads start their positions.</p> <p>“Thank you to everyone who served on the search committee and to Professer Amarasinghe for his tremendous leadership and contributions to EECS as an associate head. And please join us in congratulating Asu, Antonio, Arvind, and Joel for taking on these important new roles,” says Chandrakasan.</p> <p>“We look forward to working with the new leadership and all of the faculty in the department to help make EECS even stronger for our students and the MIT community, and more broadly, in leading this rapidly changing area,” adds Huttenlocher.&nbsp;</p> Clockwise from upper left: Asu Ozdaglar, Joel Voldman, Arvind, and Antonio TorralbaElectrical Engineering & Computer Science (eecs), Faculty, Administration, electronics, Artificial intelligence, Computer Science and Artificial Intelligence Laboratory (CSAIL), Machine learning, School of Engineering, MIT Schwarzman College of Computing Helping machines perceive some laws of physics Model registers “surprise” when objects in a scene do something unexpected, which could be used to build smarter AI. Mon, 02 Dec 2019 00:00:00 -0500 Rob Matheson | MIT News Office <p>Humans have an early understanding of the laws of physical reality. Infants, for instance, hold expectations for how objects should move and interact with each other, and will show surprise when they do something unexpected, such as disappearing in a sleight-of-hand magic trick.</p> <p>Now MIT researchers have designed a model that demonstrates an understanding of some basic “intuitive physics” about how objects should behave. The model could be used to help build smarter artificial intelligence and, in turn, provide information to help scientists understand infant cognition.</p> <p>The model, called ADEPT, observes objects moving around a scene and makes predictions about how the objects should behave, based on their underlying physics. While tracking the objects, the model outputs a signal at each video frame that correlates to a level of “surprise” — the bigger the signal, the greater the surprise. If an object ever dramatically mismatches the model’s predictions — by, say, vanishing or teleporting across a scene — its surprise levels will spike.</p> <p>In response to videos showing objects moving in physically plausible and implausible ways, the model registered levels of surprise that matched levels reported by humans who had watched the same videos. &nbsp;</p> <p>“By the time infants are 3 months old, they have some notion that objects don’t wink in and out of existence, and can’t move through each other or teleport,” says first author Kevin A. Smith, a research scientist in the Department of Brain and Cognitive Sciences (BCS) and a member of the Center for Brains, Minds, and Machines (CBMM). “We wanted to capture and formalize that knowledge to build infant cognition into artificial-intelligence agents. We’re now getting near human-like in the way models can pick apart basic implausible or plausible scenes.”</p> <p>Joining Smith on the paper are co-first authors Lingjie Mei, an undergraduate in the Department of Electrical Engineering and Computer Science, and BCS research scientist Shunyu Yao; Jiajun Wu PhD ’19; CBMM investigator Elizabeth Spelke; Joshua B. Tenenbaum, a professor of computational cognitive science, and researcher in CBMM, BCS, and the Computer Science and Artificial Intelligence Laboratory (CSAIL); and CBMM investigator Tomer D. Ullman PhD ’15.</p> <p><strong>Mismatched realities</strong></p> <p>ADEPT relies on two modules: an “inverse graphics” module that captures object representations from raw images, and a “physics engine” that predicts the objects’ future representations from a distribution of possibilities.</p> <p>Inverse graphics basically extracts information of objects —&nbsp;such as shape, pose, and velocity — from pixel inputs. This module captures frames of video as images and uses inverse graphics to extract this information from objects in the scene. But it doesn’t get bogged down in the details. ADEPT requires only some approximate geometry of each shape to function. In part, this helps the model generalize predictions to new objects, not just those it’s trained on.</p> <p>“It doesn’t matter if an object is rectangle or circle, or if it’s a truck or a duck. ADEPT just sees there’s an object with some position, moving in a certain way, to make predictions,” Smith says. “Similarly, young infants also don’t seem to care much about some properties like shape when making physical predictions.”</p> <p>These coarse object descriptions are fed into a physics engine — software that simulates behavior of physical systems, such as rigid or fluidic bodies, and is commonly used for films, video games, and computer graphics. The researchers’ physics engine “pushes the objects forward in time,” Ullman says. This creates a range of predictions, or a “belief distribution,” for what will happen to those objects in the next frame.</p> <p>Next, the model observes the actual next frame. Once again, it captures the object representations, which it then aligns to one of the predicted object representations from its belief distribution. If the object obeyed the laws of physics, there won’t be much mismatch between the two representations. On the other hand, if the object did something implausible — say, it vanished from behind a wall — there will be a major mismatch.</p> <p>ADEPT then resamples from its belief distribution and notes a very low probability that the object had simply vanished. If there’s a low enough probability, the model registers great “surprise” as a signal spike. Basically, surprise is inversely proportional to the probability of an event occurring. If the probability is very low, the signal spike is very high. &nbsp;</p> <p>“If an object goes behind a wall, your physics engine maintains a belief that the object is still behind the wall. If the wall goes down, and nothing is there, there’s a mismatch,” Ullman says. “Then, the model says, ‘There’s an object in my prediction, but I see nothing. The only explanation is that it disappeared, so that’s surprising.’”</p> <p><strong>Violation of expectations</strong></p> <p>In development psychology, researchers run “violation of expectations” tests in which infants are shown pairs of videos. One video shows a plausible event, with objects adhering to their expected notions of how the world works. The other video is the same in every way, except objects behave in a way that violates expectations in some way. Researchers will often use these tests to measure how long the infant looks at a scene after an implausible action has occurred. The longer they stare, researchers hypothesize, the more they may be surprised or interested in what just happened.</p> <p>For their experiments, the researchers created several scenarios based on classical developmental research to examine the model’s core object knowledge. They employed 60 adults to watch 64 videos of known physically plausible and physically implausible scenarios. Objects, for instance, will move behind a wall and, when the wall drops, they’ll still be there or they’ll be gone. The participants rated their surprise at various moments on an increasing scale of 0 to 100. Then, the researchers showed the same videos to the model. Specifically, the scenarios examined the model’s ability to capture notions of permanence (objects do not appear or disappear for no reason), continuity (objects move along connected trajectories), and solidity (objects cannot move through one another).</p> <p>ADEPT matched humans particularly well on videos where objects moved behind walls and disappeared when the wall was removed. Interestingly, the model also matched surprise levels on videos that humans weren’t surprised by but maybe should have been. For example, in a video where an object moving at a certain speed disappears behind a wall and immediately comes out the other side, the object might have sped up dramatically when it went behind the wall or it might have teleported to the other side. In general, humans and ADEPT were both less certain about whether that event was or wasn’t surprising. The researchers also found traditional neural networks that learn physics from observations — but don’t explicitly represent objects — are far less accurate at differentiating surprising from unsurprising scenes, and their picks for surprising scenes don’t often align with humans.</p> <p>Next, the researchers plan to delve further into how infants observe and learn about the world, with aims of incorporating any new findings into their model. Studies, for example, show that infants up until a certain age actually aren’t very surprised when objects completely change in some ways — such as if a truck disappears behind a wall, but reemerges as a duck.</p> <p>“We want to see what else needs to be built in to understand the world more like infants, and formalize what we know about psychology to build better AI agents,” Smith says.</p> An MIT-invented model demonstrates an understanding of some basic “intuitive physics” by registering “surprise” when objects in simulations move in unexpected ways, such as rolling behind a wall and not reappearing on the other side.Image: Christine Daniloff, MITResearch, Computer science and technology, Algorithms, Artificial intelligence, Machine learning, Computer vision, Computer Science and Artificial Intelligence Laboratory (CSAIL), Brain and cognitive sciences, Electrical Engineering & Computer Science (eecs), School of Engineering, Center for Brains Minds and Machines Producing better guides for medical-image analysis Model quickly generates brain scan templates that represent a given patient population. Tue, 26 Nov 2019 13:56:05 -0500 Rob Matheson | MIT News Office <p>MIT researchers have devised a method that accelerates the process for creating and customizing templates used in medical-image analysis, to guide disease diagnosis. &nbsp;</p> <p>One use of medical image analysis is to crunch datasets of patients’ medical images and capture structural relationships that may indicate the progression of diseases. In many cases, analysis requires use of a common image template, called an “atlas,” that’s an average representation of a given patient population. Atlases serve as a reference for comparison, for example to identify clinically significant changes in brain structures over time.</p> <p>Building a template is a time-consuming, laborious process, often taking days or weeks to generate, especially when using 3D brain scans. To save time, researchers often download publicly available atlases previously generated by research groups. But those don’t fully capture the diversity of individual datasets or specific subpopulations, such as those with new diseases or from young children. Ultimately, the atlas can’t be smoothly mapped onto outlier images, producing poor results.</p> <p>In a paper being presented at the Conference on Neural Information Processing Systems in December, the researchers describe an automated machine-learning model that generates “conditional” atlases based on specific patient attributes, such as age, sex, and disease. By leveraging shared information from across an entire dataset, the model can also synthesize atlases from patient subpopulations that may be completely missing in the dataset.</p> <p>“The world needs more atlases,” says first author Adrian Dalca, a former postdoc in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and now a faculty member in radiology at Harvard Medical School and Massachusetts General Hospital. “Atlases are central to many medical image analyses. This method can build a lot more of them and build conditional ones as well.”</p> <p>Joining Dalca on the paper are Marianne Rakic, a visiting researcher in CSAIL; John Guttag, the Dugald C. Jackson Professor of Computer Science and Electrical Engineering and head of CSAIL’s Data Driven Inference Group; and Mert R. Sabuncu of Cornell University.</p> <p><strong>Simultaneous alignment and atlases</strong></p> <p>Traditional atlas-building methods run lengthy, iterative optimization processes on all images in a dataset. They align, say, all 3D brain scans to an initial (often blurry) atlas, and compute a new average image from the aligned scans. They repeat this iterative process for all images. This computes a final atlas that minimizes the extent to which all scans in the dataset must deform to match the atlas. Doing this process for patient subpopulations can be complex and imprecise if there isn’t enough data available.</p> <p>Mapping an atlas to a new scan generates a “deformation field,” which characterizes the differences between the two images. This captures structural variations, which can then be further analyzed. In brain scans, for instance, structural variations can be due to tissue degeneration at different stages of a disease.</p> <p>In previous work, Dalca and other researchers developed a neural network to rapidly align these images. In part, that helped speed up the traditional atlas-building process. “We said, ‘Why can’t we build conditional atlases while learning to align images at the same time?’” Dalca says.</p> <p>To do so, the researchers combined two neural networks: One network automatically learns an atlas at each iteration, and another — adapted from the previous research — simultaneously aligns that atlas to images in a dataset.</p> <p>In training, the joint network is fed a random image from a dataset encoded with desired patient attributes. From that, it estimates an attribute-conditional atlas. The second network aligns the estimated atlas with the input image, and generates a deformation field.</p> <p>The deformation field generated for each image pair is used to train a “loss function,” a component of machine-learning models that helps minimize deviations from a given value. In this case, the function specifically learns to minimize distances between the learned atlas and each image. The network continuously refines the atlas to smoothly align to any given image across the dataset.</p> <div class="cms-placeholder-content-video"></div> <p><strong>On-demand atlases</strong></p> <p>The end result is a function that’s learned how specific attributes, such as age, correlate to structural variations across all images in a dataset. By plugging new patient attributes into the function, it leverages all learned information across the dataset to synthesize an on-demand atlas — even if that attribute data is missing or scarce in the dataset.</p> <p>Say someone wants a brain scan atlas for a 45-year-old female patient from a dataset with information from patients aged 30 to 90, but with little data for women aged 40 to 50. The function will analyze patterns of how the brain changes between the ages of 30 to 90 and incorporate what little data exists for that age and sex. Then, it will produce the most representative atlas for females of the desired age. In their paper, the researchers verified the function by generating conditional templates for various age groups from 15 to 90.</p> <p>The researchers hope clinicians can use the model to build their own atlases quickly from their own, potentially small datasets. Dalca is now collaborating with researchers at Massachusetts General Hospital, for instance, to harness a dataset of pediatric brain scans to generate conditional atlases for younger children, which are hard to come by.</p> <p>A big dream is to build one function that can generate conditional atlases for any subpopulation, spanning birth to 90 years old. Researchers could log into a webpage, input an age, sex, diseases, and other parameters, and get an on-demand conditional atlas. “That would be wonderful, because everyone can refer to this one function as a single universal atlas reference,” Dalca says.</p> <p>Another potential application beyond medical imaging is athletic training. Someone could train the function to generate an atlas for, say, a tennis player’s serve motion. The player could then compare new serves against the atlas to see exactly where they kept proper form or where things went wrong.</p> <p>“If you watch sports, it’s usually commenters saying they noticed if someone’s form was off from one time compared to another,” Dalca says. “But you can imagine that it could be much more quantitative than that.”</p> With their model, researchers were able to generate on-demand brain scan templates of various ages (pictured) that can be used in medical-image analysis to guide disease diagnosis. Image courtesy of the researchersResearch, Computer science and technology, Algorithms, Imaging, Machine learning, Health care, Medicine, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering MIT art installation aims to empower a more discerning public With “In Event of Moon Disaster,” the MIT Center for Advanced Virtuality aims to educate the public on deepfakes with an alternative history of the moon landing. Mon, 25 Nov 2019 11:30:01 -0500 Suzanne Day | MIT Open Learning <p>Videos doctored by artificial intelligence, culturally known as “deepfakes,” are being created and shared by the public at an alarming rate. Using advanced computer graphics and audio processing to realistically emulate speech and mannerisms, deepfakes have the power to distort reality, erode truth, and spread misinformation. In a troubling example, researchers around the world have sounded the alarm that they carry significant potential to influence American voters in the 2020 elections.&nbsp;</p> <p>While technology companies race to develop ways to detect and control deepfakes on social media platforms, and lawmakers search for ways to regulate them, a team of artists and computer scientists led by the MIT Center for Advanced Virtuality have designed an art installation to empower and educate the public on how to discern reality from deepfakes on their own.</p> <p>“Computer-based misinformation is a global challenge,” says Fox Harrell, professor of digital media and of artificial intelligence at MIT and director of the MIT Center for Advanced Virtuality. “We are galvanized to make a broad impact on the literacy of the public, and we are committed to using AI not for misinformation, but for truth. We are pleased to bring onboard people such as our new XR Creative Director Francesca Panetta to help further this mission.”</p> <p>Panetta is the director of “In Event of Moon Disaster,” along with co-director Halsey Burgund, a fellow in the MIT Open Documentary Lab. She says, “We hope that our work will spark critical awareness among the public. We want them to be alert to what is possible with today’s technology, to explore their own susceptibility, and to be ready to question what they see and hear as we enter a future fraught with challenges over the question of truth.”</p> <p>With “In Event of Moon Disaster,” which opened Friday at the International Documentary Festival Amsterdam, the team has reimagined the story of the moon landing. Installed in a 1960s-era living room, audiences are invited to sit on vintage furniture surrounded by three screens, including a vintage television set. The screens play an edited array of vintage footage from NASA, taking the audience on a journey from takeoff into space and to the moon. Then, on the center television, Richard Nixon reads a contingency speech written for him by his speech writer, Bill Safire, “in event of moon disaster” which he was to read if the Apollo 11 astronauts had not been able to return to Earth. In this installation, Richard Nixon reads this speech from the Oval Office.</p> <div class="cms-placeholder-content-video"></div> <p>To recreate this moving elegy that never happened, the team used deep learning techniques and the contributions of a voice actor to build the voice of Richard Nixon, producing a synthetic speech working with the Ukranian-based company Respeecher. They also worked with Israeli company Canny AI to use video dialogue replacement techniques to study and replicate the movement of Nixon’s mouth and lips, making it look as though he is reading this very speech from the Oval Office. The resulting video is highly believable, highlighting the possibilities of deepfake technology today.</p> <p>The researchers chose to create a deepfake of this historical moment for a number of reasons: Space is a widely loved topic, so potentially engaging to a wide audience; the piece is apolitical and less likely to alienate, unlike a lot of misinformation; and, as the 1969 moon landing is an event widely accepted by the general public to have taken place, the deepfake elements will be starkly obvious.&nbsp;</p> <p>Rounding out the educational experience, “In Event of Moon Disaster” transparently provides information regarding what is possible with today’s technology, and the goal of increasing public awareness and ability to identify misinformation in the form of deepfakes. This will be in the form of newspapers written especially for the exhibit which detail the making of the installation, how to spot a deepfake, and the most current work being done in algorithmic detection. Audience participants will be encouraged to take this away.</p> <p>"Our goal was to use the most advanced artificial intelligence techniques available today to create the most believable result possible — and then point to it and say, ‘This is fake; here’s how we did it; and here’s why we did it,’” says Burgund.</p> <p>While the physical installation opens in November 2019 in Amsterdam, the team is building a web-based version that is expected to go live in spring 2020.</p> "In Event of Moon Disaster" reimagines the story of the first moon landing as if the Apollo 11 astronauts had not been able to return to Earth. It was created to highlight the concern about computer-based misinformation, or "deepfakes."Photo: Chris BoebelOffice of Open Learning, Augmented and virtual reality, Machine learning, Artificial intelligence, History, Space exploration, Film and Television, Arts, Computer Science and Artificial Intelligence Laboratory (CSAIL), Comparative Media Studies/Writing, NASA, Computer science and technology, Technology and society, History of science, School of Engineering, School of Humanities Arts and Social Sciences MIT conference focuses on preparing workers for the era of artificial intelligence As automation rises in the workplace, speakers explore ways to train students and reskill workers. Fri, 22 Nov 2019 16:35:55 -0500 Rob Matheson | MIT News Office <p>In opening yesterday’s AI and the Work of the Future Congress, MIT Professor Daniela Rus presented diverging views of how artificial intelligence will impact jobs worldwide.</p> <p>By automating certain menial tasks, experts think AI is poised to improve human quality of life, boost profits, and create jobs, said Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science.</p> <p>Rus then quoted a World Economic Forum study estimating AI could help create 133 million new jobs worldwide over the next five years. Juxtaposing this optimistic view, however, she noted a recent survey that found about two-thirds of Americans believe machines will soon rob humans of their careers. “So, who is right? The economists, who predict greater productivity and new jobs? The technologists, who dream of creating better lives? Or the factory line workers who worry about unemployment?” Rus asked. “The answer is, probably all of them.”</p> <p>Her remarks kicked off an all-day conference in Kresge Auditorium that convened experts from industry and academia for panel discussions and informal talks about preparing humans of all ages and backgrounds for a future of AI automation in the workplace. The event was co-sponsored by CSAIL, the MIT Initiative on the Digital Economy (IDE), and the MIT Work of the Future Task Force, an Institute-wide effort launched in 2018 that aims to understand and shape the evolution of jobs during an age of innovation.</p> <p>Presenters were billed as “leaders and visionaries” rigorously measuring technological impact on enterprise, government, and society, and generating solutions. Apart from Rus, who also moderated a panel on dispelling AI myths, speakers included Chief Technology Officer of the United States Michael Kratsios; executives from Amazon, Nissan, Liberty Mutual, IBM, Ford, and Adobe; venture capitalists and tech entrepreneurs; representatives of nonprofits and colleges; journalists who cover AI issues; and several MIT professors and researchers.</p> <p>Rus, a self-described “technology optimist,” drove home a point that echoed throughout all discussions of the day: AI doesn’t automate jobs<em>,&nbsp;</em>it automates tasks. Rus quoted a recent McKinsey Global Institute study that estimated 45 percent of tasks that humans are paid to do can now be automated. But, she said, humans can adapt to work in concert with AI —&nbsp;meaning job tasks may change dramatically, but jobs may not disappear entirely. “If we make the right choices and the right investments, we can ensure that those benefits get distributed widely across our workforce and our planet,” Rus said.</p> <p><strong>Avoiding the “job-pocalypse”</strong></p> <p>Common topics throughout the day included reskilling veteran employees to use AI technologies; investing heavily in training young students in AI through tech apprenticeships, vocational programs, and other education initiatives; ensuring workers can make livable incomes; and promoting greater inclusivity in tech-based careers. The hope is to avoid, as one speaker put it, a “job-pocalypse,” where most humans will lose their jobs to machines.</p> <p>A panel moderated by David Mindell, the Dibner Professor of the History of Engineering and Manufacturing and a professor of aeronautics and astronautics, focused on how AI technologies are changing workflow and skills, especially within sectors resistant to change. Mindell asked panelists for specific examples of implementing AI technologies into their companies.</p> <p>In response, David Johnson, vice president of production and engineering at Nissan, shared an anecdote about pairing an MIT student with a 20-year employee in developing AI methods to autonomously predict car-part quality. In the end, the veteran employee became immersed in the technology and is now using his seasoned expertise to deploy it in other areas, while the student learned more about the technology’s real-world applications. “Only through this synergy, when you purposely pair these people with a common goal, can you really drive the skills forward … for mass new technology adoption and deployment,” Johnson said.</p> <p>In a panel about shaping public policies to ensure technology benefits society — which included U.S. CTO Kratsios — moderator Erik Brynjolfsson, director of IDE and a professor in the MIT Sloan School of Management, got straight to the point: “People have been dancing around this question: Will AI destroy jobs?”</p> <p>“Yes, it will — but not to the extent that people presume,” replied MIT Institute Professor Daron Acemoglu. AI, he said, will mostly automate mundane operations in white-collar jobs, which will free up humans to refine their creative, interpersonal, and other high-level skills for new roles. Humans, he noted, also won’t be stuck doing low-paying jobs, such as labeling data for machine-learning algorithms.</p> <p>“That’s not the future of work,” he said. “The hope is we use our amazing creativity and all these wonderful and technological platforms to create meaningful jobs in which humans can use their flexibility, creativity, and all the things … machines won’t be able to do — at least in the next 100 years.”</p> <p>Kratsios emphasized a need for public and private sectors to collaborate to reskill workers. Specifically, he pointed to the Pledge to the America’s Worker, the federal initiative that now has 370 U.S. companies committed to retraining roughly 4 million American workers for tech-based jobs over the next five years.</p> <p>Responding to an audience question about potential public policy changes, Kratsios echoed sentiments of many panelists, saying education policy should focus on all levels of education, not just college degrees. “A vast majority of our policies, and most of our departments and agencies, are targeted toward coaxing people toward a four-year degree,” Kratsios said. “There are incredible opportunities for Americans to live and work and do fantastic jobs that don’t require four-year degrees. So, [a change is] thinking about using the same pool of resources to reskill, or retrain, or [help students] go to vocational schools.”</p> <p><strong>Inclusivity and underserved populations</strong></p> <p>Entrepreneurs at the event explained how AI can help create diverse workforces. For instance, a panel about creating economically and geographically diverse workforces, moderated by Devin Cook, executive producer of IDE’s Inclusive Innovation Challenge, included Radha Basu, who founded Hewlett Packard’s operations in India in the 1970s. In 2012, Basu founded iMerit, which hires employees — half are young women and more than 80 percent come from underserved populations —&nbsp;to provide AI services for computer vision, machine learning, and other applications.</p> <p>A panel hosted by Paul Osterman, co-director of the MIT Sloan Institute for Work and Employment Research and an MIT Sloan professor, explored how labor markets are changing in the face of technological innovations. Panelist Jacob Hsu is CEO of Catalyte, which uses an AI-powered assessment test to predict a candidate’s ability to succeed as a software engineer, and hires and trains those who are most successful. Many of their employees don’t have four-year degrees, and their ages range from anywhere from 17 to 72.</p> <p>A “media spotlight” session, in which journalists discussed their reporting on the impact of AI on the workplace and the world, included David Fanning, founder and producer of the investigative documentary series FRONTLINE, which recently ran a documentary titled “In the Era of AI.” Fanning briefly discussed how, during his investigations, he learned about the profound effect AI is having on workplaces in the developing world, which rely heavily on manual labor, such as manufacturing lines.</p> <p>“What happens as automation expands, the manufacturing ladder that was opened to people in developing countries to work their way out of rural poverty — all that manufacturing gets replaced by machines,” Fanning said. “Will we end up across the world with people who have nowhere to go? Will they become the new economic migrants we have to deal with in the age of AI?”</p> <p><strong>Education: The great counterbalance</strong></p> <p>Elisabeth Reynolds, executive director for the MIT Task Force on the Work of the Future and of the MIT Industrial Performance Center, and Andrew McAfee, co-director of IDE and a principal research scientist at the MIT Sloan School of Management, closed out the conference and discussed next steps.</p> <p>Reynolds said the MIT Task Force on the Work of the Future, over the next year, will further study how AI is being adopted, diffused, and implemented across the U.S., as well as issues of race and gender bias in AI. In closing, she charged the audience with helping tackle the issues: “I would challenge everybody here to say, ‘What on Monday morning is [our] organization doing in respect to this agenda?’”&nbsp;</p> <p>In paraphrasing economist Robert Gordon, McAfee reemphasized the shifting nature of jobs in the era of AI: “We don’t have a job quantity problem, we have a job quality problem.”</p> <p>AI may generate more jobs and company profits, but it may also have numerous negative effects on employees. Proper education and training are keys to ensuring the future workforce is paid well and enjoys a high quality of life, he said: “Tech progress, we’ve known for a long time, is an engine of inequality. The great counterbalancing force is education.”</p> Daniela Rus (far right), director of the Computer Science and Artificial Intelligence Laboratory (CSAIL), moderated a panel on dispelling the myths of AI technologies in the workplace. The AI and the Work of the Future Congress was co-organized by CSAIL, the MIT Initiative on the Digital Economy, and the MIT Work of the Future Task Force.Image: Andrew KubicaResearch, Computer science and technology, Algorithms, Computer Science and Artificial Intelligence Laboratory (CSAIL), Sloan School of Management, Technology and society, Jobs, Economics, Policy, Artificial intelligence, Machine learning, Innovation and Entrepreneurship (I&E), Business and management, Manufacturing, Careers, Special events and guest speakers Bot can beat humans in multiplayer hidden-role games Using deductive reasoning, the bot identifies friend or foe to ensure victory over humans in certain online games. Tue, 19 Nov 2019 23:59:59 -0500 Rob Matheson | MIT News Office <p>MIT researchers have developed a bot equipped with artificial intelligence that can beat human players in tricky online multiplayer games where player roles and motives are kept secret.</p> <p>Many gaming bots have been built to keep up with human players. Earlier this year, a team from Carnegie Mellon University developed the world’s first bot that can beat professionals in multiplayer poker. DeepMind’s AlphaGo made headlines in 2016 for besting a professional Go player. Several bots have also been built to beat professional chess players or join forces in cooperative games such as online capture the flag. In these games, however, the bot knows its opponents and teammates from the start.</p> <p>At the Conference on Neural Information Processing Systems next month, the researchers will present DeepRole, the first gaming bot that can win online multiplayer games in which the participants’ team allegiances are initially unclear. The bot is designed with novel “deductive reasoning” added into an AI algorithm commonly used for playing poker. This helps it reason about partially observable actions, to determine the probability that a given player is a teammate or opponent. In doing so, it quickly learns whom to ally with and which actions to take to ensure its team’s victory.</p> <p>The researchers pitted DeepRole against human players in more than 4,000 rounds of the online game “The Resistance: Avalon.” In this game, players try to deduce their peers’ secret roles as the game progresses, while simultaneously hiding their own roles. As both a teammate and an opponent, DeepRole consistently outperformed human players.</p> <p>“If you replace a human teammate with a bot, you can expect a higher win rate for your team. Bots are better partners,” says first author Jack Serrino ’18, who majored in electrical engineering and computer science at MIT and is an avid online “Avalon” player.</p> <p>The work is part of a broader project to better model how humans make socially informed decisions. Doing so could help build robots that better understand, learn from, and work with humans.</p> <p>“Humans learn from and cooperate with others, and that enables us to achieve together things that none of us can achieve alone,” says co-author Max Kleiman-Weiner, a postdoc in the Center for Brains, Minds and Machines and the Department of Brain and Cognitive Sciences at MIT, and at Harvard University. “Games like ‘Avalon’ better mimic the dynamic social settings humans experience in everyday life. You have to figure out who’s on your team and will work with you, whether it’s your first day of kindergarten or another day in your office.”</p> <p>Joining Serrino and Kleiman-Weiner on the paper are David C. Parkes of Harvard and Joshua B. Tenenbaum, a professor of computational cognitive science and a member of MIT’s Computer Science and Artificial Intelligence Laboratory and the Center for Brains, Minds and Machines.</p> <p><strong>Deductive bot</strong></p> <p>In “Avalon,” three players are randomly and secretly assigned to a “resistance” team and two players to a “spy” team. Both spy players know all players’ roles. During each round, one player proposes a subset of two or three players to execute a mission. All players simultaneously and publicly vote to approve or disapprove the subset. If a majority approve, the subset secretly determines whether the mission will succeed or fail. If two “succeeds” are chosen, the mission succeeds; if one “fail” is selected, the mission fails. Resistance players must always choose to succeed, but spy players may choose either outcome. The resistance team wins after three successful missions; the spy team wins after three failed missions.</p> <p>Winning the game basically comes down to deducing who is resistance or spy, and voting for your collaborators. But that’s actually more computationally complex than playing chess and poker. “It’s a game of imperfect information,” Kleiman-Weiner says. “You’re not even sure who you’re against when you start, so there’s an additional discovery phase of finding whom to cooperate with.”</p> <p>DeepRole uses a game-planning algorithm called “counterfactual regret minimization” (CFR) — which learns to play a game by repeatedly playing against itself — augmented with deductive reasoning. At each point in a game, CFR looks ahead to create a decision “game tree” of lines and nodes describing the potential future actions of each player. Game trees represent all possible actions (lines) each player can take at each future decision point. In playing out potentially billions of game simulations, CFR notes which actions had increased or decreased its chances of winning, and iteratively revises its strategy to include more good decisions. Eventually, it plans an optimal strategy that, at worst, ties against any opponent.</p> <p>CFR works well for games like poker, with public actions — such as betting money and folding a hand — but it struggles when actions are secret. The researchers’ CFR combines public actions and consequences of private actions to determine if players are resistance or spy.</p> <p>The bot is trained by playing against itself as both resistance and spy. When playing an online game, it uses its game tree to estimate what each player is going to do. The game tree represents a strategy that gives each player the highest likelihood to win as an assigned role. The tree’s nodes contain “counterfactual values,” which are basically estimates for a payoff that player receives if they play that given strategy.</p> <p>At each mission, the bot looks at how each person played in comparison to the game tree. If, throughout the game, a player makes enough decisions that are inconsistent with the bot’s expectations, then the player is probably playing as the other role. Eventually, the bot assigns a high probability for each player’s role. These probabilities are used to update the bot’s strategy to increase its chances of victory.</p> <p>Simultaneously, it uses this same technique to estimate how a third-person observer might interpret its own actions. This helps it estimate how other players may react, helping it make more intelligent decisions. “If it’s on a two-player mission that fails, the other players know one player is a spy. The bot probably won’t propose the same team on future missions, since it knows the other players think it’s bad,” Serrino says.</p> <p><strong>Language: The next frontier</strong></p> <p>Interestingly, the bot did not need to communicate with other players, which is usually a key component of the game. “Avalon” enables players to chat on a text module during the game. “But it turns out our bot was able to work well with a team of other humans while only observing player actions,” Kleiman-Weiner says. “This is interesting, because one might think games like this require complicated communication strategies.”</p> <p>“I was thrilled to see this paper when it came out,” says Michael Bowling, a professor at the University of Alberta whose research focuses, in part, on training computers to play games. “It is really exciting seeing the ideas in DeepStack see broader application outside of poker. [DeepStack has] been so central to AI in chess and Go to situations of imperfect information. But I still wasn't expecting to see it extended so quickly into the situation of a hidden role game like Avalon. Being able to navigate a social deduction scenario, which feels so quintessentially human, is a really important step. There is still much work to be done, especially when the social interaction is more open ended, but we keep seeing that many of the fundamental AI algorithms with self-play learning can go a long way.”</p> <p>Next, the researchers may enable the bot to communicate during games with simple text, such as saying a player is good or bad. That would involve assigning text to the correlated probability that a player is resistance or spy, which the bot already uses to make its decisions. Beyond that, a future bot might be equipped with more complex communication capabilities, enabling it to play language-heavy social-deduction games — such as a popular game “Werewolf” —which involve several minutes of arguing and persuading other players about who’s on the good and bad teams.</p> <p>“Language is definitely the next frontier,” Serrino says. “But there are many challenges to attack in those games, where communication is so key.”</p> DeepRole, an MIT-invented gaming bot equipped with “deductive reasoning,” can beat human players in tricky online multiplayer games where player roles and motives are kept secret.Research, Computer science and technology, Algorithms, Video games, Artificial intelligence, Machine learning, Language, Computer Science and Artificial Intelligence Laboratory (CSAIL), Brain and cognitive sciences, Electrical Engineering & Computer Science (eecs), School of Engineering Students push to speed up artificial intelligence adoption in Latin America To help the region catch up, students organize summit to bring Latin policymakers and researchers to MIT. Tue, 19 Nov 2019 16:30:01 -0500 Kim Martineau | MIT Quest for Intelligence <p>Omar Costilla Reyes reels off all the ways that artificial intelligence might benefit his native Mexico. It could raise living standards, he says, lower health care costs, improve literacy and promote greater transparency and accountability in government.</p> <p>But Mexico, like many of its Latin American neighbors, has failed to invest as heavily in AI as other developing countries. That worries <a href="">Costilla Reyes</a>, a postdoc at MIT’s Department of Brain and Cognitive Sciences.</p> <p>To give the region a nudge, Costilla Reyes and three other MIT graduate students — <a href="" target="_blank">Guillermo Bernal</a>, <a href="">Emilia Simison</a> and <a href="">Pedro Colon-Hernandez</a> — have spent the last six months putting together a three-day event that will &nbsp;bring together policymakers and AI researchers in Latin America with AI researchers in the United States. The <a href="">AI Latin American sumMIT</a> will take place in January at the <a href="">MIT Media Lab</a>.</p> <p>“Africa is getting lots of support — Africa will eventually catch up,” Costilla Reyes says. “You don’t see anything like that in Latin America, despite the potential for AI to move the region forward socially and economically.”</p> <p><strong>Four paths to MIT and research inspired by AI</strong></p> <p>Each of the four students took a different route to MIT, where AI plays a central role in their work — on the brain, voice assistants, augmented creativity and politics. Costilla Reyes got his first computer in high school, and though it had only dial-up internet access, it exposed him to a world far beyond his home city of Toluca. He studied for a PhD &nbsp;at the University of Manchester, where he developed an <a href="">AI system</a> with applications in security and health to identify individuals by their gait. At MIT, Costilla Reyes is building computational models of how firing neurons in the brain produce memory and cognition, information he hopes can also advance AI.</p> <p>After graduating from a vocational high school in El Salvador, Bernal moved in with relatives in New Jersey and studied English at a nearby community college. He continued on to Pratt Institute, where he learned to incorporate Python into his design work. Now at the MIT Media Lab, he’s developing interactive storytelling tools like <a href="">PaperDreams</a> that uses AI to help people unlock their creativity. His work recently won a <a href="">Schnitzer Prize</a>.&nbsp;</p> <p>Simison came to MIT to study for a PhD in political science after her professors at Argentina’s University Torcuato Di Tella encouraged her to continue her studies in the United States. She is currently using text analysis tools to mine archival records in Brazil and Argentina to understand the role that political parties and unions played under the last dictatorships in both countries.</p> <p>Colon-Hernandez grew up in Puerto Rico fascinated with video games. A robotics class in high school inspired him to build a computer to play video games of his own, which led to a degree in computer engineering at the University of Puerto Rico at Mayagüez.&nbsp;After helping a friend with a project at MIT Lincoln Laboratory, Colon-Hernandez applied to a summer research program at MIT, and later, the MIT Media Lab’s graduate program. He’s currently working on intelligent voice assistants.</p> <p>It’s hard to generalize about a region as culturally diverse and geographically vast as Latin America, stretching from Mexico and the Caribbean to the tip of South America. But protests, violence and reports of entrenched corruption have dominated the news for years, and the average income per person has been <a href="">falling</a> with respect to the United States since the 1950s. All four students see AI as a means to bring stability and greater opportunity to their home countries.</p> <p><strong>AI with a humanitarian agenda</strong></p> <p>The idea to bring Latin American policymakers to MIT was hatched last December, at the world’s premier conference for AI research, <a href="">NeurIPS</a>. The organizers of NeurIPS had launched several new workshops to promote diversity in response to growing criticism of the exclusion of women and minorities in tech. At <a href="">Latinx,</a> a workshop for Latin American students, Costilla Reyes met Colon-Hernandez, who was giving a talk on voice-activated wearables. A few hours later they began drafting a plan to bring a Latinx-style event to MIT.</p> <p>Back in Cambridge, they found support from <a href="">Armando Solar-Lezama</a>, a <a href="">native of Mexico</a> and a professor at MIT’s <a href="">Department of Electrical Engineering and Computer Science</a>. They also began knocking on doors for funding, securing an initial $25,000 grant from MIT’s <a href="">Institute Community and Equity Office</a>. Other graduate students joined the cause, including, and together they set out to recruit speakers, reserve space at the MIT Media Lab and design a website. RIMAC, the MIT-IBM Watson AI Lab, X Development, and Facebook have all since offered support for the event.</p> <p>Unlike other AI conferences, this one has a practical bent, with themes that echo many of the UN Sustainable Development Goals: to end extreme poverty, develop quality education, create fair and transparent institutions, address climate change and provide good health.</p> <p>The students have set similarly concrete goals for the conference, from mapping the current state of AI-adoption across Latin America to outlining steps policymakers can take to coordinate efforts. U.S. researchers will offer tutorials on open-source AI platforms like TensorFlow and scikit-learn for Python, and the students are continuing to raise money to fly 10 of their counterparts from Latin America to attend the poster session.</p> <p>“We reinvent the wheel so much of the time,” says Simison. “If we can motivate countries to integrate their efforts, progress could move much faster.”</p> <p>The potential rewards are high. A <a href="">2017 report</a> by Accenture estimated that if AI were integrated into South America’s top five economies — Argentina, Brazil, Chile, Colombia and Peru — which generate about 85 percent of the continent’s economic output, they could each add up to 1 percent to their annual growth rate.</p> <p>In developed countries like the U.S. and in Europe, AI is sometimes viewed apprehensively for its potential to eliminate jobs, spread misinformation and perpetuate bias and inequality. But the risk of not embracing AI, especially in countries that are already lagging behind economically, is potentially far greater, says Solar-Lezama. “There’s an urgency to make sure these countries have a seat at the table and can benefit from what will be one of the big engines for economic development in the future,” he says.</p> <p>Post-conference deliverables include a set of recommendations for policymakers to move forward. “People are protesting across the entire continent due to the marginal living conditions that most face,” says Costilla Reyes. “We believe that AI plays a key role now, and in the future development of the region, if it’s used in the right way.”</p> “We believe that AI plays a key role now, and in the future development of the region, if it’s used in the right way,” says Omar Costilla Reyes, one of four MIT graduate students working to help Latin America adopt artificial intelligence technologies. Pictured here (left to right) are Costilla Reyes, Emilia Simison, Pedro Antonio Colon-Hernandez, and Guillermo Bernal.Photo: Kim MartineauQuest for Intelligence, Electrical engineering and computer science (EECS), Media Lab, Brain and cognitive sciences, Lincoln Laboratory, MIT-IBM Watson AI Lab, School of Engineering, School of Science, School of Humanities Arts and Social Sciences, Artificial intelligence, Computer science and technology, Technology and society, Machine learning, Software, Algorithms, Political science, Latin America Predicting people&#039;s driving personalities System from MIT CSAIL sizes up drivers as selfish or selfless. Could this help self-driving cars navigate in traffic? Mon, 18 Nov 2019 16:10:01 -0500 Adam Conner-Simons | Rachel Gordon | MIT CSAIL <p>Self-driving cars are coming. But for all their fancy sensors and intricate data-crunching abilities, even the most cutting-edge cars lack something that (almost) every 16-year-old with a learner’s permit has: social awareness.<br /> <br /> While autonomous technologies have improved substantially, they still ultimately view the drivers around them as obstacles made up of ones and zeros, rather than human beings with specific intentions, motivations, and personalities.<br /> <br /> But recently a team led by researchers at MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) has been exploring whether self-driving cars can be programmed to classify the social personalities of other drivers, so that they can better predict what different cars will do — and, therefore, be able to drive more safely among them.<br /> <br /> In a new paper, the scientists integrated tools from social psychology to classify driving behavior with respect to how selfish or selfless a particular driver is.<br /> <br /> Specifically, they used something called social value orientation (SVO), which represents the degree to which someone is selfish (“egoistic”) versus altruistic or cooperative (“prosocial”). The system then estimates drivers’ SVOs to create real-time driving trajectories for self-driving cars.</p> <p>Testing their algorithm on the tasks of merging lanes and making unprotected left turns, the team showed that they could better predict the behavior of other cars by a factor of 25 percent. For example, in the left-turn simulations their car knew to wait when the approaching car had a more egoistic driver, and to then make the turn when the other car was more prosocial.</p> <p>While not yet robust enough to be implemented on real roads, the system could have some intriguing use cases, and not just for the cars that drive themselves. Say you’re a human driving along and a car suddenly enters your blind spot — the system could give you a warning in the rear-view mirror that the car has an aggressive driver, allowing you to adjust accordingly. It could also allow self-driving cars to actually learn to exhibit more human-like behavior that will be easier for human drivers to understand.<br /> <br /> “Working with and around humans means figuring out their intentions to better understand their behavior,” says graduate student Wilko Schwarting, who was lead author on the new paper that will be published this week in the latest issue of the <em>Proceedings of the National Academy of Sciences</em>. “People’s tendencies to be collaborative or competitive often spills over into how they behave as drivers. In this paper, we sought to understand if this was something we could actually quantify.”<br /> <br /> Schwarting’s co-authors include MIT professors Sertac Karaman and Daniela Rus, as well as research scientist Alyssa Pierson and former CSAIL postdoc Javier Alonso-Mora.<br /> <br /> A central issue with today’s self-driving cars is that they’re programmed to assume that all humans act the same way. This means that, among other things, they’re quite conservative in their decision-making at four-way stops and other intersections.<br /> <br /> While this caution reduces the chance of fatal accidents, it also <a href="">creates bottlenecks</a> that can be frustrating for other drivers, not to mention hard for them to understand. (This may be why the majority of traffic incidents have involved <a href="">getting rear-ended by impatient drivers</a>.)<br /> <br /> “Creating more human-like behavior in autonomous vehicles (AVs) is fundamental for the safety of passengers and surrounding vehicles, since behaving in a predictable manner enables humans to understand and appropriately respond to the AV’s actions,” says Schwarting.<br /> <br /> To try to expand the car’s social awareness, the CSAIL team combined methods from social psychology with game theory, a theoretical framework for conceiving social situations among competing players.<br /> <br /> The team modeled road scenarios where each driver tried to maximize their own utility and analyzed their “best responses” given the decisions of all other agents. Based on that small snippet of motion from other cars, the team’s algorithm could then predict the surrounding cars’ behavior as cooperative, altruistic, or egoistic — grouping the first two as “prosocial.” People’s scores for these qualities rest on a continuum with respect to how much a person demonstrates care for themselves versus care for others.<br /> <br /> In the merging and left-turn scenarios, the two outcome options were to either let somebody merge into your lane (“prosocial”) or not (“egoistic”). The team’s results showed that, not surprisingly, merging cars are deemed more competitive than non-merging cars.<br /> <br /> The system was trained to try to better understand when it’s appropriate to exhibit different behaviors. For example, even the most deferential of human drivers knows that certain types of actions — like making a lane change in heavy traffic — require a moment of being more assertive and decisive.<br /> <br /> For the next phase of the research, the team plans to work to apply their model to pedestrians, bicycles, and other agents in driving environments. In addition, they will be investigating other robotic systems acting among humans, such as household robots, and integrating SVO into their prediction and decision-making algorithms. Pierson says that the ability to estimate SVO distributions directly from observed motion, instead of in laboratory conditions, will be important for fields far beyond autonomous driving.<br /> <br /> “By modeling driving personalities and incorporating the models mathematically using the SVO in the decision-making module of a robot car, this work opens the door to safer and more seamless road-sharing between human-driven and robot-driven cars,” says Rus.<br /> <br /> The research was supported by the Toyota Research Institute for the MIT team. The Netherlands Organization for Scientific Research provided support for the specific participation of Mora.</p> In lane-merging scenarios, a system developed at MIT could distinguish between altruistic and egoistic driving behavior. Image courtesy of the researchers.Electrical engineering and computer science (EECS), School of Engineering, Robotics, Robots, Artificial intelligence, Automobiles, Autonomous vehicles, Research, Computer vision, Transportation, Machine learning, Behavior, Computer Science and Artificial Intelligence Laboratory (CSAIL) Visualizing an AI model’s blind spots New tool highlights what generative models leave out when reconstructing a scene. Fri, 08 Nov 2019 13:30:02 -0500 Kim Martineau | MIT Quest for Intelligence <p>Anyone who has spent time on social media has probably noticed that GANs, or generative adversarial networks, have become remarkably good at drawing faces. They can predict what you’ll look like when you’re old and&nbsp;what you’d look like as a celebrity. But ask a GAN to draw scenes from the larger world and things get weird.</p> <p>A new&nbsp;<a href="">demo</a>&nbsp;by the&nbsp;<a href="">MIT-IBM Watson AI Lab</a> reveals what a model trained on scenes of churches and monuments decides to leave out when it draws its own version of, say, the Pantheon in Paris, or the Piazza di Spagna in Rome. The larger study,&nbsp;<a href="">Seeing What a GAN Cannot Generate</a>, was presented at the&nbsp;<a href="">International Conference on Computer Vision</a> last week.</p> <p>“Researchers typically focus on characterizing and improving what a machine-learning system can&nbsp;do&nbsp;— what it pays attention to, and how particular inputs lead to particular outputs,” says&nbsp;<a href="">David Bau</a>, a graduate student at MIT’s&nbsp;Department of Electrical Engineering and Computer Science&nbsp;and&nbsp;Computer Science and Artificial Science Laboratory (CSAIL). “With this work, we hope researchers will pay as much attention to characterizing the data that these systems ignore.”&nbsp;</p> <p>In a GAN, a pair of neural networks work together to create hyper-realistic images patterned after examples they’ve been given. Bau became interested in GANs as a way of peering inside black-box neural nets to understand the reasoning behind their decisions. An earlier tool developed with his advisor, MIT Professor&nbsp;<a href="">Antonio Torralba</a>, and IBM researcher&nbsp;<a href="">Hendrik Strobelt</a>, made it possible to identify the clusters of artificial neurons responsible for organizing the image into real-world categories like doors, trees, and clouds. A related tool,&nbsp;<a href="">GANPaint</a>, lets amateur artists add and remove those features from photos of their own.&nbsp;</p> <p>One day, while helping an artist use GANPaint, Bau hit on a problem. “As usual, we were chasing the numbers, trying to optimize numerical reconstruction loss to reconstruct the photo,” he says. “But my advisor has always encouraged us to look beyond the numbers and scrutinize the actual images. When we looked, the phenomenon jumped right out: People were getting dropped out selectively.”</p> <p>Just as GANs and other neural nets find patterns in heaps of data, they ignore patterns, too.&nbsp;Bau and his colleagues trained different types of GANs on indoor and outdoor scenes. But no matter where the pictures were taken, the GANs consistently omitted important details like people, cars, signs, fountains, and pieces of furniture, even when those objects appeared prominently in the image. In one&nbsp;<a href="">GAN reconstruction</a>, a pair of newlyweds kissing on the steps of a church are ghosted out, leaving an eerie wedding-dress texture on the cathedral door.</p> <p>“When GANs encounter objects they can’t generate, they seem to imagine what the scene would look like without them,” says Strobelt. “Sometimes people become bushes or disappear entirely into the building behind them.”</p> <p>The researchers suspect that machine laziness could&nbsp;be to blame; although a GAN is trained to create convincing images, it may learn it's easier to focus on buildings and landscapes and skip harder-to-represent people and cars. Researchers have long known that GANs have a tendency to overlook some statistically meaningful details. But this may be&nbsp;the first study to show that state-of-the-art GANs can systematically omit entire classes of objects within an image.</p> <p>An AI that drops some objects from its representations may achieve its&nbsp;numerical goals while missing the details most important to us humans, says Bau. As engineers turn to GANs to generate synthetic images to train automated systems like self-driving cars, there’s a danger that people, signs, and other critical information could be dropped without humans realizing. It shows why model performance shouldn’t be measured by accuracy alone, says Bau. “We need to understand what the networks are and aren’t doing to make sure they are making the choices we want them to make.”</p> <p>Joining Bau on the study are Jun-Yan Zhu, Jonas Wulff, William Peebles, and Torralba, of MIT; Strobelt of IBM; and Bolei Zhou of the Chinese University of Hong Kong.</p> A new tool reveals what AI models leave out in recreating a scene. Here, a GAN, or generative adversarial network, has dropped the pair of newlyweds from its reconstruction (right) of the photo it was asked to draw (left). Image courtesy of the researchers.Quest for Intelligence, MIT-IBM Watson AI Lab, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical engineering and computer science (EECS), School of Engineering, School of Science, Artificial intelligence, Computer science and technology, Machine learning, Software, Algorithms Driving toward a healthier planet Materials Day speaker Brian Storey describes how the Toyota Research Institute is embracing machine learning to advance the use of electric vehicles. Thu, 07 Nov 2019 13:20:01 -0500 Denis Paiste | Materials Research Laboratory <p>With 100 million Toyota vehicles on the planet emitting greenhouse gases at a rate roughly comparable to those of France, the Toyota Motor Corporation has set a goal of reducing all tailpipe emissions by 90 percent by 2050, according to Brian Storey, who directs the <a href="" target="_blank">Toyota Research Institute</a> (TRI) Accelerated Materials Design and Discovery program from its Kendall Square office in Cambridge, Massachusetts. He gave the keynote address at the MIT Materials Research Laboratory's Materials Day Symposium on Oct. 9.</p> <p>“A rapid shift from the traditional vehicle to electric vehicles has started,” Storey says. “And we want to enable that to happen at a faster pace.”</p> <p>“Our role at TRI is to develop tools for accelerating the development of emissions-free vehicles,” Storey said. He added that machine learning is helping to speed up those innovations, but the challenges are very great, so his team has to be a little humble about what it can actually accomplish.</p> <p>Electrification is just one of four “disrupters” to the automotive industry, which are often abbreviated CASE (connected, autonomous, shared, electric). “It’s a disrupter to the industry because Toyota has decades of experience of optimizing the combustion engine,” Storey said. “We know how to do it; it’s reliable; it’s affordable; it lasts forever. Really, the heart of the Toyota brand is the quality of the combustion engine and transmission.”</p> <p>Storey stated that as society shifts toward electrification — battery or fuel cell vehicles — new capability, technology, and know-how is needed. Storey says “while Toyota has a lot of experience in these areas, we still need to move faster if we are going to make this kind of transition.”</p> <p>To help with that acceleration,&nbsp;Toyota Research Institute&nbsp;is providing $10 million a year to support research of approximately 125 professors, postdocs, and graduate students at 10 academic institutions. About $2 million a year of that research is being&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">done at MIT</a>. Storey is also a professor of mechanical engineering at&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">Olin College of Engineering</a>.</p> <p>For example, the Battery Evaluation and Early Prediction (BEEP) project, which is a TRI collaboration with MIT and Stanford University, aims to expand the value of lithium-based battery systems. In experiments, many batteries are charged and discharged at the same time. “From that data alone, the charge and discharge data, we can extract features. It’s super practical because we get the data. We extract features from the data, and we can correlate those features with lifetime,” Storey explained.</p> <p>The traditional way of testing whether a battery is going to last for a thousand cycles is to cycle it for a thousand times. Storey noted that if each cycle takes one hour, one battery requires 1,000 hours of testing. “What we want to do is bring that time way back, and so our goal is to able to do it in five — to cycle five times and get a good estimate of what the battery’s lifetime would be at 1,000 cycles, doing it purely from data,” Storey said.</p> <p>Published&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">results</a>&nbsp;in&nbsp;<em>Nature Energy</em>&nbsp;in March 2019 show just a 4.9 percent test error using data in classifying lithium-ion batteries from the first five charge/discharge cycles.</p> <p>“This is a nice capability because it actually allows acceleration in testing,” Storey noted. “It’s using machine learning, but it’s really using it at the device scale, the ‘as-manufactured’ battery.”</p> <p>The cloud-based battery evaluation software system allows TRI to collaborate easily with colleagues at MIT, Stanford, and Toyota’s home base in Japan, he said.</p> <p>Program researchers operate it in a closed-loop, semi-autonomous way, where the computer decides and executes the next-best experiment. The system finds charging policies that are better than ones that have been published in the literature, and it finds them rapidly. “The key to this is the early prediction model, because if we want to predict the lifetime, we don’t have to do the whole test.” Storey added that the closed-loop testing “pulls the scientist up a level in terms of what questions they can ask.”</p> <p>TRI would like to use this closed-loop battery evaluation system to optimize the first charge/discharge cycle a battery goes through, which is called formation cycling. “It’s like caring for the battery when it’s a baby,” Storey explained. “How you do those first cycles actually sets it up for the rest of its life. It’s a real black art, and how do you optimize this process?”</p> <p>TRI’s long-term goal is to improve battery durability so that, from the consumer point of view, the battery capacity never goes down. Storey emphasized “we want the battery in the car to just last forever.”</p> <p>Storey notes TRI is also conducting two other research projects, AI-Assisted Catalysis Experimentation (ACE) with CalTech to improve catalysts for fuel cell vehicles such as Toyota’s Mirai, and a materials synthesis project, mostly within TRI, to use machine learning to identify whether or not the new materials predicted on the computer are likely to be synthesizable.</p> <p>For the materials synthesis project, TRI began with the phase diagrams of materials. “You build up a network of every material you’ve got in the computational database and look at features of the network. Believing that somehow those materials are connected to other materials through the relationship in this network provides a prediction of synthesizability,” Storey explained. “The way you can train the algorithm is by looking in the historical record of when certain materials were synthesized. You can virtually roll the clock back, pretending to know only what you knew in 1980, and use that to train your algorithm.” A&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">report</a>&nbsp;on the materials synthesis network was published in May in&nbsp;<em>Nature Communications</em>.</p> <p>TRI is collaborating with Lawrence Berkeley National Laboratory (LBNL) and MIT Professor Martin Z. Bazant on a project that couples highly detailed mechanics of battery particles revealed through 4D scanning tunneling electron microscopy with a continuum model that captures larger-scale materials properties. “This program figures out the reaction kinetics and thermodynamics at a continuum scale, which is otherwise unknown,” Storey said.</p> <p>“We’re putting our software tools online, so over the coming year many of these tools will start becoming available,” Storey explained. Hosted by LBNL, the Propnet materials database is already accessible to internal collaborators. Matscholar is accessible through&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">GitHub</a>. Both projects were funded by TRI.</p> <p>“Our dream, which is a work in progress, is to have a system architecture that overlies all these projects and can start to tie them together,” Storey said. “We are creating a system that’s built for machine learning from the start, allows for diverse data, allows for systems and atom-scale measurements, and is capable of this idea of AI-driven feedback and autonomy. The idea is that you launch the system and it runs on its own, and everything lives in the cloud to enable collaboration.”</p> Brian Storey, director of accelerated materials design and discovery at Toyota Research Institute, speaks at the MIT MRL Materials Day Symposium.Photo: Denis Paiste/Materials Research LaboratoryMaterials Research Laboratory, Machine learning, Artificial intelligence, Greenhouse gases, Electric vehicles, Transportation, Batteries, Emissions, Special events and guest speakers Machine learning shows no difference in angina symptoms between men and women Finding could help overturn the prevailing notion that men and women experience angina differently. Wed, 06 Nov 2019 12:10:01 -0500 Becky Ham | MIT Media Lab <div> <p>The symptoms of angina — the pain that occurs in coronary artery disease — do not differ substantially between men and women, according to the results of an unusual new clinical trial led by MIT researchers.</p> <p>The findings could help overturn the prevailing notion that men and women experience angina differently, with men experiencing “typical angina” — pain-type sensations in the chest, for instance — and women experiencing “atypical angina” symptoms such as shortness of breath and pain-type sensations in the non-chest areas such as the arms, back, and shoulders. Instead, it appears that men and women’s symptoms are largely the same, say Karthik Dinakar, a research scientist at the MIT Media Lab, and Catherine Kreatsoulas of the Harvard T.H. Chan School of Public Health.</p> <p>Dinakar and his colleagues presented the results of their HERMES angina trial at the European Society of Cardiology’s annual congress in September. Their research is one of the first clinical trials accepted at the prestigious conference to use machine learning techniques, which were used to characterize the full range of symptoms experienced by individual patients and to capture nuances in how they described their symptoms in a natural language exchange.</p> <p>The trial included 637 patients in the United States and Canada who had been referred for their first coronary angiogram, the gold-standard test to diagnose coronary artery disease. After analyzing the language expressed in recorded conversations between physicians and patients and in interviews with patients, the researchers found that almost 90 percent of women and men reported chest pain as a symptom.</p> <p>Women reported significantly more angina symptoms than men, but the machine learning algorithms identified nine clusters of symptoms, such as “chest sensations and physical limitations” and “non-chest area and associated symptoms” where there were no significant differences among men and women with blockages in their heart.</p> <p>“This work, showing no real differences between women and men in chest pain, goes against the dogma and will shake up the field of cardiology,” says Deepak L. Bhatt, executive director of Interventional Cardiovascular Programs at Brigham and Women’s Hospital and professor of medicine at Harvard Medical School, a co-author of the study. “It is also exciting to see an application of machine learning in health care that actually worked and isn’t just hype,” he adds.</p> </div> <div> <div> <div> <div> <p>“This sophisticated machine learning study suggests, alongside several other recent more conventional studies, that there may be fewer if any differences in symptomatic presentation of heart attacks in women compared to men,” says Philippe Gabriel Steg, a professor of cardiology at Université Paris- Diderot and director of the Coronary Care Unit of Hôpital Bichat in Paris, France.</p> <p>“This has important consequences in the organization of care for patients with suspected heart attacks, in whom diagnostic strategies probably need to be similar in women and men,” adds Steg, who was not involved with the MIT study.</p> <p><strong>Lensing offers a new look </strong></p> <p>The idea of applying machine learning to cardiology came when Catherine Kreatsoulas, then a Fulbright fellow and heart and stroke research fellow at the Harvard School of Public Health, met Dinakar after a talk in 2014 by noted linguist Noam Chomsky. An interest in language drew them both to the talk, and Kreatsoulas in particular was concerned about the differences in the way men and women express their symptoms, and how physicians might be understanding — or misunderstanding — the way men and women speak about their heart attack symptoms.</p> <p>In the United States and Canada, 90 percent of cardiologists are male, and Kreatsoulas thought, “‘could this be a potential case of ‘lost in translation?’,” she says.</p> <p>Kreatsoulas also was concerned that doctors might be misdiagnosing or underdiagnosing female patients — as well as men who didn’t express “typical” angina symptoms — “because doctors have this frame, given their years of medical training in cardiology, that men and women have different symptoms,” Dinakar explains.</p> <p>Dinakar thought a machine learning framework called “lensing” that he had been working on for crisis counseling might offer a new way of understanding angina symptoms. In its simplest form, lensing acknowledges that different participants bring their own viewpoint or biases to a collective problem or conversation. By developing algorithms that include these different lenses, researchers can retrieve a more complete picture of the data provided by real-world conversations.</p> <p>“When we train machine learning models in situations like the heart disease diagnosis, it is important for us to capture, in some way, the lens of the physician and the lens of the patient,” says Dinakar.</p> <p>To accomplish this, the researchers audio-recorded two clinical interviews, one of patients describing their angina symptoms in clinical consult interviews with physicians and one of patient-research assistant conversations “to capture in their own natural words their descriptions of symptoms, to see if we could use methods in machine learning to see if there are a lot of differences between women and men,” he says.</p> <p>In a typical clinical trial, researchers treat “symptoms as check boxes” in their statistical analyses, Dinakar notes. “The result is to isolate one symptom from another, and you don’t capture the entire patient symptomatology presentation — you begin to treat each symptom as if it’s the same across all patients,” says Dinakar.</p> </div> </div> </div> </div> <div> <div> <div> <div> <p>“Further, when analyzing symptoms as check boxes, you rarely see the complete picture of the constellation of symptoms that patients actually report. Often this important fact is compensated for poorly in traditional statistical analysis,” Kreatsoulas says.</p> <p>Instead, the lensing model allowed the scientists “to represent each patient as a unique fingerprint of their symptoms, based on their natural language,” says Dinakar.</p> <p>Seeing patients in this way helped to uncover clusters of symptoms that could be compared in men and women, leading to the conclusion that there were few differences in symptoms between these two groups of patients.</p> <p>"The terms ‘typical’ and ‘atypical’ angina should be abandoned, as they do not correlate with disease and may perpetuate stereotypes based on sex," Dinakar and his colleagues conclude.</p> <p><strong>Helping doctors think deeper </strong></p> <p>The goal of clinical trials like the HERMES trial is not to “replace cardiologists with an algorithm,” says Dinakar. “It’s just a more sophisticated way of doing statistics and bringing them to bear on an urgent problem like this.”</p> <p>In the medical realm, the unique lens of each patient and physician might typically be thought of as “bias” in the pejorative sense — data that should be ignored or tossed out of an analysis. But the lensing algorithms treat these biases as information that can provide a more complete picture of a problem or reveal a new way of considering a problem.</p> <p>In this case, Dinakar said, “bias is information, and it helps us to think deeper. It’s very important that we capture that and try to represent that the best we can.”</p> <p>Although machine learning in medicine is often seen as a way to “brute force” through problems, like identifying tumors by applying image recognition software and predictive algorithms, Dinakar hopes that models like lensing will help physicians break down “ossified” frames of thinking across medical challenges.</p> <p>Dinakar and Kreatsoulas are now applying the machine learning models in a clinical trial with neuro-gastroenterology researchers at Massachusetts General Hospital to compare physician lenses in diagnosing diseases such as functional gastrointestinal disease and irritable bowel syndrome.</p> <p>“Anything we can do in statistics or machine learning in medicine to help break down an ossified frame or broken logic and help both providers and patients think deeper in my opinion is a win,” he says.</p> </div> </div> </div> </div> "The terms ‘typical’ and ‘atypical’ angina should be abandoned, as they do not correlate with disease and may perpetuate stereotypes based on sex," Media Lab researcher Karthik Dinakar and his colleagues conclude.Media Lab, Machine learning, Research, Health, Health care, Medicine, Women, Algorithms, School of Architecture and Planning Materials Day talks examine the promises and challenges of AI and machine learning The ability to predict and make new materials faster highlights the need for safety, reliability, and accurate data. Tue, 05 Nov 2019 14:10:01 -0500 Denis Paiste | Materials Research Laboratory <p>The promises and challenges of artificial intelligence and machine learning highlighted the Oct. 9 MIT Materials Day Symposium, with presentations on new ways of forming zeolite compounds, faster drug synthesis, advanced optical devices, and more.</p> <p>“Machine learning is having an impact in all areas of materials research,” Materials Research Laboratory Director Carl V. Thompson said.</p> <p>“We’re increasingly able to work in tandem with machines to help us decide what materials to make,” said <a dir="ltr" href="" rel="noopener" target="_blank">Elsa A. Olivetti</a>, the Atlantic Richfield Associate Professor of Energy Studies. Machine learning is also guiding how to make those materials with new insights into synthesis methods, and, in some cases (such as with robotic systems), actually making those materials, she noted.</p> <p>Keynote speaker Brian Storey, director of accelerated materials design and discovery at Toyota Research Institute, spoke about machine learning to&nbsp;advance the switch&nbsp;from the internal combustion engine to electric vehicles, and Professor Ju Li, the Battelle Energy Alliance Professor of Nuclear Science and Engineering and professor of materials science and engineering, spoke about&nbsp;atomic engineering&nbsp;using elastic strain and radiation nudging of atoms.</p> <p><strong>Porous materials</strong></p> <p>Olivetti and&nbsp;<a href="">Rafael Gomez-Bombarelli</a>, the Toyota Assistant Professor in Materials Processing, worked together to apply machine learning to develop a better understanding of porous materials called zeolites, formed from silicon and aluminum oxide, that have a wide range of uses, from cat litter to petroleum refining.</p> <p>“Essentially, the idea is that the pore has the right size to hold organic molecules,” Gomez-Bombarelli said. While only about 250 zeolites of this class are known to engineers, physicists can calculate hundreds of thousands of possible ways these structures can form. “Some of them can be converted into each other,” he said. “So, you could mine one zeolite, put it under pressure, or heat it up, and it becomes a different one that could be more valuable for a specific application.”</p> <p>A traditional method was to interpret these crystalline structures as a combination of building blocks. However, when zeolite transformations were analyzed, more than half the time there were no building blocks in common between the original zeolite before the change and the new zeolite after the change. “Building block theory has some interesting ingredients, but doesn’t quite explain the rules to go from A to B,” Gomez-Bombarelli said.</p> <p><strong>Graph-based approach</strong></p> <p>Gomez-Bombarelli’s new&nbsp;graph-based approach&nbsp;finds that when each zeolite framework structure is represented as a graph, these graphs match before and after in zeolite transformation pairs. “Some classes of transformations only happen between zeolites that have the same graph,” he said.</p> <p>This work evolved from Olivetti’s data mining of 2.5 million materials science journal articles to uncover recipes for making different inorganic materials. The zeolite study examined 70,000 papers. “One of the challenges in learning from the literature is we publish positive examples, we publish data of things that went well,” Olivetti said. In the zeolite community, researchers also publish what doesn’t work. “That’s a valuable dataset for us to learn from,” she said. “What we’ve been able to use this dataset for is to try to predict potential synthesis pathways for making particular types of zeolites.”<br /> <br /> In earlier work with colleagues at the University of Massachusetts, Olivetti developed a system that identified common scientific words and techniques found in sentences across this large library and brought together similar findings. “One important challenge in natural language processing is to draw this linked information across a document,” Olivetti explained. “We are trying to build tools that are able to do that linking,” Olivetti says.</p> <p><strong>AI-assisted chemical synthesis</strong></p> <p><a dir="ltr" href="" rel="noopener" target="_blank">Klavs F. Jensen</a>, the Warren K. Lewis Professor of Chemical Engineering and Professor of Materials Science and Engineering, described a&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">chemical synthesis system</a>&nbsp;that combines artificial intelligence-guided processing steps with a robotically operated modular reaction system.</p> <p>For those unfamiliar with synthesis, Jensen explained that “You have reactants you start with, you have reagents that you have to add, catalysts and so forth to make the reaction go, you have intermediates, and ultimately you end up with your product.”</p> <p>The artificial intelligence system combed 12.5 million reactions, creating a set of rules, or library, from about 160,000 of the most commonly used synthesis recipes, Jensen relates. This machine learning approach suggests processing conditions such as what catalysts, solvents, and reagents to use in the reaction.</p> <p>“You can have the system take whatever information it got from the published literature about conditions and so on and you can use that to form a recipe,” he says. Because there is not enough data yet to inform the system, a chemical expert still needs to step in to specify concentrations, flow rates, and process stack configurations, and to ensure safety before sending the recipe to the robotic system.</p> <p>The researchers&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">demonstrated this system</a>&nbsp;by predicting synthesis plans for 15 drugs or drug-like molecules — the painkiller lidocaine, for example, and several high blood pressure drugs — and then making them with the system. The flow reactor system contrasts with a batch system. “In order to be able to accelerate the reactions, we use typically much more aggressive conditions than are done in batch — high temperatures and higher pressures,” Jensen says.</p> <p>The modular system consists of a processing tower with interchangeable reaction modules and a set of different reagents, which are connected together by the robot for each synthesis. These findings were reported in&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank"><em>Science</em></a>.</p> <p>Former PhD students Connor W. Coley and Dale A. Thomas built the computer-aided synthesis planner and the flow reactor system, respectively, and former postdoc Justin A. M. Lummiss did the chemistry along with a large team of MIT Undergraduate Research Opportunity Program students, PhD students, and postdocs. Jensen also notes contributions from MIT faculty colleagues Regina Barzilay, William H. Green, A. John Hart, Tommi Jaakkola, and Tim Jamison. MIT has filed a patent for the robotic handling of fluid connections. The software suite that suggests and prioritizes possible synthesis routes is open source, and an online version is at the <a href="">ASKCOS website</a>.</p> <p><strong>Robustness in machine learning</strong></p> <p>Deep learning systems perform amazingly well on benchmark tasks such as images and natural language processing applications, said Professor&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">Asu Ozdaglar</a>, who heads MIT’s Department of Electrical Engineering and Computer Science. Still, researchers are far from understanding why these deep learning systems work, when they will work, and how they generalize. And when they get things wrong, they can go completely awry.</p> <p>Ozdaglar gave an example of an image with a state-of-the-art classifier that can look at a picture of a cute pig and recognize the image as that of a pig. But, “If you add a little bit of, very little, perturbation, what happens is basically the same classifier thinks that’s an airliner,” Ozdaglar said. “So this is sort of an example where people say machine learning is so powerful, it can make pigs fly,” she said, accompanied by audience laughter. “And this immediately tells us basically we have to go beyond our standard approaches.”</p> <p>A potential solution lies in an optimization formulation known as a Minimax, or MinMax, problem. Another place where MinMax formulation arises is in generative adversarial network, or GAN, training. Using an example of images of real cars and fake images of cars, Ozdaglar explained, “We would like these fake images to be drawn from the same distribution as the training set, and this is achieved using two neural networks competing with each other, a generator network and a discriminator network. The generator network creates from random noise these fake images that the discriminator network tries to pull apart to see whether this is real or fake.”</p> <p>“It’s basically another MinMax problem whereby the generator is trying to minimize the distance between these two distributions, fake and real. And then the discriminator is trying to maximize that,” she said. The MinMax problem approach has become the backbone of robust training of deep learning systems, she noted.</p> <p>Ozdaglar added that EECS faculty are applying machine learning to new areas, including health care, citing the work of&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">Regina Barzilay</a>&nbsp;in detecting breast cancer and&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">David Sontag</a>&nbsp;in using electronic medical records for medical diagnosis and treatment.</p> <p>The EECS undergraduate machine learning course (6.036) hosted 800 students last spring, and consistently has 600 or more students enrolled, making it the most popular course at MIT. The new Stephen A. Schwarzman College of Computing provides an opportunity to create a more dynamic and adaptable structure than MIT’s traditional department structure. For example, one idea is to create several cross-departmental teaching groups. “We envision things like courses in the foundations of computing, computational science and engineering, social studies of computing, and have these courses taken by all of our students taught jointly by our faculty across MIT,” she said.</p> <p><strong>Optical advantage</strong></p> <p><a dir="ltr" href="" rel="noopener" target="_blank">Juejun "JJ" Hu</a>, associate professor of materials science and engineering, detailed his research coupling a silicon chip-based spectrometer for detecting infrared light wavelengths to a newly created machine learning algorithm. Ordinary spectrometers, going back to Isaac Newton’s first prism, work by splitting light, which reduces intensity, but Hu’s version collects all of the light at a single detector, which preserves light intensity but then poses the problem of identifying different wavelengths from a single capture.</p> <p>“If you want to solve this trade-off between the (spectral) resolution and the signal-to-noise ratio, what you have to do is resort to a new type of spectroscopy tool called wavelength multiplexing spectrometer,” Hu said. His new&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">spectrometer architecture</a>, which is called digital Fourier transform spectroscopy, incorporates tunable optical switches on a silicon chip. The device works by measuring the intensity of light at different optical switch settings and comparing the results. “What you have is essentially a group of linear equations that gives you some linear combination of the light intensity at different wavelengths in the form of a detector reading,” he said.</p> <p>A prototype device with six switches supports a total of 64 unique optical states, which can provide 64 independent readings. “The advantage of this new device architecture is that the performance doubles every time you add a new switch,” he said. Working with&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">Brando Miranda</a>&nbsp;at the Center for Brains Minds and Machines at MIT, he developed a new algorithm,&nbsp;<a dir="ltr" href="" rel="noopener" target="_blank">Elastic D1</a>, that gives a resolution down to 0.2 nanometers and gives an accurate light measurement with only two consecutive measurements.</p> <p>“We believe this kind of unique combination between the hardware of a new spectrometer architecture and the algorithm can enable a wide range of applications ranging from industrial process monitoring to medical imaging,” Hu said. Hu also is applying machine learning in his work on complex optical media such as metasurfaces, which are new optical devices featuring an array of specially designed optical antennas that add a phase delay to the incoming light.</p> <p><strong>Poster session winners</strong></p> <p>Nineteen MIT postdocs and graduate students gave two-minute talks about their research during a poster session preview. At the Materials Day&nbsp;Poster Session&nbsp;immediately following the symposium, award winners were mechanical engineering graduate student Erin Looney, media arts and sciences graduate student Bianca Datta, and materials science and engineering postdoc Michael Chon.</p> <p>The <a href="">Materials Research Laboratory</a> serves interdisciplinary groups of faculty, staff, and students, supported by industry, foundations, and government agencies to carry out fundamental engineering research on materials. Research topics include energy conversion and storage, quantum materials, spintronics, photonics, metals, integrated microsystems, materials sustainability, solid-state ionics, complex oxide electronic properties, biogels, and functional fibers.</p> Eight distinguished researchers spoke at the 2019 Materials Day Symposium. Pictured here (l-r) are MIT professors Carl Thompson, Asu Ozdaglar, Elsa Olivetti, and Ju Li.Image: Denis Paiste/Materials Research LaboratoryMaterials Research Laboratory, Materials Science and Engineering, School of Engineering, Electrical engineering and computer science (EECS), MIT Schwarzman College of Computing, Machine learning, Artificial intelligence, Center for Brains Minds and Machines Autonomous system improves environmental sampling at sea Robotic boats could more rapidly locate the most valuable sampling spots in uncharted waters. Mon, 04 Nov 2019 14:54:51 -0500 Rob Matheson | MIT News Office <p>An autonomous robotic system invented by researchers at MIT and the Woods Hole Oceanographic Institution (WHOI) efficiently sniffs out the most scientifically interesting — but hard-to-find —&nbsp;sampling spots in vast, unexplored waters.</p> <p>Environmental scientists are often interested in gathering samples at the most interesting locations, or “maxima,” in an environment. One example could be a source of leaking chemicals, where the concentration is the highest and mostly unspoiled by external factors. But a maximum can be any quantifiable value that researchers want to measure, such as water depth or parts of coral reef most exposed to air.</p> <p>Efforts to deploy maximum-seeking robots suffer from efficiency and accuracy issues. Commonly, robots will move back and forth like lawnmowers to cover an area, which is time-consuming and collects many uninteresting samples. Some robots sense and follow high-concentration trails to their leak source. But they can be misled. For example, chemicals can get trapped and accumulate in crevices far from a source. Robots may identify those high-concentration spots as the source yet be nowhere close.</p> <p>In a paper being presented at the International Conference on Intelligent Robots and Systems (IROS), the researchers describe “PLUMES,” a system that enables autonomous mobile robots to zero in on a maximum far faster and more efficiently. PLUMES leverages probabilistic techniques to predict which paths are likely to lead to the maximum, while navigating obstacles, shifting currents, and other variables. As it collects samples, it weighs what it’s learned to determine whether to continue down a promising path or search the unknown — which may harbor more valuable samples.</p> <p>Importantly, PLUMES reaches its destination without ever getting trapped in those tricky high-concentration spots. “That’s important, because it’s easy to think you’ve found gold, but really you’ve found fool’s gold,” says co-first author Victoria Preston, a PhD student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and in the MIT-WHOI Joint Program.</p> <p>The researchers built a PLUMES-powered robotic boat that successfully detected the most exposed coral head in the Bellairs Fringing Reef in Barbados —&nbsp;meaning, it was located in the shallowest spot —&nbsp;which is useful for studying how sun exposure impacts coral organisms. In 100 simulated trials in diverse underwater environments, a virtual PLUMES robot also consistently collected seven to eight times more samples of maxima than traditional coverage methods in allotted time frames.</p> <p>“PLUMES does the minimal amount of exploration necessary to find the maximum and then concentrates quickly on collecting valuable samples there,” says co-first author Genevieve Flaspohler, a PhD student and in CSAIL and the MIT-WHOI Joint Program.</p> <p>Joining Preston and Flaspohler on the paper are: Anna P.M. Michel and Yogesh Girdhar, both scientists in the Department of Applied Ocean Physics and Engineering at the WHOI; and Nicholas Roy, a professor in CSAIL and in the Department of Aeronautics and Astronautics. &nbsp;</p> <p><strong>Navigating an exploit-explore tradeoff</strong></p> <p>A key insight of PLUMES was using techniques from probability to reason about navigating the notoriously complex tradeoff between exploiting what’s learned about the environment and exploring unknown areas that may be more valuable.</p> <p>“The major challenge in maximum-seeking is allowing the robot to balance exploiting information from places it already knows to have high concentrations and exploring places it doesn’t know much about,” Flaspohler says. “If the robot explores too much, it won’t collect enough valuable samples at the maximum. If it doesn’t explore enough, it may miss the maximum entirely.”</p> <p>Dropped into a new environment, a PLUMES-powered robot uses a probabilistic statistical model called a Gaussian process to make predictions about environmental variables, such as chemical concentrations, and estimate sensing uncertainties. PLUMES then generates a distribution of possible paths the robot can take, and uses the estimated values and uncertainties to rank each path by how well it allows the robot to explore and exploit.</p> <p>At first, PLUMES will choose paths that randomly explore the environment. Each sample, however, provides new information about the targeted values in the surrounding environment — such as spots with highest concentrations of chemicals or shallowest depths. The Gaussian process model exploits that data to narrow down possible paths the robot can follow from its given position to sample from locations with even higher value. PLUMES uses a novel objective function —&nbsp;commonly used in machine-learning to maximize a reward — to make the call of whether the robot should exploit past knowledge or explore the new area.</p> <p><strong>“Hallucinating” paths</strong></p> <p>The decision where to collect the next sample relies on the system’s ability to “hallucinate” all possible future action from its current location. To do so, it leverages a modified version of Monte Carlo Tree Search (MCTS), a path-planning technique popularized for powering artificial-intelligence systems that master complex games, such as Go and Chess.</p> <p>MCTS uses a decision tree — a map of connected nodes and lines — to simulate a path, or sequence of moves, needed to reach a final winning action. But in games, the space for possible paths is finite. In unknown environments, with real-time changing dynamics, the space is effectively infinite, making planning extremely difficult. The researchers designed “continuous-observation MCTS,” which leverages the Gaussian process and the novel objective function to search over this unwieldy space of possible real paths.</p> <p>The root of this MCTS decision tree starts with a “belief” node, which is the next immediate step the robot can take. This node contains the entire history of the robot’s actions and observations up until that point. Then, the system expands the tree from the root into new lines and nodes, looking over several steps of future actions that lead to explored and unexplored areas.</p> <p>Then, the system simulates what would happen if it took a sample from each of those newly generated nodes, based on some patterns it has learned from previous observations. Depending on the value of the final simulated node, the entire path receives a reward score, with higher values equaling more promising actions. Reward scores from all paths are rolled back to the root node. The robot selects the highest-scoring path, takes a step, and collects a real sample. Then, it uses the real data to update its Gaussian process model and repeats the “hallucination” process.</p> <p>“As long as the system continues to hallucinate that there may be a higher value in unseen parts of the world, it must keep exploring,” Flaspohler says. “When it finally converges on a spot it estimates to be the maximum, because it can’t hallucinate a higher value along the path, it then stops exploring.”</p> <p>Now, the researchers are collaborating with scientists at WHOI to use PLUMES-powered robots to localize chemical plumes at volcanic sites and study methane releases in melting coastal estuaries in the Arctic. Scientists are interested in the source of chemical gases released into the atmosphere, but these test sites can span hundreds of square miles.</p> <p>“They can [use PLUMES to] spend less time exploring that huge area and really concentrate on collecting scientifically valuable samples,” Preston says.</p> Even in unexplored waters, an MIT-developed robotic system can efficiently sniff out valuable, hard-to-find spots to collect samples from. When implemented in autonomous boats deployed off the coast of Barbados (pictured), the system quickly found the most exposed coral head —meaning it was located in the shallowest spot — which is useful for studying how sun exposure impacts coral organisms.Image courtesy of the researchersResearch, Computer science and technology, Algorithms, Computer Science and Artificial Intelligence Laboratory (CSAIL), Autonomous vehicles, Machine learning, Artificial intelligence, Environment, Robots, Robotics, Oceanography and ocean engineering, Aeronautical and astronautical engineering, School of Engineering Better autonomous “reasoning” at tricky intersections Model alerts driverless cars when it’s safest to merge into traffic at intersections with obstructed views. Mon, 04 Nov 2019 12:44:49 -0500 Rob Matheson | MIT News Office <p>MIT and Toyota researchers have designed a new model to help autonomous vehicles determine when it’s safe to merge into traffic at intersections with obstructed views.</p> <p>Navigating intersections can be dangerous for driverless cars and humans alike. In 2016, roughly 23 percent of fatal and 32 percent of nonfatal U.S. traffic accidents occurred at intersections, according to a 2018 Department of Transportation study. Automated systems that help driverless cars and human drivers steer through intersections can require direct visibility of the objects they must avoid. When their line of sight is blocked by nearby buildings or other obstructions, these systems can fail.</p> <p>The researchers developed a model that instead uses its own uncertainty to estimate the risk of potential collisions or other traffic disruptions at such intersections. It weighs several critical factors, including all nearby visual obstructions, sensor noise and errors, the speed of other cars, and even the attentiveness of other drivers. Based on the measured risk, the system may advise the car to stop, pull into traffic, or nudge forward to gather more data.</p> <p>“When you approach an intersection there is potential danger for collision. Cameras and other sensors require line of sight. If there are occlusions, they don’t have enough visibility to assess whether it’s likely that something is coming,” says Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. “In this work, we use a predictive-control model that’s more robust to uncertainty, to help vehicles safely navigate these challenging road situations.”</p> <p>The researchers tested the system in more than 100 trials of remote-controlled cars turning left at a busy, obstructed intersection in a mock city, with other cars constantly driving through the cross street. Experiments involved fully autonomous cars and cars driven by humans but assisted by the system. In all cases, the system successfully helped the cars avoid collision from 70 to 100 percent of the time, depending on various factors. Other similar models implemented in the same remote-control cars sometimes couldn’t complete a single trial run without a collision.</p> <p>Joining Rus on the paper are: first author Stephen G. McGill, Guy Rosman, and Luke Fletcher of the Toyota Research Institute (TRI); graduate students Teddy Ort and Brandon Araki, researcher Alyssa Pierson, and postdoc Igor Gilitschenski, all of CSAIL; Sertac Karaman, an MIT associate professor of aeronautics and astronautics; and John J. Leonard, the Samuel C. Collins Professor of Mechanical and Ocean Engineering of MIT and a TRI technical advisor.</p> <div class="cms-placeholder-content-video"></div> <p><strong>Modeling road segments</strong></p> <p>The model is specifically designed for road junctions in which there is no stoplight and a car must yield before maneuvering into traffic at the cross street, such as taking a left turn through multiple lanes or roundabouts. In their work, the researchers split a road into small segments. This helps the model determine if any given segment is occupied to estimate a conditional risk of collision.</p> <p>Autonomous cars are equipped with sensors that measure the speed of other cars on the road. When a sensor clocks a passing car traveling into a visible segment, the model uses that speed to predict the car’s progression through all other segments. A probabilistic “Bayesian network” also considers uncertainties — such as noisy sensors or unpredictable speed changes — to determine the likelihood that each segment is occupied by a passing car.</p> <p>Because of nearby occlusions, however, this single measurement may not suffice. Basically, if a sensor can’t ever see a designated road segment, then the model assigns it a high likelihood of being occluded. From where the car is positioned, there’s increased risk of collision if the car just pulls out fast into traffic. This encourages the car to nudge forward to get a better view of all occluded segments. As the car does so, the model lowers its uncertainty and, in turn, risk.</p> <p>But even if the model does everything correctly, there’s still human error, so the model also estimates the awareness of other drivers. “These days, drivers may be texting or otherwise distracted, so the amount of time it takes to react may be a lot longer,” McGill says. “We model that conditional risk, as well.”</p> <p>That depends on computing the probability that a driver saw or didn’t see the autonomous car pulling into the intersection. To do so, the model looks at the number of segments a traveling car has passed through before the intersection. The more segments it had occupied before reaching the intersection, the higher the likelihood it has spotted the autonomous car and the lower the risk of collision.</p> <p>The model sums all risk estimates from traffic speed, occlusions, noisy sensors, and driver awareness. It also considers how long it will take the autonomous car to steer a preplanned path through the intersection, as well as all safe stopping spots for crossing traffic. This produces a total risk estimate.</p> <p>That risk estimate gets updated continuously for wherever the car is located at the intersection. In the presence of multiple occlusions, for instance, it’ll nudge forward, little by little, to reduce uncertainty. When the risk estimate is low enough, the model tells the car to drive through the intersection without stopping. Lingering in the middle of the intersection for too long, the researchers found, also increases risk of a collision.</p> <p><strong>Assistance and intervention</strong></p> <p>Running the model on remote-control cars in real-time indicates that it’s efficient and fast enough to deploy into full-scale autonomous test cars in the near future, the researchers say. (Many other models are too computationally heavy to run on those cars.) The model still needs far more rigorous testing before being used for real-world implementation in production vehicles.</p> <p>The model would serve as a supplemental risk metric that an autonomous vehicle system can use to better reason about driving through intersections safely. The model could also potentially be implemented in certain “advanced driver-assistive systems” (ADAS), where humans maintain shared control of the vehicle.</p> <p>Next, the researchers aim to include other challenging risk factors in the model, such as the presence of pedestrians in and around the road junction.</p> MIT and Toyota researchers have designed a new model that weighs various uncertainties and risks to help autonomous vehicles determine when it’s safe to merge into traffic at intersections with objects obstructing views, such as buildings blocking the line of sight. Image courtesy of the researchersResearch, Computer science and technology, Algorithms, Robotics, Robots, Autonomous vehicles, Automobiles, Artificial intelligence, Machine learning, Transportation, Technology and society, Computer Science and Artificial Intelligence Laboratory (CSAIL), Electrical Engineering & Computer Science (eecs), School of Engineering Technique helps robots find the front door Navigation method may speed up autonomous last-mile delivery. Mon, 04 Nov 2019 00:00:00 -0500 Jennifer Chu | MIT News Office <p>In the not too distant future, robots may be dispatched as last-mile delivery vehicles to drop your takeout order, package, or meal-kit subscription at your doorstep — if they can find the door.</p> <p>Standard approaches for robotic navigation involve mapping an area ahead of time, then using algorithms to guide a robot toward a specific goal or GPS coordinate on the map. While this approach might make sense for exploring specific environments, such as the layout of a particular building or planned obstacle course, it can become unwieldy in the context of last-mile delivery.</p> <p>Imagine, for instance, having to map in advance every single neighborhood within a robot’s delivery zone, including the configuration of each house within that neighborhood along with the specific coordinates of each house’s front door. Such a task can be difficult to scale to an entire city, particularly as the exteriors of houses often change with the seasons. Mapping every single house could also run into issues of security and privacy.</p> <p>Now MIT engineers have developed a navigation method that doesn’t require mapping an area in advance. Instead, their approach enables a robot to use clues in its environment to plan out a route to its destination, which can be described in general semantic terms, such as “front door” or “garage,” rather than as coordinates on a map. For example, if a robot is instructed to deliver a package to someone's front door, it might start on the road and see a driveway, which it has been trained to recognize as likely to lead toward a sidewalk, which in turn is likely to lead to the front door.</p> <div class="cms-placeholder-content-video"></div> <p>The new technique can greatly reduce the time a robot spends exploring a property before identifying its target, and it doesn’t rely on maps of specific residences.&nbsp;</p> <p>“We wouldn’t want to have to make a map of every building that we’d need to visit,” says Michael Everett, a graduate student in MIT’s Department of Mechanical Engineering. “With this technique, we hope to drop a robot at the end of any driveway and have it find a door.”</p> <p>Everett will present the group’s results this week at the International Conference on Intelligent Robots and Systems. The paper, which is co-authored by Jonathan How, professor of aeronautics and astronautics at MIT, and Justin Miller of the Ford Motor Company, is a finalist for “Best Paper for Cognitive Robots.”</p> <p><strong>“A sense of what things are”</strong></p> <p>In recent years, researchers have worked on introducing natural, semantic language to robotic systems, training robots to recognize objects by their semantic labels, so they can visually process a door as a door, for example, and not simply as a solid, rectangular obstacle.</p> <p>“Now we have an ability to give robots a sense of what things are, in real-time,” Everett says.</p> <p>Everett, How, and Miller are using similar semantic techniques as a springboard for their new navigation approach, which leverages pre-existing algorithms that extract features from visual data to generate a new map of the same scene, represented as semantic clues, or context.</p> <p>In their case, the researchers used an algorithm to build up a map of the environment as the robot moved around, using the semantic labels of each object and a depth image. This algorithm is called semantic SLAM (Simultaneous Localization and Mapping).</p> <p>While other semantic algorithms have enabled robots to recognize and map objects in their environment for what they are, they haven’t allowed a robot to make decisions in the moment while navigating a new environment, on the most efficient path to take to a semantic destination such as a “front door.”</p> <p>“Before, exploring was just, plop a robot down and say ‘go,’ and it will move around and eventually get there, but it will be slow,” How says.</p> <p><strong>The cost to go</strong></p> <p>The researchers looked to speed up a robot’s path-planning through a semantic, context-colored world. They developed a new “cost-to-go estimator,” an algorithm that converts a semantic map created by preexisting SLAM algorithms into a second map, representing the likelihood of any given location being close to the goal.</p> <p>“This was inspired by image-to-image translation, where you take a picture of a cat and make it look like a dog,” Everett says. “The same type of idea happens here where you take one image that looks like a map of the world, and turn it into this other image that looks like the map of the world but now is colored based on how close different points of the map are to the end goal.”</p> <p>This cost-to-go map is colorized, in gray-scale, to represent darker regions as locations far from a goal, and lighter regions as areas that are close to the goal. For instance, the sidewalk, coded in yellow in a semantic map, might be translated by the cost-to-go algorithm as a darker region in the new map, compared with a driveway, which is progressively lighter as it approaches the front door — the lightest region in the new map.</p> <p>The researchers trained this new algorithm on satellite images from Bing Maps containing 77 houses from one urban and three suburban neighborhoods. The system converted a semantic map into a cost-to-go map, and mapped out the most efficient path, following lighter regions in the map, to the end goal. For each satellite image, Everett assigned semantic labels and colors to context features in a typical front yard, such as grey for a front door, blue for a driveway, and green for a hedge.</p> <p>During this training process, the team also applied masks to each image to mimic the partial view that a robot’s camera would likely have as it traverses a yard.</p> <p>“Part of the trick to our approach was [giving the system] lots of partial images,” How explains. “So it really had to figure out how all this stuff was interrelated. That’s part of what makes this work robustly.”</p> <p>The researchers then tested their approach in a simulation of an image of an entirely new house, outside of the training dataset, first using the preexisting SLAM algorithm to generate a semantic map, then applying their new cost-to-go estimator to generate a second map, and path to a goal, in this case, the front door.</p> <p>The group’s new cost-to-go technique found the front door 189 percent faster than classical navigation algorithms, which do not take context or semantics into account, and instead spend excessive steps exploring areas that are unlikely to be near their goal.</p> <p>Everett says the results illustrate how robots can use context to efficiently locate a goal, even in unfamiliar, unmapped environments.</p> <p>“Even if a robot is delivering a package to an environment it’s never been to, there might be clues that will be the same as other places it’s seen,” Everett says. “So the world may be laid out a little differently, but there’s probably some things in common.”</p> <p>This research is supported, in part, by the Ford Motor Company.</p> For last-mile delivery, robots of the future may use a new MIT algorithm to find the front door, using clues in their environment.Image: MIT NewsResearch, Aeronautical and astronautical engineering, School of Engineering, Algorithms, Artifical intelligence, Machine learning, Autonomous vehicles, Robots, Robotics, Software What makes an image memorable? Ask a computer An artificial intelligence model developed at MIT shows in striking detail what makes some images stick in our minds. Fri, 01 Nov 2019 12:25:01 -0400 Kim Martineau | MIT Quest for Intelligence <p>From the "Mona Lisa" to the "Girl with a Pearl Earring," some images linger in the mind long after others have faded. Ask an artist why, and you might hear some&nbsp;generally-accepted principles for making memorable art. Now there’s an easier way to learn: ask an artificial intelligence model to draw an example.&nbsp;</p> <p>A new study using machine learning to generate images ranging from a memorable cheeseburger to a forgettable cup of coffee shows in close detail what makes a portrait or scene stand out. The images that human subjects in the study remembered best featured bright colors, simple backgrounds, and subjects that were centered prominently in the frame.&nbsp;<a href="">Results</a>&nbsp;were presented this week at the&nbsp;<a href="">International Conference on Computer Vision</a>.&nbsp;</p> <p>“A picture is worth a thousand words,” says the study’s co-senior author&nbsp;<a href="">Phillip Isola</a>, the Bonnie and Marty (1964) Tenenbaum CD Assistant Professor of Electrical Engineering and Computer Science at MIT.&nbsp;“A lot has been written&nbsp;about memorability, but this method lets us actually visualize what memorability looks like. It gives us a visual definition for something that’s hard to put into words."</p> <p>The work builds on an earlier model,&nbsp;<a href="">MemNet</a>, which rates the memorability of an image and highlights the features in the picture influencing its decision. MemNet’s predictions are based on the results of an online study in which 60,000 images were shown to human subjects and ranked by how easily they were remembered<strong>.</strong></p> <p>The model in the current study,&nbsp;<a href="">GANalyze</a>, uses a machine learning technique called generative adversarial networks, or GANs, to visualize a single image as it inches its way from "meh" to memorable. GANalyze lets viewers visualize the incremental transformation of, say, a blurry panda lost in the bamboo into a panda that dominates the frame, its black eyes, ears, and paws contrasting sharply and adorably with its white mug.</p> <p>The image-riffing GAN has three modules. An assessor, based on MemNet, turns the memorability knob on a target image and calculates how to achieve the desired effect. A transformer executes its instructions, and a generator outputs the final image.&nbsp;</p> <p>The progression has the dramatic feel of a time-lapse image. A cheeseburger shifted to the far end of the memorability scale looks fatter, brighter, and, as the authors note, “tastier,” than its earlier incarnations. A ladybug looks shinier and more purposeful. In an unexpected twist, a pepper on the vine turns chameleon-like from green to red.&nbsp;</p> <p>The researchers also looked at which features influence memorability most. In online experiments, human subjects were shown images of varying memorability and asked to flag any repeats. The duplicates that were stickiest, it turns out, featured subjects closer up, making animals or objects in the frame appear larger. The next most important factors were brightness, having the subject centered in the frame, and in&nbsp;a square or circular shape.</p> <p>“The human brain evolved to focus most on these features, and that’s what the GAN picks up on,” says study co-author&nbsp;<a href="">Lore Goetschalckx</a>, a visiting graduate student from Katholieke Universiteit&nbsp;Leuven in Belgium.</p> <p>The researchers also reconfigured GANanalyze to generate images of varying aesthetic and emotional appeal. They found that images rated higher on aesthetic and emotional grounds were brighter, more colorful, and had a shallow depth of field that blurred the background, much like the most memorable pictures. However, the most aesthetic images were not always memorable.</p> <p>GANalyze has a number of potential applications, the researchers say. It could be used to detect, and even treat, memory loss by enhancing objects in an augmented reality system.&nbsp;</p> <p>“Instead of using a drug to enhance memory, you might enhance the world through an augmented-reality device to make easily misplaced items like keys stand out,” says study co-senior author&nbsp;<a href="">Aude Oliva</a>, a principal research scientist at MIT’s&nbsp;<a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL)&nbsp;and executive director of the&nbsp;<a href="">MIT Quest for Intelligence.</a>&nbsp;&nbsp;</p> <p>GANalyze could also be used to create unforgettable graphics to help readers retain information. “It could revolutionize education,” says Oliva. Finally, GANs are already starting to be used&nbsp;to generate synthetic, realistic images of the world to help train automated systems to recognize places and objects they are unlikely to encounter in real life.&nbsp;</p> <p>Generative models offer new, creative ways for humans and machines to collaborate. Study co-author&nbsp;<a href="">Alex Andonian</a>, a graduate student at MIT’s&nbsp;<a href="">Department of Electrical Engineering and Computer Science</a>, says that's why he&nbsp;has chosen&nbsp;to make them the focus of his PhD.</p> <p>“Design software lets you adjust the brightness of an image, but not its overall memorability or aesthetic appeal — GANs let you do that,” he says. “We’re just starting to scratch the surface of what these models can do.”&nbsp;&nbsp;&nbsp;</p> <p>The study was funded by the U.S. National Science Foundation.</p> In a study using machine-generated art to understand what makes a picture memorable, researchers found that the images people remembered best had bright colors, simple backgrounds, and subjects centered prominently in the frame. Image courtesy of the researchersElectrical engineering and computer science (EECS), Computer Science and Artificial Intelligence Laboratory (CSAIL), School of Engineering, School of Science, Algorithms, Artificial intelligence, Computer science and technology, Machine learning, Software, Quest for Intelligence, Memory Bryan Reimer receives human factors innovator award MIT AgeLab research engineer directs a team that studies in-vehicle automation, robotics, AI, and the mechanics of driver attention, among other topics. Wed, 30 Oct 2019 16:00:01 -0400 Arthur Grau | Center for Transportation and Logistics <p>MIT Research Engineer Bryan Reimer recently received the Jack A. Kraft Innovator Award from the Human Factors and Ergonomics Society (HFES). Reimer directs a multidisciplinary team at MIT AgeLab that explores human-centered topics across a range of emerging technologies. His team studies in-vehicle automation, robotics, artificial intelligence, and the mechanics of driver attention, among other topics. The team’s research develops theoretical and applied insight into driver behavior and aims to find solutions to the next generation of human-factors challenges associated with the automation of transportation. Reimer received this accolade partially because of the broad applicability of his research within the field of ergonomics and technology.</p> <p>The Jack A. Kraft Innovator Award was established in 1970 by the HFES to recognize significant efforts to extend or diversify the application of human-factors principles and methods to new areas of endeavor. Reimer&nbsp; accepted the award at the HFES annual meeting on Oct. 29 in Seattle, Washington.</p> <p>“It’s quite an honor to receive a professional award of this magnitude and be recognized alongside human-factors leaders that I’ve revered, and who have shaped the profession,“ says Reimer. “I am grateful for the support of my colleagues, who for over two decades have collaborated with me on this work. This collaboration, in combination with the appetite for innovation at MIT, I believe has positioned me to receive this award.”</p> <p>Serving as the basis for the honor is Reimer’s innovative work founding and managing three industry partnerships. The Advanced Human Factors Evaluator for Attentional Demand consortium aims to develop the next generation of driver-attention measurement tools. The Advanced Vehicle Technology consortium seeks to understand how drivers use emerging, commercially available vehicle technologies, including advanced driver assistance systems and automated driving systems. Finally, the Clear Information Presentation consortium explores the impact of typography and other design features on usability in glance-based environments such as while driving or while using smartphones.</p> <p>Kermit Davis, president of the HFES, says “The Kraft Award is one of our society’s top awards and honors an individual who has made major innovation in human factors and ergonomics (HF/E). Dr. Reimer’s work in automated and operator-assisted driving stood out because of its broad scope, extensive collaboration across diverse disciplines, and highly influential impact. His focus on this new area for HF/E not only expands the reach of our profession, but also addresses an important individual and societal issue regarding the interaction between humans and technology.”&nbsp;</p> <p>The AgeLab at MIT Center for Transportation and Logistics is a multidisciplinary research program that works with business, government, and non-governmental organizations to improve the quality of life of older people and those who care for them. The HFES is the world’s largest scientific association for human factors and ergonomics professionals, with over 4,500 members in 58 countries. Reimer’s work draws together traditional psychological methods with big-data analytics, deep learning, and predictive modeling. The receipt of this award illustrates how research across disciplines may yield significant results, both for the research community and society at large.</p> <div></div> Bryan Reimer, an AgeLab research scientist and the associate director of the New England University Transportation Center, was honored for his work developing a better understanding of how people engage with vehicle automation.Photo: MIT AgeLabCenter for Transportation and Logistics, AgeLab, Autonomous vehicles, Machine learning, Design, Awards, honors and fellowships, Staff, Aging Supercomputer analyzes web traffic across entire internet Modeling web traffic could aid cybersecurity, computing infrastructure design, Internet policy, and more. Sun, 27 Oct 2019 23:59:59 -0400 Rob Matheson | MIT News Office <p>Using a supercomputing system, MIT researchers have developed a model that captures what web traffic looks like around the world on a given day, which can be used as a measurement tool for internet research and many other applications.</p> <p>Understanding web traffic patterns at such a large scale, the researchers say, is useful for informing internet policy, identifying and preventing outages, defending against cyberattacks, and designing more efficient computing infrastructure. A paper describing the approach was presented at the recent IEEE High Performance Extreme Computing Conference.</p> <p>For their work, the researchers gathered the largest publicly available internet traffic dataset, comprising 50 billion data packets exchanged in different locations across the globe over a period of several years.</p> <p>They ran the data through a novel “neural network” pipeline operating across 10,000 processors of the MIT SuperCloud, a system that combines computing resources from the MIT Lincoln Laboratory and across the Institute. That pipeline automatically trained a model that captures the relationship for all links in the dataset — from common pings to giants like Google and Facebook, to rare links that only briefly connect yet seem to have some impact on web traffic. &nbsp;</p> <p>The model can take any massive network dataset and generate some statistical measurements about how all connections in the network affect each other. That can be used to reveal insights about peer-to-peer filesharing, nefarious IP addresses and spamming behavior, the distribution of attacks in critical sectors, and traffic bottlenecks to better allocate computing resources and keep data flowing.</p> <p>In concept, the work is similar to measuring the cosmic microwave background of space, the near-uniform radio waves traveling around our universe that have been an important source of information to study phenomena in outer space. “We built an accurate model for measuring the background of the virtual universe of the Internet,” says Jeremy Kepner, a researcher at the MIT Lincoln Laboratory Supercomputing Center and an astronomer by training. “If you want to detect any variance or anomalies, you have to have a good model of the background.”</p> <p>Joining Kepner on the paper are: Kenjiro Cho of the Internet Initiative Japan; KC Claffy of the Center for Applied Internet Data Analysis at the University of California at San Diego; Vijay Gadepally and Peter Michaleas of Lincoln Laboratory’s Supercomputing Center; and Lauren Milechin, a researcher in MIT’s Department of Earth, Atmospheric and Planetary Sciences.</p> <p><strong>Breaking up data</strong></p> <p>In internet research, experts study anomalies in web traffic that may indicate, for instance, cyber threats. To do so, it helps to first understand what normal traffic looks like. But capturing that has remained challenging. Traditional “traffic-analysis” models can only analyze small samples of data packets exchanged between sources and destinations limited by location. That reduces the model’s accuracy.</p> <p>The researchers weren’t specifically looking to tackle this traffic-analysis issue. But they had been developing new techniques that could be used on the MIT SuperCloud to process massive network matrices. Internet traffic was the perfect test case.</p> <p>Networks are usually studied in the form of graphs, with actors represented by nodes, and links representing connections between the nodes. With internet traffic, the nodes vary in sizes and location. Large supernodes are popular hubs, such as Google or Facebook. Leaf nodes spread out from that supernode and have multiple connections to each other and the supernode. Located outside that “core” of supernodes and leaf nodes are isolated nodes and links, which connect to each other only rarely.</p> <p>Capturing the full extent of those graphs is infeasible for traditional models. “You can’t touch that data without access to a supercomputer,” Kepner says.</p> <p>In partnership with the Widely Integrated Distributed Environment (WIDE) project, founded by several Japanese universities, and the Center for Applied Internet Data Analysis (CAIDA), in California, the MIT researchers captured the world’s largest packet-capture dataset for internet traffic. The anonymized dataset contains nearly 50 billion unique source and destination data points between consumers and various apps and services during random days across various locations over Japan and the U.S., dating back to 2015.</p> <p>Before they could train any model on that data, they needed to do some extensive preprocessing. To do so, they utilized software they created previously, called Dynamic Distributed Dimensional Data Mode (D4M), which uses some averaging techniques to efficiently compute and sort “hypersparse data” that contains far more empty space than data points. The researchers broke the data into units of about 100,000 packets across 10,000 MIT SuperCloud processors. This generated more compact matrices of billions of rows and columns of interactions between sources and destinations.</p> <p><strong>Capturing outliers</strong></p> <p>But the vast majority of cells in this hypersparse dataset were still empty. To process the matrices, the team ran a neural network on the same 10,000 cores. Behind the scenes, a trial-and-error technique started fitting models to the entirety of the data, creating a probability distribution of potentially accurate models.</p> <p>Then, it used a modified error-correction technique to further refine the parameters of each model to capture as much data as possible. Traditionally, error-correcting techniques in machine learning will try to reduce the significance of any outlying data in order to make the model fit a normal probability distribution, which makes it more accurate overall. But the researchers used some math tricks to ensure the model still saw all outlying data — such as isolated links — as significant to the overall measurements.</p> <p>In the end, the neural network essentially generates a simple model, with only two parameters, that describes the internet traffic dataset, “from really popular nodes to isolated nodes, and the complete spectrum of everything in between,” Kepner says.</p> <p>Using supercomputing resources to efficiently process a “firehose stream of traffic” to identify meaningful patterns and web activity is “groundbreaking” work, says David Bader, a distinguished professor of computer science and director of the Institute for Data Science at the New Jersey Institute of Technology. “A grand challenge in cybersecurity is to understand the global-scale trends in Internet traffic for purposes, such as detecting nefarious sources, identifying significant flow aggregation, and vaccinating against computer viruses. [This research group has] successfully tackled this problem and presented deep analysis of global network traffic,” he says.</p> <p>The researchers are now reaching out to the scientific community to find their next application for the model. Experts, for instance, could examine the significance of the isolated links the researchers found in their experiments that are rare but seem to impact web traffic in the core nodes.</p> <p>Beyond the internet, the neural network pipeline can be used to analyze any hypersparse network, such as biological and social networks. “We’ve now given the scientific community a fantastic tool for people who want to build more robust networks or detect anomalies of networks,” Kepner says. “Those anomalies can be just normal behaviors of what users do, or it could be people doing things you don’t want.”</p> Using a supercomputing system, MIT researchers developed a model that captures what global web traffic could look like on a given day, including previously unseen isolated links (left) that rarely connect but seem to impact core web traffic (right). Image courtesy of the researchers, edited by MIT NewsResearch, EAPS, Lincoln Laboratory, School of Science, Computer science and technology, Algorithms, Artificial intelligence, Machine learning, Data, Supercomputing, Internet, cybersecurity