Skip to content
postal boxes
MaCall ManorFeb 28, 2021 4:13:00 AM3 min read

The Wonder - Machine Learning Synesthesia | SF AppWorks

A Monthly Snapshot of Life-Changing Technology

February 28th, 2021


1. Seeing Sound


Google takes a stab at mimicking synesthesia with machine learning.

Read more 

Hot Take

It's interesting to see high technology applied to an artistic endeavor so inherently and human and emotional. 


Sensory Overload


Google paired up with a cultural complex in Paris called Centre Pompidou to create a virtual exhibition of painter Vassily Kandinsky's work, complete with pieces viewable in an AR gallery. But the main attraction is the exhibit's machine learning experiment, which tries to augment synesthesia, a condition Kandinsky had, that makes a stimulation of one sense turn multi-sensory. Says the president of Centre Pompidou, "when Kandinsky painted, two senses worked systematically together: hearing and sight."


Making Sense


For the project, a hybrid team of experimental musicians and engineers analyzed Kandinsky's writing about his multi-sensory experiences and used them to train Google's Transformer neural network with music from Kandinsky's period. They also did a version using newer scores to create a more current sound perspective of the painting. Play a Kandinsky allows you to click and zoom into different areas of the painting to explore possible experiences of its shapes and colors.


2. Power Move


The pentagon has successfully tested a solar panel prototype that can beam electricity from space back to earth.

Read More


Hot Take

To creatively solve for alternative energy sources, sometimes you gotta level up ⬆️. 


Here Comes the Sun


The panel is called a "Photovoltaic Radiofrequency Antenna Module" (PRAM). It launched in May of 2020 attached to a drone operated by the Pentagon, circling the earth every 90 minutes. The panel will take advantage of the light in space, which is more powerful because it hasn't yet passed through the atmosphere. It was unnecessary to send energy back to earth in the test run, as that part of the technology has already been proven. 


Beam On


At 12x12 inches, the panel is about the size of a pizza box and can produce around 10 watts (enough energy to power a tablet). However the prototype only marks the starting point of a longer-term project wherein the team imagines dozens of panels scaled up to generate and deliver power anywhere on the globe.


3. What do you charge?


Scientists have created a tiny device that can turn your body's heat directly into electrical energy.

Read More 


Hot Take

Can it recharge your mental energy at 3pm? 


Ring of Fire?


Scientists at the University of Colorado Boulder created a thermoelectric generator (TEG) in the form of a small ring, although it could made the size of a watch or much larger depending on desired capacity for power generation. The team published its study in Science Advances, explaining how a unique material embedded inside the gadget can reconfigure to keep from breaking as the wearer moves.


More Power to You


Use cases for the battery include drug delivery pumps, pacemakers, implantable medical devices, or other wearable electronics. Jianliang Xiao, associate professor on the project, explains his vision of wearable electronics that don't need to have a battery. These TEGs use the temperature difference between your body and the surrounding air to capture energized particles as they pass through an ultra thin barrier to generate power.


4. NYPD Robot Dog


Video has emerged of the NYPD's new robot dog rushing to the scene of a crime.

Read More 


Hot Take

Does this mean any of the K-9 unit are up for adoption? 




The 70 pound robot dog was deployed alongside its human counterparts in response to a home invasion in the Bronx. The video captured "Digidog" swiftly trotting on a sidewalk toward the crime scene. It can reportedly run 3.5 mph, climb stairs, and navigate with artificial intelligence.


Good Bot


Someone commenting on behalf of the NYPD explained that the robot, which features two-way communication, lights, and cameras that allow police to view its surroundings in real time, is currently in a test phase. "This dog is going to save lives, protect people, and protect officers, and that's our goal," adds a NYPD Inspector.


Did you enjoy The Wonder? 



MaCall Manor

MaCall Manor is an editor and writer based in San Francisco. She has always been a storyteller by trade, seeking to inspire with the work and content she creates. Brilliantly imaginative in filling out the details of the innovation processes and design thinking, she's passionate about all things creative, dancing, nature, and books/movies.