Stephan Seeger

GNSS & Machine Learning Engineer

Month: December 2022

RIFFUSION: Stable Diffusion for Real-Time Music Generation

December 16, 2022 / / 0 Comments

By using the stable diffusion model v1.5 without any modifications, just fine-tuned on images of spectrograms paired with text, the software RIFFUSION (RIFF + diffusion) generates incredibly interesting music from text input. By interpolating in latent space it is possible to transition from one text prompt to the next. You can try out the model here.

The authors provide source code on GitHub for an interactive web app and an inference server. A model checkpoint is available on Hugging Face.

There is a nice video about RIFFUSION by Alan Thompson on youtube.

Even more shocking than using diffusion on spectrograms and getting great results may be a paper by Google Research published on Dec 15, 2022. They use text as an image and train their model with contrastive loss alone, thus calling their model CLIP-Pixels Only (CLIPPO). It’s a joint model for processing images and text with a single ViT (Vison Transformer) approach and astonishing performance.

RFdiffusion from Baker Lab solves the Protein Generation Problem

December 7, 2022 / / 0 Comments

While ProteinMPNN takes a protein backbone (N-CA-C-O atoms, CA = C-Alpha) and finds an amino acid sequence that would fold to that backbone structure, RFdiffusion [Twitter] instead makes the protein backbone by just providing some geometrical and functional constraints like “create a molecule that binds X”.

The authors used a guided diffusion model for generating new proteins in the same way as Dall-E produces high-quality images that have never existed before by a diffusion technique.

See also this presentation by David Baker.

If I interpret this announcement correctly it means that drug design is now basically solved (or starts to get interesting depending on the viewpoint).

This technique can be expected to significantly increase the number of potential drugs for combating diseases. However, animal tests and human studies can also be expected as the bottlenecks of the new possibilities. Techniques like organ chips from companies like emulate may be a way out of this dilemma (before one-day entire cell, tissue, or whole body computational simulations become possible).

ProteinMPNN from Baker Lab can reverse AlphaFold

December 5, 2022 / / 1 Comment

The software tool ProteinMPNN (Message Passing Neural Network) from Baker Lab can predict from a given 3D protein structure possible amino acid sequences that would fold into the given structure, in this way effectively reversing what AlphaFold from DeepMind or ESMFold from Meta can do. So the approach allows to design proteins. With a DNA/RNA printer as the BioXp from TelesisBio or the Syntax system from DNAScript it is possible to directly output the desired protein or a virus that generates the protein in a cell when injected into the body.

The source code is available on GitHub and has also already been integrated into a Hugging Face space. See also here.

Virtual Machine inside ChatGPT

December 4, 2022 / / 0 Comments

People start to realize what is possible with ChatGPT and there are already some great summaries available [1][2] . However, the currently most exciting prompt I have seen is to let ChatGPT answer like being a Linux terminal. This effectively opens up a complete virtual machine inside ChatGPT by predicting the proper text answer on user input. Doing this in a recursive way feels mind-blowing …

Lots of people are now experimenting with this idea.

More info about ChatGPT can be found in this video by Yannic Kilcher.

Scientists from Google AI, Caltech, Harvard, MIT, and Fermilab simulate a traversable wormhole with a quantum computer

December 2, 2022 / / 0 Comments

Researchers from Google AI, Caltech, Harvard, MIT, and Fermilab simulated a quantum theory on the Google Sycamore quantum processor to probe the dynamics of a quantum system equivalent to a wormhole in a gravity model.

The quantum experiment is based on the ER=EPR conjecture that states that wormholes are equivalent to quantum entanglement. ER stands for Einstein and Rosen who proposed the concept of wormholes (a term coined by Wheeler and Misner in a 1957 paper) in 1935, EPR stands for Einstein, Podolsky, and Rosen who proposed the concept of entanglement in May 1935, one month before the ER paper (see historical context). These concepts were completely unrelated until Susskind and Maldacena conjectured in 2013 that any pair of entangled quantum systems are connected by an Einstein-Rosen bridge (= non-traversable wormhole). In 2017 Jafferis, Gao, and Wall extended the ER=EPR idea to traversable wormholes. They showed that a traversable wormhole is equivalent to quantum teleportation [1][2].

The endeavor was published on Nov 30, 2022 in a Nature article. There is also a nice video on youtube explaining the experiment. Tim Andersen discusses in an interesting article whether or not a wormhole was created in the lab.

"I was born not knowing and have had only a little time to change that here and there."

Richard P. Feynman

"I have no special talent. I am only passionately curious."

Albert Einstein

"If you tell me precisely what it is a machine cannot do, then I can always make a machine which will do just that."

John von Neumann

© 2024 Stephan Seeger

Theme by Anders Noren — Up ↑