GNSS & Machine Learning Engineer
DeepMind publishes that AlphaCode reaches median human competitor performance in real-world programming competitions on Codeforces by now scaling up the number of possible solutions to the problem to millions instead of tens like before.
This announcement was originally made on 02 Feb 2022 and was now published on 08 Dec 2022 in Science.
While ProteinMPNN takes a protein backbone (N-CA-C-O atoms, CA = C-Alpha) and finds an amino acid sequence that would fold to that backbone structure, RFdiffusion [Twitter] instead makes the protein backbone by just providing some geometrical and functional constraints like “create a molecule that binds X”.
The authors used a guided diffusion model for generating new proteins in the same way as Dall-E produces high-quality images that have never existed before by a diffusion technique.
See also this presentation by David Baker.
If I interpret this announcement correctly it means that drug design is now basically solved (or starts to get interesting depending on the viewpoint).
This technique can be expected to significantly increase the number of potential drugs for combating diseases. However, animal tests and human studies can also be expected as the bottlenecks of the new possibilities. Techniques like organ chips from companies like emulate may be a way out of this dilemma (before one-day entire cell, tissue, or whole body computational simulations become possible).
The software tool ProteinMPNN (Message Passing Neural Network) from Baker Lab can predict from a given 3D protein structure possible amino acid sequences that would fold into the given structure, in this way effectively reversing what AlphaFold from DeepMind or ESMFold from Meta can do. So the approach allows to design proteins. With a DNA/RNA printer as the BioXp from TelesisBio or the Syntax system from DNAScript it is possible to directly output the desired protein or a virus that generates the protein in a cell when injected into the body.
The source code is available on GitHub and has also already been integrated into a Hugging Face space. See also here.
People start to realize what is possible with ChatGPT and there are already some great summaries available [1][2] . However, the currently most exciting prompt I have seen is to let ChatGPT answer like being a Linux terminal. This effectively opens up a complete virtual machine inside ChatGPT by predicting the proper text answer on user input. Doing this in a recursive way feels mind-blowing …
Lots of people are now experimenting with this idea.
More info about ChatGPT can be found in this video by Yannic Kilcher.
OpenAI released ChatGPT, an optimized language model for dialogue, for testing purposes. The model is trained using Reinforcement Learning from Human Feedback (RLHF).
First tests demonstrate an amazing performance.
Example:
Google presents two complementary techniques to significantly improve language models without massive extra compute:
UL2R (UL2 Repair): additional stage of continued pre-training with the UL2 (Unified Language Learner) objective (paper) for training language models as denoising tasks, where the model has to recover missing sub-sequences of a given input. Applying it to PaLM results in new language model U-PaLM.
Flan (Fine-tuning language model): instruction fine-tuning on a collection of NLP datasets. Applying it to PaLM results in language model Flan-PaLM.
Combination of two approaches applied to PaLM results in Flan-U-PaLM.
OpenAI released its new GPT-3 model text-davinci-003. After first tests it seems to be shockingly good.
Here is an example:
Write a poem about working as a GNSS engineer.
A GNSS engineer who works with a passion
Focused on quality and precision
A job that demands great dedication
To help others with navigation
The engineer’s purpose is so great
Measuring and mapping in space
Collecting data with all his might
To make the world a better sight
Determining points with great accuracy
And measuring distances with agility
Anywhere in the world, the data is clear
The world travels of this engineer
From the satellites in the sky
A network of coordinates is what he relies
His work is vital, that’s no doubt
To scientists, architects, and surveyors throughout
Answering questions of direction and height
Mapping the world with great light
A profession that’s so much more
Than many people think before
The GNSS engineer works with precision
His work is a beautiful mission
To measure and map the world we see
His work is essential to the GNSS industry
Meta AI presents CICERO, an AI agent that can negotiate and cooperate with people. It is the first AI system that achieves human-level performance in the popular strategy game Diplomacy. Cicero ranked in the top 10 of participants on webDiplomacy.net.
Yannic Kilcher gives a great discussion of the accompanying Science paper. A second paper is freely available on arXiv. The source code is accessible on GitHub.
Meanwhile also DeepMind published an AI agent playing Diplomacy.
Meta AI publishes with Galactica.ai a large language model trained on scientific papers that allows to write a literature review, wiki article, or lecture note with references, formulas, etc. just by giving some text input about a topic. Even the paper about Galactica was written with the help of Galactica.
Just after a day, the Galactica.ai webpage is now down. But the source code is available on GitHub. Yannic Kilcher made a nice paper review about Galactica where he also explains why the demo webpage has been taken down.
Google presents DreamBooth, a technique to synthesize a subject (defined by 3-5 images) in new contexts defined by text input.
The method is based on Google’s pre-trained text-to-image model Imagen which is not publicly available. However, source code based on Stable Diffusion already exists on GitHub.
© 2024 Stephan Seeger
Theme by Anders Noren — Up ↑