GNSS & Machine Learning Engineer

Month: July 2023

Room-temperature Superconductivity breakthrough?

A groundbreaking discovery has potentially been made in the field of superconductivity. Researchers from South Korea have developed a superconductor material, codenamed LK 99 (short for the authors Lee and Kim that made the first discovery of the material in 1999), that potentially operates at room temperature and atmospheric pressure. This would be a significant leap forward, overcoming the limitations of previous superconductors that required extremely low temperatures or high pressures to function.

Superconductivity, a quantum mechanical phenomenon where the electrical resistance of a material vanishes and magnetic flux fields are expelled from within the material, was first discovered by Dutch physicist Heike Kamerlingh Onnes in 1911. This discovery earned him the Nobel Prize in Physics in 1913. The implications of this phenomenon are vast, particularly for energy transmission and storage, as superconductors can conduct electricity with virtually no loss of energy.

One of the key features of superconductivity is the Meissner effect, where a superconductor in a magnetic field will expel the magnetic field within the material. This is due to the superconductor’s perfect diamagnetism, and it leads to phenomena such as magnetic levitation.

Another significant contribution to the understanding of superconductivity came from Vitaly Ginzburg and Alexei Abrikosov, who, along with Anthony Leggett, were awarded the Nobel Prize in Physics in 2003. Ginzburg and Abrikosov developed the Ginzburg-Landau theory in the 1950s, a phenomenological theory that describes superconductivity in the vicinity of the critical temperature. This theory successfully explains many properties of superconductors, including the Meissner effect, and it has been instrumental in the development of the theory of type II superconductors, which remain superconducting in the presence of strong magnetic fields.

The understanding of superconductivity took a significant leap forward in 1957 when John Bardeen, Leon Cooper, and John Robert Schrieffer proposed the BCS theory. This theory, which explains how electrical resistance in certain materials disappears at very low temperatures, earned them the Nobel Prize in Physics in 1972. The theory introduced the concept of Cooper pairs, where electrons with opposite momenta and spins pair up and move through the lattice of positive ions in the material without scattering and losing energy.

In 1986, the discovery of high-temperature superconductors by Georg Bednorz and K. Alex Müller, who were awarded the Nobel Prize in Physics in 1987, marked another milestone in the field. These materials exhibited superconducting properties at temperatures higher than those predicted by the BCS theory, opening up new possibilities for practical applications.

Each superconductor has a critical temperature below which it exhibits superconductivity, and some require a minimum pressure. Traditional superconductors need extreme cooling and sometimes high pressure. High-temperature superconductors work at warmer temperatures, but still below room level. The new material, LK 99, is groundbreaking as it remains superconducting at room temperature and atmospheric pressure.

The researchers published two papers discussing their findings on arXiv within two hours of each other on July 22, 2023. The first paper, “The First Room-Temperature Ambient-Pressure Superconductor”, was authored by Sukbae Lee, Ji-Hoon Kim, and Young-Wan Kwon. The second paper, “Superconductor Pb_10-x Cu_x (PO_4)_6 O showing levitation at room temperature and atmospheric pressure and mechanism”, was authored by the same first two researchers of the first paper along with Hyun-Tak Kim, Sungyeon Im, SooMin An, and Keun Ho Auh. The strategic authorship suggests a potential candidacy for the Nobel Prize, which can only be shared among three people.

In March 2023, the group filed for their international patent application, further solidifying their claim. However, the scientific community has expressed some skepticism due to a past incident. Randa Dias, a physicist at the University of Rochester, had a paper published in Nature in October 2020 claiming room-temperature superconductivity in a carbonaceous sulfur hydride under extreme pressure. The paper was retracted in September 2022 after other researchers were unable to replicate the results. While we await conclusive evidence supporting the claim of room-temperature superconductivity, you can monitor the scientific community’s assessment of the claim here.

The LK 99 material has a critical current of 250 mA at 300°K (27°C) that quickly drops towards almost 0 when reaching 400°K. The current generates a magnetic field that breaks down superconductivity. This is a crucial aspect as high currents for generating high magnetic fields are central for applications in MRIs and in fusion reactors, where the magnetic field is used for the confinement of the plasma.

The proposed superconductor is not only revolutionary but also simple and inexpensive to produce. The process involves three steps explicitly explained in the second paper using common materials: lead oxide, lead sulfate, copper powder, and phosphorus. The resulting compound, Pb10-xCux(PO4)6O, is achieved through a series of heating and mixing processes.

The use of copper instead of lead in the superconductor results in a shrinkage effect, which was previously achieved through high pressure. This is related to the concept of a quantum well, a potential well with discrete energy values. The quantum well effect is the underlying mechanism for superconductivity in LK-99.

The potential applications of room-temperature superconductors are transformative. They could lead to more efficient power transmission, reducing energy loss during transmission through power lines. They could also enable cheaper and simpler magnetic resonance imaging (MRI) machines, fusion reactors, high-speed magnetic trains, and quantum computers. In addition, they could lead to more efficient batteries, potentially revolutionizing the energy storage industry. A more detailed discussion of the implications of a room-temperature ambient-pressure superconductor that depends on whether strong or weak magnetic fields and currents are possible has been put together by Andrew Cote.

A comprehensive overview of this discovery has been provided in a YouTube video by ‘Two Bit da Vinci’.

The breakthrough discovery of the room-temperature superconductor LK 99 is not the only recent advancement in the field of superconductivity. In a related development, a team of scientists from MIT and their colleagues have created a simple superconducting device that could dramatically cut energy use in computing. This device, a type of diode or switch, could transfer current through electronic devices much more efficiently than is currently possible.

The team’s work, published in the July 13 online issue of Physical Review Letters, showcases a superconducting diode that is more than twice as efficient as similar ones reported by others. It could even be integral to emerging quantum computing technologies. The diode is nanoscopic, about 1,000 times thinner than the diameter of a human hair, and is easily scalable, meaning millions could be produced on a single silicon wafer.

The team discovered that the edge asymmetries within superconducting diodes, the ubiquitous Meissner screening effect found in all superconductors, and a third property of superconductors known as vortex pinning all came together to produce the diode effect. This discovery opens the door for devices whose edges could be “tuned” for even higher efficiencies.

These advancements in superconductivity, both in the creation of room-temperature superconductors and the development of highly efficient superconducting diodes, hold great promise for the future of technology and energy efficiency. They could lead to more efficient power transmission, revolutionize the energy storage industry, and dramatically cut the amount of energy used in high-power computing systems.

You can read more about the superconducting diode in the Phys.org article.

On July 29, 2023, there has been an additional announcement by Taj Quantum in Florida for a Type II room-temperature superconductor (US patent 17249094).

Update 03.01.2023 [1][2]: Two Chinese labs have now also found room-temperature superconductors.

Meta released Llama 2 free for Commercial Use

Meta open-sourced Llama 2 together with Microsoft, this time in contrast to Llama 1 free not just for research but also for commercial use.

  • Free for commercial use for businesses with less than 700 Mio monthly active users
  • Models with 70B, 13B, and 7B parameters
  • Llama-2-70B model is currently the strongest open-source LLM (Huggingface leaderboard), comparable to GPT-3.5-0301, noticeably stronger than Falcon, MPT, and Vicuna
  • Not yet at GPT-3.5 level, mainly because of its weak coding abilities
  • RLHF fine-tuned
  • Source code on GitHub, weights available on Azure, AWS, and HuggingFace
  • Llama 2 paper
  • 4K token context window
  • Trained on 2 trillion tokens with training costs of about $20M
  • Knowledge cut-off Dec 2022
  • Testing on https://www.llama2.ai

Just 4 days after this announcement, on July 22, 2023, StabilityAI released FreeWilly1 and FreeWilly2 which are fine-tuned models based on LLaMA65B and Llama-2-70B. These models took over the leadership on Hugging Face (Huggingface leaderboard). However, both models have no commercial license and are just intended for research.

GPT-4 in the top 1% of human thinkers in creativity test

In a recent study by the University of Montana, GPT-4 demonstrated remarkable performance in the Torrance Tests of Creative Thinking (TTCT, a standard test for measuring creativity), matching the top 1% of human thinkers. The model excelled in fluency and originality. These findings imply that the creative abilities of GPT-4 could potentially surpass those of humans.

For a recent benchmark on advanced reasoning capabilities of large language models take a look at the ARB (Advanced Reasoning Benchmark).

OpenAI gives all ChatGPT Plus users access to Code Interpreter

The ChatGPT code interpreter allows users to run code and upload individual data files (in .csv, .xlsx, .json format) for analysis. Multiple files can be uploaded sequentially or within one zip-file. To upload a file, click on the ‘+’ symbol located just to the left of the ‘Send a message’ box or even simpler via drag and drop.

The code interpreter functionality is accessible to ChatGPT Plus users and can be enabled in the settings under ‘Beta features’. Once enabled, this functionality will then appear in the configuration settings of any new chat under the ‘GPT-4’ section, where it also needs to be activated.

Given a prompt, the code interpreter will generate Python code that is then automatically executed in a sandboxed Python environment. If something goes wrong, for instance, if the generated source code requires the installation of a Python package or if the source code is simply incorrect, the code interpreter automatically attempts to fix these errors and tries again. This feature makes working with the code interpreter much more efficient. Before, it was necessary to paste ChatGPT’s proposal into a Jupyter notebook and run it from there. If errors occurred, these had to be fixed either independently or by manually pasting the error text back into ChatGPT so that it could provide a solution. This manual iterative procedure has now been automated with the code interpreter.

Note that the code interpreter executes the source code on OpenAI’s servers, not in the local environment. This leads to restrictions on the size of the uploaded data, as well as a very stringent time limit of 120s for the execution of the code. Given this, it becomes clear what developers truly desire. They seek the integration of this feature into their local development environment, such as VSCode, or within a cloud service, such as AWS, GCP, or Azure, without any restrictions on data size or execution times. This then leans more towards the direction of projects like AutoGPT or GPT Engineer. It’s likely only a matter of days, weeks, or months before such functionality becomes widely available. It’s also probable that complete access to your code repository will be enabled, first through a vector database solution and after some time maybe by including the entire repository within prompts, which are currently increasing dramatically in size (as exemplified in LongNet; since this requires retraining of the LLM such solutions cannot be expected to become available before GPT-4.5 or GPT-5).

For testing, try e.g. the following prompts:

  • What is the current time?
  • Plot the graphs of sin(x) and cos(x) in a single graph
  • Make a QR-code of my contact information: Stephan Seeger; Homepage: domain-seeger.de

or after uploading a data set (e.g. from Kaggle)

  • Explain the dataset.
  • Show 4 different ways of displaying the data visually.

Before, such functionality was only available via the Notable plugin or via the open-source implementation GPT-Code-UI on GitHub.

Microsoft scales Transformer sequence length to 1 billion tokens

LongNet, a new Transformer variant introduced in recent research by Microsoft, has successfully scaled sequence lengths to over 1 billion tokens without compromising shorter sequence performance. Its key innovation, dilated attention, allows an exponential expansion of the attentive field with growing distance. The model exhibits linear computational complexity and logarithmic token dependency, while also demonstrating strong performance on long-sequence modeling and general language tasks.

© 2024 Stephan Seeger

Theme by Anders NorenUp ↑