Nearly a decade after Google researchers introduced the transformer, the architecture has become the backbone of all major large language models. However, its limitations are becoming more apparent as models scale up. The dense attention mechanism grows increasingly costly with larger text inputs, and it struggles to manage extensive information simultaneously.
In response, several startups are pursuing novel approaches to replace or improve upon the transformer. These innovations aim to make LLMs faster, more efficient, and potentially more intelligent, according to MIT Technology Review's What's Next series.
Meanwhile, the landscape for academic AI research is shifting. At a recent gathering of the Schmidt Sciences AI2050 program in Mountain View, California, university researchers discussed the changing dynamics of their field. The program, funded by Eric and Wendy Schmidt, supports academics working on AI-related projects. Many fellows noted the unusual pressures and opportunities facing AI researchers in academia today.
In other technology news, Nvidia has secured $500 billion from Wall Street for AI infrastructure, with deals involving BlackRock, Goldman Sachs, and four other firms. This marks a significant investment in AI compute as an emerging asset class.
Other notable developments include Mark Zuckerberg's new manifesto advocating open-source AI as a way to save the US, Bernie Sanders calling for a pause on AI development, and a US court allowing thousands of social media lawsuits to proceed over addictive mechanisms.
Additionally, Chinese humanoid robot firm Unitree's IPO was oversubscribed by retail investors by more than 8,000 times, raising $900 million. Flock's car-tracking cameras are facing bipartisan backlash, and China has introduced new rules for emotionally interactive AI.
An AI tool claiming to select the top 1% of scientific papers has drawn skepticism from researchers, while the backlash against AI-generated content is pushing platforms to restrict it. An 82-year-old farmer rejected $26 million to turn her land into a data center, citing environmental concerns.
Senator Bernie Sanders, in a letter to AI leaders, urged: "It is not too late to avoid disaster. Stop building machines that humans cannot control."
Source: MIT Technology Review
No comments yet. Be the first to share your thoughts.