📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems are now capable of automating most engineering tasks in AI development, with research remaining the last frontier. This shift could accelerate AI progress significantly, but questions remain about the nature of research automation.
Recent empirical data from multiple AI capability benchmarks confirm that AI systems can now automate the majority of core engineering tasks involved in AI development, leaving research as the residual challenge. This shift has significant implications for the future pace of AI progress and the structure of AI research and development.
Multiple independent benchmarks, including CORE-Bench and MLE-Bench, demonstrate that AI systems have achieved near-saturation levels in core engineering skills. For example, CORE-Bench, which measures the ability to reproduce research experiments, reached 95.5% performance in December 2025, with some authors declaring it ‘solved.’ Similarly, MLE-Bench, assessing performance on Kaggle competitions, reached 64.4%, comparable to mid-tier human practitioners, by February 2026. These advances suggest that reproducing experiments and optimizing models are now primarily engineering problems that AI can handle reliably.
In contrast, the capacity of AI to automate research activities—such as hypothesis generation, novel experimentation, and creative problem-solving—remains less certain. While some progress in automating parts of research workflows is evident, the structural question of whether research itself is reducible to engineering tasks at scale is still open. Clark’s analysis indicates that the bottleneck is shifting from engineering to the research phase, which may require different approaches or remain inherently human-centric.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
![Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results](https://m.media-amazon.com/images/I/415+fSJacsL._SL500_.jpg "Engineering Is Automated. Research Is the Residual. 3")
Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
AI research automation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
AI engineering automation hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
AI experiment reproduction tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Engineering Automation for AI Development Pace
The automation of core engineering tasks in AI development suggests a potential acceleration in the overall pace of AI progress. As the bottleneck shifts from engineering to research, organizations may need to rethink their R&D strategies, investing more in AI systems capable of handling research-level tasks or re-evaluating the role of human researchers. This could lead to faster iteration cycles, more rapid deployment of advanced models, and a possible reduction in development costs. However, it also raises questions about the future of research innovation and the limits of AI’s creative capacities.
Recent Advances in AI Capabilities and Benchmark Trends
Over the past year, multiple benchmarks have shown rapid progress in AI’s ability to perform tasks traditionally associated with research and engineering. CORE-Bench, which tests research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025. MLE-Bench, evaluating Kaggle competition performance, advanced from 16.9% in October 2024 to 64.4% in February 2026. Concurrently, research papers have documented improvements in kernel design, automated code conversion, and infrastructure optimization, indicating that engineering is increasingly handled by AI systems. These trends suggest a converging pattern of rapid capability saturation across different AI skill domains.
“The pattern across these benchmarks indicates that AI is now capable of automating vast swaths of engineering tasks involved in AI development, with research remaining the last frontier.”
— Thorsten Meyer
Uncertain Limits of Research Automation
While engineering tasks are nearing full automation, the extent to which AI can autonomously conduct research—such as generating novel hypotheses, designing experiments, and interpreting results—remains unclear. The structural question of whether research can be fully reduced to engineering at scale is still open, with some experts suggesting it may be inherently more complex or require creative insight that AI has yet to master.
Next Steps for AI Capability Development and Research Automation
Researchers and organizations will likely focus on understanding the boundaries of AI-driven research automation over the next 32 months. Efforts may include developing new benchmarks to measure research-level capabilities, exploring hybrid human-AI research models, and investigating whether AI can generate truly novel scientific insights. Monitoring progress in these areas will be crucial to anticipate how AI may reshape the landscape of scientific discovery and engineering.
Key Questions
What does it mean that engineering is now automated?
It means AI systems can now reliably perform core engineering tasks involved in AI development, such as reproducing research experiments, optimizing models, and designing infrastructure, reducing the need for human intervention in these areas.
Why is research still considered the residual challenge?
Research involves creative processes like hypothesis generation, experimental design, and interpretation, which are less well-understood to be automatable and may inherently require human insight or new forms of AI capabilities yet to be developed.
How might this shift impact AI development timelines?
If research remains the bottleneck, progress may accelerate in engineering but not necessarily in scientific discovery. Conversely, if research automation advances, overall AI progress could speed up significantly.
Are there risks associated with automating engineering tasks?
Potential risks include over-reliance on AI for critical development steps, possible loss of human expertise, and challenges in ensuring quality and safety as automation increases.
What should organizations do in response to these developments?
Organizations should invest in understanding the limits of AI automation, develop new benchmarks for research capabilities, and prepare for a possible future where research is also automated, to stay competitive and innovative.
Source: ThorstenMeyerAI.com
