HTC26 Tackles 10-Terabyte Nightly Data Challenge in AI and Astrophysics
Updated
Updated · morgridge.org · Jun 22
HTC26 Tackles 10-Terabyte Nightly Data Challenge in AI and Astrophysics
1 articles · Updated · morgridge.org · Jun 22
Summary
June 9-12 at UW-Madison, HTC26 gathered international researchers to examine how throughput computing can handle fast-growing AI and astrophysics workloads rather than simply adding raw compute capacity.
10 terabytes of Rubin Observatory data each night and multimessenger astronomy pipelines are creating bottlenecks that CHTC says require adaptable tools such as HTCondor and the Pelican Platform, plus broader web-based access for everyday astronomers.
5 to 10 terabytes a week from 50 wastewater sites are also flowing through CHTC for metagenomics, where researchers said biology faces a major software gap that leaves graduate students doing both science and engineering.
2026, designated by UW-Madison as a year of AI readiness and competency, framed debate over agentic AI: researchers saw promise in tools like AlphaFold3 but said they still need investment and better integration to move beyond "party tricks."
CHTC cast the meeting as part of its translational model—turning researcher needs into shared infrastructure and then refining that infrastructure through feedback from fields ranging from small colleges to NASA-scale projects.