Nvidia wants to give AI servers more advance notice: instead of allocating resources only when individual requests arrive, agents could report their likely next model calls. Published on September 17, application US20260278406A1 describes a combination of prior workflow profiling, runtime forecasts and resource control.
The server learns what might come next
Previously recorded agent workflows provide training data for prediction models. During execution, these estimate upcoming model calls, token volumes and execution times. The application attaches this information to inference requests; the server uses it to allocate resources. Hints about recurring prompt prefixes can also help reuse previously computed data. Claim 1 describes the workflow, claim 12 the transmission and cache hints, and claim 13 planning based on expected GPU memory demand and runtime. The evidence is the original USPTO publication.
Nvidia's NeMo Agent Toolkit 1.7 documentation provides a practical connection: it describes forecasts derived from recorded workflows and hints sent to the Dynamo backend for resource management. That establishes related software functionality, not full implementation or patentability of every claim.
Analysis: useful if the forecast holds
The interesting idea is the additional view of the whole workflow. It could reduce waiting and poorly matched resource reservations. Branching, new tasks or long tool calls can invalidate forecasts, however. Whether improved planning outweighs its own overhead requires measurement in each deployment. The application alone establishes neither a specific performance gain nor product readiness.
Document and status
The applicant is NVIDIA Corporation. US application 19/358,820 was filed on October 15, 2025, claiming priority to US 63/772,593 of March 16, 2025. Its title is “Online Inference Optimization and Control of Agent Workflows”. A1 identifies the published application here, not a patent grant. We could not independently confirm the current prosecution status in the Patent Center register on September 19, 2026.