Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
Jgrubb
3 months ago
|
parent
|
context
|
favorite
| on:
Don't trust large context windows
The tokens are still being burnt, they're just doing so in a parallel dimension from the users main context window.
ajmurmann
3 months ago
|
next
[–]
It's true that the initial tool response still has the same amount of tokens but it doesn't keep dragged along in the longer-lived top context.
knollimar
3 months ago
|
parent
|
next
[–]
Don't you resend after every turn, so splitting it avoids the n^2 token usage (granted it's cached so there's some optimal amount here)
ajmurmann
3 months ago
|
root
|
parent
|
next
[–]
Yes, exactly. You resend it on every turn (assuming no cache hits). This is why using the shorter-lived subagent to take in that context and only return the useful result back to the longer-lived context safes tokens.
ViewTrick1002
3 months ago
|
prev
[–]
The real benefit is being able to use a cheaper, but good enough, model with a specific system prompt dedicated to that task.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: