Hacker Times
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
xmonkee
on June 14, 2024
|
parent
|
context
|
favorite
| on:
Cost of self hosting Llama-3 8B-Instruct
Does anyone know the impact of the prompt size in terms of throughput? If I'm only generating 10 tokens, does it matter if my initial prompt is 10 tokens or 8000 tokens? How much does it matter?
Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
