New ask Hacker News story: Ask HN: How would one go about building an API for fine tuning?
Ask HN: How would one go about building an API for fine tuning?
2 by samlhuillier | 0 comments on Hacker News.
Let's say I wanted to build an API to let users upload images and the api would fine tune stable diffusion for them returning either a checkpoint or another api that let's them run inference on the fine-tuned model. Does anyone have any architecture considerations/issues they'd suggest? Two things I'm considering: - Would the problem with this approach of ad-hoc GPUs likely be cold boot? It would take a shit ton of time to load. Though with data center networks speeds that wouldn't be too much of an issue - considering the fine-tuning itself would likely dwarf boot times. - Is it possible to launch remote GPU instances ad-hoc from code? Is there a service that provides this service? Every time a call is made we'd spin up a GPU Maybe the best approach for a V1 is to use the AWS SDK or something similar to just launch instances as calls come in. Appreciate the help!
2 by samlhuillier | 0 comments on Hacker News.
Let's say I wanted to build an API to let users upload images and the api would fine tune stable diffusion for them returning either a checkpoint or another api that let's them run inference on the fine-tuned model. Does anyone have any architecture considerations/issues they'd suggest? Two things I'm considering: - Would the problem with this approach of ad-hoc GPUs likely be cold boot? It would take a shit ton of time to load. Though with data center networks speeds that wouldn't be too much of an issue - considering the fine-tuning itself would likely dwarf boot times. - Is it possible to launch remote GPU instances ad-hoc from code? Is there a service that provides this service? Every time a call is made we'd spin up a GPU Maybe the best approach for a V1 is to use the AWS SDK or something similar to just launch instances as calls come in. Appreciate the help!
Comments
Post a Comment