New ask Hacker News story: Ask HN: How do you run long computations without interruption?
Ask HN: How do you run long computations without interruption?
3 by annowiki | 4 comments on Hacker News.
Recently I started messing around with numpy trying to write a government simulator to analyze the influence of government institution structure on the frequency of laws being passed (i.e., how liberal or conservative a government's outcomes are across similar populations) and I found the simulations to take a really long time to run (probably because the code is badly optimized). When I spent ten minutes waiting for completion only to see a completely faulty result I started wondering how most scientists and data programmers run long simulations without interruptions or disappointing results. I assume one simple thing I should have done was run a shorter simulation to see if the code was working, but I also started thinking about how I could run a really long simulation without having to worry about the simulation being scrapped after a lot of computation, either because of a runtime error or just a power outage or something. I know there are cloud services to run expensive computations, but really my question is: are there industry tools or techniques for pause/resume execution of code and simulations? I can't imagine anyone running a simulation for more than ten minutes in a Jupyter cell.
3 by annowiki | 4 comments on Hacker News.
Recently I started messing around with numpy trying to write a government simulator to analyze the influence of government institution structure on the frequency of laws being passed (i.e., how liberal or conservative a government's outcomes are across similar populations) and I found the simulations to take a really long time to run (probably because the code is badly optimized). When I spent ten minutes waiting for completion only to see a completely faulty result I started wondering how most scientists and data programmers run long simulations without interruptions or disappointing results. I assume one simple thing I should have done was run a shorter simulation to see if the code was working, but I also started thinking about how I could run a really long simulation without having to worry about the simulation being scrapped after a lot of computation, either because of a runtime error or just a power outage or something. I know there are cloud services to run expensive computations, but really my question is: are there industry tools or techniques for pause/resume execution of code and simulations? I can't imagine anyone running a simulation for more than ten minutes in a Jupyter cell.
Comments
Post a Comment