Stop A Python Script Without Losing Data

We have been running a script on partner's computer for 18 hours. We underestimated how long it would take, and now need to turn in the results. Is it possible to stop the script

Solution 1:

Well, OP doesn't seem to need an answer anymore. But I'll answer anyway for anyone else coming accross this. While it is true that stopping the program will delete all data from memory you can still save it. You can inject a debug session and save whatever you need before you kill the process.

Both PyCharm and PyDev support attaching their debugger to a running python application. See here for an explanation how it works in PyCharm. Once you've attached the debugger, you can set a breakpoint in your code and the program will stop when it hits that line the next time. Then you can inspect all variables and run some code via the 'Evaluate' feature. This code may save whatever variable you need.

I've tested this with PyCharm 2018.1.1 Community Edition and Python 3.6.4.

In order to do so I ran this code which I saved as test.py

import collections
import time

data = collections.deque(maxlen=100)
i = 0while True:
    data.append(i % 1000)
    i += 1
    time.sleep(0.001)

via the command python3 test.py from an external Windows PowerShell instance. Then I've opened that file in PyCharm and attached the debugger. I set a Breakpoint at the line i += 1 and it halted right there. Then I evaluated the following code fragment:

import json
with open('data.json', 'w') as ofile:
    json.dump(list(data), ofile)

And found all entries from data in the json file data.json.

---

Follow-up: This even works in an interactive session! I ran the very same code in a jupyter notebook cell and then attached the debugger to the kernel. Still having test.py open, I set the breakpoint again on the same line as before and the kernel halted. Then I could see all variables from the interactive notebook session.

Solution 2:

I don't think so. Stopping the program should also release all of the memory it was using.

edit: See Swenzel's comment for one way of doing it.