Python profiling

From wikinotes
Revision as of 12:49, 13 September 2020 by Will (talk | contribs) (→‎Memory)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Profiling is the act watching how long it takes for your code to run.

Memory

Even following object deletion/gc, python will not necessarily release memory to the OS.
It is critical to encapsulate memory-hungry sections of your program in separate processes.
Consider serializing data in a short-lived process, then referring to it by index to keep peak memory usage low.
See python traps for more details.


python tracemalloc memory profiler builtin to python-3.4+
python memory_profiler python wrapper that outputs a top-like table summary of memory usage
python pympler
python objgraph filter-by/produce graphviz graphs of objects in memory
python resource bulitin lib to examine/modify resources used by current program
python guppy detailed memory profiling

Performance

Profilers

python timeit
python cProfile
python pprofile

Visualizations

python vprof a truly excellent profiler. it's actually convenient.
python runSnakeRun my favourite - python2 only
python snakeviz decent profiler, python3+